Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Dec 07 19:58

    mworrell on l10n_master

    (compare)

  • Dec 07 19:58

    mworrell on master

    New Crowdin updates (#3208) (compare)

  • Dec 07 19:58
    mworrell closed #3208
  • Dec 07 18:41
    michielklonhammer opened #3208
  • Dec 07 18:41

    michielklonhammer on l10n_master

    New translations zotonic.pot (E… (compare)

  • Dec 07 18:41

    michielklonhammer on l10n_master

    New translations zotonic.pot (T… (compare)

  • Dec 07 18:41

    michielklonhammer on l10n_master

    New translations zotonic.pot (I… (compare)

  • Dec 07 18:41

    michielklonhammer on l10n_master

    New translations zotonic.pot (P… (compare)

  • Dec 07 18:41

    michielklonhammer on l10n_master

    New translations zotonic.pot (E… (compare)

  • Dec 07 18:41

    michielklonhammer on l10n_master

    New translations zotonic.pot (C… (compare)

  • Dec 07 18:41

    michielklonhammer on l10n_master

    New translations zotonic.pot (C… (compare)

  • Dec 07 18:41

    michielklonhammer on l10n_master

    New translations zotonic.pot (U… (compare)

  • Dec 07 18:41

    michielklonhammer on l10n_master

    New translations zotonic.pot (T… (compare)

  • Dec 07 18:41

    michielklonhammer on l10n_master

    New translations zotonic.pot (S… (compare)

  • Dec 07 18:41

    michielklonhammer on l10n_master

    New translations zotonic.pot (R… (compare)

  • Dec 07 18:41

    michielklonhammer on l10n_master

    New translations zotonic.pot (P… (compare)

  • Dec 07 18:41

    michielklonhammer on l10n_master

    New translations zotonic.pot (P… (compare)

  • Dec 07 18:40

    michielklonhammer on l10n_master

    New translations zotonic.pot (D… (compare)

  • Dec 07 18:40

    michielklonhammer on l10n_master

    New translations zotonic.pot (I… (compare)

  • Dec 07 18:40

    michielklonhammer on l10n_master

    New translations zotonic.pot (H… (compare)

Maas-Maarten Zeeman
@mmzeeman
As admin you should be able to see them, otherwise the user should be able use mod-backup
Also, don't forget to press the "Publish" button on the access control rule page, otherwise you new rules won't be active.
I forget that often because it is scrolled out of view.
Arthur Clemens
@ArthurClemens
I am playing the code in the console.
Dir = z_path:files_subdir_ensure("backup", Context). => returns the correct directory = ok
Files = z_utils:wildcard(filename:join(Dir, "*.sql")). => empty array
filename:join(Dir, "*.sql”). => path/to/site//files/backup/*.sql = ok
z_utils:wildcard(filename:join(Dir, "*.sql")). => empty array
Arthur Clemens
@ArthurClemens

In mod_backup line 412:

Files = z_utils:wildcard(filename:join(dir(Context), "*.sql")),

concatenates the full directory path with the wildcard. This results in an empty array.
But when calling z_utils:wildcard/2 (with the directory as 2nd parameter), it works correctly:

Files = z_utils:wildcard("*.sql", dir(Context)),
z_utils:filter_dot_files removes the files with a dot, and the path contains .local/, so...
You won’t see this on a local dev machine if it is a Mac
Arthur Clemens
@ArthurClemens
It may be that no_dot_file doesn’t do the right thing. It should remove files that start with a dot, not files with a path that contains a dot.
Maas-Maarten Zeeman
@mmzeeman
Ah.. those .local dirs where added at a later stage.
That is probably also the reason why my log files are not available... Those file routines need some tlc.
Marc Worrell
@mworrell
Maybe we should only remove .git and .. ?
or explicitly allow .local and similar?
Arthur Clemens
@ArthurClemens
To prevent future problems I suggest to make the rule as permissive as possible
Marc Worrell
@mworrell
Then only suppress a blacklist, to start .git, . and ... Agree @mmzeeman ?
Maas-Maarten Zeeman
@mmzeeman
In OTP20 filelib:find_file/3 was added. We could have a look to see if that brings any benefit for this. Otherwise restricting the wildcard to something which matches the backup filename a bit more restrictive than *.sql. (Did a rm -rf * .o in a project once. It cleaned up the directory pretty well, but ot didn't help to meet the deadline)
Maas-Maarten Zeeman
@mmzeeman
Hmm... find_file/3 is not for this kind of stuff. OTP 23 added https://www.erlang.org/doc/man/filelib.html#safe_relative_path-2 though to prevent traversal attacks... Including .git is logical I think. Although not strictly needed if the pattern against the backup files are a little bit more strict and contain the date pattern. The module can otherwise accidentally delete aribtrary .sql files.
Marc Worrell
@mworrell
Looking into the find_file issue and the crash.
Arthur Clemens
@ArthurClemens
In the logs I’m seeing another (silent) crash:
2022-01-04 00:26:07.698 [error] <0.26448.8> Cowboy stream 1 with ranch listener zotonic_https_listener_ipv4 
and connection process <0.26442.8> had its request process exit with reason: {nocatch,{stop_request,500,
{function_clause,[{cowmachine_util,parse_qs_name,[<<"=PHPB8B5F2A0-3C92-11d3-A3A9-4C7B08C10000">>,[],<<>>],[
  {file,"/home/zotonic/zotonic/_build/default/lib/cowmachine/src/cowmachine_util.erl"},{line,284}]},
  {z_context,ensure_qs,1,[{file,"/home/zotonic/zotonic/apps/zotonic_core/src/support/z_context.erl"},{line,
  523}]},{z_cowmachine_middleware,'-execute/2-fun-0-',1,[{file,"/home/zotonic/zotonic/apps/zotonic_core/src/
  support/z_cowmachine_middleware.erl"},{line,54}]},{cowmachine_decision_core,...},...]}}} in 
  cowmachine_decision_core:handle_request/2 line 40
hmm, someone is testing if the PHP server is exposing sensitive data. Still I don’t think the proces should crash.
Arthur Clemens
@ArthurClemens
This can be solved by returning an empty list if nothing can be parsed.
cowmachine_util.erl:303:
parse_qs_name(_, _Acc, _Name) -> [].
Marc Worrell
@mworrell
Can you add this as an issue in cowmachine? Then we pick it up there.
Arthur Clemens
@ArthurClemens
Sure
Marc Worrell
@mworrell
I am testing fixes for the file dirs and the query crash right now.
Arthur Clemens
@ArthurClemens

I am reading https://test.zotonic.com/page/1790/proxying-zotonic-with-nginx that states:

Remember to add X-Forwarded-Proto to proxied header so that Zotonic knows that HTTPS is used before the proxy even though HTTP is used between the proxy and backend. Also add X-Real-IP and X-Forwarded-For headers.

Isn’t this already included in the provided example?

Marc Worrell
@mworrell
I have a pull request ready for the previous issues zotonic/zotonic#2837
About the nginx config - I think we repeat it to ensure that nobody removes it.
Arthur Clemens
@ArthurClemens
check
Marc Worrell
@mworrell
Merged the cowmachine changes into zotonic master.
Arthur Clemens
@ArthurClemens
great
Marc Worrell
@mworrell
Also merged the fixes for the directory scans and the search crash
Arthur Clemens
@ArthurClemens
Nice
I’ve noticed that my paging counter is off by 2 pages at the end. On page 1 it shows the last page number “260”. When I click it, the page is empty, and so is page 259. When I arrive at page 258 this is now shown as the last page.
When I go back to page 256, the link to page 260 is visible again.
Marc Worrell
@mworrell
That is because psql estimates the number of rows now. We don’t do a physical count of all the rows anymore.
On page 258 we see from the result that it is the last page.
On the pages before we use the query planner to give an estimate.
We always fetch the current page + 1, so that we will never show a “next” if there isn’t a next page.
Arthur Clemens
@ArthurClemens
I see
Marc Worrell
@mworrell
I am thinking of adding an extra flag in the search result. Like is_count_estimated
The we can adapt the shown result to something like page 2 of about 260
Arthur Clemens
@ArthurClemens
That would not improve the pager though
Marc Worrell
@mworrell
Yes, that estimated is useful for a counter. For the pager it might be a hint to not display the “last page” navigation. Just like Google et al.
Jonathan
@steele232
Hey @mworrell Is there a good tool to find connections between different resources and the path between them?
Jonathan
@steele232
I know I could export daya to RDF type data and then use other tools to find paths but I was just wondering if there is a way to do it in Zotonic natively
Jonathan
@steele232
I have been able to put various relationships on the page for each resource so I can get pretty far with that. If I tried to do a ton of traversals, would that result in a a ton of calls to the database?
Marc Worrell
@mworrell
Hi, maybe you could so some grouped calls. Important to restrict the max traversal depth.
It is an interesting subject.
How far do you want to follow the edges?
Jonathan
@steele232
Long story short. Finding family tree / common ancestors between individuals could be a mountain of edge traversals and finding shared interests among individuals would also be quite a lot of edge traversals. I would be willing to write some custom code for it possibly as well. Can grouping db calls work with either or those two use cases or maybe you know of a better way?
And I could potentially restrict traversal depth or make it configurable.
Thanks for your help and time