Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Oct 05 12:27

    mworrell on master

    Upgrade s3filez (compare)

  • Sep 29 09:07

    mworrell on master

    mod_backup: fix for periodic up… (compare)

  • Sep 29 07:58

    mworrell on backup-upload-filestore

    (compare)

  • Sep 29 07:58

    mworrell on master

    mod_backup: less backups, uploa… (compare)

  • Sep 29 07:58
    mworrell closed #3141
  • Sep 29 07:28
    mworrell edited #3141
  • Sep 29 07:26
    mworrell synchronize #3141
  • Sep 29 07:26

    mworrell on backup-upload-filestore

    Upgrade webdavfilez (compare)

  • Sep 28 17:50
    mworrell synchronize #3141
  • Sep 28 17:50

    mworrell on backup-upload-filestore

    WebDAV fixes (compare)

  • Sep 28 16:56
    mworrell synchronize #3141
  • Sep 28 16:56

    mworrell on backup-upload-filestore

    Fix streamn_start handling (compare)

  • Sep 28 16:42
    mworrell synchronize #3141
  • Sep 28 16:41

    mworrell on backup-upload-filestore

    Add WebDAV support. (compare)

  • Sep 27 06:23
    mworrell synchronize #3141
  • Sep 27 06:23

    mworrell on backup-upload-filestore

    Extra specs; update backup list… (compare)

  • Sep 26 17:10
    mworrell demilestoned #3125
  • Sep 26 17:10
    mworrell milestoned #3125
  • Sep 26 17:10
    mworrell milestoned #3125
  • Sep 26 17:10
    mworrell demilestoned #3125
Arthur Clemens
@ArthurClemens
z_utils:filter_dot_files removes the files with a dot, and the path contains .local/, so...
You won’t see this on a local dev machine if it is a Mac
It may be that no_dot_file doesn’t do the right thing. It should remove files that start with a dot, not files with a path that contains a dot.
Maas-Maarten Zeeman
@mmzeeman
Ah.. those .local dirs where added at a later stage.
That is probably also the reason why my log files are not available... Those file routines need some tlc.
Marc Worrell
@mworrell
Maybe we should only remove .git and .. ?
or explicitly allow .local and similar?
Arthur Clemens
@ArthurClemens
To prevent future problems I suggest to make the rule as permissive as possible
Marc Worrell
@mworrell
Then only suppress a blacklist, to start .git, . and ... Agree @mmzeeman ?
Maas-Maarten Zeeman
@mmzeeman
In OTP20 filelib:find_file/3 was added. We could have a look to see if that brings any benefit for this. Otherwise restricting the wildcard to something which matches the backup filename a bit more restrictive than *.sql. (Did a rm -rf * .o in a project once. It cleaned up the directory pretty well, but ot didn't help to meet the deadline)
Maas-Maarten Zeeman
@mmzeeman
Hmm... find_file/3 is not for this kind of stuff. OTP 23 added https://www.erlang.org/doc/man/filelib.html#safe_relative_path-2 though to prevent traversal attacks... Including .git is logical I think. Although not strictly needed if the pattern against the backup files are a little bit more strict and contain the date pattern. The module can otherwise accidentally delete aribtrary .sql files.
Marc Worrell
@mworrell
Looking into the find_file issue and the crash.
Arthur Clemens
@ArthurClemens
In the logs I’m seeing another (silent) crash:
2022-01-04 00:26:07.698 [error] <0.26448.8> Cowboy stream 1 with ranch listener zotonic_https_listener_ipv4 
and connection process <0.26442.8> had its request process exit with reason: {nocatch,{stop_request,500,
{function_clause,[{cowmachine_util,parse_qs_name,[<<"=PHPB8B5F2A0-3C92-11d3-A3A9-4C7B08C10000">>,[],<<>>],[
  {file,"/home/zotonic/zotonic/_build/default/lib/cowmachine/src/cowmachine_util.erl"},{line,284}]},
  {z_context,ensure_qs,1,[{file,"/home/zotonic/zotonic/apps/zotonic_core/src/support/z_context.erl"},{line,
  523}]},{z_cowmachine_middleware,'-execute/2-fun-0-',1,[{file,"/home/zotonic/zotonic/apps/zotonic_core/src/
  support/z_cowmachine_middleware.erl"},{line,54}]},{cowmachine_decision_core,...},...]}}} in 
  cowmachine_decision_core:handle_request/2 line 40
hmm, someone is testing if the PHP server is exposing sensitive data. Still I don’t think the proces should crash.
Arthur Clemens
@ArthurClemens
This can be solved by returning an empty list if nothing can be parsed.
cowmachine_util.erl:303:
parse_qs_name(_, _Acc, _Name) -> [].
Marc Worrell
@mworrell
Can you add this as an issue in cowmachine? Then we pick it up there.
Arthur Clemens
@ArthurClemens
Sure
Marc Worrell
@mworrell
I am testing fixes for the file dirs and the query crash right now.
Arthur Clemens
@ArthurClemens

I am reading https://test.zotonic.com/page/1790/proxying-zotonic-with-nginx that states:

Remember to add X-Forwarded-Proto to proxied header so that Zotonic knows that HTTPS is used before the proxy even though HTTP is used between the proxy and backend. Also add X-Real-IP and X-Forwarded-For headers.

Isn’t this already included in the provided example?

Marc Worrell
@mworrell
I have a pull request ready for the previous issues zotonic/zotonic#2837
About the nginx config - I think we repeat it to ensure that nobody removes it.
Arthur Clemens
@ArthurClemens
check
Marc Worrell
@mworrell
Merged the cowmachine changes into zotonic master.
Arthur Clemens
@ArthurClemens
great
Marc Worrell
@mworrell
Also merged the fixes for the directory scans and the search crash
Arthur Clemens
@ArthurClemens
Nice
I’ve noticed that my paging counter is off by 2 pages at the end. On page 1 it shows the last page number “260”. When I click it, the page is empty, and so is page 259. When I arrive at page 258 this is now shown as the last page.
When I go back to page 256, the link to page 260 is visible again.
Marc Worrell
@mworrell
That is because psql estimates the number of rows now. We don’t do a physical count of all the rows anymore.
On page 258 we see from the result that it is the last page.
On the pages before we use the query planner to give an estimate.
We always fetch the current page + 1, so that we will never show a “next” if there isn’t a next page.
Arthur Clemens
@ArthurClemens
I see
Marc Worrell
@mworrell
I am thinking of adding an extra flag in the search result. Like is_count_estimated
The we can adapt the shown result to something like page 2 of about 260
Arthur Clemens
@ArthurClemens
That would not improve the pager though
Marc Worrell
@mworrell
Yes, that estimated is useful for a counter. For the pager it might be a hint to not display the “last page” navigation. Just like Google et al.
Jonathan
@steele232
Hey @mworrell Is there a good tool to find connections between different resources and the path between them?
Jonathan
@steele232
I know I could export daya to RDF type data and then use other tools to find paths but I was just wondering if there is a way to do it in Zotonic natively
Jonathan
@steele232
I have been able to put various relationships on the page for each resource so I can get pretty far with that. If I tried to do a ton of traversals, would that result in a a ton of calls to the database?
Marc Worrell
@mworrell
Hi, maybe you could so some grouped calls. Important to restrict the max traversal depth.
It is an interesting subject.
How far do you want to follow the edges?
Jonathan
@steele232
Long story short. Finding family tree / common ancestors between individuals could be a mountain of edge traversals and finding shared interests among individuals would also be quite a lot of edge traversals. I would be willing to write some custom code for it possibly as well. Can grouping db calls work with either or those two use cases or maybe you know of a better way?
And I could potentially restrict traversal depth or make it configurable.
Thanks for your help and time
Jonathan
@steele232
I could probably do family tree stuff in a different tool but I would like to do the shared interests type thing in Zotonic if possible.
And I would be willing to write code to make it happen
Gaurav
@webspeller
Hi
I am trying to make an ecommerce website and using Zotonic for the same. I am not able to install the docker image, I am sorry but I don't find instructions to be exact.
I am using Docker Desktop and pulling the image from hub