Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    aleksystasiak
    @aleksystasiak
    What do I lack ?
    extensions like @jpg @tiff or @png work only @webp does not
    Sergey Alexandrovich
    @DarthSim
    @aleksystasiak did you built libvips with webp support?
    aleksystasiak
    @aleksystasiak
    I'm checking if I missed some required packages
    aleksystasiak
    @aleksystasiak
    Indeed I missed some, after recompiling webp is supported
    But I have one more question
    Are multiple instances of imgproxy on one vm is supported scenario ?
    because I'm running 4 instances this way and observing some weird behaviour
    From one of 4 instances approximately 2h hours after start will fail
    and keep failing (stream of 504 errors)
    all other instances will work correctly without any problems
    all instances are feeded the same amount od requests, max connection number etc
    If I restart them on 2-3 hour intervals problem does not occur
    Sergey Alexandrovich
    @DarthSim
    I can't tell anything without metrics, logs, and so on.
    aleksystasiak
    @aleksystasiak
    the problem is logs do not show anything suspicious on imgproxy side
    what I see is
    Dec 8 08:01:21 srv103 haproxy[6687]: 100.100.0.1:35160 [08/Dec/2020:08:00:41.322] front80 imgproxy/imgproxy2 0/0/0/-1/40001 504 203 - - sH-- 29/29/11/9/0 0/0 "GET /i/374x560/SS20-TOM024_59A_F1.jpg@jpg HTTP/1.1"
    Dec 8 08:01:21 srv103 haproxy[6687]: 100.100.0.1:35092 [08/Dec/2020:08:00:41.470] front80 imgproxy/imgproxy2 0/0/0/-1/40001 504 203 - - sH-- 28/28/12/8/0 0/0 "GET /i/840x1260/AW20-TOD0Y2_45X_F2.jpg@jpg HTTP/1.1"
    Dec 8 08:01:21 srv103 haproxy[6687]: 100.100.0.1:35172 [08/Dec/2020:08:00:41.602] front80 imgproxy/imgproxy2 0/0/0/-1/40001 504 203 - - sH-- 27/27/8/7/0 0/0 "GET /i/261x395/KW20-BDD01H_99X_F1.jpg@webp HTTP/1.1"
    Dec 8 08:01:22 srv103 haproxy[6687]: 100.100.0.1:35168 [08/Dec/2020:08:00:42.080] front80 imgproxy/imgproxy2 0/0/0/-1/40000 504 203 - - sH-- 26/26/7/6/0 0/0 "GET /i/824x1236/AW20-KUD0KF_45X_F1.jpg@webp HTTP/1.1"
    It will be one random instance which is failing
    Without restart it will keep failing (not 100% of the time but frequently)
    My logging level is error
    Dec 8 08:04:08 srv103 imgproxy[4490]: embedded profile incompatible with image
    Dec 8 08:04:14 srv103 imgproxy[4490]: embedded profile incompatible with image
    Dec 8 08:04:31 srv103 imgproxy[12668]: embedded profile incompatible with image
    Dec 8 08:05:46 srv103 imgproxy[4490]: embedded profile incompatible with image
    Dec 8 08:05:56 srv103 imgproxy[4490]: embedded profile incompatible with image
    Dec 8 08:06:45 srv103 imgproxy[8904]: embedded profile incompatible with image
    Dec 8 08:06:51 srv103 imgproxy[4490]: embedded profile incompatible with image
    Dec 8 08:07:25 srv103 imgproxy[8904]: embedded profile incompatible with image
    Dec 8 08:08:09 srv103 imgproxy[4490]: embedded profile incompatible with image
    Dec 8 08:08:18 srv103 imgproxy[12668]: embedded profile incompatible with image
    Dec 8 08:08:24 srv103 imgproxy[12668]: embedded profile incompatible with image
    That's the only thing I see in imgproxy logs
    Earlier some of the problems were caused by :
    too large source images
    images with cmyk colorspace
    Sergey Alexandrovich
    @DarthSim
    I can only guess one of your instances is OOMed. What's the exit code of the failing instance?
    aleksystasiak
    @aleksystasiak
    no it is not oom
    there is plenty of ram available
    instance does not die, just spews couple od dozens of 504, works for a while and fail again
    in the same time other instances work correctly
    I was wandering if it's tied to default settings like concurrency and max clients as they were not touched
    I'm allowing one instance to receive 32 simultainous connections
    all instances are behind haproxy loadbalancer
    (which is also caching but that is not connected I think)
    The funny thing is, when I spot and restart misbehaving instance it will be healthy, but after some time one of 4 will start failing
    aleksystasiak
    @aleksystasiak
    and cycle repeats
    Sergey Alexandrovich
    @DarthSim
    504 means gateway timeout. Your LB doesn't receive the response from imgproxy in the expected time. What happens if you send a request to the fallen instance directly?
    aleksystasiak
    @aleksystasiak
    When I spot another occurence I will try
    But I don't think it is caused by timeout (20 seconds read|write|download on imgproxy side 40 seconds is lb tolerance)
    Sergey Alexandrovich
    @DarthSim
    I recommend setting imgproxy concurrency to num CPUs*2. You run 4 instances with 32 threads each, this means you need 64 cores to run smoothly.
    Well, 504 is gateway timeout by all means https://developer.mozilla.org/ru/docs/Web/HTTP/Status/504
    aleksystasiak
    @aleksystasiak
    I know what 504 means :) I just think in this situation i expect rather 502 if imgproxy cannot process more request (are 32 too much ?)
    But You are probably right with too much threads (although im keeping tabs on load / io / ram etc. and all is within tolerance)
    I'm in process of transfering imgproxy to dedicated vms one for instance, probably this will resolve the issue
    In other places where 1 instance can keep up with load I did not experience similiar problems
    aleksystasiak
    @aleksystasiak
    probably not multi instance problem it seems
    because I've created 3 external instances, each on it's own vm and I'm observing same behaviour
    1 hour working correctly, after it's starting to fail
    it will keep failing until restarted
    2gigs of ram free, 6 cpu and load 0.7
    only error seen in logs: Dec 8 15:31:20 imgproxy-01 imgproxy[27617]: profile incompatible with image
    grishk
    @grishk:matrix.org
    [m]
    Hi! Just to be clear — avif support hasn't been released yet? I remember reading about imgproxy supporting avif somewhere, probably on twitter, but I just tried with 2.15.0 and it doesn't know what .avif extension is.
    Sergey Alexandrovich
    @DarthSim
    Hey! No, it hasn’t. You can use the latest image to play with avif
    Ewan Higgs
    @ehiggs
    If no bg color is set when converging png with alpha channel to jpeg, is there a default colour, or is it arbitrary? It seems to be white but I have reports from the wild where it’s sometimes another color. (i know the solution is to set IMGPROXY_PRESET=default=bg:fffff just curious if I’m imagining the behaviour)
    Sergey Alexandrovich
    @DarthSim
    @ehiggs By default, imgproxy flattens images with white when saving them to jpeg. If you have a sample image where it uses another color, I'd take a look