Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Jun 26 20:54
    ziemek99 opened #561
  • Apr 19 22:52
    Travis kiss2u/FLIF (master) passed (4)
  • Apr 19 16:01
  • Apr 05 22:06

    jonsneyers on master

    Update index.html (compare)

  • Mar 20 12:23
    necros2k7 edited #560
  • Mar 20 12:22
    necros2k7 edited #560
  • Mar 20 12:21
    necros2k7 opened #560
  • Feb 09 15:36

    jonsneyers on master

    Update README.md (compare)

  • Feb 09 15:35

    jonsneyers on master

    Update index.html (compare)

  • Feb 09 12:52

    jonsneyers on master

    Update index.html (compare)

  • Feb 09 08:58

    jonsneyers on master

    Update index.html (compare)

  • Feb 09 08:52

    jonsneyers on master

    Update README.md (compare)

  • Jan 07 21:17
    charun80 opened #559
  • Nov 20 2020 07:22
    linrio opened #558
  • Nov 11 2020 09:24
    ScriptTiger commented #549
  • Nov 10 2020 17:26
    ScriptTiger commented #549
  • Jul 10 2020 14:29
    dclunie commented #557
  • Jul 10 2020 14:23

    jonsneyers on master

    Notice for JPEG XL Merge pull request #554 from sa… (compare)

  • Jul 10 2020 14:23
    jonsneyers closed #554
  • Jul 10 2020 14:23
    jonsneyers commented #554
Boris Yakubchik
@whyboris_twitter
FLIF just blew my mind!
I discovered it when I stumbled across Phew - FLIF Image Viewer for macOS: https://sveinbjorn.org/phew
Perhaps add it to the software page? http://flif.info/software.html :smile:
Boris Yakubchik
@whyboris_twitter
I'm happy to open a PR on GitHub but it doesn't look like there's anyone merging since at least January this year.
Is the project scrapped in favor of FUIF ?
Boris Yakubchik
@whyboris_twitter

It makes sense to put full force behind FUIF then ... since FLIF's 16bit images are a no-go for photographers, and FUIF has, quote:

no inherent limitations in terms of maximum image resolution, bit depth, or number of channels.

Thank you Jon Sneyers for creating FLIF and working on FUIF
https://cloudinary.com/blog/author/jon_sneyers :bow:
:heart:
The Jared Wilcurt
@TheJaredWilcurt
Phew is listed there
The Jared Wilcurt
@TheJaredWilcurt
Boris Yakubchik
@whyboris_twitter
Awesome! Thank you for creating the PR :smile:
Really hope more people hear of the two file formats.
Looks like people at ImageCon2019 just did (May 1st - 2nd) :grin:
From what I can tell by looking at Twitter, JPEG XL is the most-likely-to-succeed new format: it had an open call for proposals and FUIF is one of the two main ones (the other is Pik by Google).
If all goes well, we'll have a synthesis of some of the best ideas from FLIF, FUIF, and Pik all in there :tada:
Jon Sneyers
@jonsneyers
I should check gitter more frequently :)
Yes, I have been working together with the google pik team a lot in the past months
JPEG XL is going to be a combination of FUIF and Pik, we have been working hard to make that work
Jon Sneyers
@jonsneyers
We're following the ISO standardization process, which unfortunately implies that for now we can't put the code in the public spot since that would force potential contributors to open source their contribution and ISO doesn't allow us to force potential contributors to do that. But so far it's just Cloudinary and Google and it likely will stay that way, and we will make our code available, royalty-free and free/open source.
TobiasKnauss
@TobiasKnauss
Hi, I've used FLIF for the first time today. I downloaded the source from GitHub and compiled the DLL using the batch file, then I created a C++/CLI test project in VS 2015 and modified the test.c for my own purpose. Then I converted a set of files to flif. I discovered 3 issues:
1) it works well on reducing file size on PNG and BMP, but the sample JPG that I used was saved as a flif with 4 times the original size.
2) it failed on a 10000x10000 PNG (AccessViolationException)
3) the flif_import_image_RGBA() swaps R and B. Is there any workaround for it instead of manually copying the BitmapData into the flif image using flif_image_write_row_RGBA8() ?
Regards, Tobias
And there's a less-important 4th issue: Encoding a flif from the JPG (which is a 2400x3600 photo) takes about a minute (x86 DLL, i5-CPU). Is it really so slow?
The Jared Wilcurt
@TheJaredWilcurt
@TobiasKnauss
  1. JPG is Lossy, FLIF is Lossless. You can apply a lossy transformation in your conversion to reduce file size, the output will be stored in a lossless FLIF format but will have been degraded in quality from the original. In the CLI you would use -Q70 to set quality to ~70% of original. -Q100is the default and is lossless, all other settings are lossy.
  2. I cannot find any reference in the FLIF spec for a maximum height/width dimension. I think I recall someone saying there is no limit on dimensions. If that is the case then you like just found a bug and should report it with a means to reproduce it.
  3. Perhaps @jonsneyers or @sipa or @hrj will know more about this, I'm unfamiliar with it
  4. The current FLIF source code is one implementation of the FLIF spec. It has not been optimized at all. So it will be very slow in comparison to encoders for more mature specs that have existed for decades and had time to be heavliy optimized (GIF/PNG/JPG/Etc)
TobiasKnauss
@TobiasKnauss
@TheJaredWilcurt
Thank you very much for your reply!
Regarding 1), I created the FLIF directly from the JPG. I just doublechecked on flif.info, it compares the compression ratio of FLIF to other lossless formats, but not to JPG. So I think I should not do that either, since it would be comparing apples with oranges, isn't it?
Regarding 2), I have tested the same file again, this time with "FLIF-GUI 0.3" Filemill.exe. It worked. Now I'm testing again with my own code, it's running now. Don't know why. The exception came directly at the call of the function, not during execution, so I think it will be fine now. It will just take some time: The test with a 4096x4096 took 2:16 min, so I expect the 10000x10000 to take about 10 min or more. The difficulty on the 4096^2 file is, that every pixel is different (it contains all 24bit colors). The 10000^2 file is similar, it contains at least some identical pixels and not all of the 24bit colors.
The Jared Wilcurt
@TheJaredWilcurt
@TobiasKnauss There is a flif.exe file (along with all required Windows files for it to run on any Windows OS) here:
https://github.com/FLIF-hub/node-flif/tree/master/executables/win32
So you can run a pre-built version of the flif CLI, in case you think your build isn't working correctly
TobiasKnauss
@TobiasKnauss
@TheJaredWilcurt I am using the DLL, not the EXE, because I want to build a C# interface around it. But that's a good idea for a doublecheck. Thanks!
The encoding of the 10000^2 file has finished, it took about 11 min. Worked as it is expected to.
Now waiting for replies of other users on the issue 3 with flif_import_image_RGBA().
Regarding 4), I will mostly encode 1280x960x8bpp images from industrial GigE Cameras. It takes 5 seconds for one image (slooow...), so I'll have to use one CPU thread for every image, because I usually get 4 images (or more) every 15 seconds.
The Jared Wilcurt
@TheJaredWilcurt
FLIF is for lossless images. Most of the time if you need photographic images stored, you are better off with a lossy format
Pieter Wuille
@sipa
FLIF can be lossy too, but it's not really designed for it
TobiasKnauss
@TobiasKnauss
@TheJaredWilcurt It's not like taking photos of a landscape. In industrial applications, the main purposes are precision, repeatability and reproducability. The photos are used for detecting geometric objects, so when I have to repeat the operation for failure analysis, I need to make sure that I get the same results, so I have to store the photos without loss of details.
Jon Sneyers
@jonsneyers
JPEG XL will take JPEG as input and losslessly transcode it to something smaller. Same with PNG (like FLIF) and GIF.
TobiasKnauss
@TobiasKnauss
@jonsneyers I don't have JPEG, I receive the uncompressed pixel data from the camera. This pixel data must be stored on the disk, but I would like to avoid using 1.3MB for every image, thus I plan to compress them.
Jon Sneyers
@jonsneyers
FLIF is slow partially because the implementation is not optimized (the encoder in particular) and partially because it's inherently slow. JPEG XL will be faster, at least for lossless. For lossy in FUIF mode it's reasonably fast, for lossy in Pik mode it's currently quite slow because of the iterations of perceptual optimization.
TobiasKnauss
@TobiasKnauss

JPEG XL will be faster, at least for lossless.

So I should use this one then. But you write "will be", and what I have understood from your previous posts, it's still under development...?

Jon Sneyers
@jonsneyers
Yes, you can expect a public release by the end of the year
Pieter Wuille
@sipa
there isn't much development on FLIF anymore
oh!
ignore what i said then
i assumed jonsneyers was working on FUIF instead
Jon Sneyers
@jonsneyers
I stopped FLIF development since JPEG XL will kind of supersede it
JPEG XL = FUIF + PIK + improvements made in the last 10 months or so
TobiasKnauss
@TobiasKnauss
Hm, good to know. Then I won't spend any more time in implementing a kind of dead solution.
@jonsneyers Are you still working on basic things or are you already optimizing performance? I'm asking because I'd like to know whether you could already give some outlook on the performance compared to FLIF.
Jon Sneyers
@jonsneyers
FUIF = FLIF + Squeeze transform + quantization + MANIAC tweaks - weird scan order that allows you to predict from many neighbors but is really bad for memory locality hence speed - ColorBucket transform. For lossless it's roughly equivalent to flif -N, for progressive and lossy it's better than FLIF, and it's also faster.
We are currently polishing the committee draft, which will go to ballot to the national bodies of ISO (ANSI etc) on August 5th. If all goes well we have a draft international standard in November and a published standard in April 2020.
So it's pretty stable but we can still have bitstream breaking changes pretty much until November.
Decoder performance has mostly been optimized already (except for some simple things that we know are easy to optimize so we haven't done it yet – we focused on the things that are hard to optimize). Encoder performance has mostly been ignored so far.
TobiasKnauss
@TobiasKnauss
:-/
2 questions:
  • For my tests I mainly copied the code from test.c, including all encoder settings. From flif.exe command line help I see that -N means "no interlacing". How can I achieve that with flif_encoder_set_interlaced() ? This function currently takes "1", so I thought it was a numeric level, but based on the command line args -N and -I, it looks like its a flag, so should I use "0"?
  • What are your plans about optimizing encoder performance? Decoding actually is quite unimportant for me, because it will be done only occasionally, but encoding will be done frequently.
    Overall this sounds like I should expect a working-for-me solution by mid to end of 2020.
TobiasKnauss
@TobiasKnauss
But please don't take my question as negative criticism! I don't want to say "why does it take so long", but I need to say "THANK YOU!" for such a great development!
Jon Sneyers
@jonsneyers
I don't recall how the flif api works, but I assume -N is done by doing fuif_encoder_set_interlaced(0)
to speed flif encoding up, you can do fewer MANIAC tree learning iterations, setting -R 1 or if you want something quick -R 0 (and then later do a transcode with -R 2)
in fuif / jpeg xl you have more fine-grained control over encode speed, in flif you need an integer number of learning iterations and the default is -R 2, in fuif you can do a fractional number of learning iterations and the default is basically -R 0.5
Jon Sneyers
@jonsneyers
flif does tend to compress a bit better than fuif / jpeg xl for lossless – that's the price we have to pay to make encode/decode faster
TobiasKnauss
@TobiasKnauss
I just got the AccessViolationException again in flif_encoder_encode_file() when encoding the 10000x10000x32bpp PNG. The RAM usage is about 900MB in the moment of the exception, but it's 1400MB if no exception occurs. You may want to check whether the memory allocation is done correctly on large files; maybe not on the FLIF encoder since you stopped working on it, but on the FUIF encoder.
well, I should stop mentioning a file type. I load every file into a .Net Bitmap object, so the original file type is pretty irrelevant.
The Jared Wilcurt
@TheJaredWilcurt
@TobiasKnauss I would be curious if the flif.exe is faster or slower than your DLL approach. or if there is no noticable difference in performance
Pieter Wuille
@sipa
there shouldn't be any difference if the exe and dll are compiled with similar optimizations
The Jared Wilcurt
@TheJaredWilcurt
:+1: