Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Aug 06 05:07
    dd8 commented #541
  • Aug 06 05:05
    dd8 commented #541
  • Aug 04 16:20
    specularscholar commented #538
  • Jul 25 12:10

    jonsneyers on master

    Added FLIF-GUI in the downloads… Changed FLIF-GUI text according… Update index.html Co-Authored-… and 1 more (compare)

  • Jul 25 12:10
    jonsneyers closed #19
  • Jul 25 12:09

    jonsneyers on master

    Merge remote-tracking branch 'r… Updated Irfanview support Merge pull request #20 from The… (compare)

  • Jul 25 12:09
    jonsneyers closed #20
  • Jul 25 12:09

    jonsneyers on master

    Added Phew Merge pull request #21 from The… (compare)

  • Jul 25 12:09
    jonsneyers closed #21
  • Jul 24 11:04
    tnikolla opened #543
  • Jul 24 10:30
    tnikolla closed #542
  • Jul 24 10:30
    tnikolla opened #542
  • Jul 22 02:36
    cuanduo commented #541
  • Jul 22 02:04
    cuanduo opened #541
  • Jul 11 07:47
    DonaldTsang commented #481
  • Jul 11 07:47
    DonaldTsang commented #481
  • Jul 11 07:26
    DonaldTsang opened #540
  • Jul 10 19:37
    Sur3 commented #372
  • Jul 08 07:04
    xiaolian8325 opened #539
  • Jul 07 17:05
    lehitoskin commented #538
Pieter Wuille
@sipa
oh!
ignore what i said then
i assumed jonsneyers was working on FUIF instead
Jon Sneyers
@jonsneyers
I stopped FLIF development since JPEG XL will kind of supersede it
JPEG XL = FUIF + PIK + improvements made in the last 10 months or so
TobiasKnauss
@TobiasKnauss
Hm, good to know. Then I won't spend any more time in implementing a kind of dead solution.
@jonsneyers Are you still working on basic things or are you already optimizing performance? I'm asking because I'd like to know whether you could already give some outlook on the performance compared to FLIF.
Jon Sneyers
@jonsneyers
FUIF = FLIF + Squeeze transform + quantization + MANIAC tweaks - weird scan order that allows you to predict from many neighbors but is really bad for memory locality hence speed - ColorBucket transform. For lossless it's roughly equivalent to flif -N, for progressive and lossy it's better than FLIF, and it's also faster.
We are currently polishing the committee draft, which will go to ballot to the national bodies of ISO (ANSI etc) on August 5th. If all goes well we have a draft international standard in November and a published standard in April 2020.
So it's pretty stable but we can still have bitstream breaking changes pretty much until November.
Decoder performance has mostly been optimized already (except for some simple things that we know are easy to optimize so we haven't done it yet – we focused on the things that are hard to optimize). Encoder performance has mostly been ignored so far.
TobiasKnauss
@TobiasKnauss
:-/
2 questions:
  • For my tests I mainly copied the code from test.c, including all encoder settings. From flif.exe command line help I see that -N means "no interlacing". How can I achieve that with flif_encoder_set_interlaced() ? This function currently takes "1", so I thought it was a numeric level, but based on the command line args -N and -I, it looks like its a flag, so should I use "0"?
  • What are your plans about optimizing encoder performance? Decoding actually is quite unimportant for me, because it will be done only occasionally, but encoding will be done frequently.
    Overall this sounds like I should expect a working-for-me solution by mid to end of 2020.
TobiasKnauss
@TobiasKnauss
But please don't take my question as negative criticism! I don't want to say "why does it take so long", but I need to say "THANK YOU!" for such a great development!
Jon Sneyers
@jonsneyers
I don't recall how the flif api works, but I assume -N is done by doing fuif_encoder_set_interlaced(0)
to speed flif encoding up, you can do fewer MANIAC tree learning iterations, setting -R 1 or if you want something quick -R 0 (and then later do a transcode with -R 2)
in fuif / jpeg xl you have more fine-grained control over encode speed, in flif you need an integer number of learning iterations and the default is -R 2, in fuif you can do a fractional number of learning iterations and the default is basically -R 0.5
Jon Sneyers
@jonsneyers
flif does tend to compress a bit better than fuif / jpeg xl for lossless – that's the price we have to pay to make encode/decode faster
TobiasKnauss
@TobiasKnauss
I just got the AccessViolationException again in flif_encoder_encode_file() when encoding the 10000x10000x32bpp PNG. The RAM usage is about 900MB in the moment of the exception, but it's 1400MB if no exception occurs. You may want to check whether the memory allocation is done correctly on large files; maybe not on the FLIF encoder since you stopped working on it, but on the FUIF encoder.
well, I should stop mentioning a file type. I load every file into a .Net Bitmap object, so the original file type is pretty irrelevant.
The Jared Wilcurt
@TheJaredWilcurt
@TobiasKnauss I would be curious if the flif.exe is faster or slower than your DLL approach. or if there is no noticable difference in performance
Pieter Wuille
@sipa
there shouldn't be any difference if the exe and dll are compiled with similar optimizations
The Jared Wilcurt
@TheJaredWilcurt
:+1:
Pieter Wuille
@sipa
the overhead of calling an exe is probably negligable compared to the actual compression algorithm
Jon Sneyers
@jonsneyers
yes, it is
Jon Sneyers
@jonsneyers
by the way @TobiasKnauss, about "3) the flif_import_image_RGBA() swaps R and B. " – this is probably because on Windows, pixel buffers are usually BGR instead of RGB (it's the old little-endian vs big-endian thing).
TobiasKnauss
@TobiasKnauss
@jonsneyers Yes, on Windows it's BGRA (https://stackoverflow.com/a/8104553). Is it different on other systems? Is there a way to make FLIF "compatible" to the Windows byte order? This would speed up the copying of image data a lot prior to the actual encoding.
@TheJaredWilcurt : I compiled the flif library using the "dl_make_vs.bat"
@sipa "the overhead of calling an exe" should be some tens of milliseconds, which is nothing even on my medium-sized images that take 5 seconds. It would only be relevant at encoding of tiny PNG images that take about 100ms.
Tests with different values of interlace and learn-repeat finished. Results on various kinds of image data are too long to post it here, but I can send it to anybody who's interested.
Summary for the 1280x960x8bpp:
image.bmp    1280x960xFormat8bppIndexed : image=1228800 bytes, sourcefile=1229878 bytes, sourcefile/image=100,087728%
Interlaced  = 1, LearnRepeat = 3, encoding: 00:00:04.4502546, file=430089 bytes, file/image=35,000732%  file/sourcefile=34,970%
Interlaced  = 0, LearnRepeat = 3, encoding: 00:00:04.2122409, file=433344 bytes, file/image=35,265625%  file/sourcefile=35,235%
Interlaced  = 0, LearnRepeat = 1, encoding: 00:00:01.5380880, file=435377 bytes, file/image=35,431071%  file/sourcefile=35,400%
TobiasKnauss
@TobiasKnauss
I just tested with Interladed=0 and LearnRepeat=0, and... wow! This is fast now! And it's still quite small:
Interlaced  = 0, LearnRepeat = 0, encoding: 00:00:00.1940111,  file=477835  bytes, file/image=38,886312%  file/sourcefile=38,852%
Daniel Griffen
@dgriffen
IS the spec for JPEG XL released already or is that a WIP too?
ika4
@ika4
I'm having some trouble getting good results out of FLIF. I have a set of BMP images at 128x128, so every image is 12kb. If I convert these images to WEBP they all compress and have filesizes lower than 12kb. But if I convert them to FLIF they are all larger than 12kb. On bigger files FLIF always seems to beat WEBP but on this imageset FLIF is doing worse than the uncompressed BMP files.
with my WEBP conversion I use -z 9 (highest lossy compression level) and with FLIF I use -E=100
ika4
@ika4
using more iterations and stripping some data seems to help , but now it's kind of a toss-up between the two, certain images are doing better than others
The Jared Wilcurt
@TheJaredWilcurt
@ika4 I think if you turn of interlacing it may help for smaller dimensions. Though I believe be default interlacing is disabled on tiny images. but I'm not sure how "tiny" is defined in the spec
Jon Sneyers
@jonsneyers
JPEG XL draft spec is here: https://arxiv.org/abs/1908.03565
Pieter Wuille
@sipa
5 different entropy coders?
Jon Sneyers
@jonsneyers
@ika4 you can try -N -B
Daniel Griffen
@dgriffen
how close is this to the final spec do you know?
Jon Sneyers
@jonsneyers
@dgriffen that will depend on ISO ballot comments and core experiment results
but probably there will still be quite some difference
Daniel Griffen
@dgriffen
fair enough. It would probably be a waste of time for me to write a rust implementation based on the draft then I'm guessing?
Pieter Wuille
@sipa
oh, no 6
brotli, ANS, sANS, BAC, huffman, ABRAC
Jon Sneyers
@jonsneyers
I personally would like to get rid of some entropy coders (I think 3 different ones are needed, but no more), the Brunsli mode (dedicated jpeg recompression; this can also be done in the normal kPasses mode), and the dedicated lossless mode
basically everything after page 135 should be removed in my opinion, but I'm having a hard time convincing the Google team :)
I want to keep Brotli, ANS and ABRAC.
Daniel Griffen
@dgriffen
well at the very least I'll reserve the rust crate name so no one else poaches it
Jon Sneyers
@jonsneyers
(MABEGABRAC is basically the same thing as MANIAC, by the way)
The Jared Wilcurt
@TheJaredWilcurt
:tada:
This is good progress
mo0nsniper
@mo0nsniper
How is the compression from PIK used in Jpeg XL? In FUIF I understand that the same algorithm is used both for Loossless and Lossy compression.