Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • 00:39
    johannesvollmer opened #1626
  • Nov 26 16:46
    johannesvollmer commented #1404
  • Nov 26 16:45
    johannesvollmer commented #1404
  • Nov 26 16:41
    johannesvollmer commented #1404
  • Nov 26 16:37
    johannesvollmer commented #1404
  • Nov 26 00:39
    tristanphease synchronize #1624
  • Nov 26 00:15
    tristanphease synchronize #1624
  • Nov 26 00:15
    tristanphease synchronize #1624
  • Nov 26 00:15
    tristanphease synchronize #1624
  • Nov 26 00:14
    tristanphease synchronize #1624
  • Nov 25 22:12
    DmitrijVC closed #1625
  • Nov 25 22:12
    DmitrijVC commented #1625
  • Nov 25 21:41
    HeroicKatora synchronize #1620
  • Nov 25 20:26
    johannesvollmer commented #1625
  • Nov 25 19:49
    fintelia commented #1625
  • Nov 25 18:52
    DmitrijVC commented #1625
  • Nov 25 16:58
    johannesvollmer commented #1511
  • Nov 25 16:58
    johannesvollmer closed #1511
  • Nov 25 16:54
    johannesvollmer commented #1625
  • Nov 25 16:16
    DmitrijVC opened #1625
HeroicKatora
@heroic_katora:matrix.org
[m]
Yup, that's how cargo fuzz works. So there's a reason not to build another separate layer of scripts around it ^^
5225225
@r522:matrix.org
[m]
so what would be the proposal there, guessing the length to do what?
HeroicKatora
@heroic_katora:matrix.org
[m]
So, add a flag --minimum-throughput=4kB/s to dynamically adjust the timeout based on its guess of maximum length.
I mean, in the truly best case it would adjust the limit based on the actual test case but this would probably require quite a bit more integration with libfuzzer.
5225225
@r522:matrix.org
[m]

also, i'm not entirely sure how useful the shell script that just sets the timeout is
since often when i'm running fuzzers i want to change the number of processes it runs / sanitisers

I could add a copy-pastable command line in the readme in the root (or a new README.md in the fuzz/ directory), but I'm not entirely sure how useful the shell script would be directly

ahh, yeah
HeroicKatora
@heroic_katora:matrix.org
[m]
With the motivation that throughput is something that can be a requirement (think: realtime).
Of course if you specify an explicit timeout it should override it. (Or take the minimum?)
5225225
@r522:matrix.org
[m]
libfuzzer would probably need sub-second timeouts (and to probably use CPU time, not real time) for that to be more useful
idk
HeroicKatora
@heroic_katora:matrix.org
[m]
Oh true, would really appreciate it if cargo-fuzz spawned libfuzzer in a cgroup that's in full control over a set of cores on Linux-RT :P
Yeah ^^ I'm daydreaming about cargo-fuzz so a copy-paste command line should be good enough for now
5225225
@r522:matrix.org
[m]
or run through valgrind and count CPU cycles and estimate a runtime from that
1 reply
make your fuzzing runs 10 times slower with this one easy trick
HeroicKatora
@heroic_katora:matrix.org
[m]
And a Fuzz-OS that could do this more precisely without the overhead
In any case, regarding CI failure, feel free to bump the minimum rustc version to 1.48
5225225
@r522:matrix.org
[m]
okay cool
CI now passes and i added a short note in the readme about fuzzing
HeroicKatora
@heroic_katora:matrix.org
[m]
Nice, thank you so much for that :)
5225225
@r522:matrix.org
[m]
and hopefully the turnaround for my PRs can only get faster from here :D
one thing that might be worth doing is getting a sample of known valid images and testing that we do parse them fine
because all fuzzing can really do is test that we don't panic on bad input
1 reply
but always returning Err even for valid input would be invisible to the fuzzer
HeroicKatora
@heroic_katora:matrix.org
[m]
As an experience report the turnaround will be slow and fast, depending on mood 😂
It's also far more likely that Issues are opened for erroneous errors than for pathological cases.
And we can more easily throw a fuzzer run in than run against the vastness of images on the internet.
5225225
@r522:matrix.org
[m]
heads up HeroicKatora : ravif 0.7.0 has been yanked so cargo check on a clean checkout of image-rs/image doesn't build, bumping it to 0.8.0 looks to work fine
can make a PR to do that bump if you want
HeroicKatora
@heroic_katora:matrix.org
[m]
Sure, feel free to do.
Do you happen to know why it was yanked? Not that it really matters if the bump is a clean one to do
5225225
@r522:matrix.org
[m]
no idea
no mention of it in the issue tracker
there is a yank mentioned because it didn't build, but i'm assuming 0.7.0 built fine
HeroicKatora
@heroic_katora:matrix.org
[m]
Yeah, although a weird reason to yank. It always required a pretty precise/recent version of nasm to build so maybe that's a nudge to use the correct version
5225225
@r522:matrix.org
[m]

image-rs/image#1564

bumped it from 0.7.0 to 0.8.0, and marked the issue about it as Fixes:

tests look fine on my side but i'd definitely check if CI has any problems with it

oh wait yeah
avif isn't run by default
okay it's not a clean bump, i just didn't see the error
looks like just a single type changed tho
HeroicKatora
@heroic_katora:matrix.org
[m]
Ah well, that's still good.
5225225
@r522:matrix.org
[m]
the afl-fuzz target doesn't compile on CI
but i can't seem to reproduce that locally
HeroicKatora
@heroic_katora:matrix.org
[m]

error: failed to run LLVM passes: unknown pass name 'sancov'

Huh, I guess something about the new pass manager in nightly?

5225225
@r522:matrix.org
[m]
unless it's very new
HeroicKatora
@heroic_katora:matrix.org
[m]
Yeah, that one is very new
5225225
@r522:matrix.org
[m]
i'll try updating my nightly and trying again
HeroicKatora
@heroic_katora:matrix.org
[m]
Like 3 days old
5225225
@r522:matrix.org
[m]
looks to be unfixed upstream
HeroicKatora
@heroic_katora:matrix.org
[m]
Heh, the afl library also moved on 7 versions since it was last updated.
I've only been using the rust-fuzz target :)
(libfuzzer/cargo-fuzz) I mean
5225225
@r522:matrix.org
[m]
yep
i don't know if it's worth keeping around afl if we already use libfuzzer
not used afl but if it's not significantly better at finding stuff, we could remove it?
HeroicKatora
@heroic_katora:matrix.org
[m]
Yeah, I was considering the same just now. Let me try just bumping to latest and if that works I'll PR it quickly.
And forcibly merge :)
Which I'll do for the ravif one as well
5225225
@r522:matrix.org
[m]
cool
i have a fix for image-rs/image#1501 in a draft PR, just running the fuzzer for a bit to see if it finds any other issues, then I'll open up the PR (image-rs/image#1563)
5225225
@r522:matrix.org
[m]

also, thoughts on adding limits to jpeg-decoder?

https://github.com/image-rs/jpeg-decoder/blob/ab6d326a7b194568725731a19603fb580814452d/src/decoder.rs#L277 is at least one source of unbounded allocation, but I saw discussion on trying to make it library-wide image-rs/image#938.

HeroicKatora
@heroic_katora:matrix.org
[m]
So the library wide version is 'advisory' first and it's not necessary to implement for decoders. We're still looking for real world feedback if it's okay to reject some of the image formats if a limit is requested (I mean, there is no version with it released so far..).