Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • 13:46
    mpizenberg commented #1571
  • 13:31
    mpizenberg commented #1571
  • 13:19
    mpizenberg commented #1571
  • 12:18
    HarryLovesCode commented #1571
  • 12:17
    HarryLovesCode commented #1571
  • Oct 23 14:43
    HeroicKatora edited #1433
  • Oct 23 14:43
    HeroicKatora edited #1433
  • Oct 23 13:24
    HeroicKatora commented #1215
  • Oct 23 13:22
    HeroicKatora commented #1099
  • Oct 23 13:21
    HeroicKatora commented #1516
  • Oct 23 13:21
    HeroicKatora opened #1585
  • Oct 23 13:18

    HeroicKatora on master

    Update mp4parse to 0.12 The ma… Merge pull request #1584 from H… (compare)

  • Oct 23 13:18
    HeroicKatora closed #1584
  • Oct 23 13:05
    HeroicKatora edited #1433
  • Oct 23 12:59
    HeroicKatora opened #1584
  • Oct 23 12:56
    HeroicKatora closed #1516
  • Oct 23 12:56
    HeroicKatora closed #1215
  • Oct 23 12:56
    HeroicKatora closed #1104
  • Oct 23 12:56
    HeroicKatora closed #1099
  • Oct 23 12:56
    HeroicKatora closed #1094
5225225
@r522:matrix.org
[m]
they're limited to 4096 bytes by default anyways
unless you have a larger file in your corpus directory, in which case the size of the largest file is taken to be the limit
HeroicKatora
@heroic_katora:matrix.org
[m]
Can we query that? Well, or since the limit is manually configured specify it in the readme.
Basically we know 4kB should be done in < 4s if we enforce 1kB throughput.
Or make a PR to cargo fuzz for that functionality :) (super optional)
5225225
@r522:matrix.org
[m]
i think it would be a libfuzzer thing, since that's the thing that's actually guesssing the length
1 reply
and libfuzzer is LLVM
cargo fuzz, as i understand it, is basically just scripts to make working with libfuzzer nicer
HeroicKatora
@heroic_katora:matrix.org
[m]
Yup, that's how cargo fuzz works. So there's a reason not to build another separate layer of scripts around it ^^
5225225
@r522:matrix.org
[m]
so what would be the proposal there, guessing the length to do what?
HeroicKatora
@heroic_katora:matrix.org
[m]
So, add a flag --minimum-throughput=4kB/s to dynamically adjust the timeout based on its guess of maximum length.
I mean, in the truly best case it would adjust the limit based on the actual test case but this would probably require quite a bit more integration with libfuzzer.
5225225
@r522:matrix.org
[m]

also, i'm not entirely sure how useful the shell script that just sets the timeout is
since often when i'm running fuzzers i want to change the number of processes it runs / sanitisers

I could add a copy-pastable command line in the readme in the root (or a new README.md in the fuzz/ directory), but I'm not entirely sure how useful the shell script would be directly

ahh, yeah
HeroicKatora
@heroic_katora:matrix.org
[m]
With the motivation that throughput is something that can be a requirement (think: realtime).
Of course if you specify an explicit timeout it should override it. (Or take the minimum?)
5225225
@r522:matrix.org
[m]
libfuzzer would probably need sub-second timeouts (and to probably use CPU time, not real time) for that to be more useful
idk
HeroicKatora
@heroic_katora:matrix.org
[m]
Oh true, would really appreciate it if cargo-fuzz spawned libfuzzer in a cgroup that's in full control over a set of cores on Linux-RT :P
Yeah ^^ I'm daydreaming about cargo-fuzz so a copy-paste command line should be good enough for now
5225225
@r522:matrix.org
[m]
or run through valgrind and count CPU cycles and estimate a runtime from that
1 reply
make your fuzzing runs 10 times slower with this one easy trick
HeroicKatora
@heroic_katora:matrix.org
[m]
And a Fuzz-OS that could do this more precisely without the overhead
In any case, regarding CI failure, feel free to bump the minimum rustc version to 1.48
5225225
@r522:matrix.org
[m]
okay cool
CI now passes and i added a short note in the readme about fuzzing
HeroicKatora
@heroic_katora:matrix.org
[m]
Nice, thank you so much for that :)
5225225
@r522:matrix.org
[m]
and hopefully the turnaround for my PRs can only get faster from here :D
one thing that might be worth doing is getting a sample of known valid images and testing that we do parse them fine
because all fuzzing can really do is test that we don't panic on bad input
1 reply
but always returning Err even for valid input would be invisible to the fuzzer
HeroicKatora
@heroic_katora:matrix.org
[m]
As an experience report the turnaround will be slow and fast, depending on mood 😂
It's also far more likely that Issues are opened for erroneous errors than for pathological cases.
And we can more easily throw a fuzzer run in than run against the vastness of images on the internet.
5225225
@r522:matrix.org
[m]
heads up HeroicKatora : ravif 0.7.0 has been yanked so cargo check on a clean checkout of image-rs/image doesn't build, bumping it to 0.8.0 looks to work fine
can make a PR to do that bump if you want
HeroicKatora
@heroic_katora:matrix.org
[m]
Sure, feel free to do.
Do you happen to know why it was yanked? Not that it really matters if the bump is a clean one to do
5225225
@r522:matrix.org
[m]
no idea
no mention of it in the issue tracker
there is a yank mentioned because it didn't build, but i'm assuming 0.7.0 built fine
HeroicKatora
@heroic_katora:matrix.org
[m]
Yeah, although a weird reason to yank. It always required a pretty precise/recent version of nasm to build so maybe that's a nudge to use the correct version
5225225
@r522:matrix.org
[m]

image-rs/image#1564

bumped it from 0.7.0 to 0.8.0, and marked the issue about it as Fixes:

tests look fine on my side but i'd definitely check if CI has any problems with it

oh wait yeah
avif isn't run by default
okay it's not a clean bump, i just didn't see the error
looks like just a single type changed tho
HeroicKatora
@heroic_katora:matrix.org
[m]
Ah well, that's still good.
5225225
@r522:matrix.org
[m]
the afl-fuzz target doesn't compile on CI
but i can't seem to reproduce that locally
HeroicKatora
@heroic_katora:matrix.org
[m]

error: failed to run LLVM passes: unknown pass name 'sancov'

Huh, I guess something about the new pass manager in nightly?

5225225
@r522:matrix.org
[m]
unless it's very new
HeroicKatora
@heroic_katora:matrix.org
[m]
Yeah, that one is very new
5225225
@r522:matrix.org
[m]
i'll try updating my nightly and trying again
HeroicKatora
@heroic_katora:matrix.org
[m]
Like 3 days old
5225225
@r522:matrix.org
[m]
looks to be unfixed upstream
HeroicKatora
@heroic_katora:matrix.org
[m]
Heh, the afl library also moved on 7 versions since it was last updated.
I've only been using the rust-fuzz target :)
(libfuzzer/cargo-fuzz) I mean
5225225
@r522:matrix.org
[m]
yep
i don't know if it's worth keeping around afl if we already use libfuzzer
not used afl but if it's not significantly better at finding stuff, we could remove it?