Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Paulo Matos
    @pmatos
    I was not aware that the usage of the FileWriterDecorator depended on how the fuzzer generates the test cases. Makes more sense now.
    Regarding the subprocess_runner, that's an interesting point. My current fuzzer generates batches of 25 tests (25 is the constant I chose), unsure how this is actually working. It works so transparently that I am afraid something is going wrong, or fuzzinator is smart enough to generate the 25 tests and transparently run them. Better check.
    Paulo Matos
    @pmatos
    Ah yes, I am using the subprocess_runner but have not set the contents, which is why it worked with the FileWriterDecorator but not without it. I can simplify this.
    Renáta Hodován
    @renatahodovan

    @renatahodovan ahah - I didn't know that I could access saved fields in issue during the validate_ call, that makes sense. And I guess I need to do the same for the reduce_call because the reduce will need the same options as well. Thanks

    exactly! what's more, if your validate_call and reduce_call looks the same, then it's enough to define the reduce_call since validate_call will fallback to it if there is no separate validate_call defined in the configs.

    Paulo Matos
    @pmatos
    Awesome!
    Something is off though...
    Suddently the command I am getting in my SubprocessCall (the custom one) is bytes instead of str
    command: b'./WebKitBuild/Debug/bin/jsc {options} {test}\n'
    This was working before but after today's changes it's a byte string instead.
    Renáta Hodován
    @renatahodovan

    Regarding the subprocess_runner, that's an interesting point. My current fuzzer generates batches of 25 tests (25 is the constant I chose), unsure how this is actually working. It works so transparently that I am afraid something is going wrong, or fuzzinator is smart enough to generate the 25 tests and transparently run them. Better check.

    I'm not sure I understand the meaning of "transparent" in this context, but you need to define the batch size in the config to inform fuzzinator how many tests cases to look for (https://github.com/renatahodovan/fuzzinator/blob/master/fuzzinator/job/fuzz_job.py#L48). Batch size can be defined with the batch option of the fuzzer config: https://github.com/renatahodovan/fuzzinator-configs/blob/master/sut/jsc/jsc-grammarinator-cli.ini#L20

    If your fuzzer enables setting the number of generated test cases through CLI arguments, then you can synchronize these values:

    [fuzz.jsc-grammarinator]
    batch=25
    ...
    
    [fuzz.jsc-grammarinator.fuzzer.init]
    command=grammarinator-generate -n=${fuzz.jsc-grammarinator:batch} ...
    Paulo Matos
    @pmatos
    fuzzinator backtraces are funny ... they show up a huge nested list of decorators until getting to the real problem. :)
    Renáta Hodován
    @renatahodovan
    One corner case for the sake of completeness, but I don't think that you will need it now: you can set the batchsize to inf in case your fuzzer generates test cases continuously.
    Paulo Matos
    @pmatos
    Thanks - as you mention, not needed at the moment.
    Renáta Hodován
    @renatahodovan

    This was working before but after today's changes it's a byte string instead.

    what were the changes?

    Renáta Hodován
    @renatahodovan

    fuzzinator backtraces are funny ... they show up a huge nested list of decorators until getting to the real problem. :)

    yeah, this comes from the nature of decorator chains :)

    Paulo Matos
    @pmatos

    This was working before but after today's changes it's a byte string instead.

    what were the changes?

    Still trying to determine when this broke.

    fuzzinator backtraces are funny ... they show up a huge nested list of decorators until getting to the real problem. :)

    yeah, this comes from the nature of decorator chains :)

    :thumbsup:

    this morning I woke up and was like : "ok, will work 1 hr on fuzzing and move to other projects afterwards..." My son just got home from school wanting lunch and I am still on fuzzing. :eyes:
    Thanks so much for your help on this.
    Renáta Hodován
    @renatahodovan
    yeah, it's familiar. I also keep switching between hobby fuzzing and bubble blowing with my daughter :P

    Thanks so much for your help on this.

    you're welcome :)

    Paulo Matos
    @pmatos

    This was working before but after today's changes it's a byte string instead.

    what were the changes?

    Still trying to determine when this broke.

    I am starting to think that validation never worked properly. I think that the command is a string when we first run the SUT. Then for validation we load the SUT from the database, create a path and the command becomes bytes.

    I have absolutely no evidence of the above. Trying to grok the code atm.
    Nah - that can't be right. The load from the database is for the test which forms a test file_path in the file_writer_decorator.py.
    What I am seeing is the command as bytes.
    Paulo Matos
    @pmatos
    fuzzer_1  | Exception in <fuzzinator.job.validate_job.ValidateJob object at 0xf48e6370>: expected str, got bytes
    fuzzer_1  | Traceback (most recent call last):
    fuzzer_1  |   File "/jscfuzz/venv/lib/python3.7/site-packages/fuzzinator/controller.py", line 413, in _run_job
    fuzzer_1  |     for issue in job.run():
    fuzzer_1  |   File "/jscfuzz/venv/lib/python3.7/site-packages/fuzzinator/job/validate_job.py", line 29, in run
    fuzzer_1  |     _, new_issues = self.validate()
    fuzzer_1  |   File "/jscfuzz/venv/lib/python3.7/site-packages/fuzzinator/job/validate_job.py", line 37, in validate
    fuzzer_1  |     issue = sut_call(**sut_call_kwargs)
    fuzzer_1  |   File "/jscfuzz/venv/lib/python3.7/site-packages/fuzzinator/config.py", line 42, in __call__
    fuzzer_1  |     return self._callable(*args, **kwargs)
    fuzzer_1  |   File "/jscfuzz/venv/lib/python3.7/site-packages/fuzzinator/call/file_writer_decorator.py", line 60, in writer
    fuzzer_1  |     issue = fn(*args, **kwargs)
    fuzzer_1  |   File "/jscfuzz/venv/lib/python3.7/site-packages/fuzzinator/call/anonymize_decorator.py", line 46, in filter
    fuzzer_1  |     issue = fn(*args, **kwargs)
    fuzzer_1  |   File "/jscfuzz/venv/lib/python3.7/site-packages/fuzzinator/call/subprocess_property_decorator.py", line 59, in filter
    fuzzer_1  |     issue = fn(*args, **kwargs)
    fuzzer_1  |   File "/jscfuzz/venv/lib/python3.7/site-packages/fuzzinator/call/subprocess_property_decorator.py", line 59, in filter
    fuzzer_1  |     issue = fn(*args, **kwargs)
    fuzzer_1  |   File "/jscfuzz/venv/lib/python3.7/site-packages/fuzzinator/call/subprocess_property_decorator.py", line 59, in filter
    fuzzer_1  |     issue = fn(*args, **kwargs)
    fuzzer_1  |   [Previous line repeated 1 more time]
    fuzzer_1  |   File "/jscfuzz/venv/lib/python3.7/site-packages/fuzzinator/call/platform_info_decorator.py", line 34, in filter
    fuzzer_1  |     issue = fn(*args, **kwargs)
    fuzzer_1  |   File "/jscfuzz/venv/lib/python3.7/site-packages/fuzzinator/call/unique_id_decorator.py", line 45, in filter
    fuzzer_1  |     issue = fn(*args, **kwargs)
    fuzzer_1  |   File "/jscfuzz/venv/lib/python3.7/site-packages/fuzzinator/call/gdb_backtrace_decorator.py", line 60, in filter
    fuzzer_1  |     issue = fn(*args, **kwargs)
    fuzzer_1  |   File "/jscfuzz/venv/lib/python3.7/site-packages/fuzzinator/call/exit_code_filter.py", line 53, in filter
    fuzzer_1  |     issue = fn(*args, **kwargs)
    fuzzer_1  |   File "/jscfuzz/jsc32-fuzz/fuzzinator/igalia/fuzzinator/call/subprocess_jsccall.py", line 66, in SubprocessJSCCall
    fuzzer_1  |     command = formatter.vformat(command, (), mapping)
    fuzzer_1  |   File "/usr/lib/python3.7/string.py", line 190, in vformat
    fuzzer_1  |     result, _ = self._vformat(format_string, args, kwargs, used_args, 2)
    fuzzer_1  |   File "/usr/lib/python3.7/string.py", line 200, in _vformat
    fuzzer_1  |     self.parse(format_string):
    fuzzer_1  |   File "/usr/lib/python3.7/string.py", line 284, in parse
    fuzzer_1  |     return _string.formatter_parser(format_string)
    fuzzer_1  | TypeError: expected str, got bytes
    Paulo Matos
    @pmatos
    Reproducing this is quite a pain, unless I manage to inject something into the db manually but without knowing the db format it's hard.
    Paulo Matos
    @pmatos
    I think I found the problem by inspecting the database. This is a simple database:
    > use fuzzinator
    switched to db fuzzinator
    > db.getCollection('fuzzinator_issues').find({})
    { "_id" : ObjectId("60d1ea728c198c90cb14ffd5"), "id" : "  ", "sut" : "jsc", "backtrace" : BinData(0,""), "build_command" : BinData(0,"L2pzY2Z1enovanNjMzItZnV6ei9jb25maWdzL2pzYy1idWlsZC5zaCBsaW51eDMyCg=="), "build_name" : BinData(0,"ZGVidWdPcHQgSlNDT25seQo="), "command" : BinData(0,"Li9XZWJLaXRCdWlsZC9EZWJ1Zy9iaW4vanNjIHtvcHRpb25zfSB7dGVzdH0K"), "count" : 67, "exit_code" : -6, "first_seen" : ISODate("2021-06-22T13:49:38.486Z"), "fuzzer" : "js-fuzzer", "last_seen" : ISODate("2021-06-22T14:05:05.959Z"), "node" : "bc861183bc96", "options" : "--returnEarlyFromInfiniteLoopsForFuzzing=1 --earlyReturnFromInfiniteLoopsLimit=1000000 --useConcurrentGC=0", "platform" : "Linux-5.10.0-0.bpo.7-arm64-aarch64-with-debian-10.9", "reduced" : null, "reported" : false, "stderr" : BinData(0,""), "stdout" : BinData(0,"QlVHTlVNQkVSOiAKU1RBVFVTOiBTd2l0Y2ggd2l0aCBtb3JlIHRoYW4gNjRrIGF0b21zCiBQQVNTRUQhIFN3aXRjaCB3aXRoIG1vcmUgdGhhbiA2NGsgYXRvbXMK"), "subconfig" : { "subconfig" : "c7b4b647b" }, "test" : "/jscfuzz/fuzzinator-tmp/js_fuzzer/10855-4102971824/fuzz-9.js", "version" : BinData(0,"YTZhZmZlYTdmZDEyMzUyMGM2MGI3MzExZjI1YThlZmRjMDFjZGU3Ygo=") }
    command is saved in the database as a BinData and when it is read, it's read as bytes.
    So either it's saved as a string to begin with
    or we convert the bytes to string when reading.
    I think it makes sense to save it as a string like we do for the sut key for example.
    So currently backtrace, build_command, build_name, command, stderr, stdout are currently bindata.
    I wonder if any of those have a reason for being bindata.
    Paulo Matos
    @pmatos
    somewhere something must transform command into bytes or something. :)
    Renáta Hodován
    @renatahodovan
    I'm inspecting the code and your example database in the meantime and trying to undestand what's going on
    the first thing I don't understand, who does save the command into the database? it should not be there in the first place, it should be read from the config during the validation as well.
    second, is this a test database or a real fuzzer entry? (since there are a few interesting/wrong entry in it)
    Paulo Matos
    @pmatos
    it's a real fuzzer entry!
    however, your last message on the "who saves the command into the database" triggered me to look at my config and I found at least one problem:
    Renáta Hodován
    @renatahodovan
    third: is it the master branch that you are currently using? https://github.com/pmatos/jsc32-fuzz
    Paulo Matos
    @pmatos
    # Saves the execution command as command
    [sut.jsc.call.decorate(8)]
    property=command
    command=echo "${sut.jsc.call:command}"

    third: is it the master branch that you are currently using? https://github.com/pmatos/jsc32-fuzz

    yes

    Renáta Hodován
    @renatahodovan
    yes, I guess that this is a subprocessPropertyDecorator which save the stdout as byte array
    Paulo Matos
    @pmatos
    This should fix the command issue: pmatos/jsc32-fuzz@e4e3a64
    which other issues did you see?
    Renáta Hodován
    @renatahodovan
    yes, this should solve it, since right now your SubprocessJSCCall receives two command arguments, one from the config and one from the database. the first one is a string and the second one is a byte array (I think this way at least)
    Paulo Matos
    @pmatos
    I was actually thinking that you might read the command from the config as a string and then somehow when you import the database, you overwrite the sut['command'] with the bytes and then the validation fails.
    I wonder if some property names should be marked as special and not allowed.