Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Paulo Matos
    @pmatos
    Strangely there's also no backtrace.
    But I don't see a reason for properties to be BinData. Like, build_command, build_name, error_type, etc.
    Renáta Hodován
    @renatahodovan

    oh - not working...

    wow :o what was the original and the modified url you tried?

    btw if you click on Export button and Export as JSON/XML you can download database entry of the issue
    404: api not found
    always fails.
    Renáta Hodován
    @renatahodovan
    remove the / from the end
    it's a typo in my previous comment, sorry
    Paulo Matos
    @pmatos
    wot? it worked. :no_mouth:
    Thanks.
    Renáta Hodován
    @renatahodovan
    my bad
    Paulo Matos
    @pmatos
    No worries, should have tried.
    Oh my, I think because validation is not working, I am getting totally unreliable bugs. Where if I run jsc 3 times, I get 3 different results in stderr. Really hard to reproduce.
    main pmatos@bbox-10-armhf ~/dev/webkit $ WebKitBuild/Debug/bin/jsc ~/test.js
    Failed
    Segmentation fault
    main pmatos@bbox-10-armhf ~/dev/webkit $ WebKitBuild/Debug/bin/jsc ~/test.js
    Failed
    ASSERTION FAILED: index < numBits()
    WTF/Headers/wtf/FastBitVector.h(571) : WTF::FastBitReference WTF::FastBitVector::at(size_t)
    Aborted
    main pmatos@bbox-10-armhf ~/dev/webkit $ WebKitBuild/Debug/bin/jsc ~/test.js
    Failed
    ASSERTION FAILED: from.isCell() && from.asCell()->JSCell::inherits(from.asCell()->vm(), std::remove_pointer<To>::type::info())
    ../../Source/JavaScriptCore/runtime/JSCast.h(55) : To JSC::jsCast(JSC::JSValue) [with To = JSC::JSObject*]
    Aborted
    that's just so I can share my pain... :)
    hehehe
    Renáta Hodován
    @renatahodovan
    I know your pain :P
    Paulo Matos
    @pmatos
    I feel better already. :)
    Renáta Hodován
    @renatahodovan
    welcome in the world of fuzzing :P
    Paulo Matos
    @pmatos
    best thing is... if I compile JSC with gcc-10 (instead of gcc-8.3, which is what our embedded clients are using), I don't get a failure at all.
    :cry:
    Renáta Hodován
    @renatahodovan
    nice :|
    Renáta Hodován
    @renatahodovan

    But I don't see a reason for properties to be BinData. Like, build_command, build_name, error_type, etc.

    storing the test content and maybe the stdout and stderr as bytes was a decision, since we wanted to ensure to validate the original context without any encoding/decoding magic which could change the test in some corner cases (real-life experience)

    storing other things like the result of SubprocessPropertyDecorator as bytes is not necessarily the best option.

    Paulo Matos
    @pmatos
    Would you be ok if I get a PR upstream to change this?
    As in store properties from SubprocessPropertyDecorator as strings?
    Renáta Hodován
    @renatahodovan
    Yeah, sure! Thanks.
    Paulo Matos
    @pmatos
    Just found out something surprising, although it's embarrasing I only just found it out.
    Decorators with higher numbers are ran first, they encapsulate earlier decorators. For definitions like call.decorate(0) ... call.decorate(10), the 10th decorator runs first, calls decorator 9, calls decorator 8, and so on.
    This is quite embarrassing because the order of the definition for decorators is quite important and I am sure some of the weird behaviour I am seeing is due to this.
    Paulo Matos
    @pmatos
    @renatahodovan i am confused about some code in fuzzinator. Sometimes properties are assigned to issue, other times things are assigned to kwargs.
    How does this relate to each other and when should you do one or the other?
    For example, in file_writer_decorator.py, we assign test to kwargs, but filename to issue.
    However, on the beginning of the same file we do if 'filename' in kwargs: so we sort of expect filename to be in kwargs, not issue.
    Could you please clarify this?
    Renáta Hodován
    @renatahodovan

    Hi @pmatos!

    Sure! The example of FileWriterDecorator is perfect to understand this issue vs kwargs mechanism, so I'll use that one to explain.

    So, the purpose of FileWriterDecorator is to write the test content (defined as bytes or str) to a file and use that filename as input instead, i.e., the purpose of FileWriterDecorator is to change the original value of the test parameter before the SUT is executed. Use case: the fuzzer generates str but the SUT needs file input.

    The test parameter is the only mandatory parameter of a SUT call, all the additional parameters - coming either from config or from the issue dict by validation - are optional. As a consequence, kwargs in SUT calls are always containing this field. Similarly, the fuzzer callable always has the index parameter.

    Let's see the decorator now:

    # The `filename` parameter comes from the ini config, containing the file path pattern to save the test case. It may contain a {uid} placeholder that will be filled by this decorator.
    def decorator(self, filename, **kwargs):
        def wrapper(fn):
            def writer(*args, **kwargs):
                # At this point the mandatory `test` keyword argument contains the test case as bytes or string. This will be saved to file.
                file_content = kwargs['test']
                # Define `file_path` with filling the {uid} placeholder with unique identifier.
                file_path = as_path(filename.format(uid='{pid}-{id}'.format(pid=os.getpid(), id=id(self))))
                # `filename` is in kwargs only in validation step, when the kwargs are composed from ini config and from the issue to be validated: [link](https://github.com/renatahodovan/fuzzinator/blob/master/fuzzinator/job/validate_job.py#L36).
                if 'filename' in kwargs:
                    # Ensure that the test case will be saved to the directory defined by the
                    # config file and its name will be what is expected by the kwargs.
                    file_path = os.path.join(os.path.dirname(file_path), kwargs['filename'])
    
                os.makedirs(os.path.dirname(file_path), exist_ok=True)
                with open(file_path, 'w' if not isinstance(file_content, bytes) else 'wb') as f:
                    f.write(file_content)
    
                # After saving the test content to the path specified in the `file_path` variable, let's use this path as test, i.e., change the `test` keyword argument to that value. Notice, that this kwargs is used in the next line, to execute the SUT call.
                kwargs['test'] = file_path
                # Execute the SUT call with the updated test.
                issue = fn(*args, **kwargs)
                # If an issue was triggered, save the basename of the testcase to the issue dict. This way, in a follow-up validation step we can use exactly the same file name (See the `if 'filename' in kwargs:` branch above).
                if issue:
                    issue['filename'] = os.path.basename(file_path)
    
                os.remove(file_path)
                return issue
    
            return writer
        return wrapper

    So, in short, kwargs is usually changed in the decorators before the actual SUT execution to change one of its argument. While issue is updated to define what to save into the database.

    Renáta Hodován
    @renatahodovan
    hi @pmatos !
    just wanted to tell you, that I landed several patches into Fuzzinator. Among others, every metadata is saved as string into the database, hence db inspection became easier. So, if you want to try out, then you need to update your repo.