Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Maks Rafalko
    @maks-rafalko
    @computamike thanks for letting me know. Glad you found an issue !
    Mike Hingley
    @computamike

    Hi - Question about false positives - so I have the following mutant output :

        ____      ____          __  _
       /  _/___  / __/__  _____/ /_(_)___  ____ 
       / // __ \/ /_/ _ \/ ___/ __/ / __ \/ __ \
     _/ // / / / __/  __/ /__/ /_/ / /_/ / / / /
    /___/_/ /_/_/  \___/\___/\__/_/\____/_/ /_/
    
    Running initial test suite...
    
    Phpunit version: 6.2.4
    
       61 [============================] 3 mins
    
    Generate mutants...
    
    Processing source code files: 4/4
    Creating mutated files and processes: 4/4
    .: killed, M: escaped, S: uncovered, E: fatal error, T: timed out
    
    SMMM                                                 (4 / 4)
    
    4 mutations were generated:
           0 mutants were killed
           1 mutants were not covered by tests
           3 covered mutants were not detected
           0 errors were encountered
           0 time outs were encountered
    
    Metrics:
             Mutation Score Indicator (MSI): 0%
             Mutation Code Coverage: 75%
             Covered Code MSI: 0%
    
    Please note that some mutants will inevitably be harmless (i.e. false positives).

    so I took one of the mutants that i found in my infection log :

    3) /Users/mikehingley/Code/octopus/pacific/wordpress/web/app/plugins/pacific-octopus/pacific-octopus.php:131    [M] FalseValue
    exec /usr/local/Cellar/php71/7.1.14_25/bin/php -c /private/var/folders/6h/jk4qf_555dd4561g67lw8z600000gp/T/infectionq5df5x /Users/mikehingley/Code/octopus/pacific/wordpress/vendor/phpunit/phpunit/phpunit --configuration /var/folders/6h/jk4qf_555dd4561g67lw8z600000gp/T/infection/phpunitConfiguration.882e7e50be1628ccb7cd7e543af8b36a.infection.xml --stop-on-failure 
    --- Original
    +++ New
    @@ @@
                 if (is_wp_error($response)) {
    -                return false;
    +                return true;
                 }
                 return $response->ID;
             }
         }
     }
     /**
    PHPUnit 6.2.4 by Sebastian Bergmann and contributors.
    
    Runtime:       PHP 7.1.14
    Configuration: /private/var/folders/6h/jk4qf_555dd4561g67lw8z600000gp/T/infection/phpunitConfiguration.882e7e50be1628ccb7cd7e543af8b36a.infection.xml
    
    ......                                                              6 / 6 (100%)
    
    Time: 1.63 seconds, Memory: 4.00MB
    
    OK (6 tests, 6 assertions)

    Which I think is a false positive (somehow) because when I made that mutation to my code, and ran the tests for that file it failed a test, as one of the assertions does check that the return value is false - so I'm confused by the test would pass. I'll investigate further.

    Mike Hingley
    @computamike
    Ok - I have done more research - here is what I have found :
    • I cleared out the /private/var/folders//6h/jk4qf_555dd4561g67lw8z600000gp/T/infection folder - so that I could be sure that my results were accurate.
    • Re-ran the mutation - same result.
    • The infection folder was re-created - so that was good.
    • Found the infection log - and found the line that was run for the 3rd mutant - which I ran.
      exec /usr/local/Cellar/php71/7.1.14_25/bin/php -c /private/var/folders/6h/jk4qf_555dd4561g67lw8z600000gp/T/infectionb4u3ig /Users/mikehingley/Code/octopus/pacific/wordpress/vendor/phpunit/phpunit/phpunit --configuration /var/folders/6h/jk4qf_555dd4561g67lw8z600000gp/T/infection/phpunitConfiguration.882e7e50be1628ccb7cd7e543af8b36a.infection.xml &
    • I ran it with & to allow it to run in the background, so that I wouldn’t have to restart a new terminal.
    • I also didn’t include the stop-on-failure option - I could but I didn’t copy the line properly.
    Mike Hingley
    @computamike
    The interception doesn’t seem to be wokring - running the test and ading output to the original unit present the out put from that unit - therefore instead of running the mutant, this test is actually running the original unit. This is why the tests pass, because the original code is being called.
    Mike Hingley
    @computamike
    I’m not sure - but i am currently working on the idea / premise that the issue is related to unit testing tags : * @runInSeparateProcess
     * @preserveGlobalState disabled
    Mike Hingley
    @computamike
    There doesn’t seem to be any coverage around infection/src/StreamWrapper/IncludeInterceptor.php - I’ll write a test project to confirm my theory and try and write a patch… I’m not a php guy so wish me luck...
    Maks Rafalko
    @maks-rafalko
    Thanks @computamike for the info. You can create a repo where tje issue is reproduced and create an issue on githib, we will look into it
    Looks really weird
    Mike Hingley
    @computamike
    Thanks Maks - I’ve started trying to create a really simple version of this - to see if I can create a minimum test example. - I’ll let you know when I have something to demo
    Maks Rafalko
    @maks-rafalko
    :+1:
    Mike Hingley
    @computamike

    Hi Maks - I have a git hub repo set up - but I’m struggling a little - I have some basic tests that in theory should allow true/false mutants to exist. I have 100% coverage, but should still have mutatnts here. Strangely infection doesn’t generate any mutants -

    ➜  infection-test git:(master) vendor/bin/infection --coverage=coverage
    You are running Infection with xdebug enabled.
        ____      ____          __  _
       /  _/___  / __/__  _____/ /_(_)___  ____ 
       / // __ \/ /_/ _ \/ ___/ __/ / __ \/ __ \
     _/ // / / / __/  __/ /__/ /_/ / /_/ / / / /
    /___/_/ /_/_/  \___/\___/\__/_/\____/_/ /_/
    
    Running initial test suite...
    
    PHPUnit version: 7.0.2
    
        7 [============================] < 1 sec
    
    Generate mutants...
    
    Processing source code files: 1/1
    Creating mutated files and processes:    0/0
    .: killed, M: escaped, S: uncovered, E: fatal error, T: timed out
    
    
    
    0 mutations were generated:
           0 mutants were killed
           0 mutants were not covered by tests
           0 covered mutants were not detected
           0 errors were encountered
           0 time outs were encountered
    
    Metrics:
             Mutation Score Indicator (MSI): 0%
             Mutation Code Coverage: 0%
             Covered Code MSI: 0%
    
    Please note that some mutants will inevitably be harmless (i.e. false positives).

    GitHub repo can be found here : https://github.com/computamike/infection-test

    Maks Rafalko
    @maks-rafalko
    Thanj you @computamike , I will have a look
    Gert de Pagter
    @BackEndTea
    Hi @computamike I'm not sure if anyone got back to you already, but due to auto-loading reasons, infection can't cover functions, only class methods.
    Gert de Pagter
    @BackEndTea
    From what i saw in your code snippets, you are working with wordpress. I must admit that i haven't worked with wordpress before, but from what i have seen it works a lot with functions, rather than classes and methods, which may be where the issue lies.
    Grummfy
    @Grummfy
    anyone on line?
    Maks Rafalko
    @maks-rafalko
    yes
    Grummfy
    @Grummfy
    hi, just to speak here about the atoum integration instead of all on the issue, I think it will be faster, if you ar eok
    Maks Rafalko
    @maks-rafalko
    no problem
    Grummfy
    @Grummfy
    regarding what I see now, I think it could interesting to perhaps refactor a bit (later) the way the framework are integrated. Perhaps to be seen more as plugins (if possible). Because a lot of code seems redundant . But perhaps I see optimization where there is not ;)
    otherwise, regarding the implementation of my integration of atoum. is there any specific point that require attention. I see some comment about being sure it's isolated from the testing framework to avoid dependency if possible. but is there some other points to be carefull?
    Grummfy
    @Grummfy
    one other question, if I see some refactoring that make integration easier (I speak about small one), do you prefer to have a pull request about it on the side instead of all together?
    Maks Rafalko
    @maks-rafalko

    is there any specific point that require attention

    you can take a look at the comments inside Codeception integration. Many of them are still not addressed. For example - there is no need to update public API of existing interfaces, adding $mutant parameter where it is not needed (https://github.com/infection/infection/pull/143#discussion_r165826810)

    do you prefer to have a pull request about it on the side instead of all together?

    I would prefer to have it separated to reduce complexity of the code review

    Grummfy
    @Grummfy
    oki, thanks.
    I read the pr about codeception a lot are common and still valid, so it was helpfull ;
    Maks Rafalko
    @maks-rafalko

    but is there some other points to be carefull?

    I have no other points at the moment. Just make sure to generate needed coverage reports to make it fast

    btw, the Slack channel is much more popular for Infection
    Grummfy
    @Grummfy
    regarding the header of file with the license, Did I put the same as the rest of the app or do you prefer I put my own like in the codeception PR
    • Copyright © 2018 Grummfy OR Copyright © 2017-2018 Maks Rafalko => for me it's the same
    I miss the slack channel ;)

    regarding license - we have a php-cs-fixer rule for it

    just run make cs-fix

    Grummfy
    @Grummfy
    yep, cs rule and other stuff like that I was thinking to run it after ;)
    regarding slack, is there any autoinvitation stuff or did I need to send you my email address?
    Maks Rafalko
    @maks-rafalko
    Grummfy
    @Grummfy
    thanks, well I miss nothing ;)
    Maks Rafalko
    @maks-rafalko

    regarding slack, is there any autoinvitation stuff or did I need to send you my email address?

    https://symfony.com/slack-invite - our channel is inside symfony-dev account

    or whatever it's named ;)
    Grummfy
    @Grummfy
    thanks
    Maks Rafalko
    @maks-rafalko
    you are welcome
    Stanislav Krasnoyarov
    @alvin777
    Hi there! Does infection still claim php 7.0 support despite using Symphony 4 which requires PHP 7.1.3+ to run?
    Owen Voke
    @pxgamer
    Looks like it's using Symfony ^3.2 or ^4.0 so it should support PHP 7.0
    Gert de Pagter
    @BackEndTea

    Hey @alvin777 Infection uses symfony 3.2, but supports 4.0 as well. If your project requires php 7.0 you could consider adding

    "config": {
      "platform": {
        "php": "7.0.8"
      }
    }

    This to your composer.json, which forces dependencies to be compatible with that php version

    Stanislav Krasnoyarov
    @alvin777
    Thanks, that seemed to help!
    Edvin Malinovskis
    @nCrazed
    Is it possible to disable xsd check for phpunit? the remote xsd storage appears to be broken again with no ETA on a fix which means that we have to either wait indefinitely or disable mutation tests in our CI.
    <phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
             xsi:noNamespaceSchemaLocation="./vendor/phpunit/phpunit/phpunit.xsd"
    Edvin Malinovskis
    @nCrazed
    @borNfreee perfect, thanks
    Joe Watkins
    @krakjoe
    Hi
    I hear you have a problem with phpdbg ... tell me all about it ...
    Bernhard Breytenbach
    @Xethron
    haha. I just came here to ask about phpdbg :/
    I can't seem to set the memory limit. I believe infection spins off another thread for the phpunit tests? Is it possible that the parameters aren't carried across?
    I tried the following command which didn't work: phpdbg -qrr -dmemory_limit=1G vendor/bin/infection --min-msi=95 --min-covered-msi=98 --threads=4