Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Ian Fleming
    @midniteio
    Hey there
    Let me take a look now and see how difficult thatd be to add
    Actually, this already is implemented, I'm going to be closing out this issue if it's ok with you
    There is a --logdir flag that you can pass a path to, thats where json logs end up
    You can see the cucumber command that gets run here, we always export a json file actually, it's what we use to parse the results per-scenario and then aggregate for results https://github.com/midniteio/multi-cuke/blob/master/src/lib/worker.js#L35
    So each log will end up there
    Ian Fleming
    @midniteio
    Also, all the scenarios do get merged back into a single results json file here once all are finished https://github.com/midniteio/multi-cuke/blob/master/src/lib/test-handler.js#L76
    So this should provide the granularity of individual logs and a single overall log
    @guilhotarras if you'd like to reopen the issue please do and ping me here, but I believe this is what you're looking for.
    Ian Fleming
    @midniteio
    In regards to midniteio/multi-cuke#63, let me look to see how cucumber itself interprets feature-level tags
    Ian Fleming
    @midniteio
    Yeah we just aren't supporting feature tags at this point, I can get this worked out
    Ian Fleming
    @midniteio
    @guilhotarras I've tagged you in a PR to resolve this second issue, midniteio/multi-cuke#65
    Ian Fleming
    @midniteio
    This is resolved now in version 0.13.4, feel free to ping here again if that isnt working for you or if you have other questions
    guilhotarras
    @guilhotarras
    Thanks ! It really works. But there is also another bug. When I take a screenshot, the screenshot is not being shown on the html generated report. If you check the results.json file, you will see there that it isnt any reference to the taken screenshot.
    here is my code to take the screenshot:
    this.After({tags: ['@01_web_app_build_number']}, function(scenario) {
    this.browser.takeScreenshot()
    .then( buffer => scenario.attach(new Buffer(buffer, 'base64'), 'image/png') )
    });
    Ian Fleming
    @midniteio
    @guilhotarras good to hear. Re: screenshots, can you make an issue for this? I can try and take a look at it today. Functionally speaking, multicuke uses the json output that comes directly from cucumber, so should have anything that cucumber itself provides there, can you gist/link a snippet of the results you're getting output? If not for company/project reasons, that's fine- I can still try to repro
    Ian Fleming
    @midniteio
    I just attempted this by adding scenario.attach(new Buffer([100, 97, 116, 97]), 'image/png'); to the beforeHooks in one of our projects, I am seeing this in the logs multi-cuke produces:
    "embeddings": [
      {
        "data": "ZGF0YQ==",
        "mime_type": "image/png"
      }
    ],
    guilhotarras
    @guilhotarras
    It looks like that now its working :) Maybe I did something wrong on the last time I tried it. Thanks !
    guilhotarras
    @guilhotarras
    Do you have planned to have a re-run failed scenarios feature ?
    guilhotarras
    @guilhotarras
    Another question, when using multi-cuke, I write console.log(''some text") on my tests, but nothing is shown on the console
    that is a problem, when I want to make debugs
    Ian Fleming
    @midniteio
    Re: console.logs, that's mentioned here: midniteio/multi-cuke#56
    The problem here is that we are explicitly silencing stdout on test runs because we run X number of tests at once all trying to write to console and it becomes a jumbled mess of incoherent text. Our solution is to silence that output, and then output the results of the tests once the test has finished based on its results given by the cucumber json- but that means debug text gets stuck in a bad place
    Ian Fleming
    @midniteio
    I do have plans to refactor a bit so that we can have the best of both worlds. Ther'es an option we could take to pipe the output per-scenario rather than ignore it, but your console would be flooded with sumnmary output on every single scenario you run and would be unwieldy. And scraping that summary away from the text isn't a great solution because that could vary with a given version of cucumber and that'd be a bummer for compatability
    In the meantime, instead of console.log you could simply have a debug log file that you fs.appendFile to if that help? :/ Not the solution you probably were looking for, but I have this issue noted and planned to take care of soon
    This issue's also listed within the readme, and we have another solution you could try listed here if you like: https://github.com/midniteio/multi-cuke#differences-from-standard-cucumber-js
    but will be looking to support you're standard console.log/warn/error commands when I get to refactoring this
    Ian Fleming
    @midniteio
    Re: failure reruns, issue added here: midniteio/multi-cuke#67 will address this one asap
    Ian Fleming
    @midniteio
    So in regards to console logs, this is solvable on our end with stubs, in a way that you could then use console log/warn/error/etc as normal and we'd be able to output it at test time.
    However associating it with a step would be a little more difficult without explicitly changing the console calls, which I'd like to avoid so that step definitions work regardless of cucumber/multi-cuke/parallel/serial
    It will print them in order, but theyd display in order after all of the steps
    I suppose we could get a timestamp at the time of console log, and then try to appropriately place it within the order of steps based on the timestamp of each step
    Ian Fleming
    @midniteio
    But in any case, this would require a refactor worker.jsfrom spawning cucumber to instead spawning a worker that runs cucumber from node (The difference being spawning cucumber as a process won't let us capture console logs with a stub because all output comes from stdout, where from node we can actively monitor the function call)
    Ian Fleming
    @midniteio
    I may have spoken too soon, I forgot the reason we moved away from the above approach was an issue a coworker and I discovered with cucumber itself when calling it from it's own api (See issue #2 here: https://github.com/cucumber/cucumber-js/issues/565). It's fixed in Cucumber v2.0, but that makes supporting both v1 and v2+ of cucumber difficult with that approach.
    guilhotarras
    @guilhotarras
    OK Thanks. So currently the only way to do some output is to a file, right ?
    Ian Fleming
    @midniteio
    That's probably the most advisable way at the moment unfortunately
    I'll look into other possibilities on ha doing this across all cucumber versions
    guilhotarras
    @guilhotarras
    Hi, can you tell me how workerEnvVars works ? How can I access on world.js file the properties that I define on workerEnvVars ? thanks
    Ian Fleming
    @midniteio
    @guilhotarras Sure, workerEnvVars will do pretty much what it sounds like: it exports environment variables to the spawned process. I actually just looked at this to double check and realized it wasn't setting them properly, just inheritting you're current environment variables (which apparently is why we missed this, since our team duplicates that code to both export the variables and pass them :angry:) In any case, that fix lands here: midniteio/multi-cuke#68
    Moving past that though, assuming that fix gets in, you can access those in your world.js just as you would any other environment variable with process.env.PROPERTYNAME or process.env['PROPERTYNAME'] or, if using es6, you can do object deconstruction and just simply let {PROPERTYNAME} = process.env (assuming in this case your workerEnvVars looked something like
    {
      PROPERTYNAME: 'Some value'
    }
    Ian Fleming
    @midniteio
    That above PR has landed, so these directions should get you what you're looking for.
    guilhotarras
    @guilhotarras
    Ok Thanks !
    guilhotarras
    @guilhotarras
    I have one more question: On my world.js file I have defined several variables that I use on the step definitions
    this.afterHooks = {
    "preAssessmentId": null,
    "iebModule": null,
    "deleteStudyPlan": false,
    "userLogin": null,
    "userPassword": null,
    "screenShots": []
    };
    My question is, when I run multi-cuke, each thread has its own this.afterHooks variable ?
    or everything on world.js is shared between the threads ?
    guilhotarras
    @guilhotarras
    And another thing, how many workers do you recommend being used, to not affect to much the tests perfomance ?
    guilhotarras
    @guilhotarras
    I have noticed that when you run test 1 and test2 and the report files are generated, next you run test3 and test 4, but the merged result file uses all test1, test2, test3 and test4. I would like to have the merged result file as an output of the last execution test suite. In this example, would be for only test 3 and test 4. Is this possible ?
    Current behavior is grabbing all json files to generate the results.json, and in my case I would like to grab only the json files of the last execution.