Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Ian Fleming
    @midniteio
    This is resolved now in version 0.13.4, feel free to ping here again if that isnt working for you or if you have other questions
    guilhotarras
    @guilhotarras
    Thanks ! It really works. But there is also another bug. When I take a screenshot, the screenshot is not being shown on the html generated report. If you check the results.json file, you will see there that it isnt any reference to the taken screenshot.
    here is my code to take the screenshot:
    this.After({tags: ['@01_web_app_build_number']}, function(scenario) {
    this.browser.takeScreenshot()
    .then( buffer => scenario.attach(new Buffer(buffer, 'base64'), 'image/png') )
    });
    Ian Fleming
    @midniteio
    @guilhotarras good to hear. Re: screenshots, can you make an issue for this? I can try and take a look at it today. Functionally speaking, multicuke uses the json output that comes directly from cucumber, so should have anything that cucumber itself provides there, can you gist/link a snippet of the results you're getting output? If not for company/project reasons, that's fine- I can still try to repro
    Ian Fleming
    @midniteio
    I just attempted this by adding scenario.attach(new Buffer([100, 97, 116, 97]), 'image/png'); to the beforeHooks in one of our projects, I am seeing this in the logs multi-cuke produces:
    "embeddings": [
      {
        "data": "ZGF0YQ==",
        "mime_type": "image/png"
      }
    ],
    guilhotarras
    @guilhotarras
    It looks like that now its working :) Maybe I did something wrong on the last time I tried it. Thanks !
    guilhotarras
    @guilhotarras
    Do you have planned to have a re-run failed scenarios feature ?
    guilhotarras
    @guilhotarras
    Another question, when using multi-cuke, I write console.log(''some text") on my tests, but nothing is shown on the console
    that is a problem, when I want to make debugs
    Ian Fleming
    @midniteio
    Re: console.logs, that's mentioned here: midniteio/multi-cuke#56
    The problem here is that we are explicitly silencing stdout on test runs because we run X number of tests at once all trying to write to console and it becomes a jumbled mess of incoherent text. Our solution is to silence that output, and then output the results of the tests once the test has finished based on its results given by the cucumber json- but that means debug text gets stuck in a bad place
    Ian Fleming
    @midniteio
    I do have plans to refactor a bit so that we can have the best of both worlds. Ther'es an option we could take to pipe the output per-scenario rather than ignore it, but your console would be flooded with sumnmary output on every single scenario you run and would be unwieldy. And scraping that summary away from the text isn't a great solution because that could vary with a given version of cucumber and that'd be a bummer for compatability
    In the meantime, instead of console.log you could simply have a debug log file that you fs.appendFile to if that help? :/ Not the solution you probably were looking for, but I have this issue noted and planned to take care of soon
    This issue's also listed within the readme, and we have another solution you could try listed here if you like: https://github.com/midniteio/multi-cuke#differences-from-standard-cucumber-js
    but will be looking to support you're standard console.log/warn/error commands when I get to refactoring this
    Ian Fleming
    @midniteio
    Re: failure reruns, issue added here: midniteio/multi-cuke#67 will address this one asap
    Ian Fleming
    @midniteio
    So in regards to console logs, this is solvable on our end with stubs, in a way that you could then use console log/warn/error/etc as normal and we'd be able to output it at test time.
    However associating it with a step would be a little more difficult without explicitly changing the console calls, which I'd like to avoid so that step definitions work regardless of cucumber/multi-cuke/parallel/serial
    It will print them in order, but theyd display in order after all of the steps
    I suppose we could get a timestamp at the time of console log, and then try to appropriately place it within the order of steps based on the timestamp of each step
    Ian Fleming
    @midniteio
    But in any case, this would require a refactor worker.jsfrom spawning cucumber to instead spawning a worker that runs cucumber from node (The difference being spawning cucumber as a process won't let us capture console logs with a stub because all output comes from stdout, where from node we can actively monitor the function call)
    Ian Fleming
    @midniteio
    I may have spoken too soon, I forgot the reason we moved away from the above approach was an issue a coworker and I discovered with cucumber itself when calling it from it's own api (See issue #2 here: https://github.com/cucumber/cucumber-js/issues/565). It's fixed in Cucumber v2.0, but that makes supporting both v1 and v2+ of cucumber difficult with that approach.
    guilhotarras
    @guilhotarras
    OK Thanks. So currently the only way to do some output is to a file, right ?
    Ian Fleming
    @midniteio
    That's probably the most advisable way at the moment unfortunately
    I'll look into other possibilities on ha doing this across all cucumber versions
    guilhotarras
    @guilhotarras
    Hi, can you tell me how workerEnvVars works ? How can I access on world.js file the properties that I define on workerEnvVars ? thanks
    Ian Fleming
    @midniteio
    @guilhotarras Sure, workerEnvVars will do pretty much what it sounds like: it exports environment variables to the spawned process. I actually just looked at this to double check and realized it wasn't setting them properly, just inheritting you're current environment variables (which apparently is why we missed this, since our team duplicates that code to both export the variables and pass them :angry:) In any case, that fix lands here: midniteio/multi-cuke#68
    Moving past that though, assuming that fix gets in, you can access those in your world.js just as you would any other environment variable with process.env.PROPERTYNAME or process.env['PROPERTYNAME'] or, if using es6, you can do object deconstruction and just simply let {PROPERTYNAME} = process.env (assuming in this case your workerEnvVars looked something like
    {
      PROPERTYNAME: 'Some value'
    }
    Ian Fleming
    @midniteio
    That above PR has landed, so these directions should get you what you're looking for.
    guilhotarras
    @guilhotarras
    Ok Thanks !
    guilhotarras
    @guilhotarras
    I have one more question: On my world.js file I have defined several variables that I use on the step definitions
    this.afterHooks = {
    "preAssessmentId": null,
    "iebModule": null,
    "deleteStudyPlan": false,
    "userLogin": null,
    "userPassword": null,
    "screenShots": []
    };
    My question is, when I run multi-cuke, each thread has its own this.afterHooks variable ?
    or everything on world.js is shared between the threads ?
    guilhotarras
    @guilhotarras
    And another thing, how many workers do you recommend being used, to not affect to much the tests perfomance ?
    guilhotarras
    @guilhotarras
    I have noticed that when you run test 1 and test2 and the report files are generated, next you run test3 and test 4, but the merged result file uses all test1, test2, test3 and test4. I would like to have the merged result file as an output of the last execution test suite. In this example, would be for only test 3 and test 4. Is this possible ?
    Current behavior is grabbing all json files to generate the results.json, and in my case I would like to grab only the json files of the last execution.
    Ian Fleming
    @midniteio
    I suppose it would rebuild it with all runs after running again... hmm... that seems like not what you'd want at all. Will fix tonight
    The number of worker is pretty dependent on the machine running it. I have a long-standing issue here: midniteio/multi-cuke#11 that would sort of address this by introducing an abstract cost/resource system (where you could then basically spin up clients up until you reach some threshhold of performance basically and keep you leveled there automatically). That's not been worked on yet though
    I'm not sure if I have recommendations on how many you should run because it's both machine and browser dependent. As an example, we use this in my company actually testing a custom version of CEF that has more overhead than Chrome or Firefox etc would have, so our number in parallel would differ
    Ian Fleming
    @midniteio
    Re: afterhooks- yeah each thread runs its own instance of Cucumber so each has its own instance of world/afterhooks/etc. For a given scenario, you'll still have access to it across each step, but you wouldn't be able to pass data from scenario to scenario (which you shouldn't do fundamentally anyway really, and in Cucumber itself I believe it will reset world and hooks on each scenario as well)
    guilhotarras
    @guilhotarras
    Alright ! I will be waiting for the re run feature that is very important. When do you expect it to be done ?
    Ian Fleming
    @midniteio
    @guilhotarras sorry for the delayed response. I will try to get that worked up and merged I tomorrow (March 7)
    guilhotarras
    @guilhotarras
    Thanks. Please ping me when its done.
    guilhotarras
    @guilhotarras
    Also, the expected behavior is that when a test fails, the json output is ignored, until a new run is made and successful. The json output will only represent a failure when all the number of re runs of that scenario fails, right ? Thank you very much
    guilhotarras
    @guilhotarras
    Hi there, there is also another bug, when passing tags: ['@01_web_app_build_number', '@02_eula_check'] (more than one tag), multi-cuke says that there are no scenarios that match this
    Ian Fleming
    @midniteio
    FYI, New versino of Gherkin released today breaks multicuke it would seem. I will be bumping version to address this
    guilhotarras
    @guilhotarras
    Hi ... how does that affect multi-cuke ?
    guilhotarras
    @guilhotarras
    I am having a very strange issue on my bamboo server (locally it works well) - when I run 2 scenarios, there are runned 2 times the same tests ! If I am running 3 scenarios, they run 3 times the same tests. Can you help ?