These are chat archives for mojotech/pioneer

28th
May 2015
Steven Langbroek
@StevenLangbroek
May 28 2015 08:36
so, did anyone ever get saucelabs working with pioneer + jenkins?
Ricardo Machado
@mAiNiNfEcTiOn
May 28 2015 08:55
Well I only made Pioneeer and Sauce working together… With Jenkins, no
Tom Hicks
@tomhicks
May 28 2015 09:31
How does it not work with Jenkins? It just runs scripts! What is not working?
Steven Langbroek
@StevenLangbroek
May 28 2015 09:45
the tunnel appears to not be working...
i'll contact saucelabs support, seems to be on their end
Tom Hicks
@tomhicks
May 28 2015 09:48
To be honest, although I don't know how much investigation you've done, it's unlikely to be a problem their end as they have thousands of customers all using the tunnel without issue. What makes you think it's their issue? (Also, I'm trying to save you from dealing with their crappy support!)
Steven Langbroek
@StevenLangbroek
May 28 2015 09:50
haha true
well, whenever i run the tests, they just show a browser default "connection refused" screen
Tom Hicks
@tomhicks
May 28 2015 09:51
Ok have you tried running the tunnel locally and testing against localhost and everything works?
Steven Langbroek
@StevenLangbroek
May 28 2015 09:52
yup
works like a charm
i'm just trying to find some basic documentation on what to use as a tunnel-identifier
the configuration i get is which devices / os'es to launch
i don't want that
i wanna do that in my saucelabs config in pioneer
Tom Hicks
@tomhicks
May 28 2015 09:53
You don't need a tunnel identifier.
Steven Langbroek
@StevenLangbroek
May 28 2015 09:54
yeah i understand that
Tom Hicks
@tomhicks
May 28 2015 09:55
Ok are you making sure you wait for the 'you may start your tests' message?
Because that can take a while
And have you tried a manual sauce labs session while the tunnel is up and running on Jenkins?
Steven Langbroek
@StevenLangbroek
May 28 2015 09:59
haven't yet, no
Tom Hicks
@tomhicks
May 28 2015 10:01
That's usually a quicker debugging path than running the job, waiting for the tunnel, waiting for the failures, etc. just ssh into the machine, start the tunnel and play on saucelabs.com. Much easier
Steven Langbroek
@StevenLangbroek
May 28 2015 11:24
alright, got it working
apparently you can just ignore all the device selection malarkey, enable it and the tunnel will work
now, how do i configure multiple "devices"?
    new SeleniumDriver.Builder()
      .usingServer("http://ondemand.saucelabs.com:80/wd/hub")
      .withCapabilities
        browserName: 'firefox'
        platform: 'OS X 10.10'
        version: '32.0'
        username: process.env.SAUCE_USERNAME
        accessKey: process.env.SAUCE_ACCESS_KEY
      .build()
put an array into withCapabilities?
Tom Hicks
@tomhicks
May 28 2015 14:06
No you'll need multiple tests runs with multiple configure files. Pioneer-ios7.json, etc. then run each from the command line
Steven Langbroek
@StevenLangbroek
May 28 2015 14:06
:/
sad panda
Tom Hicks
@tomhicks
May 28 2015 14:07
That's really not that big of a deal!
Steven Langbroek
@StevenLangbroek
May 28 2015 14:07
no i know
it isn't
but i hoped this would've been a bit more streamlined
Tom Hicks
@tomhicks
May 28 2015 14:08
But that would add a load more complexity. It's really just running a program and saving its output to file. It shouldn't do much more hand holding than it does in my opinion. Simple !== easy
Steven Langbroek
@StevenLangbroek
May 28 2015 14:18
i'm just a bit surprised, cause in karma-sauce-launcher it will run 'em all in parallel
these test-runs are gonna take forever
eh wait no not in parallel
mean this:
var customLaunchers = {
    sl_chrome: {
      base: 'SauceLabs',
      browserName: 'chrome',
      platform: 'Windows 7',
      version: '35'
    },
    sl_firefox: {
      base: 'SauceLabs',
      browserName: 'firefox',
      version: '30'
    },
    sl_ios_safari: {
      base: 'SauceLabs',
      browserName: 'iphone',
      platform: 'OS X 10.9',
      version: '7.1'
    },
    sl_ie_11: {
      base: 'SauceLabs',
      browserName: 'internet explorer',
      platform: 'Windows 8.1',
      version: '11'
    }
  };
Tom Hicks
@tomhicks
May 28 2015 14:31
Ok that's great but that wouldn't suffice for our purposes, as we need different setup for different devices. So what works for you might not be worth the effort for other people. Having multiple configure files is slightly more tricky than what you've just written above, but waaaay more flexible
I guess I'm saying that sticking to composition over 1 hundred configuration options is better. It's more Linux-y. Do one thing and do it well. If you want multiple runs, spawn multiple processes
Steven Langbroek
@StevenLangbroek
May 28 2015 14:32
we're gonna need about 12 runs for this project :/
ios/android devices + versions
that's gonna be a long run
Tom Hicks
@tomhicks
May 28 2015 14:33
Well they can run in parallel
Steven Langbroek
@StevenLangbroek
May 28 2015 14:33
yeah might have to grab some help
Tom Hicks
@tomhicks
May 28 2015 14:33
Ok
Steven Langbroek
@StevenLangbroek
May 28 2015 14:33
this is a bit beyond my knowledge of jenkins/selenium
luckily there's a selenium meetup in our lounge today :D
Tom Hicks
@tomhicks
May 28 2015 14:34
It's not either of those two things that you need to know to get parallel test tuns
Steven Langbroek
@StevenLangbroek
May 28 2015 14:34
how then?
Tom Hicks
@tomhicks
May 28 2015 14:34
It's bash scripting, or a simple node app to spawn multiple child processes
Selenium just automates a single browser
Jenkins just runs scripts
You just need to write a script that runs multiple pioneer --configPath=whatever commands
When I'm back in the office I'll send over our parallel runner that we use to run multiple tests in parallel.
I'm out until Monday though
Steven Langbroek
@StevenLangbroek
May 28 2015 14:37
wow rly? awesome dude, thanks alot!
no problem at all
Tom Hicks
@tomhicks
May 28 2015 14:42
It's important to understand the job of each of the tools in your chain. Asking for a selenium test runner that runs cucumber features and reports them for Jenkins and runs them on sauce labs on multiple devices is very specific! The tools we have (pioneer [itself being cucumber, selenium, page object model] and Jenkins) already have this capability via piping on the command line, or using stdin and stdout in a node script, so we should use those rather than trying to get a massive load of configuration that will do all that inside a black box.
its the equivalent to the difference between grunt and gulp
Steven Langbroek
@StevenLangbroek
May 28 2015 14:42
which is why i opened #325
Tom Hicks
@tomhicks
May 28 2015 14:43
Grunt is simple to get going, but a disaster for maintenance and extensibility. Gulp on the other hand is a little more tricky to get started with, but beats the crap out of grunt when you need to alter, extend or join together tasks
Steven Langbroek
@StevenLangbroek
May 28 2015 14:43
yeah i'm a gulp user
Tom Hicks
@tomhicks
May 28 2015 14:45
So if you ever used grunt you'll know how everything in grunt exists as an non-package specifically written to use in grunt, whereas that kind of 'plugin' just isn't necessary as often for gulp.
Npm sorry, not non-package. Autocomplete
So yeah, compose existing tools into the chain you want, rather than expecting a pre-built configurable chain to exist
....and....I'm done!
Steven Langbroek
@StevenLangbroek
May 28 2015 14:47
haha
i'm with you, but feel this could've been made alot easier with a proper plugin architecture and better (or better documented) hooks
the script you guys use at work, can i write a tutorial based on my experience last couple of days and include that?
Tom Hicks
@tomhicks
May 28 2015 14:54
Sure. I've tried to write a pluggable test runner for pioneer, but it unfortunately hair doesn't work because of cucumber, so I had to resort to this script instead. And it's not so bad really
I would have preferred to use pioneer via its API rather than the command line, but for whatever reason cucumber just doesn't want to play
I might try again next week, because it really would be better. Then you could share test executors with each other. Like ones for parallel execution, or running certain tests based on tags with different browser configuration, etc