falsifiable.us/popper

Q&A related to the Popper experimentation protocol

Jayjeet Chakraborty

@JayjeetAtGithub

You can express your interest in the issue by leaving a comment. Or your thoughts. Then we can discuss on specific details there with @ivotron and take it from there . Does that sound good ?

Vipul Chhabra

@vipulchhabra99

Hi @ivotron @JayjeetAtGithub , I am writing integration test cases and unit test cases for substitutions flag but I think I can't use it for needs attribute and If I use it for secrets it gives me key error in Integration Testing do I need to solve these issues?

Ivo Jimenez

@ivotron

@vipulchhabra99 can you please update the PR with the code you're referring to?

Vipul Chhabra

@vipulchhabra99

Yeah sure

Vipul Chhabra

@vipulchhabra99

I have updated the pr but his contains pep8 styling errors and also without appropriate messages . After making more changes I will clean the commits history and update the pr

Vipul Chhabra

@vipulchhabra99

hi @ivotron, I found out the error why I was unable to use substitutions for secrets that was because I didn't set that variable as environment variable. When I set that variable then It worked. Sorry for the inconvenience caused by me.

Vipul Chhabra

@vipulchhabra99

hi @ivotron, I found out the error why I was unable to use substitutions for secrets because I didn't export those variables during Integration Testing When I set that variable then It worked. Sorry for the inconvenience caused by me.

Vipul Chhabra

@vipulchhabra99

To make substitutions usable for needs, I have migrated the parsing substitutions step before the validation of the action block step and now It's working for all possible arguments.

Thanks alot @JayjeetAtGithub for helping me out.

Vipul Chhabra

@vipulchhabra99

Hi, @ivotron @JayjeetAtGithub I tried a lot and searched for setting the environment variables in Travis there were 2 possible solutions. One was to add the key through Travis settings for that I would require repo settings access and the other by encrypting the token and adding it in Travis file but since Travis doesn't allows the pr to use the environment variables that are encrypted or are not to be exposed to everyone. So if I add them anyhow then whenever any new user will send pr then the tests would fail.

Quincy Wofford

@qwofford

Hey Ivo! Apologies if I've asked this before, does Popper support parallel pipeline stage execution? Specifically wrt the main.workflow file, I'd like to specify two actions with one "needs" item, and I'd like to specify the next action with two "needs" items.

I can do this in my own scripts (and do), but perhaps it's better to do this with Popper if I can

Quincy Wofford

@qwofford

To give you an impression about what's going on now: I'm launching two jobs on Summit in one stage, and then I'm waiting for those jobs to finish in the next stage. Once the jobs are done I move on to packaging the research artifact. This works, but if my popper pipeline is to represent an experiment's flow, then I think it might make sense to move this abstraction up to main.workflow.

Ivo Jimenez

@ivotron

hey Quincy! Yeah, Popper can execute them in parallel by doing:

popper run --parallel

and the workflow looks something like:

workflow "foo" {
  resolves = "packaging"
}

action "a" {
  ...
}

action "b" {
  ...
}

action "packaging" {
  needs = ["a", "b"]
}

Screen Shot 2020-01-15 at 3.13.41 PM.png

Vipul Chhabra

@vipulchhabra99

When I am running unit test for singularity , I am getting test_singularity_strat() failed due to assertion error of 255 != 0 can anyone help me to solve it?

Quincy Wofford

@qwofford

Sweet! I'm going to try this soon, Ivo. Thanks

Ivo Jimenez

@ivotron

dear @/all, we've added a preliminary list of gsoc 2020 projects here: https://github.com/uccross/uccross.github.io/blob/master/gsoc20.md#popper

Please note that participating organizations have not been announced yet. We plan to apply, and the list of project ideas we posted above is what we would like to propose as GSoC'2020 projects (plus other ideas that students might come up with), but we (CROSS) need to be officially accepted first. See here for the timeline: https://summerofcode.withgoogle.com/how-it-works/#timeline

Edidiong Etuk

@edeediong

@ivotron Hi,, I installed popper but I'm presently having issues running the basic Popper commands

I get

Error: Command 'run' doesn't exist. Type 'popper --help' for more.

The most similar commands are: run, version No module named 'pwd'

What could I be doing wrong?

Edidiong Etuk

@edeediong

So the thing is I'm using Windows, should I switch to Linux? Just tried it on a VM and it works perfectly.

Ivo Jimenez

@ivotron

@edeediong on windows, you can use the WSL https://docs.microsoft.com/en-us/windows/wsl/install-win10

Popper doesn't work when executed directly on powershell or CMD

Edidiong Etuk

@edeediong

Thanks... Ama use the Linux Machine I used before

Aditya Prajapati

@Syn3rman

Hey,
I'm a junior year undergrad interested in contributing to popper under gsoc this year. What would be some good resources to get started?
Thanks!

Edidiong Etuk

@edeediong

welcome @Syn3rman . You can check https://github.com/uccross/uccross.github.io/blob/master/gsoc20.md#popper for Good First Issue. Then you either try to fix them or any other bugs. More info will be decimated in due time

Aditya Prajapati

@Syn3rman

Thanks! I'll familiarise myself with the code and start working on an issue asap :)

Ivo Jimenez

@ivotron

thanks for your interest @Syn3rman , as @edeediong mentioned, we have a list of issues tagged with Good First Issue here: https://github.com/systemslab/popper/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22

Jayesh Sharma

@wjayesh

Hello, I am a sophomore from IIT Bhubaneswar and love working on open source software and building communities. I am so pumped to have joined this channel and look forward to learning a lot from all of you folks! I have asked to be assigned to an issue so that I can quickly get to speed with working on Popper :)

Ivo Jimenez

@ivotron

welcome @WJayesh ! thanks for your interest

Simon Worthington

@mrchristian

@ivotron Thanks for the presentation yesterday on the #elife open source community call, great stuff your doing here, I'll give it a spin

Vipul Chhabra

@vipulchhabra99

Hi everyone, I am getting mk_repo is not an attribute in utils module. Can anyone help me how to fix this I tried different ways but none worked.

Ivo Jimenez

@ivotron

@ivotron Thanks for the presentation yesterday on the #elife open source community call, great stuff your doing here, I'll give it a spin

thanks a lot @mrchristian happy to help with any questions you might have

@vipulchhabra99 can you please provide a bit more context on the error you're observing?

Vipul Chhabra

@vipulchhabra99

@ivotron, Actually I am running unit-test cases and I am observing this error. This is probably due to the error of the utils module is not installed. I tried installing the popper again but still, that error remains the same and I tried various modules with a similar name but none worked.

But I also think that since the last build of popper on Travis was successful. So It can be due to some other reasons.

Vipul Chhabra

@vipulchhabra99

Hi @ivotron , I tried the method being used in travis.yml .It worked, thanks for the help.

Vipul Chhabra

@vipulchhabra99

Hi @ivotron, I was going through the conversion of validating integration test but there are workflows that are not valid in HCL syntax since all the features are not currently available for YAML syntax that are present in HCL so should I convert it using .worflow file or .yml file?

Ivo Jimenez

@ivotron

hi @vipulchhabra99 , yeah, testing those using HCL syntax is fine

Vipul Chhabra

@vipulchhabra99

ok, thanks @ivotron

Vipul Chhabra

@vipulchhabra99

Hi @ivotron, the parallel flag seems to have a missing implementation, is it removed?

Vipul Chhabra

@vipulchhabra99

similarly, the test for inject is also not working should I open issue for the same?

Ivo Jimenez

@ivotron

hi @vipulchhabra99 , yeah, you can leave those out from the tests. Thanks a lot!

Ivo Jimenez

@ivotron

hi @/all just to let you know that we also have a slack workspace in addition to this gitter channel: https://join.slack.com/t/getpopper/shared_invite/zt-dtn0se2s-c50myMHNpeoikQXDeNbPew

bertbesser

@bertbesser

Hi everybody, recently I started using popper for private 'dockerized batch jobs'. Now I wonder if popper allows to define DAGs of steps (instead of just linear sequence), but I did not find any documentation that shows how or that clearly points out that it's not possible. Do you have pointers?

bertbesser

@bertbesser

Also, when implementing support for k8s, how will the concept of a workspace mount be changed in order to pass on intermediate results from one step to the next? Perhaps sth similar to Argo artifacts?

Thx.

Where communities thrive

People

Repo info

Activity