falsifiable.us/popper

Q&A related to the Popper experimentation protocol

Vipul Chhabra

@vipulchhabra99

https://github.com/systemslab/popper/pull/744/files in actions-demo file line 126

These are the merge conflicts that you haven't resolved while rebasing with master and pushing them back

Vipul Chhabra

@vipulchhabra99

Hi, I am working on the issue to support custom arguments in popper run, The approach that I am currently thinking of is by setting these arguments as environment variables and the variables that are used in workflows would also be declared separately and during run time I can fetch them. If those environment variables are not declared then log the appropriate error. I would appreciate some feedback for the same.

Ivo Jimenez

@ivotron

thanks @vipulchhabra99, really appreciate your time. I left a comment on that issue.

there aren't any merge conflicts with the base

thanks a lot for your patience @sarthak0906. Regarding systemslab/popper#744, there is an issue related to the the additions of --no-color in the tests

sarthak khandelwal

@sarthak0906

No problem related to that, I am here to contribute and learn so all this is really amazing 🤓

Vipul Chhabra

@vipulchhabra99

Hi @ivotron, I am working on the issue of adding docstring. I had the following doubts :
1) Ctx variable is used in many functions but hasn't been used in function declaration so what actually should I write as its description.
2) Wf variable similarly.
I have updated the docstrings of the commands and now working on other parts of it.

Vipul Chhabra

@vipulchhabra99

Hi, I am working on the issue of adding docstring. I have completed almost 90 % of the work. I am stuck upon the description of a few variables that are used in function. They are as follows:
1) ctx variable is used in many commands.
2) wf in command_dot.py file in add_to_graph function.
3) frame(class) in signal_handler function in ci.py
4) mod.cli being returned in cli.py
5) wf in check_secrets in gha.py
6) wid in download_actions in gha.py
7) r in get_default_branch in scm.py
It would be great if my queries would be resolved so that I can fix the issue.

Ivo Jimenez

@ivotron

sorry for the delay @vipulchhabra99, I left comments on systemslab/popper#749

Vipul Chhabra

@vipulchhabra99

No issues @ivotron,Thanks alot.

@ivotron I am currently working on adding features for custom arguments, I currently used string replacement for implementing it which will require special naming of those substitution varibles. Is that fine?

Vipul Chhabra

@vipulchhabra99

Like google cloud substitutions uses

Ivo Jimenez

@ivotron

@vipulchhabra99 do you mean prefixing variable names with an underscore?

Vipul Chhabra

@vipulchhabra99

Yeah and having all the characters in capitals

Ivo Jimenez

@ivotron

yeah, that works

Vipul Chhabra

@vipulchhabra99

Hi @ivotron , I have added the pr for custom args. Can you please review it once?

Vipul Chhabra

@vipulchhabra99

I was thinking about how could we support both HCl and YAML syntax. One of the ideas I could think of is we can define specifications differently for both types of files and whenever put our file in HCl parser or YAML parser if our file matches the style required by parsing library. It would work using the specifications defined for that particular type of file else would use other specifications. I would appreciate some feedback for the same.

Jayjeet Chakraborty

@JayjeetAtGithub

@vipulchhabra99 What do you mean by defining specs differently? Whether its yaml or hcl, the structure will have to be same. The ideal scenario is where we turn either yaml or hcl into a same python dict or object.

Vipul Chhabra

@vipulchhabra99

ya exactly, the only difference that we will encounter is which parsing library to be used and converting it to standard format which would be further used by other functions of the program rest other parts of parsing would remain same.

Jayjeet Chakraborty

@JayjeetAtGithub

Yeah, so there will be just another layer between the yaml or hcl file and the generated dict, which will identify whether its yaml or workflow format and use the respective parsers.All validation rules will be same.

Vipul Chhabra

@vipulchhabra99

yes exactly

Jayjeet Chakraborty

@JayjeetAtGithub

great.

Vipul Chhabra

@vipulchhabra99

Can I work on this if this issue hasn't been assigned to anyone else?

Jayjeet Chakraborty

@JayjeetAtGithub

You can express your interest in the issue by leaving a comment. Or your thoughts. Then we can discuss on specific details there with @ivotron and take it from there . Does that sound good ?

Vipul Chhabra

@vipulchhabra99

Hi @ivotron @JayjeetAtGithub , I am writing integration test cases and unit test cases for substitutions flag but I think I can't use it for needs attribute and If I use it for secrets it gives me key error in Integration Testing do I need to solve these issues?

Ivo Jimenez

@ivotron

@vipulchhabra99 can you please update the PR with the code you're referring to?

Vipul Chhabra

@vipulchhabra99

Yeah sure

Vipul Chhabra

@vipulchhabra99

I have updated the pr but his contains pep8 styling errors and also without appropriate messages . After making more changes I will clean the commits history and update the pr

Vipul Chhabra

@vipulchhabra99

hi @ivotron, I found out the error why I was unable to use substitutions for secrets that was because I didn't set that variable as environment variable. When I set that variable then It worked. Sorry for the inconvenience caused by me.

Vipul Chhabra

@vipulchhabra99

hi @ivotron, I found out the error why I was unable to use substitutions for secrets because I didn't export those variables during Integration Testing When I set that variable then It worked. Sorry for the inconvenience caused by me.

Vipul Chhabra

@vipulchhabra99

To make substitutions usable for needs, I have migrated the parsing substitutions step before the validation of the action block step and now It's working for all possible arguments.

Thanks alot @JayjeetAtGithub for helping me out.

Vipul Chhabra

@vipulchhabra99

Hi, @ivotron @JayjeetAtGithub I tried a lot and searched for setting the environment variables in Travis there were 2 possible solutions. One was to add the key through Travis settings for that I would require repo settings access and the other by encrypting the token and adding it in Travis file but since Travis doesn't allows the pr to use the environment variables that are encrypted or are not to be exposed to everyone. So if I add them anyhow then whenever any new user will send pr then the tests would fail.

Quincy Wofford

@qwofford

Hey Ivo! Apologies if I've asked this before, does Popper support parallel pipeline stage execution? Specifically wrt the main.workflow file, I'd like to specify two actions with one "needs" item, and I'd like to specify the next action with two "needs" items.

I can do this in my own scripts (and do), but perhaps it's better to do this with Popper if I can

Quincy Wofford

@qwofford

To give you an impression about what's going on now: I'm launching two jobs on Summit in one stage, and then I'm waiting for those jobs to finish in the next stage. Once the jobs are done I move on to packaging the research artifact. This works, but if my popper pipeline is to represent an experiment's flow, then I think it might make sense to move this abstraction up to main.workflow.

Ivo Jimenez

@ivotron

hey Quincy! Yeah, Popper can execute them in parallel by doing:

popper run --parallel

and the workflow looks something like:

workflow "foo" {
  resolves = "packaging"
}

action "a" {
  ...
}

action "b" {
  ...
}

action "packaging" {
  needs = ["a", "b"]
}

Screen Shot 2020-01-15 at 3.13.41 PM.png

Vipul Chhabra

@vipulchhabra99

When I am running unit test for singularity , I am getting test_singularity_strat() failed due to assertion error of 255 != 0 can anyone help me to solve it?

Quincy Wofford

@qwofford

Sweet! I'm going to try this soon, Ivo. Thanks

Ivo Jimenez

@ivotron

dear @/all, we've added a preliminary list of gsoc 2020 projects here: https://github.com/uccross/uccross.github.io/blob/master/gsoc20.md#popper

Please note that participating organizations have not been announced yet. We plan to apply, and the list of project ideas we posted above is what we would like to propose as GSoC'2020 projects (plus other ideas that students might come up with), but we (CROSS) need to be officially accepted first. See here for the timeline: https://summerofcode.withgoogle.com/how-it-works/#timeline

Edidiong Etuk

@edeediong

@ivotron Hi,, I installed popper but I'm presently having issues running the basic Popper commands

I get

Error: Command 'run' doesn't exist. Type 'popper --help' for more.

The most similar commands are: run, version No module named 'pwd'

What could I be doing wrong?

Edidiong Etuk

@edeediong

So the thing is I'm using Windows, should I switch to Linux? Just tried it on a VM and it works perfectly.

Ivo Jimenez

@ivotron

@edeediong on windows, you can use the WSL https://docs.microsoft.com/en-us/windows/wsl/install-win10

Popper doesn't work when executed directly on powershell or CMD

Edidiong Etuk

@edeediong

Thanks... Ama use the Linux Machine I used before

Aditya Prajapati

@Syn3rman

Hey,
I'm a junior year undergrad interested in contributing to popper under gsoc this year. What would be some good resources to get started?
Thanks!

Where communities thrive

People

Repo info

Activity