by

Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
Owen Barton
@grugnog_gitlab
Added my notes to http://codeearth.net/wiki/index.php/Hackathon_Wed_Notes - feel free to add other notes to this page also!
Denton Gentry
@DentonGentry

One thought, though I think the solution is simple: when Researcher A goes to work on a model and needs to update code on the HTTP server to match, those changes should not go live for everyone.

Researcher A might be expected to spin up a dev instance of the HTTP server, update a URL in their Excel sheet to point to their dev instance, and go.

Alternately, there could be URL parameters to pass some kind of version indication, and Researcher A might use the same HTTP server as everyone else but set their Excel sheet to request "?v=dgentry.9.5.2018" in their URLs (for example).

Versioning might also get used to resolve some of the compatibility issues mentioned earlier today, like how many regions there are. Honestly though, I think we have other options there like named fields in JSON rather than fixed-size arrays where Excel code needs to know the definition of each column.

Owen Barton
@grugnog_gitlab
Owen Barton
@grugnog_gitlab
Denton Gentry
@DentonGentry
Nifty. I imagine the eventual interface will need to be a POST due to the size of the data structures which may be passed around, but still nifty to see it working.
Robert L. Read
@RobertLRead
@DentonGentry Yes, we were just hacking furiously to get something working. In fact before our energy was exhausted Owen got the solution above working, and I got (thanks to a free software project) the ability to make a request to his code from Excel. However, I have not yet integrated this with the DD model spreadsheet, nor have I even got it reading from cells; I am working on that now.
Robert L. Read
@RobertLRead
FYI: The VBA code that makes the request using the "MIT License", which is a open source license that does not require programmers to give their code back, unlike the GPL. Marc, the attorney, and Henry can explain all this tomorrow.
Robert L. Read
@RobertLRead
@marctjones_gitlab Can you please load this repo onto your windows machine: git@gitlab.com:RobertLRead/drawdown.git
Owen Barton
@grugnog_gitlab
@gregelin are you planning on joining us this morning - we are almost ready to start
Greg Elin
@gregelin
Yes!
Denton Gentry
@DentonGentry
We're getting zoom set up.
Greg Elin
@gregelin
Very quiet on mic right now...
Denton Gentry
@DentonGentry
Ryan is trying the mic now.
Greg Elin
@gregelin
Audio is good from Robert.
Greg Elin
@gregelin
@marctjones_gitlab There's an echo when you talk
Echo is bad...
Owen Barton
@grugnog_gitlab
@gregelin better?
Greg Elin
@gregelin
Much better. Thanks.
Greg Elin
@gregelin
+1 on the forking risk!
Nick Peters
@nickpeters_drawdown_gitlab
Thanks Greg - thoughts on this plan (essentially Owen's option b)? Any clarification needed?
Greg Elin
@gregelin
(BRB, I have a scrum call)
I'm back.
Nick Peters
@nickpeters_drawdown_gitlab
Thanks - we are taking a short break, back in ~5min
Denton Gentry
@DentonGentry

For me at least, the git@gitlab.com user identification didn't work. Cloning the repo without a user did work:

git clone https://gitlab.com/RobertLRead/drawdown.git

Denton Gentry
@DentonGentry

Possibility for integration tests of Excel: http://docs.xlwings.org/en/stable/quickstart.html

For example, a test could be:

import xlwings as xw
with xw.Book(r'/path/to/drawdown.xlsx') as workbook:
sheet1 = workbook.sheets['Sheet1']
cell_A1 = sheet1.range('A1').value
ASSERT_EQ(cell_A1, 37)

...and gitter removed all of the spaces.
xlwings is distributed under a BSD 3-clause license.
Greg: we're all currently sitting around the table, typing.
Greg Elin
@gregelin
(Ok, away from keyboard for a moment...)
Owen Barton
@grugnog_gitlab
Greg Elin
@gregelin
@grugnog_gitlab @marctjones_gitlab @DentonGentry I'm working on tests directory in a new branch. Will make a PR when the basic framework is in place.
Robert L. Read
@RobertLRead
Thanks @gregelin
Owen Barton
@grugnog_gitlab
@gregelin great :) - what type of tests are you working on?
@gregelin I added a simple dummy pytest in the Gitlab CI pipeline in https://gitlab.com/codeearth/drawdown/blob/master/.gitlab-ci.yml - we should replace or add your tests into this pipeline when they are ready also
Owen Barton
@grugnog_gitlab
@RobertLRead @marctjones_gitlab @DentonGentry The Gitlab CI pipeline is now running the build and tests (just the dummy one for now) on each merge request, then it is automatically deploying all changes in master to https://drawdown-staging.herokuapp.com/ and deploying tags to https://drawdown.herokuapp.com/ (I cut a "0.1" tag to do an initial deploy). You can see the pipeline results at https://gitlab.com/codeearth/drawdown/pipelines
I will add these details to the README :)
Owen Barton
@grugnog_gitlab
Also you can see Gitlab activity in this Gitter sidebar
Robert L. Read
@RobertLRead
Good work @grugnog_gitlab
Let's plan to have everybody compare notes and plan for the evening at 5:15. I assume we will talk to 5:30, and then discuss plans for the evening, both work and pleasure.
Denton Gentry
@DentonGentry

Current status:

  1. integration test using xlwings and Excel can work with some compromises:
    a) on MacOS, may have to enable macros within Excel. Otherwise it prompts to Enable Macros every time the sheet is opened, and I haven't found a good way to handle this automatically.
    b) on MacOS, can not have a workbook already open. Otherwise it tries to launch another instance of Excel, which raises a weird protected worksheet error.

  2. Expecting to have tests/excel_integration_test.py with a setup() function to start a flask process on an unassigned TCP port, check whether Excel already has a workbook open and fail if it does (to survive 1.b.), and then test cases for each xlsm file which will open the file, write the URL="http://localhost:port/path" to a cell, and then check the values of a few output cells again known golden values.

  3. I will be heading with daughter to track practice at 5:30

Oh, also: will see if xlwings can install in a python venv, to minimize dependencies.
Robert L. Read
@RobertLRead
@DentonGentry That all sounds fantastic! I'm not sure I grok it fully, but maybe you can explain it in the morning. Owen and I are going to try to work another hour, although we are both tired. We are close to "completing the circuit" from Excel to python and back for the Net Funcitonal Unit stuff, and Owen and ready to make good progress on the rest of the model (that you participated with today.)
Robert L. Read
@RobertLRead
@grugnog_gitlab Here is the requestbin link: https://requestbin.fullcontact.com/1jwwq0i1?inspect
Owen Barton
@grugnog_gitlab
@RobertLRead codeearth/drawdown!6
Robert L. Read
@RobertLRead
Team, some more progress last night: We can now move entire tables from the Net Functional Unit calculation into our Python model and back out and onto the spreadsheet. Thanks to Owen for improving the robustness of this. Here are some Todos: 1) Get the "kernel" model working in Python 2) I have to iron out some configuraiton issues, 3) Try @DentonGentry 's "gold standard" test system 4) Create a "Model Eveolution Configuration tab" where Model users can configure URLs and debugging stuff on a Tab in the model instead of VBA, 5) Document all of this, 6) Document all of this some more.
Nick Peters
@nickpeters_drawdown_gitlab
Thanks for the update Rob and to everyone for all of the hard work. Lots of progress!! See everyone soon for day 3
Denton Gentry
@DentonGentry

An initial version of the gold standard test is in the cloned repo https://gitlab.com/DentonGentry/drawdown

There are at least two things left to do:

  1. determine whether and where to check in excel files. This will make them public, as the repository is public.
  2. Get the flask app to reliably stop and start in the test. The excel file I've been testing with is the original, unmodified file, so it has no dependency on the flask server running.
Owen Barton
@grugnog
I also added a unit test and documentation for the the one function we implemented, so we have a pattern to replicate for rest of the "kernel" functions/tables.
Denton Gentry
@DentonGentry
Oh, also: CI wants to run the test to merge, but this particular test will only work on a system with Excel. Need to figure out how to mark the test as not to be automatically run.