Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
Denton Gentry
@DentonGentry
We're getting zoom set up.
Greg Elin
@gregelin
Very quiet on mic right now...
Denton Gentry
@DentonGentry
Ryan is trying the mic now.
Greg Elin
@gregelin
Audio is good from Robert.
Greg Elin
@gregelin
@marctjones_gitlab There's an echo when you talk
Echo is bad...
Owen Barton
@grugnog_gitlab
@gregelin better?
Greg Elin
@gregelin
Much better. Thanks.
Greg Elin
@gregelin
+1 on the forking risk!
Nick Peters
@nickpeters_drawdown_gitlab
Thanks Greg - thoughts on this plan (essentially Owen's option b)? Any clarification needed?
Greg Elin
@gregelin
(BRB, I have a scrum call)
I'm back.
Nick Peters
@nickpeters_drawdown_gitlab
Thanks - we are taking a short break, back in ~5min
Denton Gentry
@DentonGentry

For me at least, the git@gitlab.com user identification didn't work. Cloning the repo without a user did work:

git clone https://gitlab.com/RobertLRead/drawdown.git

Denton Gentry
@DentonGentry

Possibility for integration tests of Excel: http://docs.xlwings.org/en/stable/quickstart.html

For example, a test could be:

import xlwings as xw
with xw.Book(r'/path/to/drawdown.xlsx') as workbook:
sheet1 = workbook.sheets['Sheet1']
cell_A1 = sheet1.range('A1').value
ASSERT_EQ(cell_A1, 37)

...and gitter removed all of the spaces.
xlwings is distributed under a BSD 3-clause license.
Greg: we're all currently sitting around the table, typing.
Greg Elin
@gregelin
(Ok, away from keyboard for a moment...)
Owen Barton
@grugnog_gitlab
Greg Elin
@gregelin
@grugnog_gitlab @marctjones_gitlab @DentonGentry I'm working on tests directory in a new branch. Will make a PR when the basic framework is in place.
Robert L. Read
@RobertLRead
Thanks @gregelin
Owen Barton
@grugnog_gitlab
@gregelin great :) - what type of tests are you working on?
@gregelin I added a simple dummy pytest in the Gitlab CI pipeline in https://gitlab.com/codeearth/drawdown/blob/master/.gitlab-ci.yml - we should replace or add your tests into this pipeline when they are ready also
Owen Barton
@grugnog_gitlab
@RobertLRead @marctjones_gitlab @DentonGentry The Gitlab CI pipeline is now running the build and tests (just the dummy one for now) on each merge request, then it is automatically deploying all changes in master to https://drawdown-staging.herokuapp.com/ and deploying tags to https://drawdown.herokuapp.com/ (I cut a "0.1" tag to do an initial deploy). You can see the pipeline results at https://gitlab.com/codeearth/drawdown/pipelines
I will add these details to the README :)
Owen Barton
@grugnog_gitlab
Also you can see Gitlab activity in this Gitter sidebar
Robert L. Read
@RobertLRead
Good work @grugnog_gitlab
Let's plan to have everybody compare notes and plan for the evening at 5:15. I assume we will talk to 5:30, and then discuss plans for the evening, both work and pleasure.
Denton Gentry
@DentonGentry

Current status:

  1. integration test using xlwings and Excel can work with some compromises:
    a) on MacOS, may have to enable macros within Excel. Otherwise it prompts to Enable Macros every time the sheet is opened, and I haven't found a good way to handle this automatically.
    b) on MacOS, can not have a workbook already open. Otherwise it tries to launch another instance of Excel, which raises a weird protected worksheet error.

  2. Expecting to have tests/excel_integration_test.py with a setup() function to start a flask process on an unassigned TCP port, check whether Excel already has a workbook open and fail if it does (to survive 1.b.), and then test cases for each xlsm file which will open the file, write the URL="http://localhost:port/path" to a cell, and then check the values of a few output cells again known golden values.

  3. I will be heading with daughter to track practice at 5:30

Oh, also: will see if xlwings can install in a python venv, to minimize dependencies.
Robert L. Read
@RobertLRead
@DentonGentry That all sounds fantastic! I'm not sure I grok it fully, but maybe you can explain it in the morning. Owen and I are going to try to work another hour, although we are both tired. We are close to "completing the circuit" from Excel to python and back for the Net Funcitonal Unit stuff, and Owen and ready to make good progress on the rest of the model (that you participated with today.)
Robert L. Read
@RobertLRead
@grugnog_gitlab Here is the requestbin link: https://requestbin.fullcontact.com/1jwwq0i1?inspect
Owen Barton
@grugnog_gitlab
@RobertLRead codeearth/drawdown!6
Robert L. Read
@RobertLRead
Team, some more progress last night: We can now move entire tables from the Net Functional Unit calculation into our Python model and back out and onto the spreadsheet. Thanks to Owen for improving the robustness of this. Here are some Todos: 1) Get the "kernel" model working in Python 2) I have to iron out some configuraiton issues, 3) Try @DentonGentry 's "gold standard" test system 4) Create a "Model Eveolution Configuration tab" where Model users can configure URLs and debugging stuff on a Tab in the model instead of VBA, 5) Document all of this, 6) Document all of this some more.
Nick Peters
@nickpeters_drawdown_gitlab
Thanks for the update Rob and to everyone for all of the hard work. Lots of progress!! See everyone soon for day 3
Denton Gentry
@DentonGentry

An initial version of the gold standard test is in the cloned repo https://gitlab.com/DentonGentry/drawdown

There are at least two things left to do:

  1. determine whether and where to check in excel files. This will make them public, as the repository is public.
  2. Get the flask app to reliably stop and start in the test. The excel file I've been testing with is the original, unmodified file, so it has no dependency on the flask server running.
Owen Barton
@grugnog
I also added a unit test and documentation for the the one function we implemented, so we have a pattern to replicate for rest of the "kernel" functions/tables.
Denton Gentry
@DentonGentry
Oh, also: CI wants to run the test to merge, but this particular test will only work on a system with Excel. Need to figure out how to mark the test as not to be automatically run.
Robert L. Read
@RobertLRead
@DentonGentry I believe we have decided you can check in excel files; just don't put a license on them yet. The fact that people can see one is a not a problem; the license is yet to be finalized.
Denton Gentry
@DentonGentry

I made some more progress on a) starting and stopping a flask app reliably and b) skipping the test on systems where Excel is not present.

I have a videoconference at 10am which wasn't amenable to rescheduling. I reserved a conference room downstairs, and will be back up by 11am (probably earlier).

Robert L. Read
@RobertLRead
Denton Gentry
@DentonGentry

Continuous Integration is going to be tricky. Adding xlwings to requirements.txt fails because:
OSError: currently only Windows and OSX are supported.

No adding it to requirements.txt makes the excel_integration_test.py fail when trying to import modules.

I think excel_integration_test.py should put the import xlwings inside a try block, and skip the tests if it fails.

Greg Elin
@gregelin
@grugnog Thanks for that unittest. I'm sorry I did not make more progress with the tests yesterday. I first started with Flask-Test, then switched to pytest, but I was not familiar with the DataFrame library and was trying to figure it out.
Greg Elin
@gregelin
@DentonGentry Did you look at other python excel libraries?
Owen Barton
@grugnog
@gregelin no probs - I thought about testing the flask serve also, but I think will Denton's end-to-end functional test we should have that base sufficiently covered, and the model unit tests provide more useful example code for model developers.
Denton Gentry
@DentonGentry
@gregelin I looked at libraries which could read Excel files, but that isn't what we really need. We want to start Excel, having the VBA code send a request to Python, and put the results back into the spreadsheet. xlwings seemed the best option for controlling Excel.
Marc Jones
@marctjones
Denton Gentry
@DentonGentry

The excel integration test is in https://gitlab.com/DentonGentry/drawdown.

It fails continuous integration:
https://gitlab.com/DentonGentry/drawdown/-/jobs/95821051
because I don't have the rights to have the Heroku API key, so staging deployment fails and fails the commit. We'll figure this out later, and get it merged into the code.earth drawdown repository.

Marc Jones
@marctjones_gitlab
\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\