Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Francesco Lombardi
    @FLomb
    apparently the problem only occurs for some random locations. But all of them are defined in the same way
    Bryn Pickering
    @brynpickering
    It could be that one of CNOR or CSUD hasn't actually been defined in the model, so Calliope thinks that the user has deleted the location (whereas it just hasn't been defined). If those locations are definitely in the model, you could try explicitly setting them to exist (which is the default anyway), e.g. locations.CNOR.exists: true
    Francesco Lombardi
    @FLomb
    Thanks Bryn, I managed to fix it, there was a typo in a location definition. Now it works, but still there is something weird happening: everything works fine if running in "planning" mode, but if trying to run in "operational" mode (as I normally always do) it returns an error saying that I need to specify I finite capacity on each of the defined transmission lines. Which of course I did define, using finite numbers (which just work as expected in planning mode). Any hints? It's possible that there is still something wrong in how I defined such techs (considering that I used to work on a single-node model before), but it is quite weird that I have no problem if running in planning mode.
    a finite capacity*
    Bryn Pickering
    @brynpickering
    Operation mode is quite rigid in its requirements, since it is so easy to define an optimisation problem which is unphysical. All capacity maximum values are set as the decision variable value, instead of acting as a maximum. So any infinite capacity won't work. The easiest thing here is to check model.inputs.energy_cap_max and/or model.inputs.energy_cap_equals to see if there is a rogue infinite or NaN value. NaN values will default to inf (as that is the default for energy_cap_max)
    Francesco Lombardi
    @FLomb
    yes, in fact (in line with the warning and error) the energy_cap of transmission lines (and demands also) is set to "nan". The question is: why? I have defined finite capacity numbers
    Also, making the same test in plan mode, transmission lines are no more set to "nan". They are given the specified capacity limit
    there's just something that makes he remove cap_equals constraints on tramission lines in operation mode
    it*
    Bryn Pickering
    @brynpickering
    demands shouldn't be a problem, since capacity doesn't really make sense for them. For transmission lines, this reminds me of a problem that came up before. Are you setting the transmission capacities at a technology or a link level?
    Francesco Lombardi
    @FLomb
    well, for the "big" transmission lines between bidding zones I'm setting the capacities at link level. Conversely, for the "small" lines that are conceived as free transmission I am setting a capacity constraint at tech level, the same for all of them, and it's like a big number because I originally wanted it to be "inf", but then I put like 10GW to try overcome this error, without success
    I think the latter lines are those giving the problem
    Bryn Pickering
    @brynpickering
    There's no reason for the inputs dataset to have different values for different solving modes, since there is nothing in the code that changes until after the model is sent to the solver. Are you sure that model.inputs gives you two different arrays between loading the model with plan vs operate(i.e. without ever telling the model to run the optimisation)?
    Francesco Lombardi
    @FLomb
    yes, absolutely. Anyway, I just tried to follow your hint and specify capacities at link level; also, I realised I was using energy_cap_max instead of energy_cap_equals for the "big" lineas (even if Calliope should automatically set that to equals if running in operation mode, shouldn't it?). Well, after these two changes, it is happy to solve the model. The only thing that is still problematic is that it does the following: Resource capacity constraint removed from SICI::hydro_dam as force_resource is applied for every single supply_plus tech I have, in every region, even if for most of them I specifically stated "forse_resource: False".
    lines*
    accordingly, results vary between plan and operate because supply_plus are free to supply much more than they should
    this is something I used to have also in single-node
    and that I avoided by applying some further tricks on those techs I wanted to keep under control in terms of resource (i.e. applying also a resource_cap_equals constraint, which I cannot do anymore 'cause I'm using timeseries for these techs now).
    well, I guess I could still hack it in a similar way by applying twice the timeseries to both "resource" and "resource_cap_equals", but it's not super elegant to see
    Francesco Lombardi
    @FLomb
    Ok, I managed to keep it under control
    Francesco Lombardi
    @FLomb
    Hi guys, urgent issue: sometimes ago we were discussing about having variable COPs for heat pumps in a Calliope model, and @brynpickering said this is currently already possible by simply assigning a csv file with a time-step-dependent COP. Now, has anybody actually tried that? Because I get a quite explicit error saying that "ValueError: can only convert an array of size 1 to a Python scalar". Which means, I think, that the code is conceived to take only a scalar for the "efficiency" constraint of a conversion_plus.
    some time ago*
    Francesco Lombardi
    @FLomb
    even though, looking at Calliope's code it seems indeed conceived for taking time-step-dependent efficiencies: (this is from the conversion_plus constraints code) energy_eff = get_param(backend_model, 'energy_eff', (loc_tech, timestep))
    and also the documentation reports that this should be possible without problems
    Francesco Lombardi
    @FLomb
    could there be a bug? Should I open an issue?
    Bryn Pickering
    @brynpickering
    It doesn't look like it's a problem with it accepting timeseries per se
    I would open an issue with the full error log, then we can dig into it some more. Can you confirm that this also happens when assigning a timeseries to energy_eff for the urban scale model chp?
    Francesco Lombardi
    @FLomb
    uhm, actually it doesn't. I tried to put a timeseries for that and it works without error. The output seems reasonable (I put a timeseries with efficiency of 0.2 for every time step and the result is that it doesn't choose to use the chp at all anymore)
    but let me try to put something greater than 1 as for HPs
    nope, it works pretty well
    so what could make it unhappy with my case?
    Francesco Lombardi
    @FLomb
    @brynpickering updates about this: I was wondering, what's different between my model and the urban_scale one? the run mode. I always use "operate", and many times when I have problems they disappear switching to "plan". The same happens for this issue with the time series. Is this helpful?
    Bryn Pickering
    @brynpickering
    Hi @FLomb, the issue is in checks we make before things go to pyomo, which you should be able to see in the error traceback. Not sure when a solution to this will be pushed to master, but for your local copy, you can just update line 47 of calliope/backend/checks.py to if _is_in(loc_tech, var) and not any((pd.isnull((model_data[var].loc[loc_tech].values, )),)):
    Francesco Lombardi
    @FLomb
    Hi guys, I know I still have to update about previous issues, but we're hugely overloaded in these days. There's a funny thing happening with a calliope_3.6 installation: we installed it on 2 different laptops and it just works fine, providing the same output for the same script and model formulation, just great. Now, when installing exactly the same thing on our workstation (same OS, more or less, same Conda version etc.), the same sript/model.yaml work (i.e. no errors raised by Calliope, model solved) but there's a weird bug. The bug is that results (if displayed with the calliope.get_formatted_array func) are very different from those obtained if saving .to_csv. In particular, the latter are just ok and identical to those obtained on the other two laptops. The ones displayed with the calliope API fail to account for the fact that the solar resource is multiplied by an energy_cap and area, so it's like totally negligible (0.X instead of XXX kW). As a consequence, we also fail to have storage (no VRES to be stored), so it's not just a matter of plotting or something like that. Any idea??
    Bryn Pickering
    @brynpickering
    As an input parameter, the resource variable in xarray that you see is not multiplied by anything like resource_area or energy_cap (since those are decision variables which are not known when you load the model inputs). It would also only be multiplied by one of resource_area or energy_cap, depending on whether you have set your resource_unit as energy_per_area or energy_per_cap, respectively. It isn't really possible for results to be different between the xarray dataset and the CSV files, since the latter is saved directly from the former, so you should see that energy_cap values, for instance, are the same in both. If results are significantly different to your other devices, I would check the versions of all your packages (and your solver, particularly if it is GLPK) and that you are definitely using calliope 0.6.3 (the current stable release), and not the current master branch development version (calliope.__version__ should say 0.6.3).
    Francesco Lombardi
    @FLomb
    thanks Bryn, of course I know that I have to pay attention to the resource_unit, but that was not the problem really. As strange as it may seem - I agree and understand it is virtually impossible that csv and get_formatted_array provide different things - this is what was happening. Same model instance, one run, got solved (CPLEX, btw). Then I did model.to_csv and model.get_formatted_array: having different timeseries for the variables that I previously mentioned. And the version of calliope is the same between the working station and at least the second laptopt (not mine, which has a dev version). But: we are realising that the issue is probably with the working station and with its version of Anaconda (the latest one) which is also giving tons of problems with other open-source models that we have and which are based on the same libraries and solvers. It looks like there's a retrocompatibility issue with things developed in Python 3.6, even whereas you specifically tell conda to create environments based on Py 3.6. It is possible that something gets corrupted in the installation. I will try to reinstall a safer conda version on the working station
    Francesco Lombardi
    @FLomb
    Ok, a fresh new installation solved all problems
    now, something instead that is probably more interesting for everybody. When modelling thermal storage in a standard way (i.e. 0.1-0.2% losses per timestep, efficiency = 1, small or no cost of operation), it happens that the model finds optimal to keep the storage in full charge even in long periods of non-use (e.g. summer for space heating). Any hints about how to easily avoid that (apart from putting some hard constraints into the code)? I was guessing putting some penalisation to the storage operation would be enough, but it looks like it's not enough. I think I also tried penalising a little bit the efficiency, but not sure if I dreamed of it or actually did it. Thanks as usual
    Bryn Pickering
    @brynpickering
    How have you modelled your storage over the year, i.e. is it assuming cyclic storage (storage at time t(T) = storage just before time t(0))? It isn't too strange in a multi-energy system for the storage to be used as a way of getting rid of excess energy, by storing it when it is abundant and letting it dissipate due to standing losses while there is less demand. You could try adding a hard constraint for non-summer use, or have the storage technology separated from the system using conversion technologies. If those conversion techs have 0% efficiency in the summer then they won't be able to provide the storage with any energy... Although I would say it is always better practice to try and work out why the storage system operating that way proves to be the cheapest way of running the system ;)
    Francesco Lombardi
    @FLomb
    I kept cyclic storage as it is by default, but I don't think that has a significant influence. It is indeed clear why the model thinks it is more cost-effective to charge it, and the reason is basically that otherwise you would have some excess PV to just waste, so why not charging a little bit the storage? The point is that the storages for cooling and SH, modelled for simplicity as two different techs for two different HPs, in reality are a single object (a single water tank), with different behaviour in summer/winter. As such, it is non-sense to have it charged with heat/cooling at the same time. The idea of the hard constraint is interesting, I might try to put that into the code, but also the trick of the "foo" converion techs is nice, very similar to something I tried for another case
    Nicolò Stevanato
    @Stevogallo

    Hi Guys! Happy to (virtually) meet everybody! I am a colleague of Francesco Lombardi here at Politecnico di Milano and I just started my PhD on the themes of energy modelling, in particular on the impact evaluation of developing countries energy policies. I am making my first steps with calliope as I see it as a great instrument that suits my necessities of research.
    While using it to build a model of Kenyan power system, something really similar to what Dr. Pfenninger proposed for UK, I encountered a problem that I think deserves your attention. The model was originally built by a student I am tutoring for her Master Thesis, in Calliope 0.6.2, the version the student is using, and in that environment it works perfectly, we don't get any error from the IPython console and we are able to extract all the datas we need, but when I tried to run it in Calliope 0.6.3, the version I have on my laptop, the model works but I get this error:

    DuplicateKeyFutureWarning:

    while constructing a mapping
    in "<unicode string>", line 5, column 5:
    WSTR,NBOR:
    ^ (line: 5)
    found duplicate key "WSTR,NBOR" with value "{}" (original value: "{}")
    in "<unicode string>", line 104, column 5:
    WSTR,NBOR:
    ^ (line: 104)

    To suppress this check see:
    http://yaml.readthedocs.io/en/latest/api.html#duplicate-keys

    Duplicate keys will become an error in future releases, and are errors
    by default when using the new API.

    This seems to be related to how the transmission lines are designed, in fact NBOR and WSTR are two different nodes of the system, connected by two different voltages transmission lines, and they are two-way lines. When i try to extract the data about transmission lines with the get_formatted_array, it doesn't tecognize some of the transmission techs, which perfectly recognizes when i use the netcdf from the student, with the model built in 0.6.2 with the same imputs and model structure. Do you think this problem is worth opening an issue in github or should I just change the way the transmission lines are defined in order to avoid "duplicate keys"?

    Thanks everubody for your time, I hope somebody can help :)

    Bryn Pickering
    @brynpickering

    Hello and welcome @Stevogallo! Looking forward to seeing the Kenyan model in due course - it could even make it into our published model library once you're happy with it.

    As to your issue, that is an issue with your YAML files, rather than Calliope (I guess that ruamel.yaml changed their warnings between <0.15 and >0.15, since we use the latter from 0.6.3). How many times are you referring to WSTR,NBOR in the same YAML file / across files? If you have two techs, they should be nested under the same key:

    links:
        WSTR,NBOR:
            techs:
                tech1: ...
                tech2: ...

    If you import two files which refer to the same link, but give different techs, that may cause issues. It is only as overrides/scenarios that you can add to / change values associated with keys in the model.

    Francesco Lombardi
    @FLomb
    @Stevogallo I told you, the support team is amazing ;) haha
    Bryn Pickering
    @brynpickering
    I would warn you that I'm going on holiday in a few days, so will hope others can chip in during my absence ;)
    Francesco Lombardi
    @FLomb
    @brynpickering don't worry and have a nice holiday, I am next to his office and I am handling ordinary doubts already haha :)
    Bryn Pickering
    @brynpickering
    :+1:
    Nicolò Stevanato
    @Stevogallo
    Thanks a lot for thr hint @brynpickering ! I will follow that right away. Enjoy your holiday! I will try to limit as much as I can my questions on ordinary doubts :)
    Tom Harris
    @tomdeallycat
    Hello everyone, may I introduce myself, Tom Harris, and my colleague, Rob Spencer. We are users of Calliope and developers of UI+backend "wrapper" for Calliope. For the first time today we are using (in Calliope 0.5.4) 'e_cap.total_max' which generates an error apparently from Pyomo/Numpy:
      pv:
        carrier_out: power
        constraints:
          e_cap: {total_max: 2768}
        parent: renewables
      renewables: {group: true, parent: supply_plus}

    gives

    ERROR: Rule failed when generating expression for constraint c_e_cap_total_systemwide with index pv:
            KeyError: "Error accessing indexed component: Index '('pv', 'CapEx_Oahu')' is not valid for array component 'e_cap'"
    ERROR: Constructing component 'c_e_cap_total_systemwide' from data=None failed:
            KeyError: "Error accessing indexed component: Index '('pv', 'CapEx_Oahu')' is not valid for array component 'e_cap'"

    Has anyone seen this issue? Wonder if we are specifying e_cap.total_max incocorrectly?
    THANKS!