calliope/backend/checks.py to if _is_in(loc_tech, var) and not any((pd.isnull((model_data[var].loc[loc_tech].values, )),)):
resource variable in xarray that you see is not multiplied by anything like resource_area or energy_cap (since those are decision variables which are not known when you load the model inputs). It would also only be multiplied by one of resource_area or energy_cap, depending on whether you have set your resource_unit as energy_per_area or energy_per_cap, respectively. It isn't really possible for results to be different between the xarray dataset and the CSV files, since the latter is saved directly from the former, so you should see that energy_cap values, for instance, are the same in both. If results are significantly different to your other devices, I would check the versions of all your packages (and your solver, particularly if it is GLPK) and that you are definitely using calliope 0.6.3 (the current stable release), and not the current master branch development version (calliope.__version__ should say 0.6.3).
Hi Guys! Happy to (virtually) meet everybody! I am a colleague of Francesco Lombardi here at Politecnico di Milano and I just started my PhD on the themes of energy modelling, in particular on the impact evaluation of developing countries energy policies. I am making my first steps with calliope as I see it as a great instrument that suits my necessities of research.
While using it to build a model of Kenyan power system, something really similar to what Dr. Pfenninger proposed for UK, I encountered a problem that I think deserves your attention. The model was originally built by a student I am tutoring for her Master Thesis, in Calliope 0.6.2, the version the student is using, and in that environment it works perfectly, we don't get any error from the IPython console and we are able to extract all the datas we need, but when I tried to run it in Calliope 0.6.3, the version I have on my laptop, the model works but I get this error:
DuplicateKeyFutureWarning:
while constructing a mapping
in "<unicode string>", line 5, column 5:
WSTR,NBOR:
^ (line: 5)
found duplicate key "WSTR,NBOR" with value "{}" (original value: "{}")
in "<unicode string>", line 104, column 5:
WSTR,NBOR:
^ (line: 104)
To suppress this check see:
http://yaml.readthedocs.io/en/latest/api.html#duplicate-keys
Duplicate keys will become an error in future releases, and are errors
by default when using the new API.
This seems to be related to how the transmission lines are designed, in fact NBOR and WSTR are two different nodes of the system, connected by two different voltages transmission lines, and they are two-way lines. When i try to extract the data about transmission lines with the get_formatted_array, it doesn't tecognize some of the transmission techs, which perfectly recognizes when i use the netcdf from the student, with the model built in 0.6.2 with the same imputs and model structure. Do you think this problem is worth opening an issue in github or should I just change the way the transmission lines are defined in order to avoid "duplicate keys"?
Thanks everubody for your time, I hope somebody can help :)
Hello and welcome @Stevogallo! Looking forward to seeing the Kenyan model in due course - it could even make it into our published model library once you're happy with it.
As to your issue, that is an issue with your YAML files, rather than Calliope (I guess that ruamel.yaml changed their warnings between <0.15 and >0.15, since we use the latter from 0.6.3). How many times are you referring to WSTR,NBOR in the same YAML file / across files? If you have two techs, they should be nested under the same key:
links:
WSTR,NBOR:
techs:
tech1: ...
tech2: ...If you import two files which refer to the same link, but give different techs, that may cause issues. It is only as overrides/scenarios that you can add to / change values associated with keys in the model.
pv:
carrier_out: power
constraints:
e_cap: {total_max: 2768}
parent: renewables
renewables: {group: true, parent: supply_plus}gives
ERROR: Rule failed when generating expression for constraint c_e_cap_total_systemwide with index pv:
KeyError: "Error accessing indexed component: Index '('pv', 'CapEx_Oahu')' is not valid for array component 'e_cap'"
ERROR: Constructing component 'c_e_cap_total_systemwide' from data=None failed:
KeyError: "Error accessing indexed component: Index '('pv', 'CapEx_Oahu')' is not valid for array component 'e_cap'"Has anyone seen this issue? Wonder if we are specifying e_cap.total_max incocorrectly?
THANKS!
I have a question, if you've considered using the NEOS server yet: https://neos-server.org/neos/
This could facilitate to use many different solvers without the need of getting a licence for them 1 by 1.
Thanks for your comments!
Thanks for pointing us to this Agnes! I like the idea of a web platform for submitting and running jobs, without worrying about sorting everything out on your own device. I hadn't come across NEOS before. But I'm not so sure about what I think of the terms and conditions of using NEOS. Particularly the 'license to your submission':
License to Your Submission. You hereby grant to MIR a royalty-free, non-exclusive, perpetual license to reproduce, prepare derivative works, distribute, and otherwise use the Submission in order to perform the Services. In addition, You hereby grant to MIR a right to store, archive and distribute the Submission, as well as use the Submission for non-commercial academic research.
Subject to Your rights in the underlying copyright to the Submissions, You hereby waive any right, title or interest in and to any results, improvements, research results, inventions, discoveries, or other materials (whether patentable or copyrightable) that are developed, created, or discovered from or as a result of any licensed use of the Submission including any rights in academic publication and improvement to optimization algorithms.
Although just an LP file, so a lot of the contextual information about a submission isn't present, I feel a bit uncomfortable with the idea that the data of all my jobs is being processed by the NEOS team for their research purposes without it being transparent as to how they will use it.
You should also check whether your institute has a high performance computing cluster you could access to run your jobs. It works in the same way as NEOS, really, with at least one commercial solver usually available.
Whether or not you choose to use NEOS, we'd love to hear how you end up using calliope and to be pointed towards anything that you make publicly available (e.g. a GitHub repository of your model)! And, of course, we welcome any contributions to the code and/or documentation that you might make along the way :)
Hello all, Ross Ring-Jarvi here. I have been working with Calliope 0.6.3 and I have two questions.
Question 1: I'm attempting to model a mix fuel plant that burns bio-diesel or diesel. There is a mandate that the more expensive bio-diesel must be burned 30% of the time. I currently am attempting to model this with 2 supply techs:
supply_diesel:
essentials:
carrier: fossil_fuelsupply_biodiesel:
essentials:
carrier: rew_fuelthat connects to a conversion plus tech:
bi_fuel_conversion:
essentials:
parent: conversion_plus
carrier_out: power
carrier_in: [fossil_fuel, rew_fuel]
primary_carrier_in: rew_fuel
carrier_ratios:
carrier_in: {rew_fuel: .75, fossil_fuel: 1}When optimizing costs but still forcing the more expensive fuel to be used at the 30% rate is the problem?
Question 2: I would like to be able to shift between different levels of detail in locations, from a copper plate model (modeling the whole system as one location) to fully distributed system of locations with techs and demands. Is there a hierarchical system of locations, parent and child or such that could help with this?
Thank you!
@ringjarvi this is something we hope to be able to cover with our (mostly work in progress) group constraints. At the moment you can set constraints on how much of a specific carrier is produced by a specific technology (e.g. saying that 20% of 'power' must be produced by the 'bi_fuel_conversion' tech), but we haven't produced that for carrier consumption. You could make a workaround by having the two techs 'supply_diesel' and 'supply_biodiesel' produce the same fuel type (say 'fuel') but with different efficiencies. Then 'bi_fuel_conversion' can be a simple conversion technology, taking in 'fuel' and producing 'power'. The group constraint would then work, to force a minimum production of 30% of 'fuel' coming from 'supply_biodiesel' (see the documentation for more info).
RE the different levels of locations. Sadly, we removed hierarchical locations in 0.6.0, since it was a pain to have in the codebase relative to how much we felt people were using it. At the moment, your best bet would be to have different scenarios for different numbers of locations, and use the comma notation in defining locations to simplify the definition of higher resolution locations which share the same technology sets. See e.g. the use of this in the UK-calliope model
I have a question regarding storage technologies:
Is it possible to force the model to not fall below a certain filling level of the storage?
In the case of a battery it would make sense that the stored energy is always above ~10% of the battery capacity (additionally maybe as well <80% to increase lifetime). I am asking because I am deadling with hydrogen intermediate storage for my next calliope model. Usually you have a cushion gas which is defined as the volume of the tank you're not allowed to empty in order to keep a certain pressure level. This is pretty much the same as with the battery example.
Thanks and congratulations to 0.6.4. Incredible improvements in my opinion! Already tested some of the new features. What I like the most is the multi-objective optimisation feature and the new cost_max etc. constraints, which really adds a lot of value to my analyses! Thanks for all the good work!
