I have a question, if you've considered using the NEOS server yet: https://neos-server.org/neos/
This could facilitate to use many different solvers without the need of getting a licence for them 1 by 1.
Thanks for your comments!
Thanks for pointing us to this Agnes! I like the idea of a web platform for submitting and running jobs, without worrying about sorting everything out on your own device. I hadn't come across NEOS before. But I'm not so sure about what I think of the terms and conditions of using NEOS. Particularly the 'license to your submission':
License to Your Submission. You hereby grant to MIR a royalty-free, non-exclusive, perpetual license to reproduce, prepare derivative works, distribute, and otherwise use the Submission in order to perform the Services. In addition, You hereby grant to MIR a right to store, archive and distribute the Submission, as well as use the Submission for non-commercial academic research.
Subject to Your rights in the underlying copyright to the Submissions, You hereby waive any right, title or interest in and to any results, improvements, research results, inventions, discoveries, or other materials (whether patentable or copyrightable) that are developed, created, or discovered from or as a result of any licensed use of the Submission including any rights in academic publication and improvement to optimization algorithms.
Although just an LP file, so a lot of the contextual information about a submission isn't present, I feel a bit uncomfortable with the idea that the data of all my jobs is being processed by the NEOS team for their research purposes without it being transparent as to how they will use it.
You should also check whether your institute has a high performance computing cluster you could access to run your jobs. It works in the same way as NEOS, really, with at least one commercial solver usually available.
Whether or not you choose to use NEOS, we'd love to hear how you end up using calliope and to be pointed towards anything that you make publicly available (e.g. a GitHub repository of your model)! And, of course, we welcome any contributions to the code and/or documentation that you might make along the way :)
Hello all, Ross Ring-Jarvi here. I have been working with Calliope 0.6.3 and I have two questions.
Question 1: I'm attempting to model a mix fuel plant that burns bio-diesel or diesel. There is a mandate that the more expensive bio-diesel must be burned 30% of the time. I currently am attempting to model this with 2 supply techs:
supply_diesel:
essentials:
carrier: fossil_fuelsupply_biodiesel:
essentials:
carrier: rew_fuelthat connects to a conversion plus tech:
bi_fuel_conversion:
essentials:
parent: conversion_plus
carrier_out: power
carrier_in: [fossil_fuel, rew_fuel]
primary_carrier_in: rew_fuel
carrier_ratios:
carrier_in: {rew_fuel: .75, fossil_fuel: 1}When optimizing costs but still forcing the more expensive fuel to be used at the 30% rate is the problem?
Question 2: I would like to be able to shift between different levels of detail in locations, from a copper plate model (modeling the whole system as one location) to fully distributed system of locations with techs and demands. Is there a hierarchical system of locations, parent and child or such that could help with this?
Thank you!
@ringjarvi this is something we hope to be able to cover with our (mostly work in progress) group constraints. At the moment you can set constraints on how much of a specific carrier is produced by a specific technology (e.g. saying that 20% of 'power' must be produced by the 'bi_fuel_conversion' tech), but we haven't produced that for carrier consumption. You could make a workaround by having the two techs 'supply_diesel' and 'supply_biodiesel' produce the same fuel type (say 'fuel') but with different efficiencies. Then 'bi_fuel_conversion' can be a simple conversion technology, taking in 'fuel' and producing 'power'. The group constraint would then work, to force a minimum production of 30% of 'fuel' coming from 'supply_biodiesel' (see the documentation for more info).
RE the different levels of locations. Sadly, we removed hierarchical locations in 0.6.0, since it was a pain to have in the codebase relative to how much we felt people were using it. At the moment, your best bet would be to have different scenarios for different numbers of locations, and use the comma notation in defining locations to simplify the definition of higher resolution locations which share the same technology sets. See e.g. the use of this in the UK-calliope model
I have a question regarding storage technologies:
Is it possible to force the model to not fall below a certain filling level of the storage?
In the case of a battery it would make sense that the stored energy is always above ~10% of the battery capacity (additionally maybe as well <80% to increase lifetime). I am asking because I am deadling with hydrogen intermediate storage for my next calliope model. Usually you have a cushion gas which is defined as the volume of the tank you're not allowed to empty in order to keep a certain pressure level. This is pretty much the same as with the battery example.
Thanks and congratulations to 0.6.4. Incredible improvements in my opinion! Already tested some of the new features. What I like the most is the multi-objective optimisation feature and the new cost_max etc. constraints, which really adds a lot of value to my analyses! Thanks for all the good work!
@brynpickering Hi Bryn. I've now got things up and running w/ 0.6.3. I've implemented (separate from Calliope) the optimization method introduced by Poncelet (2017) to select representative days and their associated weights. I'd like to test these days out in the uk-calliope much like Stefan did in his similar paper. I've followed the directions outlined in the docs to load user-defined representative days. Some questions:
1) It's not clear to me how to set the timestep_weight values as an input parameter. My input file would ideally have columns of i) Timestep; Cluster Number; Weight.
2) It seems when I use the apply_clustering function (pointing to the file containing the representative days), all of the raw data is first read in during Calliope's first preprocessing step. This requires GBs worth of memory when only the data from the pre-selected representative days need be read in. Not the end of the world, just makes troubleshooting tedious.
Any tips would be helpful. Again, all I'm trying to do is take the output from the separate optimization model (i.e. clusters and their associated weights) and use that as an input to the Calliope-UK model. Thanks!
1) You can give information on the timestep and cluster number, but it will automatically take the timestep_weight as being equals to the number of timesteps associated with that weight. If you have specific timestep weights that override this, then you'll need to update the resulting data in
model._model_data. Keep in mind that if timestep weights are updated and the sum of weights doesn't equal to the sum of all original timesteps, calculations further down the line will be erroneous.2) Yep, that's because it does time clustering after loading in the data. Since there are different ways of selecting the representative days, it needs access to all timesteps to make that decision (and the inbuilt clustering methods work with all timesteps). It also doesn't really work with pandas to only load in 'representative days' into memory when it loads in a CSV; without a lot of work, it's going to have to load in all data, then select a subset of it. So you'll always have a reasonable amount of memory consumption with a model like uk-calliope.
If you're doing all the clustering independently of the in-built calliope functions, the best might be to create CSVs with only the representative days. You'll need to give them a monotonically increasing timeseries (i.e. if it's 5 RDs, the timeseries is the first 5 days of a random year) and then assign the timestep weights as your custom weights after the model has been loaded in. That will reduce memory consumption too. Would require some playing around to make it all work properly, but might be your best bet.
ERROR: Rule failed when generating expression for constraint group_supply_share_equals_constraint with index ('Wind_HU', 'NG', 'equals'): KeyError: ('NG', 'Wind_HU')
ERROR: Constructing component 'group_supply_share_equals_constraint' from data=None failed: KeyError: ('NG', 'Wind_HU')
The constraint I use looks like this:
Wind_HU:
techs: ['ConELC-PP_WON']
locs: ['HU']
supply_share_equals:
electricity: 0.02
It seems to me, that the problem lies with my other carriers (i.e. NG). Is there a good way to fix this?
I have 4 regions in my model and two different kinds of transmission lines connecting them, namely 132kV and 220kV, now in the 6.4 version I defined for every couple of regions two different techs per each voltage, one for one direction and one for the other, since they have different capacities in the two directions (an example here)
WSTR,MTKR:
techs:
hvdc_132:
constraints:
energy_cap_equals: 133183
one_way: true
distance: 0.52 # in hundreds of km
MTKR,WSTR:
techs:
hvdc_132_back:
constraints:
energy_cap_equals: 127868
one_way: true
distance: 1.13 # in hundreds of km But i get the error:
"Index '('MTKR::hvdc_220_new:NBOR::power', Timestamp('2035-06-01 00:00:00'))' is not valid for indexed component 'carrier_prod'"
and I don't get it without "one_way: true", but I get one bidirectional line connecting the two regions with the second values inserted,
has anybody faced this problem before? Any hint will be super appreciated thanks everybody :)
Hi there,
I'm trying to set up a new model in Calliope to understand when it makes sense to connect two microgrids in a remote context. So far I'm just setting up a simple toy model with two nodes before starting to put in some more realistic data, and have been running into some errors.
I was wondering if anyone might have seen similar and be able to offer any advice?
$ ./run_single.sh
Attempting run...
Traceback (most recent call last):
File "/home/spf310/anaconda3/envs/calliope/bin/calliope", line 5, in <module>
from calliope.cli import cli
File "/home/spf310/anaconda3/envs/calliope/lib/python3.6/site-packages/calliope/__init__.py", line 2, in <module>
from calliope.core import AttrDict, Model, read_netcdf, _logger, set_log_level
File "/home/spf310/anaconda3/envs/calliope/lib/python3.6/site-packages/calliope/core/__init__.py", line 1, in <module>
from calliope.core.model import Model, read_netcdf
File "/home/spf310/anaconda3/envs/calliope/lib/python3.6/site-packages/calliope/core/model.py", line 18, in <module>
from calliope.analysis import plotting, postprocess
File "/home/spf310/anaconda3/envs/calliope/lib/python3.6/site-packages/calliope/analysis/plotting/__init__.py", line 1, in <module>
from calliope.analysis.plotting.plotting import plot_summary, ModelPlotMethods
File "/home/spf310/anaconda3/envs/calliope/lib/python3.6/site-packages/calliope/analysis/plotting/plotting.py", line 21, in <module>
from calliope.analysis.plotting.transmission import plot_transmission
File "/home/spf310/anaconda3/envs/calliope/lib/python3.6/site-packages/calliope/analysis/plotting/transmission.py", line 16, in <module>
from calliope.core.preprocess.util import vincenty
File "/home/spf310/anaconda3/envs/calliope/lib/python3.6/site-packages/calliope/core/preprocess/__init__.py", line 2, in <module>
from calliope.core.preprocess.model_run import \
File "/home/spf310/anaconda3/envs/calliope/lib/python3.6/site-packages/calliope/core/preprocess/model_run.py", line 24, in <module>
from calliope.core.preprocess import locations, sets, checks, constraint_sets, util
File "/home/spf310/anaconda3/envs/calliope/lib/python3.6/site-packages/calliope/core/preprocess/locations.py", line 17, in <module>
from calliope.core.preprocess.checks import DEFAULTS, POSSIBLE_COSTS
File "/home/spf310/anaconda3/envs/calliope/lib/python3.6/site-packages/calliope/core/preprocess/checks.py", line 31, in <module>
DEFAULTS = AttrDict.from_yaml(_defaults_files['defaults'])
File "/home/spf310/anaconda3/envs/calliope/lib/python3.6/site-packages/calliope/core/attrdict.py", line 167, in from_yaml
loaded = cls(_yaml_load(src))
File "/home/spf310/anaconda3/envs/calliope/lib/python3.6/site-packages/calliope/core/attrdict.py", line 40, in _yaml_load
src = src.read()
File "/home/spf310/anaconda3/envs/calliope/lib/python3.6/encodings/ascii.py", line 26, in decode
return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 3403: ordinal not in range(128)
Run finishedI've uploaded a minimal example of the model I've been running here https://drive.google.com/drive/folders/1EeHV9BNWP2Zvc6TeOOmBLq_p-m9ZFDeO?usp=sharing
Any advice much appreciated,
Sheridan
NBOR,MTKR:
techs:
hvdc_220_new:
constraints:
energy_cap_equals: 133183
one_way: true
MTKR,NBOR:
techs:
hvdc_220_new:
constraints:
energy_cap_equals: 127868
one_way: true
The question is that my model has almost 200 nodes so possibly creating that file is a hard task. Does anyone have experienced this?
--save-plots), then either use the Calliope plotting API or your own custom plotting functions in python interactively
I'm working on a water-energy-nexus model, integrating hydrological balance in hydroelectric technology implementation.
My focus is the Zambezi River basin, in which there are 4 cascade reservoirs. It means that the upstream turbined flows go into the downstream reservoirs.
I modeled the system describing the reservoir as storage tech, the inflow as supply tech and a conversion plus tech to model the electricity produced by the hydropower plant and
to track the turbined flow as a cascade inflow to the downstream reservoir (scheme attached).
The unit of measure of the storage has to be considered in m^3 of water that are converted in kWh through an energy efficiency in the conversion plus tech.
A more precise description wants the hydraulic head varying with the storage water level, it means that the energy efficiency should be a function of the storage variable.
Is it feasible to implement this dependance?
Thanks everybody for your time
carrier_prod(t) <= storage(t) * some_constant). I can imagine that dependence being implementable.
Hi everyone. I'm currently learning to use Calliope to model renewable integration in some key areas in Portugal.
My question for the day is: after loading a "model" variable using calliope.read_netcdf(), where in that variable can I find the parent for each tech that was described in the .yaml?