by

Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Francesco Lombardi
    @FLomb
    @brynpickering don't worry and have a nice holiday, I am next to his office and I am handling ordinary doubts already haha :)
    Bryn Pickering
    @brynpickering
    :+1:
    Nicolò Stevanato
    @Stevogallo
    Thanks a lot for thr hint @brynpickering ! I will follow that right away. Enjoy your holiday! I will try to limit as much as I can my questions on ordinary doubts :)
    Tom Harris
    @tomdeallycat
    Hello everyone, may I introduce myself, Tom Harris, and my colleague, Rob Spencer. We are users of Calliope and developers of UI+backend "wrapper" for Calliope. For the first time today we are using (in Calliope 0.5.4) 'e_cap.total_max' which generates an error apparently from Pyomo/Numpy:
      pv:
        carrier_out: power
        constraints:
          e_cap: {total_max: 2768}
        parent: renewables
      renewables: {group: true, parent: supply_plus}

    gives

    ERROR: Rule failed when generating expression for constraint c_e_cap_total_systemwide with index pv:
            KeyError: "Error accessing indexed component: Index '('pv', 'CapEx_Oahu')' is not valid for array component 'e_cap'"
    ERROR: Constructing component 'c_e_cap_total_systemwide' from data=None failed:
            KeyError: "Error accessing indexed component: Index '('pv', 'CapEx_Oahu')' is not valid for array component 'e_cap'"

    Has anyone seen this issue? Wonder if we are specifying e_cap.total_max incocorrectly?
    THANKS!

    awelsz
    @awelsz
    Hi All! Let me introduce myself, I'm Agnes, currently I'm writing my master's thesis about interconnected and islanded microgrids (islands, small scale national grids), how to model these systems in order to capture the value of BESS. And in the next months I'd like to contribute to your work.
    I have a question, if you've considered using the NEOS server yet: https://neos-server.org/neos/
    This could facilitate to use many different solvers without the need of getting a licence for them 1 by 1.
    Thanks for your comments!
    Bryn Pickering
    @brynpickering

    Thanks for pointing us to this Agnes! I like the idea of a web platform for submitting and running jobs, without worrying about sorting everything out on your own device. I hadn't come across NEOS before. But I'm not so sure about what I think of the terms and conditions of using NEOS. Particularly the 'license to your submission':

    License to Your Submission. You hereby grant to MIR a royalty-free, non-exclusive, perpetual license to reproduce, prepare derivative works, distribute, and otherwise use the Submission in order to perform the Services. In addition, You hereby grant to MIR a right to store, archive and distribute the Submission, as well as use the Submission for non-commercial academic research.
    Subject to Your rights in the underlying copyright to the Submissions, You hereby waive any right, title or interest in and to any results, improvements, research results, inventions, discoveries, or other materials (whether patentable or copyrightable) that are developed, created, or discovered from or as a result of any licensed use of the Submission including any rights in academic publication and improvement to optimization algorithms.

    Although just an LP file, so a lot of the contextual information about a submission isn't present, I feel a bit uncomfortable with the idea that the data of all my jobs is being processed by the NEOS team for their research purposes without it being transparent as to how they will use it.

    You should also check whether your institute has a high performance computing cluster you could access to run your jobs. It works in the same way as NEOS, really, with at least one commercial solver usually available.

    Whether or not you choose to use NEOS, we'd love to hear how you end up using calliope and to be pointed towards anything that you make publicly available (e.g. a GitHub repository of your model)! And, of course, we welcome any contributions to the code and/or documentation that you might make along the way :)

    @tomdeallycat sorry for the radio silence on your question, I've been away from gitter for a while. I'm afraid I haven't dealt with 0.5.x in several years, so I'm not sure I can offer much assistance on how to fix your problem. My advice would be to work solely with 0.6.x ;)
    awelsz
    @awelsz
    @brynpickering Thanks for the feedback! Yes, I plan to make some changes for my work, I have my repository all set up, and hopefully you'll find my contributions useful! :)
    ringjarvi
    @ringjarvi

    Hello all, Ross Ring-Jarvi here. I have been working with Calliope 0.6.3 and I have two questions.
    Question 1: I'm attempting to model a mix fuel plant that burns bio-diesel or diesel. There is a mandate that the more expensive bio-diesel must be burned 30% of the time. I currently am attempting to model this with 2 supply techs:

    supply_diesel:
            essentials:
                carrier: fossil_fuel
    supply_biodiesel:
            essentials:
                carrier: rew_fuel

    that connects to a conversion plus tech:

     bi_fuel_conversion:
            essentials:
                parent: conversion_plus
                carrier_out: power
                carrier_in: [fossil_fuel, rew_fuel]
                primary_carrier_in: rew_fuel
                carrier_ratios:
                    carrier_in: {rew_fuel: .75, fossil_fuel: 1}

    When optimizing costs but still forcing the more expensive fuel to be used at the 30% rate is the problem?

    Question 2: I would like to be able to shift between different levels of detail in locations, from a copper plate model (modeling the whole system as one location) to fully distributed system of locations with techs and demands. Is there a hierarchical system of locations, parent and child or such that could help with this?
    Thank you!

    Graeme Hawker
    @GraemeHawker
    Hi all - I've been somewhat quiet for the past 6 months as due to funding etc I've been working on non-energy modelling things (sadly). However, I just wanted to chime in that I'm just starting a 6 month placement at the Scottish Government, where I will be using calliope to model the future of gas networks in Scotland in the context of getting to net zero across all sectors by 2045. So this means finally moving forward from calliope 0.5.4! One of my intentions (as I've previously stated) is to look at getting some physical network constraints into the model (such as compressor/linepack representations for hydrogen transmission) so would be interested in talking to anyone else who has tried to do this. Anway, good to be back in the fold, and hopefully I can make some contributions back to the codebase again.
    Bryn Pickering
    @brynpickering

    @ringjarvi this is something we hope to be able to cover with our (mostly work in progress) group constraints. At the moment you can set constraints on how much of a specific carrier is produced by a specific technology (e.g. saying that 20% of 'power' must be produced by the 'bi_fuel_conversion' tech), but we haven't produced that for carrier consumption. You could make a workaround by having the two techs 'supply_diesel' and 'supply_biodiesel' produce the same fuel type (say 'fuel') but with different efficiencies. Then 'bi_fuel_conversion' can be a simple conversion technology, taking in 'fuel' and producing 'power'. The group constraint would then work, to force a minimum production of 30% of 'fuel' coming from 'supply_biodiesel' (see the documentation for more info).

    RE the different levels of locations. Sadly, we removed hierarchical locations in 0.6.0, since it was a pain to have in the codebase relative to how much we felt people were using it. At the moment, your best bet would be to have different scenarios for different numbers of locations, and use the comma notation in defining locations to simplify the definition of higher resolution locations which share the same technology sets. See e.g. the use of this in the UK-calliope model

    @GraemeHawker Glad to hear it! We're not actively working on physical network constraints, but I remember chatting with @FLomb at some point, so they might be hoping to improve matters in that domain too.
    pmmeyourmodel
    @pmmeyourmodel

    I have a question regarding storage technologies:
    Is it possible to force the model to not fall below a certain filling level of the storage?
    In the case of a battery it would make sense that the stored energy is always above ~10% of the battery capacity (additionally maybe as well <80% to increase lifetime). I am asking because I am deadling with hydrogen intermediate storage for my next calliope model. Usually you have a cushion gas which is defined as the volume of the tank you're not allowed to empty in order to keep a certain pressure level. This is pretty much the same as with the battery example.

    Thanks and congratulations to 0.6.4. Incredible improvements in my opinion! Already tested some of the new features. What I like the most is the multi-objective optimisation feature and the new cost_max etc. constraints, which really adds a lot of value to my analyses! Thanks for all the good work!

    Francesco Lombardi
    @FLomb
    @pmmeyourmodel We've been actually working exactly on that, there's a pull request pending on the GitHub called "storage dod" (dod stands for "depth of discharge"). All tests passed, so it's a matter of time before @brynpickering or @sjpfenninger approve that and make it part of the master code
    glad to hear that the constraint is useful for somebody else
    @GraemeHawker that's interesting. I'm not sure if the kind of constraint you have in mind pose problems because they would be non-linear or for some other reason, but if that is the case, as @brynpickering said, we made some experiments about how to overcome the need of non-linear constraints without affecting the LP formulation.
    Francesco Lombardi
    @FLomb
    Basically, the idea is to start with a Calliope LP formulation that finds the optimal configuration or dispatch disregarding non-linearities. Subsequently, the found optimal solutions is fed into a non-linear simulation model that tells what happens to non-linear "hidden" variables and allows to re-calibrate some of the parameters of the LP model. The two models iterate until "convergence" or sufficient degree of stability.
    We published a conference paper about that; unfortunately the proceedings are still not online, but I might recover the final paper we sent and share it, if you are interested. In the while, you can have a look at the GitHub repo of the integrated model. Our case of application was a closed-loop heat network, whose temperature was influenced by HPs operation in a non-linear way. Here's the GitHub, to start with: https://github.com/SESAM-Polimi/ICCEP-Calliope_iterative
    Francesco Lombardi
    @FLomb
    Feel free to contact me privately if you need more info
    b-jesse
    @b-jesse
    I have a question about the transmissions in version 0.6.4. First, I think the new options for asynchronous mode are really great and that I now only need one transmission for both directions between two locations. However, I have the problem that the capacity of the transmissions in both directions is not always the same. Is there any way to model this?
    Stefan Pfenninger
    @sjpfenninger
    @b-jesse currently you'd need to define two different transmission technologies, but we're looking into improving this for a future release
    Guilherme Luz
    @luzgui
    Hello @tomdeallycat, my name is Guilherme Luz from Lisbon University. Me and my colleague Rodrigo are working on calliope to model and optimize a small town energy system. We got interested in the Calliope wrapper you mentioned and we wanted to ask if it is available. Thanks
    Cameron Wade
    @_clameron_twitter
    Hello! The uk-calliope model (https://github.com/sjpfenninger/uk-calliope) doesn't seem to be compatible w/ 0.6.4: A number of warnings appear when instantiating the model, and an error is thrown during the run. If I'm wanting to continue using the uk model, I'm wondering if the easiest fix is to downgrade my calliope version to 0.6.3 and continue from there? Or does there seem to be a bug, since the uk documentation states it is compatible w/ 0.6. Thanks!
    Tom Harris
    @tomdeallycat
    Hello, @luzgui. It is currently available hosted (considered Alpha version) on NREL servers and called Engage. We welcome your registration and use. We haven't yet made the code available on github but intend to do so. Nonetheless, if you'd like to run the Calliope problems locally after using the web application to configure them, they are downloadable. If you'd like to use the hosted version, please reach out to me at tom.harris@nrel.gov, copying brian.hartman@nrel.gov, and we will provide registration and information for you to access it. We appreciate your patience as our Comms team are still working on web presence for informational materials, forum, bug reporting, etc., and we are working on additional materials such as tutorials. Furthermore, the web application is still in very active development with, e.g., an inline help system to be integrated soon, and a new, powerful solver integration currently working its way through NREL cybersecurity approval. All the best!
    Guilherme Luz
    @luzgui
    Thank you very much @tomdeallycat for the info. All the best to you
    Bryn Pickering
    @brynpickering
    @_clameron_twitter we made some potentially breaking changes in 0.6.4 in how we define scenarios, so I expect that it is simply that the UK model was only updated for compatibility with an earlier version of 0.6.x. If it works with 0.6.3, then go with that. If you want the 0.6.4 functionality with an instance of the UK model, then I would check out the changelog to see what changes were made that you could use to update a copy of the UK model. We would obviously welcome any UK model updates as pull requests on the model repository :)
    Cameron Wade
    @_clameron_twitter
    @brynpickering Thanks, Bryn. I'll likely just roll with the most recent compatible version. I'm playing around with some time resolution methods so I'm not overly concerned at this point with the added features of 0.6.4...
    Cameron Wade
    @_clameron_twitter

    @brynpickering Hi Bryn. I've now got things up and running w/ 0.6.3. I've implemented (separate from Calliope) the optimization method introduced by Poncelet (2017) to select representative days and their associated weights. I'd like to test these days out in the uk-calliope much like Stefan did in his similar paper. I've followed the directions outlined in the docs to load user-defined representative days. Some questions:

    1) It's not clear to me how to set the timestep_weight values as an input parameter. My input file would ideally have columns of i) Timestep; Cluster Number; Weight.
    2) It seems when I use the apply_clustering function (pointing to the file containing the representative days), all of the raw data is first read in during Calliope's first preprocessing step. This requires GBs worth of memory when only the data from the pre-selected representative days need be read in. Not the end of the world, just makes troubleshooting tedious.

    Any tips would be helpful. Again, all I'm trying to do is take the output from the separate optimization model (i.e. clusters and their associated weights) and use that as an input to the Calliope-UK model. Thanks!

    Bryn Pickering
    @brynpickering
    @_clameron_twitter
    1) You can give information on the timestep and cluster number, but it will automatically take the timestep_weight as being equals to the number of timesteps associated with that weight. If you have specific timestep weights that override this, then you'll need to update the resulting data in model._model_data. Keep in mind that if timestep weights are updated and the sum of weights doesn't equal to the sum of all original timesteps, calculations further down the line will be erroneous.
    2) Yep, that's because it does time clustering after loading in the data. Since there are different ways of selecting the representative days, it needs access to all timesteps to make that decision (and the inbuilt clustering methods work with all timesteps). It also doesn't really work with pandas to only load in 'representative days' into memory when it loads in a CSV; without a lot of work, it's going to have to load in all data, then select a subset of it. So you'll always have a reasonable amount of memory consumption with a model like uk-calliope.
    If you're doing all the clustering independently of the in-built calliope functions, the best might be to create CSVs with only the representative days. You'll need to give them a monotonically increasing timeseries (i.e. if it's 5 RDs, the timeseries is the first 5 days of a random year) and then assign the timestep weights as your custom weights after the model has been loaded in. That will reduce memory consumption too. Would require some playing around to make it all work properly, but might be your best bet.
    Cameron Wade
    @_clameron_twitter
    @brynpickering Thanks so much. I'll have a go a it.
    b-jesse
    @b-jesse
    @sjpfenninger Thank you for your input regarding the transmissions. Unfortuneally I ran into a problem with the new constraints. I'm using the supply_share_equals constraint and get following error message:
    ERROR: Rule failed when generating expression for constraint group_supply_share_equals_constraint with index ('Wind_HU', 'NG', 'equals'): KeyError: ('NG', 'Wind_HU')
    ERROR: Constructing component 'group_supply_share_equals_constraint' from data=None failed: KeyError: ('NG', 'Wind_HU')
    The constraint I use looks like this:
    Wind_HU:
    techs: ['ConELC-PP_WON']
    locs: ['HU']
    supply_share_equals:
    electricity: 0.02
    It seems to me, that the problem lies with my other carriers (i.e. NG). Is there a good way to fix this?
    Nicolò Stevanato
    @Stevogallo
    Hello everybody, I am still working on the Kenya model that I talked to you about some months ago and I am finalizing it. I just have one small problem with transmission lines definitions that I hope you might help me to solve.
    I have 4 regions in my model and two different kinds of transmission lines connecting them, namely 132kV and 220kV, now in the 6.4 version I defined for every couple of regions two different techs per each voltage, one for one direction and one for the other, since they have different capacities in the two directions (an example here)
    WSTR,MTKR:
        techs:
            hvdc_132:
                constraints:
                    energy_cap_equals: 133183
                    one_way: true                   
                distance: 0.52                   # in hundreds of km                 
    
    MTKR,WSTR:
        techs:
            hvdc_132_back:
                constraints:
                    energy_cap_equals: 127868
                    one_way: true                    
                distance: 1.13                # in hundreds of km    

    But i get the error:

    "Index '('MTKR::hvdc_220_new:NBOR::power', Timestamp('2035-06-01 00:00:00'))' is not valid for indexed component 'carrier_prod'"

    and I don't get it without "one_way: true", but I get one bidirectional line connecting the two regions with the second values inserted,

    has anybody faced this problem before? Any hint will be super appreciated thanks everybody :)

    Nicolò Stevanato
    @Stevogallo
    (I get the same error with version 6.3)
    sheridanfew
    @sheridanfew

    Hi there,

    I'm trying to set up a new model in Calliope to understand when it makes sense to connect two microgrids in a remote context. So far I'm just setting up a simple toy model with two nodes before starting to put in some more realistic data, and have been running into some errors.

    I was wondering if anyone might have seen similar and be able to offer any advice?

    $ ./run_single.sh 
    Attempting run...
    Traceback (most recent call last):
      File "/home/spf310/anaconda3/envs/calliope/bin/calliope", line 5, in <module>
        from calliope.cli import cli
      File "/home/spf310/anaconda3/envs/calliope/lib/python3.6/site-packages/calliope/__init__.py", line 2, in <module>
        from calliope.core import AttrDict, Model, read_netcdf, _logger, set_log_level
      File "/home/spf310/anaconda3/envs/calliope/lib/python3.6/site-packages/calliope/core/__init__.py", line 1, in <module>
        from calliope.core.model import Model, read_netcdf
      File "/home/spf310/anaconda3/envs/calliope/lib/python3.6/site-packages/calliope/core/model.py", line 18, in <module>
        from calliope.analysis import plotting, postprocess
      File "/home/spf310/anaconda3/envs/calliope/lib/python3.6/site-packages/calliope/analysis/plotting/__init__.py", line 1, in <module>
        from calliope.analysis.plotting.plotting import plot_summary, ModelPlotMethods
      File "/home/spf310/anaconda3/envs/calliope/lib/python3.6/site-packages/calliope/analysis/plotting/plotting.py", line 21, in <module>
        from calliope.analysis.plotting.transmission import plot_transmission
      File "/home/spf310/anaconda3/envs/calliope/lib/python3.6/site-packages/calliope/analysis/plotting/transmission.py", line 16, in <module>
        from calliope.core.preprocess.util import vincenty
      File "/home/spf310/anaconda3/envs/calliope/lib/python3.6/site-packages/calliope/core/preprocess/__init__.py", line 2, in <module>
        from calliope.core.preprocess.model_run import \
      File "/home/spf310/anaconda3/envs/calliope/lib/python3.6/site-packages/calliope/core/preprocess/model_run.py", line 24, in <module>
        from calliope.core.preprocess import locations, sets, checks, constraint_sets, util
      File "/home/spf310/anaconda3/envs/calliope/lib/python3.6/site-packages/calliope/core/preprocess/locations.py", line 17, in <module>
        from calliope.core.preprocess.checks import DEFAULTS, POSSIBLE_COSTS
      File "/home/spf310/anaconda3/envs/calliope/lib/python3.6/site-packages/calliope/core/preprocess/checks.py", line 31, in <module>
        DEFAULTS = AttrDict.from_yaml(_defaults_files['defaults'])
      File "/home/spf310/anaconda3/envs/calliope/lib/python3.6/site-packages/calliope/core/attrdict.py", line 167, in from_yaml
        loaded = cls(_yaml_load(src))
      File "/home/spf310/anaconda3/envs/calliope/lib/python3.6/site-packages/calliope/core/attrdict.py", line 40, in _yaml_load
        src = src.read()
      File "/home/spf310/anaconda3/envs/calliope/lib/python3.6/encodings/ascii.py", line 26, in decode
        return codecs.ascii_decode(input, self.errors)[0]
    UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 3403: ordinal not in range(128)
    Run finished

    I've uploaded a minimal example of the model I've been running here https://drive.google.com/drive/folders/1EeHV9BNWP2Zvc6TeOOmBLq_p-m9ZFDeO?usp=sharing

    Any advice much appreciated,

    Sheridan

    sheridanfew
    @sheridanfew
    Alternatively if anyone has/can point me to a similar minimal working example of a Calliope model (eg. with just a couple of nodes), I'd be interested to try it out - partly to check if it's my Calliope setup or my model that's at fault.
    Stefan Pfenninger
    @sjpfenninger
    @sheridanfew that sounds like an encoding issue with one or several of your input files
    so not a problem with the model per se
    can you check that all your files are saved as UTF-8 encoded text
    Bryn Pickering
    @brynpickering
    @Stevogallo it's difficult to tell why there would be an issue with your model, but can you check that you never define the same technology in both directions of a link? e.g.
    NBOR,MTKR:
        techs:
            hvdc_220_new:
                constraints:
                    energy_cap_equals: 133183
                    one_way: true                        
    
    MTKR,NBOR:
        techs:
            hvdc_220_new:
                constraints:
                    energy_cap_equals: 127868
                    one_way: true
    Guilherme Luz
    @luzgui
    Hello to you all. I would like to ask if anyone had the following problem and if there is any workaround: During the calliope run and after the solving rpocess it remains an infinite time saving plots in HTML. Specifically it stays in the "Saving HTML file with plots to: file.html" process.
    The question is that my model has almost 200 nodes so possibly creating that file is a hard task. Does anyone have experienced this?
    sheridanfew
    @sheridanfew

    @sheridanfew that sounds like an encoding issue with one or several of your input files

    Thanks Stefan, I deleted a couple of £ signs (from commented out sections) and seems to be running now.

    Bryn Pickering
    @brynpickering
    @luzgui Creating HTMLs of large problems will take a very long time, it's just the price paid for interactive plots. I would suggest not producing the HTML files automatically (i.e., no use of --save-plots), then either use the Calliope plotting API or your own custom plotting functions in python interactively
    Alessandro-Barbieri95
    @Alessandro-Barbieri95
    image.png
    Goodmorning, I'm Alessandro, a master thesis student at Politecnico di Milano.
    I'm working on a water-energy-nexus model, integrating hydrological balance in hydroelectric technology implementation.
    My focus is the Zambezi River basin, in which there are 4 cascade reservoirs. It means that the upstream turbined flows go into the downstream reservoirs.
    I modeled the system describing the reservoir as storage tech, the inflow as supply tech and a conversion plus tech to model the electricity produced by the hydropower plant and
    to track the turbined flow as a cascade inflow to the downstream reservoir (scheme attached).
    The unit of measure of the storage has to be considered in m^3 of water that are converted in kWh through an energy efficiency in the conversion plus tech.
    A more precise description wants the hydraulic head varying with the storage water level, it means that the energy efficiency should be a function of the storage variable.
    Is it feasible to implement this dependance?
    Thanks everybody for your time
    Bryn Pickering
    @brynpickering
    @Alessandro-Barbieri95 nice - certainly the kind of description hydropower systems should have! As to the question of hydraulic head, there is no way I can think of to link electricity generation efficiency with quantity of stored energy in a linear model (it would require multiplication of decision variables). But I also wouldn't expect the difference in hydraulic head to have such a large impact on output efficiency. Instead, it may noticeably impact the maximum power output (carrier_prod(t) <= storage(t) * some_constant). I can imagine that dependence being implementable.
    Francesco Lombardi
    @FLomb
    @Alessandro-Barbieri95 I confirm what @brynpickering was suggesting, you cannot link the energy storage level with an efficiency without going non-linear. I have similar problems modelling thermal storage and power-to-heat conversion techs. There are ways you can escape that, and we did it for non-linear thermal modelling, but probably not ones that would make you happy and your problem easy to solve. And I also agree that the efficiency shouldn't be particularly pivotal here compared to other things, such as the maximum power output.
    Rodrigo Amaro e Silva
    @ramaroesilva

    Hi everyone. I'm currently learning to use Calliope to model renewable integration in some key areas in Portugal.

    My question for the day is: after loading a "model" variable using calliope.read_netcdf(), where in that variable can I find the parent for each tech that was described in the .yaml?