Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Francesco Lombardi
    @FLomb
    Hi guys, urgent issue: sometimes ago we were discussing about having variable COPs for heat pumps in a Calliope model, and @brynpickering said this is currently already possible by simply assigning a csv file with a time-step-dependent COP. Now, has anybody actually tried that? Because I get a quite explicit error saying that "ValueError: can only convert an array of size 1 to a Python scalar". Which means, I think, that the code is conceived to take only a scalar for the "efficiency" constraint of a conversion_plus.
    some time ago*
    Francesco Lombardi
    @FLomb
    even though, looking at Calliope's code it seems indeed conceived for taking time-step-dependent efficiencies: (this is from the conversion_plus constraints code) energy_eff = get_param(backend_model, 'energy_eff', (loc_tech, timestep))
    and also the documentation reports that this should be possible without problems
    Francesco Lombardi
    @FLomb
    could there be a bug? Should I open an issue?
    Bryn Pickering
    @brynpickering
    It doesn't look like it's a problem with it accepting timeseries per se
    I would open an issue with the full error log, then we can dig into it some more. Can you confirm that this also happens when assigning a timeseries to energy_eff for the urban scale model chp?
    Francesco Lombardi
    @FLomb
    uhm, actually it doesn't. I tried to put a timeseries for that and it works without error. The output seems reasonable (I put a timeseries with efficiency of 0.2 for every time step and the result is that it doesn't choose to use the chp at all anymore)
    but let me try to put something greater than 1 as for HPs
    nope, it works pretty well
    so what could make it unhappy with my case?
    Francesco Lombardi
    @FLomb
    @brynpickering updates about this: I was wondering, what's different between my model and the urban_scale one? the run mode. I always use "operate", and many times when I have problems they disappear switching to "plan". The same happens for this issue with the time series. Is this helpful?
    Bryn Pickering
    @brynpickering
    Hi @FLomb, the issue is in checks we make before things go to pyomo, which you should be able to see in the error traceback. Not sure when a solution to this will be pushed to master, but for your local copy, you can just update line 47 of calliope/backend/checks.py to if _is_in(loc_tech, var) and not any((pd.isnull((model_data[var].loc[loc_tech].values, )),)):
    Francesco Lombardi
    @FLomb
    Hi guys, I know I still have to update about previous issues, but we're hugely overloaded in these days. There's a funny thing happening with a calliope_3.6 installation: we installed it on 2 different laptops and it just works fine, providing the same output for the same script and model formulation, just great. Now, when installing exactly the same thing on our workstation (same OS, more or less, same Conda version etc.), the same sript/model.yaml work (i.e. no errors raised by Calliope, model solved) but there's a weird bug. The bug is that results (if displayed with the calliope.get_formatted_array func) are very different from those obtained if saving .to_csv. In particular, the latter are just ok and identical to those obtained on the other two laptops. The ones displayed with the calliope API fail to account for the fact that the solar resource is multiplied by an energy_cap and area, so it's like totally negligible (0.X instead of XXX kW). As a consequence, we also fail to have storage (no VRES to be stored), so it's not just a matter of plotting or something like that. Any idea??
    Bryn Pickering
    @brynpickering
    As an input parameter, the resource variable in xarray that you see is not multiplied by anything like resource_area or energy_cap (since those are decision variables which are not known when you load the model inputs). It would also only be multiplied by one of resource_area or energy_cap, depending on whether you have set your resource_unit as energy_per_area or energy_per_cap, respectively. It isn't really possible for results to be different between the xarray dataset and the CSV files, since the latter is saved directly from the former, so you should see that energy_cap values, for instance, are the same in both. If results are significantly different to your other devices, I would check the versions of all your packages (and your solver, particularly if it is GLPK) and that you are definitely using calliope 0.6.3 (the current stable release), and not the current master branch development version (calliope.__version__ should say 0.6.3).
    Francesco Lombardi
    @FLomb
    thanks Bryn, of course I know that I have to pay attention to the resource_unit, but that was not the problem really. As strange as it may seem - I agree and understand it is virtually impossible that csv and get_formatted_array provide different things - this is what was happening. Same model instance, one run, got solved (CPLEX, btw). Then I did model.to_csv and model.get_formatted_array: having different timeseries for the variables that I previously mentioned. And the version of calliope is the same between the working station and at least the second laptopt (not mine, which has a dev version). But: we are realising that the issue is probably with the working station and with its version of Anaconda (the latest one) which is also giving tons of problems with other open-source models that we have and which are based on the same libraries and solvers. It looks like there's a retrocompatibility issue with things developed in Python 3.6, even whereas you specifically tell conda to create environments based on Py 3.6. It is possible that something gets corrupted in the installation. I will try to reinstall a safer conda version on the working station
    Francesco Lombardi
    @FLomb
    Ok, a fresh new installation solved all problems
    now, something instead that is probably more interesting for everybody. When modelling thermal storage in a standard way (i.e. 0.1-0.2% losses per timestep, efficiency = 1, small or no cost of operation), it happens that the model finds optimal to keep the storage in full charge even in long periods of non-use (e.g. summer for space heating). Any hints about how to easily avoid that (apart from putting some hard constraints into the code)? I was guessing putting some penalisation to the storage operation would be enough, but it looks like it's not enough. I think I also tried penalising a little bit the efficiency, but not sure if I dreamed of it or actually did it. Thanks as usual
    Bryn Pickering
    @brynpickering
    How have you modelled your storage over the year, i.e. is it assuming cyclic storage (storage at time t(T) = storage just before time t(0))? It isn't too strange in a multi-energy system for the storage to be used as a way of getting rid of excess energy, by storing it when it is abundant and letting it dissipate due to standing losses while there is less demand. You could try adding a hard constraint for non-summer use, or have the storage technology separated from the system using conversion technologies. If those conversion techs have 0% efficiency in the summer then they won't be able to provide the storage with any energy... Although I would say it is always better practice to try and work out why the storage system operating that way proves to be the cheapest way of running the system ;)
    Francesco Lombardi
    @FLomb
    I kept cyclic storage as it is by default, but I don't think that has a significant influence. It is indeed clear why the model thinks it is more cost-effective to charge it, and the reason is basically that otherwise you would have some excess PV to just waste, so why not charging a little bit the storage? The point is that the storages for cooling and SH, modelled for simplicity as two different techs for two different HPs, in reality are a single object (a single water tank), with different behaviour in summer/winter. As such, it is non-sense to have it charged with heat/cooling at the same time. The idea of the hard constraint is interesting, I might try to put that into the code, but also the trick of the "foo" converion techs is nice, very similar to something I tried for another case
    Nicolò Stevanato
    @Stevogallo

    Hi Guys! Happy to (virtually) meet everybody! I am a colleague of Francesco Lombardi here at Politecnico di Milano and I just started my PhD on the themes of energy modelling, in particular on the impact evaluation of developing countries energy policies. I am making my first steps with calliope as I see it as a great instrument that suits my necessities of research.
    While using it to build a model of Kenyan power system, something really similar to what Dr. Pfenninger proposed for UK, I encountered a problem that I think deserves your attention. The model was originally built by a student I am tutoring for her Master Thesis, in Calliope 0.6.2, the version the student is using, and in that environment it works perfectly, we don't get any error from the IPython console and we are able to extract all the datas we need, but when I tried to run it in Calliope 0.6.3, the version I have on my laptop, the model works but I get this error:

    DuplicateKeyFutureWarning:

    while constructing a mapping
    in "<unicode string>", line 5, column 5:
    WSTR,NBOR:
    ^ (line: 5)
    found duplicate key "WSTR,NBOR" with value "{}" (original value: "{}")
    in "<unicode string>", line 104, column 5:
    WSTR,NBOR:
    ^ (line: 104)

    To suppress this check see:
    http://yaml.readthedocs.io/en/latest/api.html#duplicate-keys

    Duplicate keys will become an error in future releases, and are errors
    by default when using the new API.

    This seems to be related to how the transmission lines are designed, in fact NBOR and WSTR are two different nodes of the system, connected by two different voltages transmission lines, and they are two-way lines. When i try to extract the data about transmission lines with the get_formatted_array, it doesn't tecognize some of the transmission techs, which perfectly recognizes when i use the netcdf from the student, with the model built in 0.6.2 with the same imputs and model structure. Do you think this problem is worth opening an issue in github or should I just change the way the transmission lines are defined in order to avoid "duplicate keys"?

    Thanks everubody for your time, I hope somebody can help :)

    Bryn Pickering
    @brynpickering

    Hello and welcome @Stevogallo! Looking forward to seeing the Kenyan model in due course - it could even make it into our published model library once you're happy with it.

    As to your issue, that is an issue with your YAML files, rather than Calliope (I guess that ruamel.yaml changed their warnings between <0.15 and >0.15, since we use the latter from 0.6.3). How many times are you referring to WSTR,NBOR in the same YAML file / across files? If you have two techs, they should be nested under the same key:

    links:
        WSTR,NBOR:
            techs:
                tech1: ...
                tech2: ...

    If you import two files which refer to the same link, but give different techs, that may cause issues. It is only as overrides/scenarios that you can add to / change values associated with keys in the model.

    Francesco Lombardi
    @FLomb
    @Stevogallo I told you, the support team is amazing ;) haha
    Bryn Pickering
    @brynpickering
    I would warn you that I'm going on holiday in a few days, so will hope others can chip in during my absence ;)
    Francesco Lombardi
    @FLomb
    @brynpickering don't worry and have a nice holiday, I am next to his office and I am handling ordinary doubts already haha :)
    Bryn Pickering
    @brynpickering
    :+1:
    Nicolò Stevanato
    @Stevogallo
    Thanks a lot for thr hint @brynpickering ! I will follow that right away. Enjoy your holiday! I will try to limit as much as I can my questions on ordinary doubts :)
    Tom Harris
    @tomdeallycat
    Hello everyone, may I introduce myself, Tom Harris, and my colleague, Rob Spencer. We are users of Calliope and developers of UI+backend "wrapper" for Calliope. For the first time today we are using (in Calliope 0.5.4) 'e_cap.total_max' which generates an error apparently from Pyomo/Numpy:
      pv:
        carrier_out: power
        constraints:
          e_cap: {total_max: 2768}
        parent: renewables
      renewables: {group: true, parent: supply_plus}

    gives

    ERROR: Rule failed when generating expression for constraint c_e_cap_total_systemwide with index pv:
            KeyError: "Error accessing indexed component: Index '('pv', 'CapEx_Oahu')' is not valid for array component 'e_cap'"
    ERROR: Constructing component 'c_e_cap_total_systemwide' from data=None failed:
            KeyError: "Error accessing indexed component: Index '('pv', 'CapEx_Oahu')' is not valid for array component 'e_cap'"

    Has anyone seen this issue? Wonder if we are specifying e_cap.total_max incocorrectly?
    THANKS!

    awelsz
    @awelsz
    Hi All! Let me introduce myself, I'm Agnes, currently I'm writing my master's thesis about interconnected and islanded microgrids (islands, small scale national grids), how to model these systems in order to capture the value of BESS. And in the next months I'd like to contribute to your work.
    I have a question, if you've considered using the NEOS server yet: https://neos-server.org/neos/
    This could facilitate to use many different solvers without the need of getting a licence for them 1 by 1.
    Thanks for your comments!
    Bryn Pickering
    @brynpickering

    Thanks for pointing us to this Agnes! I like the idea of a web platform for submitting and running jobs, without worrying about sorting everything out on your own device. I hadn't come across NEOS before. But I'm not so sure about what I think of the terms and conditions of using NEOS. Particularly the 'license to your submission':

    License to Your Submission. You hereby grant to MIR a royalty-free, non-exclusive, perpetual license to reproduce, prepare derivative works, distribute, and otherwise use the Submission in order to perform the Services. In addition, You hereby grant to MIR a right to store, archive and distribute the Submission, as well as use the Submission for non-commercial academic research.
    Subject to Your rights in the underlying copyright to the Submissions, You hereby waive any right, title or interest in and to any results, improvements, research results, inventions, discoveries, or other materials (whether patentable or copyrightable) that are developed, created, or discovered from or as a result of any licensed use of the Submission including any rights in academic publication and improvement to optimization algorithms.

    Although just an LP file, so a lot of the contextual information about a submission isn't present, I feel a bit uncomfortable with the idea that the data of all my jobs is being processed by the NEOS team for their research purposes without it being transparent as to how they will use it.

    You should also check whether your institute has a high performance computing cluster you could access to run your jobs. It works in the same way as NEOS, really, with at least one commercial solver usually available.

    Whether or not you choose to use NEOS, we'd love to hear how you end up using calliope and to be pointed towards anything that you make publicly available (e.g. a GitHub repository of your model)! And, of course, we welcome any contributions to the code and/or documentation that you might make along the way :)

    @tomdeallycat sorry for the radio silence on your question, I've been away from gitter for a while. I'm afraid I haven't dealt with 0.5.x in several years, so I'm not sure I can offer much assistance on how to fix your problem. My advice would be to work solely with 0.6.x ;)
    awelsz
    @awelsz
    @brynpickering Thanks for the feedback! Yes, I plan to make some changes for my work, I have my repository all set up, and hopefully you'll find my contributions useful! :)
    ringjarvi
    @ringjarvi

    Hello all, Ross Ring-Jarvi here. I have been working with Calliope 0.6.3 and I have two questions.
    Question 1: I'm attempting to model a mix fuel plant that burns bio-diesel or diesel. There is a mandate that the more expensive bio-diesel must be burned 30% of the time. I currently am attempting to model this with 2 supply techs:

    supply_diesel:
            essentials:
                carrier: fossil_fuel
    supply_biodiesel:
            essentials:
                carrier: rew_fuel

    that connects to a conversion plus tech:

     bi_fuel_conversion:
            essentials:
                parent: conversion_plus
                carrier_out: power
                carrier_in: [fossil_fuel, rew_fuel]
                primary_carrier_in: rew_fuel
                carrier_ratios:
                    carrier_in: {rew_fuel: .75, fossil_fuel: 1}

    When optimizing costs but still forcing the more expensive fuel to be used at the 30% rate is the problem?

    Question 2: I would like to be able to shift between different levels of detail in locations, from a copper plate model (modeling the whole system as one location) to fully distributed system of locations with techs and demands. Is there a hierarchical system of locations, parent and child or such that could help with this?
    Thank you!

    Graeme Hawker
    @GraemeHawker
    Hi all - I've been somewhat quiet for the past 6 months as due to funding etc I've been working on non-energy modelling things (sadly). However, I just wanted to chime in that I'm just starting a 6 month placement at the Scottish Government, where I will be using calliope to model the future of gas networks in Scotland in the context of getting to net zero across all sectors by 2045. So this means finally moving forward from calliope 0.5.4! One of my intentions (as I've previously stated) is to look at getting some physical network constraints into the model (such as compressor/linepack representations for hydrogen transmission) so would be interested in talking to anyone else who has tried to do this. Anway, good to be back in the fold, and hopefully I can make some contributions back to the codebase again.
    Bryn Pickering
    @brynpickering

    @ringjarvi this is something we hope to be able to cover with our (mostly work in progress) group constraints. At the moment you can set constraints on how much of a specific carrier is produced by a specific technology (e.g. saying that 20% of 'power' must be produced by the 'bi_fuel_conversion' tech), but we haven't produced that for carrier consumption. You could make a workaround by having the two techs 'supply_diesel' and 'supply_biodiesel' produce the same fuel type (say 'fuel') but with different efficiencies. Then 'bi_fuel_conversion' can be a simple conversion technology, taking in 'fuel' and producing 'power'. The group constraint would then work, to force a minimum production of 30% of 'fuel' coming from 'supply_biodiesel' (see the documentation for more info).

    RE the different levels of locations. Sadly, we removed hierarchical locations in 0.6.0, since it was a pain to have in the codebase relative to how much we felt people were using it. At the moment, your best bet would be to have different scenarios for different numbers of locations, and use the comma notation in defining locations to simplify the definition of higher resolution locations which share the same technology sets. See e.g. the use of this in the UK-calliope model

    @GraemeHawker Glad to hear it! We're not actively working on physical network constraints, but I remember chatting with @FLomb at some point, so they might be hoping to improve matters in that domain too.
    pmmeyourmodel
    @pmmeyourmodel

    I have a question regarding storage technologies:
    Is it possible to force the model to not fall below a certain filling level of the storage?
    In the case of a battery it would make sense that the stored energy is always above ~10% of the battery capacity (additionally maybe as well <80% to increase lifetime). I am asking because I am deadling with hydrogen intermediate storage for my next calliope model. Usually you have a cushion gas which is defined as the volume of the tank you're not allowed to empty in order to keep a certain pressure level. This is pretty much the same as with the battery example.

    Thanks and congratulations to 0.6.4. Incredible improvements in my opinion! Already tested some of the new features. What I like the most is the multi-objective optimisation feature and the new cost_max etc. constraints, which really adds a lot of value to my analyses! Thanks for all the good work!

    Francesco Lombardi
    @FLomb
    @pmmeyourmodel We've been actually working exactly on that, there's a pull request pending on the GitHub called "storage dod" (dod stands for "depth of discharge"). All tests passed, so it's a matter of time before @brynpickering or @sjpfenninger approve that and make it part of the master code
    glad to hear that the constraint is useful for somebody else
    @GraemeHawker that's interesting. I'm not sure if the kind of constraint you have in mind pose problems because they would be non-linear or for some other reason, but if that is the case, as @brynpickering said, we made some experiments about how to overcome the need of non-linear constraints without affecting the LP formulation.
    Francesco Lombardi
    @FLomb
    Basically, the idea is to start with a Calliope LP formulation that finds the optimal configuration or dispatch disregarding non-linearities. Subsequently, the found optimal solutions is fed into a non-linear simulation model that tells what happens to non-linear "hidden" variables and allows to re-calibrate some of the parameters of the LP model. The two models iterate until "convergence" or sufficient degree of stability.
    We published a conference paper about that; unfortunately the proceedings are still not online, but I might recover the final paper we sent and share it, if you are interested. In the while, you can have a look at the GitHub repo of the integrated model. Our case of application was a closed-loop heat network, whose temperature was influenced by HPs operation in a non-linear way. Here's the GitHub, to start with: https://github.com/SESAM-Polimi/ICCEP-Calliope_iterative
    Francesco Lombardi
    @FLomb
    Feel free to contact me privately if you need more info
    b-jesse
    @b-jesse
    I have a question about the transmissions in version 0.6.4. First, I think the new options for asynchronous mode are really great and that I now only need one transmission for both directions between two locations. However, I have the problem that the capacity of the transmissions in both directions is not always the same. Is there any way to model this?
    Stefan Pfenninger
    @sjpfenninger
    @b-jesse currently you'd need to define two different transmission technologies, but we're looking into improving this for a future release
    Guilherme Luz
    @luzgui
    Hello @tomdeallycat, my name is Guilherme Luz from Lisbon University. Me and my colleague Rodrigo are working on calliope to model and optimize a small town energy system. We got interested in the Calliope wrapper you mentioned and we wanted to ask if it is available. Thanks
    Cameron Wade
    @_clameron_twitter
    Hello! The uk-calliope model (https://github.com/sjpfenninger/uk-calliope) doesn't seem to be compatible w/ 0.6.4: A number of warnings appear when instantiating the model, and an error is thrown during the run. If I'm wanting to continue using the uk model, I'm wondering if the easiest fix is to downgrade my calliope version to 0.6.3 and continue from there? Or does there seem to be a bug, since the uk documentation states it is compatible w/ 0.6. Thanks!
    Tom Harris
    @tomdeallycat
    Hello, @luzgui. It is currently available hosted (considered Alpha version) on NREL servers and called Engage. We welcome your registration and use. We haven't yet made the code available on github but intend to do so. Nonetheless, if you'd like to run the Calliope problems locally after using the web application to configure them, they are downloadable. If you'd like to use the hosted version, please reach out to me at tom.harris@nrel.gov, copying brian.hartman@nrel.gov, and we will provide registration and information for you to access it. We appreciate your patience as our Comms team are still working on web presence for informational materials, forum, bug reporting, etc., and we are working on additional materials such as tutorials. Furthermore, the web application is still in very active development with, e.g., an inline help system to be integrated soon, and a new, powerful solver integration currently working its way through NREL cybersecurity approval. All the best!