Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    H. Thomson Comer
    @thomcom
    Doh! Sad to miss it, I ended up adding it to my personal calendar and not my work calendar. :p
    rraymondgh
    @rraymondgh
    seems that CI is broken ci/envs/39-latest-conda-forge.yaml leads to this error requests.exceptions.ConnectionError: HTTPSConnectionPool(host='conda.anaconda.org', port=443): Max retries exceeded with url: /conda-forge/label/shapely_dev/notices.json (Caused by ReadTimeoutError("HTTPSConnectionPool(host='conda.anaconda.org', port=443): Read timed out. (read timeout=5)"))
    Joris Van den Bossche
    @jorisvandenbossche
    Yes, I noticed that as well. Not directly sure what is going wrong, since it worked until recently
    rraymondgh
    @rraymondgh
    I'm not at all familiar with conda or mamba, but my guess is this channel conda-forge/label/shapely_dev is no longer valid
    Joris Van den Bossche
    @jorisvandenbossche
    mamba install shapely -c conda-forge/label/shapely_dev still seems to find the package, though
    Shirzart Enwer
    @Aniwax

    Hey guys, quick question about runtime performance improvement for pygeos:

    is it possible to store pygeos objects, such as Polygons, as memory-mapped files? Given that I have bunch of planes growing throughout simulation, but not every part of polygon is accessed or written at each time step, it would be great if it can be mapped to memory, such as numpy memory mapped arrays...

    Joris Van den Bossche
    @jorisvandenbossche
    No, that's currently not possible. Geometries in memory are represented as GEOS objects (the underlying C++ library), and those objects can't be created from memory mapped data (well it can, but it will do a copy anyway)
    Shirzart Enwer
    @Aniwax

    Ahhhh alright... wil think about other ways..thanks a lot!

    But opening a multi-processing of a GEOS object is possible right? There is a way to pass the Polygon instance between different processes? Do I need to define that explicitly?

    Shirzart Enwer
    @Aniwax
    A stupid question: how do you check which GEOS version is my local pygeos is using?
    Joris Van den Bossche
    @jorisvandenbossche
    pygeos.geos_version
    1 reply

    But opening a multi-processing of a GEOS object is possible right? There is a way to pass the Polygon instance between different processes? Do I need to define that explicitly?

    Between different processes you will have to actually copy (serialize/deserialize) the object, you can't share such a python object between processes. You can do that in multithreading though (we do that in dask-geopandas)

    Shirzart Enwer
    @Aniwax

    But opening a multi-processing of a GEOS object is possible right? There is a way to pass the Polygon instance between different processes? Do I need to define that explicitly?

    Between different processes you will have to actually copy (serialize/deserialize) the object, you can't share such a python object between processes. You can do that in multithreading though (we do that in dask-geopandas)

    Once you share the copy, you would have to merge the final version of all parallel processed polygons together?

    Shirzart Enwer
    @Aniwax
    How do I check if Cython was used to compile pygeos into C, when I install pygeos? I tried to uninstall Cython from my environment, and then reinstall pygeos, but the _geometry.pyx and _geometry.c file is still inside the pygeos/ folder
    Joris Van den Bossche
    @jorisvandenbossche
    @Aniwax cython is always used, so not fully sure what you? Or do you want to know if the package was compiled locally on your machine, or whether you installed a pre-compiled wheel?
    Shirzart Enwer
    @Aniwax

    @Aniwax cython is always used, so not fully sure what you? Or do you want to know if the package was compiled locally on your machine, or whether you installed a pre-compiled wheel?

    Ahh I meant I wasn't sure, if I don't have Cython locally, how is pygeos complied by Cython?

    But opening a multi-processing of a GEOS object is possible right? There is a way to pass the Polygon instance between different processes? Do I need to define that explicitly?

    Between different processes you will have to actually copy (serialize/deserialize) the object, you can't share such a python object between processes. You can do that in multithreading though (we do that in dask-geopandas)

    Following up on this one, can I use Dask-Geopandas or Geopandas to do operations that are done in pygeos? such as polygon.intersect(), polygon.union_all() ?
    And if I use Geopands to check the element-wise intersection between a list of polygons and another geometry, is this faster than using list comprehension to check those intersections?

    Martin Fleischmann
    @martinfleis

    You don't need cython if you install pygeos from pip or conda but you do need it if you want to install from source.

    Following up on this one, can I use Dask-Geopandas or Geopandas to do operations that are done in pygeos? such as polygon.intersect(), polygon.union_all() ?

    Yes, if you have pygeos installed, it is automatically used under the hood to do all these. Not that the unary_union is the name for union_all in geopandas.

    And if I use Geopands to check the element-wise intersection between a list of polygons and another geometry, is this faster than using list comprehension to check those intersections?

    Yes, much faster as it is using vectorized approach of pygeos.

    2 replies
    Shirzart Enwer
    @Aniwax
    I experienced failure of installation for pygeos with python 3.11
    Shirzart Enwer
    @Aniwax
    I raised the issue here shapely/shapely#1584
    rraymondgh
    @rraymondgh

    I'm trying to create all the ci envs on a vagrant box, all work except

    (base) vagrant@gpd:~$ mamba env update -q -n 38-pd11-defaults -f /home/vagrant/geopandas/ci/envs/38-pd11-defaults.yaml
    Encountered problems while solving:
      - nothing provides gdal 1.11.4 needed by fiona-1.6.3-np110py27_0

    Is there a different approach to creating these envs that works for all?

    rraymondgh
    @rraymondgh

    I'm trying to create all the ci envs on a vagrant box, all work except

    (base) vagrant@gpd:~$ mamba env update -q -n 38-pd11-defaults -f /home/vagrant/geopandas/ci/envs/38-pd11-defaults.yaml
    Encountered problems while solving:
      - nothing provides gdal 1.11.4 needed by fiona-1.6.3-np110py27_0

    Is there a different approach to creating these envs that works for all?

    FYI using mambaforge instead of miniconda resolves

    Brendan Ward
    @brendan-ward
    @rraymondgh those are very old versions of Fiona and GDAL, which I believe have been unsupported for quite some time now. Is there something specific about your configuration that requires such old versions?
    rraymondgh
    @rraymondgh
    @brendan-ward I'm using the latest versions of the yaml files from the main branch so I have a full set of environments that match the CI github actions. So in theory I'll never see a failure against a checkin I do. What surprised me is that mamba resolved these yaml files in this way. Moving from miniconda to mambaforge means they resolve
    rraymondgh
    @rraymondgh
    it appears this release https://github.com/python-visualization/branca/releases has broken a test case in test_explore() I'll look into resolving this tmr
    Shirzart Enwer
    @Aniwax
    Guys, is there any plan on making pygeos release for python 3.11 version?
    Joris Van den Bossche
    @jorisvandenbossche
    You mean a python 3.11 wheel, right? (since pygeos itself works fine with python 3.11)
    2 replies
    Shirzart Enwer
    @Aniwax
    yes, 3.11 wheel
    Martin Fleischmann
    @martinfleis

    A reminder that we have a scheduled GeoPandas community meeting today Thursday, November 24 at 17:00 UTC.

    Tentative agenda and notes (feel free to add points ahead of the meeting): https://hackmd.io/pq8MqmtxTFqqaiayFuuEYQ

    Link to the Zoom meeting: https://us06web.zoom.us/j/88360697466?pwd=ZXlxSXd6WWI2bEoxS2ZzbjZSa3d5Zz09

    As always, we welcome not only the geopandas collaborators but everyone willing to spend some time talking about GeoPandas (and related packages) and its future. Also, feel free to join as observers only.

    Shirzart Enwer
    @Aniwax
    Looking forward to GeoPython Conference!
    rraymondgh
    @rraymondgh
    I meant to attend community meeting. Happy to take on changing natural earth to use api instead of static copy. (In last update I did update code to download in a more automated way)
    Martin Fleischmann
    @martinfleis
    @rraymondgh coordinate that with @jorisvandenbossche, he mentioned he'll have a look at that
    hannah
    @story645
    Hi @martinfleis @jorisvandenbossche - wanted to know if you'd find it helpful to have the borders update message translated into Russian and Ukrainian -if so I can ask around for folks who can translate (& my Russian isn't awesome but I can probably help if auto translate is being fuzzy on some of the responses you're getting)
    Martin Fleischmann
    @martinfleis
    Thanks for the kind offer @story645 ! Do you think it would help? I, personally, would prefer if this whole matter went away as soon as possible. The damage control wasn’t easy. But we can still do this if you thinks it’s worth. You’re the communication expert here :)
    1 reply
    Joris Van den Bossche
    @jorisvandenbossche
    I mentioned yesterday during the S2 chat about GEOS prepared predicates being faster, even when preparing each time for a single call, and so why GEOS then doesn't just use that version by default. It seems that they started exploring/implementing just that: https://github.com/libgeos/geos/pull/777#issuecomment-1353744775 (for GEOSIntersects, it has already been merged)
    Martin Fleischmann
    @martinfleis
    Hey, I spent some time on that package replacing datasets module. See martinfleis/xyzdata#1 and let me know your thoughts there please. If we warn about deprecation in 0.13, docs shall not be using it anymore. This can replace bits.
    Joris Van den Bossche
    @jorisvandenbossche
    @martinfleis the polygonize is failing because pandas doesn't recognize the result as "scalar" and then assumes the result is an array
    Martin Fleischmann
    @martinfleis
    @jorisvandenbossche that is something to be fixed on our side then, right?
    Joris Van den Bossche
    @jorisvandenbossche
    It would be easier to fix on the pandas side, but that gives some more delays
    Martin Fleischmann
    @martinfleis
    I think there's no rush. better to have a proper fix