geopandas/geopandas

Python tools for geographic data

H. Thomson Comer

@thomcom

Hey @jorisvandenbossche @martinfleis @brendan-ward @m-richards can I hop in a couple of GeoPandas meetings?

Brendan Ward

@brendan-ward

@sabeerahameds you should use a filename (e.g,. <...path...>/test.shp) instead of a directory.

@thomcom yes! See geopandas/community#15 for more info on today's meeting and geopandas/community#12 for tentative schedule of upcoming meetings

H. Thomson Comer

@thomcom

Thanks so much for the time at the GeoPandas meeting, I'll see you again soon.

Arin

@lotsaocelots1

Hi, I have a question regarding the pyogrio engine in Geopandas. Is it possible to use the pyogrio where parameter with geopandas.read_file ?

Joris Van den Bossche

@jorisvandenbossche

Yes, normally all additional **kwargs are passed through to pyogrio.read_dataframe

Henrikki Tenkanen

@HTenkanen

Hi guys, I know this might be a bit incorrect forum to ask, but I still try my luck here as you might have experienced something similar. We have had some issues in getting geopandas/fiona installing correctly on Github Actions tests for Windows, as it complains about ImportError: DLL load failed while importing ogrext: The specified procedure could not be found.Everything seems to work fine on local machines when installing the environment, so it might be a GA issue. Has any of you experienced something similar and if yes, have you found a solution to fix this? You can find the error logs here: https://github.com/r5py/r5py/runs/7856124475?check_suite_focus=true
And the PR where we discuss about this issue is here: r5py/r5py#149
Thanks already in advance!

Henrikki Tenkanen

@HTenkanen

Update on this ^. Seems to somehow relate to pytest: https://github.com/r5py/r5py/pull/149#issuecomment-1216617678

Joris Van den Bossche

@jorisvandenbossche

Question: is this PR adding a first test that actually uses geopandas.read_file? (it's not that this was working in the past, and for some reason started to fail now)

Henrikki Tenkanen

@HTenkanen

@jorisvandenbossche : Thanks for chipping in! Yes this is actually the first test that reads a file with geopandas. At the moment, it seems that this somehow relates to pytest, as the issue only appears once the test suite is launched in the pipeline. We are porting quite a bit of Java code to Python using jpype library, so not sure whether this also causes some weird behavior.

Martin Fleischmann

@martinfleis

Hi @/all . We are proposing a formalisation of the governance structure of the GeoPandas project and would love to have your feedback, or at least +1 or approval on the PR if you are happy with the current proposal. If you are interested, you will find the current version in this PR geopandas/governance#1. Let us know what you think! Thanks!

Gareth Simons

@songololo

Any advice for how to use shapely==2.0a1 alongside GeoPandas? Would be nice to start using some of the new features from shapely v2 but currently clashes with GeoPandas. I see there is a pull request geopandas/geopandas#2275 working on this, so maybe this will be released into a prerelease at some point in the near-ish future?

Joris Van den Bossche

@jorisvandenbossche

@songololo yes, that's the idea. For now you will have to get that branch of that PR to have shapely 2 working with geopandas (we should merge that PR so you can test with geopandas main branch at least. Will try to do that early next week). The plan is to (pre)release a next geopandas once shapely 2.0b1 is out (my guess is that will be in 1 or 2 weeks).

rraymondgh

@rraymondgh

both v0.11 of geopandas and main branch (pytest tests/test_overlay.py) start failing with pygeos 0.13. is this expected? I can furnish further details in an issue if required

rraymondgh

@rraymondgh

I'm finding make html fails with pydata-sphinx-theme==0.10.1, downgrade to 0.9.0 and it runs ok. any other contributors finding same version issue? How do I run doctests in a MacOS env? Have tried pytest --doctest-only

rraymondgh

@rraymondgh

worked out docttests, need to install pytest-doctestplus. pydata-sphinx-theme needs to be 0.9.0

Joris Van den Bossche

@jorisvandenbossche

Indeed, also the online doc build started to fail recently since pydata-sphinx-theme released 0.10 (https://readthedocs.org/projects/geopandas/builds/17963754/). So we should pin that version for now in doc/environment.yml, and look into fixing it

Pinning -> geopandas/geopandas#2560

@rraymondgh thanks for noticing!

rraymondgh

@rraymondgh

I'll raise an issue and look into what's needed to use latest version of pydata-sphinx-theme

Taylor Hietpas

@sonderyugen

hello all I am curious if there is any knowledge on an sjoin_nearest for dask-geopandas?

Joris Van den Bossche

@jorisvandenbossche

@sonderyugen that's not yet implemented

It's mentioned in geopandas/dask-geopandas#130, but feel free to open a dedicated issue about supporting sjoin_nearest (the complication here is that the "nearest" geometry might live on a different partition, although I think if we require that the user passes a max search distance, it should be doable to implement this)

Joris Van den Bossche

@jorisvandenbossche

@brendan-ward @martinfleis for the meeting on Thursday, would one hour later or one hour earlier also work? (have another overlapping meeting; the geopandas meeting takes priority if it is difficult to move an hour, though)

Brendan Ward

@brendan-ward

@jorisvandenbossche either option works fine for me.

Martin Fleischmann

@martinfleis

@jorisvandenbossche same for me. Both are fine.

Joris Van den Bossche

@jorisvandenbossche

One hour later then?

Joris Van den Bossche

@jorisvandenbossche

@martinfleis I edited the google calendar invite to be one hour later (only for this time). Can you check that is reflected in your calendar as well?

Taylor Hietpas

@sonderyugen

@jorisvandenbossche hey thanks for the reply. will checkout that other thread. i suspected that the partitioning might make nearest determinations less than trivial but I think I can see what you are saying about enforcing effective use of the max_distance parameter. thank you for the update

Martin Fleischmann

@martinfleis

@jorisvandenbossche it did on my side.

H. Thomson Comer

@thomcom

Heya, is it time for another GeoPandas steering meeting? Can I hop in again? :)

Joris Van den Bossche

@jorisvandenbossche

@thomcom certainly!

H. Thomson Comer

@thomcom

This message was deleted

Doh! Sad to miss it, I ended up adding it to my personal calendar and not my work calendar. :p

rraymondgh

@rraymondgh

seems that CI is broken ci/envs/39-latest-conda-forge.yaml leads to this error requests.exceptions.ConnectionError: HTTPSConnectionPool(host='conda.anaconda.org', port=443): Max retries exceeded with url: /conda-forge/label/shapely_dev/notices.json (Caused by ReadTimeoutError("HTTPSConnectionPool(host='conda.anaconda.org', port=443): Read timed out. (read timeout=5)"))

Joris Van den Bossche

@jorisvandenbossche

Yes, I noticed that as well. Not directly sure what is going wrong, since it worked until recently

rraymondgh

@rraymondgh

I'm not at all familiar with conda or mamba, but my guess is this channel conda-forge/label/shapely_dev is no longer valid

Joris Van den Bossche

@jorisvandenbossche

mamba install shapely -c conda-forge/label/shapely_dev still seems to find the package, though

Shirzart Enwer

@Aniwax

Hey guys, quick question about runtime performance improvement for pygeos:

is it possible to store pygeos objects, such as Polygons, as memory-mapped files? Given that I have bunch of planes growing throughout simulation, but not every part of polygon is accessed or written at each time step, it would be great if it can be mapped to memory, such as numpy memory mapped arrays...

Joris Van den Bossche

@jorisvandenbossche

No, that's currently not possible. Geometries in memory are represented as GEOS objects (the underlying C++ library), and those objects can't be created from memory mapped data (well it can, but it will do a copy anyway)

Shirzart Enwer

@Aniwax

Ahhhh alright... wil think about other ways..thanks a lot!

But opening a multi-processing of a GEOS object is possible right? There is a way to pass the Polygon instance between different processes? Do I need to define that explicitly?

Shirzart Enwer

@Aniwax

A stupid question: how do you check which GEOS version is my local pygeos is using?

Joris Van den Bossche

@jorisvandenbossche

pygeos.geos_version

1 reply

But opening a multi-processing of a GEOS object is possible right? There is a way to pass the Polygon instance between different processes? Do I need to define that explicitly?

Between different processes you will have to actually copy (serialize/deserialize) the object, you can't share such a python object between processes. You can do that in multithreading though (we do that in dask-geopandas)

Shirzart Enwer

@Aniwax

But opening a multi-processing of a GEOS object is possible right? There is a way to pass the Polygon instance between different processes? Do I need to define that explicitly?

Between different processes you will have to actually copy (serialize/deserialize) the object, you can't share such a python object between processes. You can do that in multithreading though (we do that in dask-geopandas)

Once you share the copy, you would have to merge the final version of all parallel processed polygons together?

Shirzart Enwer

@Aniwax

How do I check if Cython was used to compile pygeos into C, when I install pygeos? I tried to uninstall Cython from my environment, and then reinstall pygeos, but the _geometry.pyx and _geometry.c file is still inside the pygeos/ folder

Joris Van den Bossche

@jorisvandenbossche

@Aniwax cython is always used, so not fully sure what you? Or do you want to know if the package was compiled locally on your machine, or whether you installed a pre-compiled wheel?

Shirzart Enwer

@Aniwax

@Aniwax cython is always used, so not fully sure what you? Or do you want to know if the package was compiled locally on your machine, or whether you installed a pre-compiled wheel?

Ahh I meant I wasn't sure, if I don't have Cython locally, how is pygeos complied by Cython?

But opening a multi-processing of a GEOS object is possible right? There is a way to pass the Polygon instance between different processes? Do I need to define that explicitly?

Between different processes you will have to actually copy (serialize/deserialize) the object, you can't share such a python object between processes. You can do that in multithreading though (we do that in dask-geopandas)

Following up on this one, can I use Dask-Geopandas or Geopandas to do operations that are done in pygeos? such as polygon.intersect(), polygon.union_all() ?
And if I use Geopands to check the element-wise intersection between a list of polygons and another geometry, is this faster than using list comprehension to check those intersections?

Martin Fleischmann

@martinfleis

You don't need cython if you install pygeos from pip or conda but you do need it if you want to install from source.

Following up on this one, can I use Dask-Geopandas or Geopandas to do operations that are done in pygeos? such as polygon.intersect(), polygon.union_all() ?

Yes, if you have pygeos installed, it is automatically used under the hood to do all these. Not that the unary_union is the name for union_all in geopandas.

And if I use Geopands to check the element-wise intersection between a list of polygons and another geometry, is this faster than using list comprehension to check those intersections?

Yes, much faster as it is using vectorized approach of pygeos.

2 replies

Where communities thrive

People

Repo info

Activity