Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Martin Fleischmann
    @martinfleis
    And fill, of course :).
    Joris Van den Bossche
    @jorisvandenbossche
    And the link to the tweet, in case you want to share it on twitter: https://twitter.com/geopandas/status/1308375400897277956
    deeplook
    @deeplook
    Hi, is anybody here using GeoPandas on PyPy3? I’m trying but I cannot get shapely installed. Details upon request, but maybe somebody has a pointer to share? Thanks!
    Martin Fleischmann
    @martinfleis
    That can be a bit complicated. Shapely, fiona and rtree, which are required dependencies have C components, which apparently need to be recompiled for PyPy3.
    Joris Van den Bossche
    @jorisvandenbossche
    On conda-forge they are recompiling many packages for PyPy, but I don't know the state of the geospatial packages
    deeplook
    @deeplook
    Not sure how to search https://conda-forge.org for pypy packages. http://packages.pypy.org says pip install shapely==1.5.17 should work, but it complains about missing libgeos_c. Rtree is listed as buggy, fiona and geopandas are not listed at all. Not sure how well supported that list is. But I can pip install pandas at least which takes a looong time, though. ;)
    Joris Van den Bossche
    @jorisvandenbossche
    Conda-forge has a status page, which lists a "pypy migration", where you can see which packages are done / in progress (not necessarily a handy overview, but still has some information)
    For example, it seems Shapely is "in-PR" -> conda-forge/shapely-feedstock#70
    Pandas is listed as done, but it seems the pypy builds are failing though (eg you look at the status checks of the last commit at https://github.com/conda-forge/pandas-feedstock)
    deeplook
    @deeplook
    Ok, thanks! I’m still missing how I would conda-install something for pypy3? To use pip I need to download it first: wget https://bootstrap.pypa.io/get-pip.py; pypy3 get-pip.py.
    deeplook
    @deeplook
    Henrikki Tenkanen
    @HTenkanen
    Hi guys! I'm developing something where I would need to add some additional metadata to gdf. What would be the best way to do this, so that this metadata "travels" with the GeoDataFrame in a similar manner as gdf.crs? Basically, I would like to be able to give a name for the frame. It's of course possible to add an extra column for this metadata but was wondering if there would be a better way, ie. doing something like: gdf._name = "this_is_me!". <-- this works but gets dropped out after slicing, copying etc. typical operations. This would of course be easy to add into the GeoDataFrame class as a property but not sure if this is something you've discussed as a no-go. Any thoughts on this @martinfleis @jorisvandenbossche?
    Levi John Wolf
    @ljwolf
    @HTenkanen, Does this do what you need?
    Martin Fleischmann
    @martinfleis
    Screenshot 2020-10-13 at 09.24.26.png
    Hi, this is something that should be implemented in pandas. Adding your meta to _metadata does preserve it, but I am not sure how stable that is
    Pandas also has attrs which looks that it should do the trick, but does not atm. https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.attrs.html
    Joris Van den Bossche
    @jorisvandenbossche
    Yeah, just wanted to mention .attrs, which is in theory added to pandas for this kind of things
    but so right now there are still quite some places where .attrs is not preserved
    Martin Fleischmann
    @martinfleis
    Screenshot 2020-10-13 at 09.27.24.png
    yeah, even slicing does not work. maybe only for gdf?
    Joris Van den Bossche
    @jorisvandenbossche
    Basic slicing does work:
    In [28]: df = pd.DataFrame({'a': range(10)})                                                                                                                                                                       
    
    In [30]: df.attrs['name'] = "my_name"                                                                                                                                                                              
    
    In [31]: df.attrs                                                                                                                                                                                                  
    Out[31]: {'name': 'my_name'}
    
    In [32]: df[:5].attrs                                                                                                                                                                                              
    Out[32]: {'name': 'my_name'}
    
    In [33]: df[df['a'] > 5].attrs                                                                                                                                                                                     
    Out[33]: {'name': 'my_name'}
    but maybe only on master
    Adding it to _metadata should be relatively robust, I think (since it's also what geopandas uses), but in principle this is only meant to be used by subclasses, not for a normal user adding a property to it dynamically
    Martin Fleischmann
    @martinfleis
    Slicing works on DataFrame but not GeoDataFrame
    gdf looses attrs
    Joris Van den Bossche
    @jorisvandenbossche
    Ah, that's an interesting observation. We should find out how that comes, and fix that
    Maybe our overridden __finalize__ needs to be updated for that
    Martin Fleischmann
    @martinfleis
    I'll open an issue so we don't forget
    Joris Van den Bossche
    @jorisvandenbossche
    yes, so those attrs are passed through using the __finalize__ mechanism (so the same as _metadata actually)
    but we override __finalize__
    Martin Fleischmann
    @martinfleis
    do we actually have to do that now since we have crs on array level?
    Joris Van den Bossche
    @jorisvandenbossche
    and we added several types of functionality to that recently (attrs but also flags)
    (I should have thought about this earlier duing the discussions to add those about the impact / relation to geopandas ..)
    In the GeoPandas implementation, we have some custom logic for merge and concat (to use the metadata of the first object)
    but would need to think that through if this is still needed with the "crs on the array level" change
    Martin Fleischmann
    @martinfleis
    yes, I just tested that and we need that to preserve geometry column name
    few tests fail without it
    Joris Van den Bossche
    @jorisvandenbossche
    (the GeoSeries.__finalize__ version can probably be removed, since that doesn't have custom logic, and it has a comment that says this was copied from pandas which gets the method only starting in 0.13)
    Hmm, so to avoid needing to "copy" the implementation of pandas, we need some way to have our own logic for _metadata, but call super() for the handling of attrs/flags
    That doesn't seem easily possible with the current pandas implementation
    Martin Fleischmann
    @martinfleis
    maybe override merge and concat instead? use super(), then pass meta and remove custom __finalize__
    Joris Van den Bossche
    @jorisvandenbossche
    we can't override pd.merge and pd.concat since that are functions, not methods (only eg DataFrame.merge, which we actually already override)
    Martin Fleischmann
    @martinfleis
    oh, sure. yeah.
    Joris Van den Bossche
    @jorisvandenbossche
    short term fix might be to copy part of pandas' __finalize__ to geopandas (eg to deal with attrs), but we should also discuss with the pandas devs how to enable easier integration / customization
    (will open an issue on the pandas side for this)
    Martin Fleischmann
    @martinfleis
    geopandas/geopandas#1654 Maybe better to move the discussion there, so we don't loose it.
    Levi John Wolf
    @ljwolf
    :thumbsup: attributes would be very nice to preserve!
    Henrikki Tenkanen
    @HTenkanen
    Wow, a lot happened here during my lunch break! :D Thanks guys! :+1: I will use the _metadata for now and update it to attrs once that's fixed to geopandas.