by

Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Joris Van den Bossche
    @jorisvandenbossche
    certainly!
    For me, a main milestone would be to have a Shapely 2.0 released, so we can require that and can remove all the shapely/pygeos compatibility code
    But that of course makes it dependent on the progress in another package ..
    Martin Fleischmann
    @martinfleis
    I agree. It does not make sense to make 1.0 with optional pygeos layer
    Adrian Garcia Badaracco
    @adriangb
    Hi folks, is there anything I can do to move the geopandas/geopandas#1343 forward? I just want to make sure I'm making it as easy to review as possible. Thanks!
    Joris Van den Bossche
    @jorisvandenbossche
    @adriangb thanks for the ping!
    Brendan Ward
    @brendan-ward
    Is there a trick to get asv to recognize newly added functionality for use in benchmarks? I'm trying to add benchmarks for the new parquet functionality, and keep running into this: ImportError: cannot import name 'read_parquet' from 'geopandas'
    Yet all the parquet tests run fine...
    I don't see anything in the asv config that would cause it to install geopandas from conda instead of using the local package.
    Adrian Garcia Badaracco
    @adriangb
    If you want to run using your current source you need to use asv dev. I think that is what you're asking right?
    Brendan Ward
    @brendan-ward
    That worked! I had tried it previously to no avail, but worked when I tried it again just now. Thanks @adriangb !
    Adrian Garcia Badaracco
    @adriangb
    šŸ‘
    Krishna Chaitanya
    @chaitan94
    Hey @jorisvandenbossche! I have recently been trying to implement a vectorized function over a C API for a personal project, similar to what @caspervdw and you guys have done over GEOS C API via PyGEOS (Really amazing work btw, I am learning a lot :thumbsup: ). So I am new to all this, so I was trying to understand your approach of using Python Extenstion Types, I also wanted to validate an alternate approach using pybind11 (https://pybind11.readthedocs.io/en/stable/advanced/pycpp/numpy.html#vectorizing-functions). From what I could gather so far, this approach seems to be much easier to implement, and I feel it should have similar performance too - but I am not sure as I have not tried implementing either as of yet. I was just wondering if you guys considered the pybind11 approach too, and if so what would be the reasons to not prefer it?
    Joris Van den Bossche
    @jorisvandenbossche
    Hi @chaitan94 !
    I don't think we considered it (we mostly followed numpy's guidelines from writing your own ufuncs, which is using the C API)
    But, that's for the actual vectorized functions
    The reason we use a Python Extension Type is because we need to handle external C++ objects (GEOSGeometry objects), so not just ints and floats like in pybind11 example you linked to.
    By wrapping those GEOSGeometry objects in a PyObject extension type, we can rely on Python for memory management (ensuring the objects are kept alive / are destroyed when needed) and don't need to expose raw pointers.
    Joris Van den Bossche
    @jorisvandenbossche
    So for us, the question would be: does pybind also support vectorizing with python objects, and is it then still easier to write than the templated ufuncs we do now (eg you can't just vectorize the GEOS functions like GEOSIntersection_r, as that expects pointers, and not the python object).
    Krishna Chaitanya
    @chaitan94
    You are right, pybind11's vectorize seems to be only for simple types. Specifically primitives like ints and floats, and for Plain Old Data (POD) types.
    So from what you are saying, in order to deal with objects like GEOSGeometry, we, in the end, would have to resort to Python Extension Types, is it? Even in case of pybind that is.
    I think will have dive much deeper into both to understand it clearly. Perhaps I will try out a minimal implementation of both. Thanks for your help!
    Joris Van den Bossche
    @jorisvandenbossche
    Yeah, if you are able to set it up with pybind11, I would be very interested to see it!

    So from what you are saying, in order to deal with objects like GEOSGeometry, we, in the end, would have to resort to Python Extension Types, is it?

    There will be other ways for sure (eg before I just stored the pointers in an integer numpy array, but that is not very robust), but in the case of PyGEOS using the python object helps us to have proper memory management in Python without having to worry too much about it

    Krishna Chaitanya
    @chaitan94
    Sure if I find anything promising in the pybind approach, I will let you know.
    btw I am quite excited about the Shapely 2.0 project :) . If you guys are in need of any help where I could pitch in, I would love to contribute. Would be great learning exercise for me too.
    Joris Van den Bossche
    @jorisvandenbossche
    There are several issues on the PyGEOS side that can use help, if you are interested!
    Currently the active development is still there, until we start actually moving it to shapely
    We should probably start using a label for issues that are somewhat straightforward to do (or are not in a state of discussion)
    pygeos/pygeos#126 also has a list of things to add
    Krishna Chaitanya
    @chaitan94
    Yeah I would be interested in PyGEOS as well, will go through pygeos/pygeos#126 tomorrow and ping you privately regarding the same
    Joris Van den Bossche
    @jorisvandenbossche
    (basically what would get done now in PyGEOS would end up in Shapely 2.0)
    Dani Arribas-Bel
    @darribas

    I'm trying to ready this SpatiaLite DB with geopandas but run into the following error:

    engine = create_engine("sqlite:///dm_bcn.sqlite")
    geopandas.read_postgis("SELECT * FROM 'data_out.dm_bcn' LIMIT 5;", engine, geom_col="GEOMETRY")

    which returns:

    ParseException: Unknown WKB type 96
    ---------------------------------------------------------------------------
    WKBReadingError                           Traceback (most recent call last)
    <ipython-input-33-d7e99b258e62> in <module>
    ----> 1 geopandas.read_postgis("SELECT * FROM 'data_out.dm_bcn' LIMIT 5;", engine, geom_col="GEOMETRY")
    
    /opt/conda/lib/python3.7/site-packages/geopandas/io/sql.py in read_postgis(sql, con, geom_col, crs, index_col, coerce_float, parse_dates, params)
         89             load_geom = load_geom_text
         90 
    ---> 91         df[geom_col] = geoms = geoms.apply(load_geom)
         92         if crs is None:
         93             srid = shapely.geos.lgeos.GEOSGetSRID(geoms.iat[0]._geom)
    
    /opt/conda/lib/python3.7/site-packages/pandas/core/series.py in apply(self, func, convert_dtype, args, **kwds)
       3846             else:
       3847                 values = self.astype(object).values
    -> 3848                 mapped = lib.map_infer(values, f, convert=convert_dtype)
       3849 
       3850         if len(mapped) and isinstance(mapped[0], Series):
    
    pandas/_libs/lib.pyx in pandas._libs.lib.map_infer()
    
    /opt/conda/lib/python3.7/site-packages/shapely/wkb.py in loads(data, hex)
         15         return reader.read_hex(data)
         16     else:
    ---> 17         return reader.read(data)
         18 
         19 def load(fp, hex=False):
    
    /opt/conda/lib/python3.7/site-packages/shapely/geos.py in read(self, data)
        425         if not geom:
        426             raise WKBReadingError(
    --> 427                 "Could not create geometry because of errors "
        428                 "while reading input.")
        429         # avoid circular import dependency
    
    WKBReadingError: Could not create geometry because of errors while reading input.

    am I missing a setting? I can open it right away with QGIS. (Also, this is not top priority, but I thought I'd report it)

    Joris Van den Bossche
    @jorisvandenbossche
    With spatialite, I think you need to convert to WKB manually in the query
    See its docstring:
    (but it doesn't render in the online docs, I suppose because of a wrong header)
    Dani Arribas-Bel
    @darribas
    :+1: thanks! I'll give it a go
    Andrew Annex
    @AndrewAnnex
    hey all, I am running into a api question, I have a function which takes a geodataframe group, selects N random points using 'interpolate' , and for each random point that I select I keep the columns by copying the dataframe of the group and overriding the geometry of that copy. So for each group, I get a list of dicts by calling .to_dict('r') . I am having trouble now taking those records and generating a geodataframe from each group, and probably will have the same issue when trying to combine the groups into one big geodataframe
    I would expect that given a list of dicts, with one key being geomtry, that geopandas should be able to construct a geodataframe
    but from_records just makes a dataframe with 1 column containing the dict
    and from_features returns a type error, I'll look into the source to better understand that
    Andrew Annex
    @AndrewAnnex
    wow nvm, turns out to_dict was returning a list of 1 dict,
    Joris Van den Bossche
    @jorisvandenbossche
    to_dict has several options on the output structure you want
    Now, I don't fully follow, so a small example of the data object you have in the end that you want to turn into a GeoDataFrame might help
    Joris Van den Bossche
    @jorisvandenbossche
    For those that didn't do it yet, it would be welcome to downvote the first answer and upvote the second (mine) answer, at https://stackoverflow.com/questions/34427788/how-to-successfully-install-pyproj-and-geopandas/37635303?noredirect=1#comment109719531_37635303
    Nothing against the person who posted that answer ;), but it's simply outdated nowadays
    (and my answer can certainly still be improved to give more details on the conda install)
    Martin Fleischmann
    @martinfleis
    @jorisvandenbossche any updates on pandas position regarding logo? Iā€™d like to take it into account in docs refactor plan
    Joris Van den Bossche
    @jorisvandenbossche
    they are in principle fine with it, but need to check with NumFOCUS (didn't yet get a reply from them)
    will send a mail with more details