Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Shirzart Enwer
    @Aniwax
    from this I am guessing it's not fromnumpy, but I can be wrong
    Joris Van den Bossche
    @jorisvandenbossche
    (with "from numpy", I didn't mean from a direct numpy call, but we are using numpy's C API to implement those ufuncs in C, so it's still running numpy's machinery although calling a pygeos function)
    As for a reproducible example, trying something very simple:
    In [5]: polygons1 = pygeos.polygons(np.random.randn(10, 3, 2))
    
    In [6]: polygons2 = pygeos.polygons(np.random.randn(10, 3, 2))
    
    In [7]: pygeos.union(polygons1, polygons2)
    Out[7]: 
    array([<pygeos.Geometry MULTIPOLYGON (((1.12 1.27, 0.677 -1.4, 1.27 -0.521, 1.12 1....>,
           <pygeos.Geometry POLYGON ((-2.33 -0.959, -1.35 -0.317, 0.53 0.064, -0.602 1....>,
           <pygeos.Geometry POLYGON ((0.69 -1.4, 0.454 -0.704, 0.0886 -0.934, 0.448 -0....>,
           <pygeos.Geometry POLYGON ((-0.78 -1.96, -0.967 -0.353, -0.731 -0.221, -1.01 ...>,
           <pygeos.Geometry POLYGON ((0.885 0.607, -0.62 -0.964, -0.213 -0.0648, -0.318...>,
           <pygeos.Geometry POLYGON ((0.682 1.38, 0.68 -0.76, 0.215 -1.09, 0.755 -1.52,...>,
           <pygeos.Geometry MULTIPOLYGON (((-1 0.0676, 1.21 -1.35, -0.0311 -1.23, -1 0....>,
           <pygeos.Geometry MULTIPOLYGON (((-1.55 1.55, -0.198 -0.242, -0.614 0.613, -1...>,
           <pygeos.Geometry POLYGON ((-0.661 0.291, -0.181 0.0644, 0.246 0.118, 0.861 -...>,
           <pygeos.Geometry MULTIPOLYGON (((-0.702 2.27, 0.874 -1.25, -0.751 0.967, -0....>],
          dtype=object)
    So that doesn't raise a warning. But now it would be interesting to know what characteristic in polygons1/polygons2 would trigger such a warning
    Although you are actually using a unary union, I see. So the example would be:
    In [8]: polygons = pygeos.polygons(np.random.randn(10, 3, 2))
    
    In [9]: pygeos.union_all(polygons)
    Out[9]: <pygeos.Geometry POLYGON ((-0.973 0.0936, -0.976 0.0886, -1.03 0.138, -1.1 0...>
    Shirzart Enwer
    @Aniwax
    wait, I will get all the numbers
    Shirzart Enwer
    @Aniwax
    Here is a list of polygons, that when I executed the following lines, a RuntimeWarning was catched:
    try: area2d.area = pygeos.constructive.simplify( pygeos.set_operations.union_all(polygons), tolerance=default_precision) except RuntimeWarning: print(polygons)
    POLYGON ((-0.961 0.109, -0.16 -0.98, -0.782 -0.319, -0.961 0.109))
    POLYGON ((-0.782 -0.319, -0.622 0.661, -0.961 0.109, -0.782 -0.319))
    POLYGON ((-0.179 0.428, -0.961 0.109, -0.622 0.661, -0.179 0.428))
    POLYGON ((0.179 -0.428, -0.16 -0.98, 0.622 -0.661, 0.179 -0.428))
    POLYGON ((0.179 -0.428, -0.782 -0.319, -0.16 -0.98, 0.179 -0.428))
    POLYGON ((0.179 -0.428, -0.622 0.661, -0.782 -0.319, 0.179 -0.428))
    POLYGON ((0.782 0.319, 0.622 -0.661, -0.179 0.428, 0.782 0.319))
    POLYGON ((0.16 0.98, -0.179 0.428, -0.622 0.661, 0.16 0.98))
    POLYGON ((0.16 0.98, 0.782 0.319, -0.179 0.428, 0.16 0.98))
    POLYGON ((-0.339 -0.552, -0.179 0.428, 0.622 -0.661, -0.339 -0.552))
    POLYGON ((-0.339 -0.552, -0.16 -0.98, -0.961 0.109, -0.339 -0.552))
    but when I just run pygeos.constructive.simplify( pygeos.set_operations.union_all(polygons), tolerance=default_precision) in the debug console, there is no warning message shown
    Shirzart Enwer
    @Aniwax
    Could you reproduce this scenario?
    Joris Van den Bossche
    @jorisvandenbossche
    Yes
    arr = pygeos.from_wkt(
          ['POLYGON ((-0.961 0.109, -0.16 -0.98, -0.782 -0.319, -0.961 0.109))',
           'POLYGON ((-0.782 -0.319, -0.622 0.661, -0.961 0.109, -0.782 -0.319))',
           'POLYGON ((-0.179 0.428, -0.961 0.109, -0.622 0.661, -0.179 0.428))',
           'POLYGON ((0.179 -0.428, -0.16 -0.98, 0.622 -0.661, 0.179 -0.428))',
           'POLYGON ((0.179 -0.428, -0.782 -0.319, -0.16 -0.98, 0.179 -0.428))',
           'POLYGON ((0.179 -0.428, -0.622 0.661, -0.782 -0.319, 0.179 -0.428))',
           'POLYGON ((0.782 0.319, 0.622 -0.661, -0.179 0.428, 0.782 0.319))',
           'POLYGON ((0.16 0.98, -0.179 0.428, -0.622 0.661, 0.16 0.98))',
           'POLYGON ((0.16 0.98, 0.782 0.319, -0.179 0.428, 0.16 0.98))',
           'POLYGON ((-0.339 -0.552, -0.179 0.428, 0.622 -0.661, -0.339 -0.552))',
           'POLYGON ((-0.339 -0.552, -0.16 -0.98, -0.961 0.109, -0.339 -0.552))'
          ])
    In [19]: pygeos.union_all(arr)
    /home/joris/miniconda3/envs/geo/lib/python3.9/site-packages/pygeos/set_operations.py:388: RuntimeWarning: invalid value encountered in unary_union
      result = lib.unary_union(collections, **kwargs)
    Out[19]: <pygeos.Geometry POLYGON ((-0.961 0.109, -0.622 0.661, 0.16 0.98, 0.782 0.31...>
    Thanks!
    Shirzart Enwer
    @Aniwax
    You're welcome :)
    Do I need to post this on pygeos repo as issue?
    Joris Van den Bossche
    @jorisvandenbossche
    Although it gives "invalid value" for me, and not "divide by zero encountered"
    But in any case also the "invalid value" warning is confusing
    Shirzart Enwer
    @Aniwax
    I had both cases.
    But don't know how to catch them differently
    In [19]: pygeos.union_all(arr)
    /home/joris/miniconda3/envs/geo/lib/python3.9/site-packages/pygeos/set_operations.py:388: RuntimeWarning: invalid value encountered in unary_union
      result = lib.unary_union(collections, **kwargs)
    Out[19]: <pygeos.Geometry POLYGON ((-0.961 0.109, -0.622 0.661, 0.16 0.98, 0.782 0.31...>
    for me, if I run this cell, it gave me both errors:
    c:\Users\aniwax\Documents\GitHub\ibrus_new_design\ibrus-env\lib\site-packages\pygeos\set_operations.py:388: RuntimeWarning: divide by zero encountered in unary_union
      result = lib.unary_union(collections, **kwargs)
    c:\Users\aniwax\Documents\GitHub\ibrus_new_design\ibrus-env\lib\site-packages\pygeos\set_operations.py:388: RuntimeWarning: invalid value encountered in unary_union
      result = lib.unary_union(collections, **kwargs)
    <pygeos.Geometry POLYGON ((-0.961 0.109, -0.622 0.661, 0.16 0.98, 0.782 0.31...>
    Joris Van den Bossche
    @jorisvandenbossche
    I think it's worth opening a pygeos issue for it
    Shirzart Enwer
    @Aniwax
    :thumbsup:
    Shirzart Enwer
    @Aniwax
    Joris Van den Bossche
    @jorisvandenbossche
    thanks!
    Shirzart Enwer
    @Aniwax
    Hey guys, I have a question out of curiosity: how much faster is it to run pygeos module with pypy instead of python?
    Do you have any experience or examples to share? Thanks a lot :)
    Shirzart Enwer
    @Aniwax
    PS C:\Users\aniwax\Downloads> pypy3 -m pip install '.\pygeos-0.10 (1).tar.gz'
    WARNING: Value for scheme.headers does not match. Please report this to <https://github.com/pypa/pip/issues/9617>
    distutils: C:\Users\aniwax\pypy3.7\include\UNKNOWN
    sysconfig: C:\Users\aniwax\pypy3.7\include
    WARNING: Additional context:
    user = False
    home = None
    root = None
    prefix = None
    Processing c:\users\aniwax\downloads\pygeos-0.10 (1).tar.gz
      Installing build dependencies ... done
      Getting requirements to build wheel ... done
        Preparing wheel metadata ... done
    Requirement already satisfied: numpy>=1.13 in c:\users\aniwax\pypy3.7\site-packages (from pygeos==0.10) (1.20.2)
    Building wheels for collected packages: pygeos
      Building wheel for pygeos (PEP 517) ... error
      ERROR: Command errored out with exit status 1:
       command: 'C:\Users\aniwax\pypy3.7\pypy3.exe' 'C:\Users\aniwax\pypy3.7\site-packages\pip\_vendor\pep517\in_process\_in_process.py' build_wheel 'C:\Users\aniwax\AppData\Local\Temp\tmp2561_abp'
           cwd: C:\Users\aniwax\AppData\Local\Temp\pip-req-build-r9lnxq0h
      Complete output (48 lines):
      Could not find geos-config executable. Either append the path to geos-config to PATH or manually provide the include_dirs, library_dirs, libraries and other link args for compiling against a GEOS version >=3.5.
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build\lib.win-amd64-3.7
      creating build\lib.win-amd64-3.7\pygeos
      copying pygeos\constructive.py -> build\lib.win-amd64-3.7\pygeos
      copying pygeos\coordinates.py -> build\lib.win-amd64-3.7\pygeos
      copying pygeos\creation.py -> build\lib.win-amd64-3.7\pygeos
      copying pygeos\decorators.py -> build\lib.win-amd64-3.7\pygeos
      copying pygeos\enum.py -> build\lib.win-amd64-3.7\pygeos
      copying pygeos\geometry.py -> build\lib.win-amd64-3.7\pygeos
      copying pygeos\io.py -> build\lib.win-amd64-3.7\pygeos
      copying pygeos\linear.py -> build\lib.win-amd64-3.7\pygeos
      copying pygeos\measurement.py -> build\lib.win-amd64-3.7\pygeos
      copying pygeos\predicates.py -> build\lib.win-amd64-3.7\pygeos
      copying pygeos\set_operations.py -> build\lib.win-amd64-3.7\pygeos
      copying pygeos\strtree.py -> build\lib.win-amd64-3.7\pygeos
      copying pygeos\_version.py -> build\lib.win-amd64-3.7\pygeos
      copying pygeos\__init__.py -> build\lib.win-amd64-3.7\pygeos
      running egg_info
      writing pygeos.egg-info\PKG-INFO
      writing dependency_links to pygeos.egg-info\dependency_links.txt
      writing requirements to pygeos.egg-info\requires.txt
      writing top-level names to pygeos.egg-info\top_level.txt
      reading manifest file 'pygeos.egg-info\SOURCES.txt'
      adding license file 'LICENSE'
      writing manifest file 'pygeos.egg-info\SOURCES.txt'
      copying pygeos\_geometry.c -> build\lib.win-amd64-3.7\pygeos
      copying pygeos\_geometry.pyx -> build\lib.win-amd64-3.7\pygeos
      copying pygeos\_geos.c -> build\lib.win-amd64-3.7\pygeos
      copying pygeos\_geos.pxd -> build\lib.win-amd64-3.7\pygeos
      copying pygeos\_geos.pyx -> build\lib.win-amd64-3.7\pygeos
      copying pygeos\_pygeos_api.pxd -> build\lib.win-amd64-3.7\pygeos
      UPDATING build\lib.win-amd64-3.7\pygeos/_version.py
      set build\lib.win-amd64-3.7\pygeos/_version.py to '0.10'
      running build_ext
      building 'pygeos.lib' extension
      creating build\temp.win-amd64-3.7
      creating build\temp.win-amd64-3.7\Release
      creating build\temp.win-amd64-3.7\Release\src
      C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.28.29910\bin\HostX86\x64\cl.exe /c 
    /nologo /Ox /W3 /GL /DNDEBUG /MD -IC:\Users\aniwax\pypy3.7\include -IC:\Users\aniwax\AppData\Local\Temp\pip-build-env-hqf0nndx\overlay\site-packages\numpy\core\include -IC:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.28.29910\ATLMFC\include -IC:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.28.29910\include -IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\ucrt -IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\sh
    I ran into this problem, when I try to install pygeos with pypy
    Shirzart Enwer
    @Aniwax
    This problem occured on [PyPy 7.3.5 with MSC v.1927 64 bit (AMD64)] on win32 and also [PyPy 7.3.4 with MSC v.1927 64 bit (AMD64)] on win32
    Martin Fleischmann
    @martinfleis
    The main bulk of time is spent in GEOS, so I don’t expect much of a difference
    Shirzart Enwer
    @Aniwax
    Thanks! Do you guys have any idea how to bypass the issue on installing with pypy?
    Brendan Ward
    @brendan-ward
    @Aniwax I haven't tested with pypy, but given that the slow things in pygeos either come directly from GEOS or have already largely been optimized into C or Cython extensions, I wouldn't expect much benefit from using it for pygeos.
    I may be mistaken, but it looks like pygeos on conda-forge should support pypy; perhaps try installing it that way?
    Shirzart Enwer
    @Aniwax

    @brendan-ward Thanks for the tip! I understand that pygeos won't be much faster with pypy. But I wanted to improve the other part of my python code. So I still want to try it out. If I want to use conda forge, it looks like I need to reinstall pypy and every other packages with conda forge? Or is there a way to just install packages using conda forge to my pypy environment?

    I still think having a wheel built for pypy would be very appreciated :)
    btw, I also opened an issue on the repo, in case other people have similar problems. Maybe we can move the discussion there pygeos/pygeos#358

    Shirzart Enwer
    @Aniwax
    Conda-forge is also having trouble with pypy installation.... I am really crying out loud for a pip wheel file of pygeos
    Shirzart Enwer
    @Aniwax
    Sorry guys, I just tried out pypy + Shapely vs python + Shapely, and it turned out using pypy with Shapelyand numpy together, actually slow down the code x10 times. So I guess there isn't really a use case for using pypy with pygeos.
    Joris Van den Bossche
    @jorisvandenbossche
    That could also exactly be a reason to look into pygeos, if you want to use pypy.
    (without having practical experience with it, so the below is just guessing)
    In general, I think pypy doesn't yet work super well with Python C extensions (in the sense that those are often slower (defeating part of the purpose of pypy), because the Python <-> C API interactions in pypy can be slower. I don't know the current state of this, but I seem to remember this certainly being an issue in the past.
    When using an array of Shapely objects and doing operations on those objects, you constantly have Python <-> C interactions (through ctypes) because Shapely only implements this for scalar objects.
    While with pygeos, because of the vectorized operations, larger chunks of computations are done in one go at the C level, and in general you thus have less Python <-> C cross-overs
    So "pygeos on pypy" might give a smaller slowdown than "shapely on pypy"

    Now, related to

    I am really crying out loud for a pip wheel file of pygeos

    You are certainly welcome to look into what it would take to build pygeos wheels for pypy (I don't think any of the maintainers has experience with this)

    Shirzart Enwer
    @Aniwax
    Thanks a lot for the insight @jorisvandenbossche :) I was actually curious if there would be "speed-up" rather than "smaller slow down" when using pypy+pygeos...hahaha but I guess without an experiment, you can't answer that with a confidence. But anyway, thanks for the insight!
    Joris Van den Bossche
    @jorisvandenbossche
    I don't think there can ever be a speed-up. Whenever a pygeos functions takes a significant amount of time, this is largely spent inside the GEOS C++ library. Using python vs pypy will not affect that.
    (and this is exactly the benefit of pygeos -> it removes a lot of the Python overhead that Shapely has, and as a result most of the remaining time is actually spent in the spatial algorithms in GEOS)
    Shirzart Enwer
    @Aniwax
    Aha....I need to look into other area of my code then :)
    Thanks a lot !
    Shirzart Enwer
    @Aniwax
    Hey guys, I am back again, with a slightly weird question: what kind of polygon is couldn't be fix by pygeos.make_valid() method?