Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Walter Lorenzetti
    @wlorenzetti
    Hi all, I'm newone, nice to meet you!
    I've a timing calculation problen with pyproj
    I use pyproj to project official Italy cadastrian data...
    by on a customer VM istallation I notice a 40 times lower then on my laptop
    this is my test code: 
    from pyproj import Proj, transform
import time

start = time.time()
outProj = Proj(init='epsg:3003')
inProj = Proj("+proj=cass +lat_0=43.318293 +lon_0=11.332212 +x_0=0.000000000000000 +y_0=0.000000000000000 +datum=WGS84 +units=m +no_defs")
x = 61759.831
y = -41125.585
ret = transform(inProj, outProj, y, x)
stop = time.time()
print "TIME: {}".format(stop - start)
print ret
    on my laptop I obtain this: 
    TIME: 0.00243186950684
(1646302.7175562638, 4859396.617420217)
    on my customer VM server this:
    Walter Lorenzetti
    @wlorenzetti
    TIME: 0.0984871387482
(1646302.7175560538, 4859396.61741957)
    any idea about this lower behaviour?
    thanks in advance and good weekend
    Alan D. Snow
    @snowman2
    How did you install each one? What version of pyproj/PROJ are you using?
    Walter Lorenzetti
    @wlorenzetti
    Hi Alan, I've different version on my laptop and on remote VM: laptop Ubuntu 16.04 PROJ: 4.9.2 pyproj: 1.9.5.1 installe into virtualenv by pip ; Vm remote ubuntu 18.04 PROJ:4.9.3 pyproj: 2.2.2 installed into virtualenv by pip.
    Walter Lorenzetti
    @wlorenzetti
    Hi Alan, another step for understanding the possibile problem. I installed on VM remote machine pyproj 1.9.1.5 e remove 2.2.2 version and now the reprojection are more kindly :) : 
    TIME: 0.000527143478394
(1646302.7175562638, 4859396.617420217)
    I hope this can help..
    Walter Lorenzetti
    @wlorenzetti
    i continue analisys:
    on my laptop

    v 1.9.6:

    TIME: 0.000527143478394
(1646302.7175562638, 4859396.617420217)

    v 2.0.0

    TIME: 0.0132768154144
(1646279.7128513593, 4859469.284666889)

    v 2.0.1

    TIME: 0.0472819805145
(1646279.7128513593, 4859469.284666889)

    v 2.1.0

    TIME: 0.0480349063873
(1646279.7128513593, 4859469.284666889)

    v 2.1.1

    TIME: 0.0479390621185
(1646279.7128513593, 4859469.284666889)

    v 2.1.2

    TIME: 0.142739057541
(1646302.7175560538, 4859396.61741957)

    v 2.1.3

    TIME: 0.149616956711
(1646302.7175560538, 4859396.61741957)

    v 2.2.0

    TIME: 0.0712070465088
(1646302.7175560538, 4859396.61741957)

    v 2.2.1

    TIME: 0.0708930492401
(1646302.7175560538, 4859396.61741957)

    v 2.2.2

    TIME: 0.0708050727844
(1646302.7175560538, 4859396.61741957)
    Schermata del 2019-12-02 11-53-50.png
    Alan D. Snow
    @snowman2
    That makes sense. There is a little more overhead in the pyproj 2+ to create the Proj object due to the changes in PROJ 6+.
    Walter Lorenzetti
    @wlorenzetti
    So with PROJ4 is better stay lower then pyproj 2
    David Hoese
    @djhoese
    @snowman2 Am I correct that in pyproj 2.4.2 the transform always returns float64s?
    Nevermind, this has likely been this way for a while
    Alan D. Snow
    @snowman2
    So with PROJ4 is better stay lower then pyproj 2
    It depends on your use case. Personally I would recommend pyproj 2+.
    Walter Lorenzetti
    @wlorenzetti
    Thnks @snowman2 !
    Joris Van den Bossche
    @jorisvandenbossche
    @wlorenzetti the difference in speed between pyproj 1.9 and pyproj 2 is only for single transformations
    if you do repeated transformations using the same inProj and outProj, you can use the Transformer object in pyproj 2+, which will be much faster
    Walter Lorenzetti
    @wlorenzetti
    Thanks @jorisvandenbossche I'll try!
    Joris Van den Bossche
    @jorisvandenbossche
    With the transform function, each time you call transform(inProj, outProj, y, x), pyproj (PROJ) needs to find the best transformation between inProj and outProj, which makes it slower. But the Transform object can remember that
    See https://pyproj4.github.io/pyproj/stable/gotchas.html#upgrading-to-pyproj-2-from-pyproj-1
    Walter Lorenzetti
    @wlorenzetti
    Good it's clear
    Walter Lorenzetti
    @wlorenzetti
    I just try with pyproj 2.2.2 with Transformer object: 
    # https://pyproj4.github.io/pyproj/stable/gotchas.html#upgrading-to-pyproj-2-from-pyproj-1

from pyproj import Transformer
import time

start = time.time()
x = 61759.831
y = -41125.585
transformer = Transformer.from_crs("+proj=cass +lat_0=43.318293 +lon_0=11.332212 +x_0=0.000000000000000 +y_0=0.000000000000000 +datum=WGS84 +units=m +no_defs", "epsg:3003")
ret = transformer.transform(y, x)
stop = time.time()
print "TIME: {}".format(stop - start)
print ret
    but timing in not good again.. 
    TIME: 0.0580408573151
(1646302.7175560538, 4859396.61741957)
    I' tryee inside a 20 time cycle to test memory effect: 
    TIME: 0.0537190437317
(1646302.7175560538, 4859396.61741957)
TIME: 0.0591440200806
(1646302.7175560538, 4859396.61741957)
TIME: 0.0580699443817
(1646302.7175560538, 4859396.61741957)
TIME: 0.0595941543579
(1646302.7175560538, 4859396.61741957)
TIME: 0.0600090026855
(1646302.7175560538, 4859396.61741957)
TIME: 0.059278011322
(1646302.7175560538, 4859396.61741957)
TIME: 0.0588028430939
(1646302.7175560538, 4859396.61741957)
TIME: 0.0584590435028
(1646302.7175560538, 4859396.61741957)
TIME: 0.0585939884186
(1646302.7175560538, 4859396.61741957)
TIME: 0.0572869777679
(1646302.7175560538, 4859396.61741957)
TIME: 0.0574839115143
(1646302.7175560538, 4859396.61741957)
TIME: 0.0569059848785
(1646302.7175560538, 4859396.61741957)
TIME: 0.0589108467102
(1646302.7175560538, 4859396.61741957)
TIME: 0.0600910186768
(1646302.7175560538, 4859396.61741957)
TIME: 0.0592269897461
(1646302.7175560538, 4859396.61741957)
TIME: 0.058445930481
(1646302.7175560538, 4859396.61741957)
TIME: 0.0585761070251
(1646302.7175560538, 4859396.61741957)
TIME: 0.0573010444641
(1646302.7175560538, 4859396.61741957)
TIME: 0.067663192749
(1646302.7175560538, 4859396.61741957)
TIME: 0.0576071739197
(1646302.7175560538, 4859396.61741957)
    But I can see a better timing...
    Joris Van den Bossche
    @jorisvandenbossche
    So what I said above: it will only be faster if you use the transform object repeatedly. For a single transformation, it will still be slower compared to pyproj 1.9.6
    Walter Lorenzetti
    @wlorenzetti
    Oh sorry I have putted 'transformer' instance creation inside the cycle, now the timing is good! Thanks: 
    TIME: 6.41345977783e-05
(1646302.7175560538, 4859396.61741957)
TIME: 1.50203704834e-05
(1646302.7175560538, 4859396.61741957)
TIME: 1.21593475342e-05
(1646302.7175560538, 4859396.61741957)
TIME: 1.31130218506e-05
(1646302.7175560538, 4859396.61741957)
TIME: 9.77516174316e-06
(1646302.7175560538, 4859396.61741957)
TIME: 1.09672546387e-05
(1646302.7175560538, 4859396.61741957)
TIME: 1.38282775879e-05
(1646302.7175560538, 4859396.61741957)
TIME: 9.05990600586e-06
(1646302.7175560538, 4859396.61741957)
TIME: 1.00135803223e-05
(1646302.7175560538, 4859396.61741957)
TIME: 1.09672546387e-05
(1646302.7175560538, 4859396.61741957)
TIME: 8.10623168945e-06
(1646302.7175560538, 4859396.61741957)
TIME: 7.86781311035e-06
(1646302.7175560538, 4859396.61741957)
TIME: 1.69277191162e-05
(1646302.7175560538, 4859396.61741957)
TIME: 1.09672546387e-05
(1646302.7175560538, 4859396.61741957)
TIME: 8.10623168945e-06
(1646302.7175560538, 4859396.61741957)
TIME: 1.4066696167e-05
(1646302.7175560538, 4859396.61741957)
TIME: 1.4066696167e-05
(1646302.7175560538, 4859396.61741957)
TIME: 1.09672546387e-05
(1646302.7175560538, 4859396.61741957)
TIME: 1.71661376953e-05
(1646302.7175560538, 4859396.61741957)
TIME: 1.69277191162e-05
(1646302.7175560538, 4859396.61741957)
    donglx2018
    @donglx2018
    hi
    pyproj fails to compille when I pip install basemap
    donglx2018
    @donglx2018
    the error message is as follows:
    src/_proj.c:6754:24: error: 'PyThreadState' {aka 'struct _ts'} has no member named 'exc_type'; did you mean 'curexc_type'?
    tmp_type = tstate->exc_type;
    ^~~~
    curexc_type
    any idea about this error?
    Alan D. Snow
    @snowman2
    What is your operating system version? Python version?
    David Hoese
    @djhoese
    @snowman2 I'm bugging you here since it seems like the quickest way to find an answer, sorry. I'm hacking together a custom MapServer where I want to serve data in the +proj=geos projection. I was thinking of defining my own EPSG code for this, but with the changes to PROJ where it uses a sqlite DB now, is it possible to do this any more?