Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Mar 01 21:30
    dependabot[bot] labeled #3339
  • Mar 01 21:30

    dependabot[bot] on npm_and_yarn

    Bump prismjs from 1.22.0 to 1.2… (compare)

  • Mar 01 21:30
    dependabot[bot] opened #3339
  • Feb 22 21:58

    pomadchin on master

    Fix spelling (#3337) Stiching … (compare)

  • Feb 22 21:58
    pomadchin closed #3337
  • Feb 22 20:08
    nertzy synchronize #3337
  • Feb 22 20:08
    nertzy synchronize #3337
  • Feb 22 20:07
    nertzy synchronize #3337
  • Feb 16 23:00
    pomadchin synchronize #3338
  • Feb 16 23:00
    pomadchin edited #3338
  • Feb 16 22:58
    pomadchin commented #3337
  • Feb 16 22:57
    pomadchin commented #3337
  • Feb 16 22:39
    pomadchin review_requested #3338
  • Feb 16 22:37
    pomadchin opened #3338
  • Feb 09 18:34
    nertzy edited #3337
  • Feb 09 18:34
    nertzy opened #3337
  • Feb 06 02:55
    pomadchin labeled #3336
  • Feb 05 11:57
    tosen1990 opened #3336
  • Feb 04 14:57
    pomadchin labeled #3335
  • Feb 04 12:39
    tosen1990 opened #3335
John Smith
@GintokiYs
Can Geotrellis-3.0 be used now?
Grigory
@pomadchin
@15952026052 yes, it is released
John Smith
@GintokiYs
My spark version is 2.3.1. Must I have spark-2.4?
iceland1906
@iceland1906
@pomadchin checked two tiles inside the bottom polygon, printed their coordinates, both should be inside but the contain function return false
multipoly2.png
metricLayerRdd.filter(
      rddKeyVal => {
        val key = rddKeyVal._1
        val tileExtentPolygon = key.extent(params.metadata.layout).jtsGeom
        val contained = shapePolygon.contains(tileExtentPolygon)
        if(tileExtentPolygon.contains(pnt.jtsGeom)) {
          println(tileExtentPolygon)
          println(shapePolygon)
          println(contained)
          System.exit(1)
        }
        contained
      })
Grigory
@pomadchin
can you throw here polygons that should be inside?
as geojsons
iceland1906
@iceland1906
sure
Grigory
@pomadchin
println(tileExtentPolygon.asJson); println(shapePolygon.asJson)
^ I think (don't really remember the API but can look into it in a while if that would be a problem)
Simeon H.K. Fitch
@metasim
Congratulations to all Geotrellis contributors :clap: :clap: :clap: . 3.0 is a massive leap forward, particularly in fundamental module architecture, providing a fantastic foundation for the future. A substantial achievement in which you should all be very proud. :smile: :smile: :smile:
David Landry
@davidlandry93
Hello all, congrats on 3.0
I have an issue I wonder if you can help me with
I keep runnning in a StackOverflow when trying to reproject and RDD to WebMercator
Exception in thread "main" java.lang.StackOverflowError
        at org.locationtech.jts.geom.CoordinateArrays.dimension(CoordinateArrays.java:44)
        at org.locationtech.jts.geom.impl.CoordinateArraySequence.<init>(CoordinateArraySequence.java:65)
        at org.locationtech.jts.geom.impl.CoordinateArraySequenceFactory.create(CoordinateArraySequenceFactory.java:55)
        at org.locationtech.jts.geom.GeometryFactory.createPoint(GeometryFactory.java:267)
        at geotrellis.vector.Point$.apply(Point.scala:25)
        at geotrellis.vector.reproject.Reproject$.refine$1(Reproject.scala:87)
        at geotrellis.vector.reproject.Reproject$.refine$1(Reproject.scala:97)
        at geotrellis.vector.reproject.Reproject$.refine$1(Reproject.scala:97)
        at geotrellis.vector.reproject.Reproject$.refine$1(Reproject.scala:97)
        at geotrellis.vector.reproject.Reproject$.refine$1(Reproject.scala:97)
        at geotrellis.vector.reproject.Reproject$.refine$1(Reproject.scala:97)
        at geotrellis.vector.reproject.Reproject$.refine$1(Reproject.scala:97)
        at geotrellis.vector.reproject.Reproject$.refine$1(Reproject.scala:97)
        at geotrellis.vector.reproject.Reproject$.refine$1(Reproject.scala:97)
# Continued...
The offending call
  def assimilate(path: String, catalog: String)(implicit sc: SparkContext): Unit = {
    val scheme = ZoomedLayoutScheme(LatLng, tileSize = 256)

    val (zoom, rdd) = netcdfToRDD(path, scheme)
    val reprojected = rdd.reproject(WebMercator, scheme, Bilinear)

    val attributeStore = new FileAttributeStore(catalog)
    val writer = new FileLayerWriter(attributeStore, catalog)
    writer.write(LayerId("precipitation", zoom), rdd, ZCurveKeyIndexMethod.byHour())
  }
Are you guys familiar with this problem? Otherwise I'll try to send you a snippet to reproduce
iceland1906
@iceland1906

@pomadchin here are the shapePolygon and the tileExtent that is inside the polygon

    metricLayerRdd.filter(  rddKeyVal => {
        val key = rddKeyVal._1
        val tileExtentPolygon = key.extent(params.metadata.layout)
        val contained = shapePolygon.contains(tileExtentPolygon)
        if(tileExtentPolygon.contains(pnt.jtsGeom)) {
          println(tileExtentPolygon.toGeoJson())
          println(shapePolygon.toGeoJson())
          println(contained)
          System.exit(1)
        }
        contained
      })

output

iceland1906
@iceland1906

{"type":"Polygon","coordinates":[[[301599.466697037,4932229.096487437],[301599.466697037,4932230.096487457],[301600.46669720486,4932230.096487457],[301600.46669720486,4932229.096487437],[301599.466697037,4932229.096487437]]]}
{"type":"MultiPolygon","coordinates":[[[[301570.4514820544,4932330.550698328],[301608.4739502942,4932329.050574216],[301615.7797098104,4932283.3908159025],[301567.98736547644,4932285.527373691],[301570.4514820544,4932330.550698328]],[[301568.33936377685,4932265.41646293],[301616.3991255134,4932264.116287277],[301607.37711448024,4932221.9959483575],[301576.71921868704,4932223.2709015105],[301568.33936377685,4932265.41646293]]]]}
false
Grigory
@pomadchin
cool let me see
thanks @metasim :tada:
@davidlandry93 yes it is a known bug, first of all try to update up to 3.0 it will write you a better exception :D but it looks like your initial CRS of the rasters you’re reprojecting is in incorrect CRS
for instance you have an extent is some UTM zone and it is in RDD with a LatLng CRS and you’re trying to reproject it into webmercator
@davidlandry93 locationtech/geotrellis#3083 and the related issue with a similar problem you had locationtech/geotrellis#3023
David Landry
@davidlandry93
@pomadchin I'll update to 3.0. I made the input RDD myself so I might have gotten mixed up in the CRS for sure.
My extent is valid to latlng though it you compare to this https://github.com/locationtech/geotrellis/issues/3023#issuecomment-534733560
Grigory
@pomadchin
@iceland1906 what is the CRS of these polygons?
David Landry
@davidlandry93
Extent(210.0, 40.0, 309.8999938964844, 69.94000244140625)
Grigory
@pomadchin
@davidlandry93 210 and 309 are problematic here O:
David Landry
@davidlandry93
Extents are lat lon lat lon?
Grigory
@pomadchin
in LatLng longs are in [-180, 180] & lats are in [-90, 90] degree ranges
David Landry
@davidlandry93
Ohhhhhh
iceland1906
@iceland1906
@pomadchin EPSG:32616
Grigory
@pomadchin
@iceland1906 ha this looks like a bug(?)
Will look closer into it today
iceland1906
@iceland1906
thanks @pomadchin
Grigory
@pomadchin
what is tileExtentPolygon.contains(pnt.jtsGeom)?
in your code above
and what is the type of shapePolygon?
ah nvm I think I fugred out how to reproduce ghe behavior you had
will create a unit test for it
iceland1906
@iceland1906
@pomadchin tileExtentPolygon.contains(pnt.jtsGeom) is just a condition to only show one tile in the target polygon, for demo purpose, not related with the issue
shapePolygon is geotrellis.vector.MultiPolygon type
Ricardo Yrupailla Meza
@stg101
Hi everyone, congratulations on 3.0 !
How can I save
a geotrellis.spark.TileLayerRDD[geotrellis.layer.SpatialKey] as a single tif file
?
btw I am starting to use geotrellis, so excuse me if the question is too simple.
iceland1906
@iceland1906
@stg101 try Section "Stiching Tiles into a single GeoTiff" at this link https://geotrellis.readthedocs.io/en/latest/guide/spark.html
Ricardo Yrupailla Meza
@stg101
Thanks I'm going to try that
I got the following error
[error]  No implicit view available from geotrellis.layer.SpatialKey => geotrellis.spark.tiling.TilerKeyMethods[geotrellis.layer.SpatialKey,geotrellis.layer.SpatialKey].
[error]       rdd.collectMetadata[SpatialKey](layoutScheme)
[error]
In this line
    val (_: Int, metadata: TileLayerMetadata[SpatialKey]) =
      rdd.collectMetadata[SpatialKey](layoutScheme)
Grigory
@pomadchin
hey @stg101 yea, but it depends on what do you want: you can stitch TileLayerRDD into a single raster and persist as a tiff (but it would trigger the entire rdd reduce on a driver) or you can group it by and save by chunks