Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • May 12 18:20
    axb21 commented #3464
  • May 12 18:16

    pomadchin on master

    Expose Charset in a ShapeFileRe… (compare)

  • May 12 18:16
    pomadchin closed #3464
  • May 12 18:16
    pomadchin closed #3445
  • May 12 17:30
    vlulla commented #3464
  • May 12 17:13
    pomadchin edited #3464
  • May 12 17:09
    pomadchin edited #3464
  • May 12 17:09
    pomadchin synchronize #3464
  • May 12 17:03
    vlulla synchronize #3464
  • May 12 16:33
    pomadchin edited #3464
  • May 12 15:36
    vlulla review_requested #3464
  • May 12 13:25
    vlulla commented #3464
  • May 12 13:22
    vlulla synchronize #3464
  • May 11 15:21
    pomadchin synchronize #3464
  • May 11 15:18
    vlulla commented #3464
  • May 11 15:16
    vlulla synchronize #3464
  • May 11 15:12
    pomadchin edited #3464
  • May 11 15:12
    pomadchin edited #3464
  • May 11 15:12
    pomadchin edited #3464
  • May 11 15:10
    pomadchin commented #3464
Grigory
@pomadchin
:+1: good plan
btw if you will endup looking into it more dont heistate to drop stacktraces
I’m also interested in the native image so mb we can craft smth together :rocket:
hey @GofferdoXu
$ sbt
$ project ogc-example
$ run http://localhost:port/
@Max-AR oooooh one thing
an issue can be related to jts
Max
@Max-AR

raster and vector packages only; and without SPI usage

Sounds interesting, maybe I can cut down the geotrellis deps a bit more and do a minimal install

Grigory
@pomadchin
:shrug: still worth trying you don’t really need any reflection hopefully

raster and vector packages only; and without SPI usage

Sounds interesting, maybe I can cut down the geotrellis deps a bit more and do a minimal install

makes sense +

nice nested quotes
:D
Max
@Max-AR
:D
Grigory
@pomadchin
how deep we can go?
Max
@Max-AR

raster and vector packages only; and without SPI usage

Sounds interesting, maybe I can cut down the geotrellis deps a bit more and do a minimal install

makes sense +
nice nested quotes
:D

A long way haha

Thanks again for the gudience, I'll let you know how i go
Grigory
@pomadchin
leo.jpg

Thanks again for the gudience, I'll let you know how i go

:+1: :rocket:

James Hughes
@jnh5y
@pomadchin what issues do you think JTS would cause with GraalVM?
Grigory
@pomadchin
@jnh5y not with GraalVM but with its Native Image https://www.graalvm.org/reference-manual/native-image/Limitations/
I think JTS uses some reflection; but mb it will 'just work'
Mr.Gordon
@GofferdoXu

Hi I want to read the RDD by time. This is my code

val attributeStore = FileAttributeStore("/Users/xugaofeng/tif/LC08TIME/123")
    val layerReader = FileLayerReader(attributeStore)
    val queryResult: TileLayerRDD[SpaceTimeKey] = layerReader.query[SpaceTimeKey, Tile, TileLayerMetadata[SpaceTimeKey]](LayerId("TESTTIME", 2))
      .where(Between(ZonedDateTime.of(2018, 12, 27, 7, 0, 0, 0, ZoneOffset.UTC), ZonedDateTime.of(2018, 12, 28, 9, 0, 0, 0, ZoneOffset.UTC)))
      .result
    val maximumTemperature: RDD[(SpatialKey, Tile)] = queryResult.map { case (key, tile) => (key.getComponent[SpatialKey], tile) }.reduceByKey(_.localMax(_))
    val tile = maximumTemperature.stitch()
    val png = tile.renderPng(colorMapPm25)
    png.write("/Users/xugaofeng/tif/LC08TIME/123/time.png")

Error message

Exception in thread "dispatcher-event-loop-1" java.lang.StackOverflowError
    at com.esotericsoftware.kryo.util.ObjectMap.get(ObjectMap.java:351)
    at com.esotericsoftware.kryo.util.DefaultClassResolver.getRegistration(DefaultClassResolver.java:79)
    at com.esotericsoftware.kryo.Kryo.getRegistration(Kryo.java:488)
    at com.esotericsoftware.kryo.util.DefaultClassResolver.writeClass(DefaultClassResolver.java:97)
    at com.esotericsoftware.kryo.Kryo.writeClass(Kryo.java:540)
    at com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:75)
    at com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:508)
    at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:651)
    at com.twitter.chill.Tuple2Serializer.write(TupleSerializers.scala:36)
    at com.twitter.chill.Tuple2Serializer.write(TupleSerializers.scala:33)
    at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:575)
Grigory
@pomadchin
hey @GofferdoXu what GT version is that and what spark version?
is it the full stack trace?
Mr.Gordon
@GofferdoXu
@pomadchin It seems to be a problem with the temporal_resolution parameter. The parameter value is changed to 86400000, and the error will no longer occur.
Grigory
@pomadchin
Oh mb that was some runtime exception which has not been properly handled by Spark and exceptions are non serializable
Mr.Gordon
@GofferdoXu
Hey @pomadchin How to convert RDD to Tile? I know stitch at the moment.
Grigory
@pomadchin

@GofferdoXu depends on what you need

stitch stitches everything into a single raster on a driver => a pretty expensive operation

Mr.Gordon
@GofferdoXu
Is there any other way? @pomadchin
Grigory
@pomadchin
@GofferdoXu what way you need?
you can save them as chips without stitching
Mr.Gordon
@GofferdoXu
This is my code
val tile = value
      .where(Intersects(extent.center))
      .where(Between(ZonedDateTime.of(2017, 12, 27, 7, 0, 0, 0, ZoneOffset.UTC), ZonedDateTime.of(2021, 12, 28, 9, 0, 0, 0, ZoneOffset.UTC)))
      .result.sortByKey(false).map { case (key, tile) => (key.getComponent[SpatialKey], tile) }.reduceByKey(_.merge(_)).stitch()
    val png = tile.renderPng()
    png
@pomadchin I want to generate png in real time and display it on the map
Grigory
@pomadchin
so usually; you need to use Value reader in this case
and if you have the TMS map than each key would correspond to a tile on a map
Mr.Gordon
@GofferdoXu
But I want to perform a query operation😅
Grigory
@pomadchin
in this case that is the only way
if you’re sure that the result will always be small you can use the CollectionLayerReader for the temporal query to make it faster; the API would be the same but you wont need Spark
Mr.Gordon
@GofferdoXu
@pomadchin Thank you.
Grigory
@pomadchin
@GofferdoXu :+1:
Mr.Gordon
@GofferdoXu
Hey @pomadchin I still have a question. I use the pipeline, and the picture that comes out looks like this, and it doesn’t look normal.
image.png
😅
Grigory
@pomadchin
hey @GofferdoXu what do you mean by “doesnt look normal” ?
how should it look like?
Mr.Gordon
@GofferdoXu
@pomadchin I think it should be a remote sensing image, not such a dotted image
😅
Grigory
@pomadchin
not a ‘dotted' image?
looks pretty much normal to me
I guess to get some support from me you’d need to share 1. the source, 2. the set of operations you’re doing and 3. the result you’re gettings vs expecting