Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Jun 15 22:29
    pomadchin unassigned #3388
  • Jun 15 07:10
    qw845602 commented #3465
  • Jun 15 06:49
    qw845602 commented #3465
  • Jun 15 06:47
    qw845602 commented #3465
  • Jun 02 01:04

    pomadchin on npm_and_yarn

    (compare)

  • Jun 02 01:04

    pomadchin on master

    Bump eventsource from 1.0.7 to … (compare)

  • Jun 02 01:04
    pomadchin closed #3466
  • Jun 01 20:52
    dependabot[bot] labeled #3466
  • Jun 01 20:52
    dependabot[bot] opened #3466
  • Jun 01 20:52

    dependabot[bot] on npm_and_yarn

    Bump eventsource from 1.0.7 to … (compare)

  • May 27 12:47
    pomadchin synchronize #3435
  • May 22 01:34
    pomadchin commented #3465
  • May 22 01:33
    pomadchin labeled #3465
  • May 21 16:17
    pomadchin commented #3465
  • May 21 16:10
    qw845602 commented #3465
  • May 21 16:07
    qw845602 commented #3465
  • May 21 15:59
    qw845602 commented #3465
  • May 21 15:57
    qw845602 commented #3465
  • May 21 15:56
    qw845602 commented #3465
  • May 21 15:56
    pomadchin commented #3465
Jeroen Dries
@jdries
FYI, last week at Phi week in Rome, we released an early adopter release of a new ESA platform for EO data processing. One of the backends is built entirely with GeoTrellis (https://github.com/Open-EO/openeo-geotrellis-extensions/).
We're also giving away 3-month accounts to interested beta testers: https://openeo.cloud/early-adopters/
Grigory
@pomadchin
@jdries :tada:
Mr.Gordon
@GofferdoXu
Hey @pomadchin I want to run the ogc-example project, https://github.com/geotrellis/geotrellis-server/blob/main/ogc-example,How to write url?
Max
@Max-AR
@pomadchin Using graalvm in JIT mode (not a native binary) gave about a 7-10% boost in perf, but I wasn't able to figure out how to get a native compile working to really test the differences between java byte code and graalvms binary. Do you think it would be possible to do a native-image compile of geotrellis? (not using spark that is)
Grigory
@pomadchin
Hey @Max-AR I think I mentioned somehwere above, I don’t think it is possible right now to build native image: spark has complicated dependencies and java reflection calls, uses SPI etc
the same is applicable to some GeoTrellis dependencies (like geotools) and almost every GeoTrellis package: cassandra backend, raster package - uses SPI, etc
but what errors do you have?
I think i’ve seen before a docker image with Spark
Max
@Max-AR
Yeah do you did, that is all good. Basically just having problems finding libraries at runtime. I didn't dig too much into it cause it felt like a complete rabbit hole of complications haha, I'd need to be across the geotrellis codebase a lot better to resolve those problems, i think.
Grigory
@pomadchin
Yea; I think it is impossible
the whole point of the native image is to preload classes at the build time
Spark relies on the dynamic classload
Max
@Max-AR
Yeah exactly. I was hoping that since spark/geotrellis-spark isn't a dependency in the project I am working on right now i coud skirt the issue :D
But all good
Grigory
@pomadchin
well you can try the raster and vector packages only; and without SPI usage
:+1: good plan
btw if you will endup looking into it more dont heistate to drop stacktraces
I’m also interested in the native image so mb we can craft smth together :rocket:
hey @GofferdoXu
$ sbt
$ project ogc-example
$ run http://localhost:port/
@Max-AR oooooh one thing
an issue can be related to jts
Max
@Max-AR

raster and vector packages only; and without SPI usage

Sounds interesting, maybe I can cut down the geotrellis deps a bit more and do a minimal install

Grigory
@pomadchin
:shrug: still worth trying you don’t really need any reflection hopefully

raster and vector packages only; and without SPI usage

Sounds interesting, maybe I can cut down the geotrellis deps a bit more and do a minimal install

makes sense +

nice nested quotes
:D
Max
@Max-AR
:D
Grigory
@pomadchin
how deep we can go?
Max
@Max-AR

raster and vector packages only; and without SPI usage

Sounds interesting, maybe I can cut down the geotrellis deps a bit more and do a minimal install

makes sense +
nice nested quotes
:D

A long way haha

Thanks again for the gudience, I'll let you know how i go
Grigory
@pomadchin
leo.jpg

Thanks again for the gudience, I'll let you know how i go

:+1: :rocket:

James Hughes
@jnh5y
@pomadchin what issues do you think JTS would cause with GraalVM?
Grigory
@pomadchin
@jnh5y not with GraalVM but with its Native Image https://www.graalvm.org/reference-manual/native-image/Limitations/
I think JTS uses some reflection; but mb it will 'just work'
Mr.Gordon
@GofferdoXu

Hi I want to read the RDD by time. This is my code

val attributeStore = FileAttributeStore("/Users/xugaofeng/tif/LC08TIME/123")
    val layerReader = FileLayerReader(attributeStore)
    val queryResult: TileLayerRDD[SpaceTimeKey] = layerReader.query[SpaceTimeKey, Tile, TileLayerMetadata[SpaceTimeKey]](LayerId("TESTTIME", 2))
      .where(Between(ZonedDateTime.of(2018, 12, 27, 7, 0, 0, 0, ZoneOffset.UTC), ZonedDateTime.of(2018, 12, 28, 9, 0, 0, 0, ZoneOffset.UTC)))
      .result
    val maximumTemperature: RDD[(SpatialKey, Tile)] = queryResult.map { case (key, tile) => (key.getComponent[SpatialKey], tile) }.reduceByKey(_.localMax(_))
    val tile = maximumTemperature.stitch()
    val png = tile.renderPng(colorMapPm25)
    png.write("/Users/xugaofeng/tif/LC08TIME/123/time.png")

Error message

Exception in thread "dispatcher-event-loop-1" java.lang.StackOverflowError
    at com.esotericsoftware.kryo.util.ObjectMap.get(ObjectMap.java:351)
    at com.esotericsoftware.kryo.util.DefaultClassResolver.getRegistration(DefaultClassResolver.java:79)
    at com.esotericsoftware.kryo.Kryo.getRegistration(Kryo.java:488)
    at com.esotericsoftware.kryo.util.DefaultClassResolver.writeClass(DefaultClassResolver.java:97)
    at com.esotericsoftware.kryo.Kryo.writeClass(Kryo.java:540)
    at com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:75)
    at com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:508)
    at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:651)
    at com.twitter.chill.Tuple2Serializer.write(TupleSerializers.scala:36)
    at com.twitter.chill.Tuple2Serializer.write(TupleSerializers.scala:33)
    at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:575)
Grigory
@pomadchin
hey @GofferdoXu what GT version is that and what spark version?
is it the full stack trace?
Mr.Gordon
@GofferdoXu
@pomadchin It seems to be a problem with the temporal_resolution parameter. The parameter value is changed to 86400000, and the error will no longer occur.
Grigory
@pomadchin
Oh mb that was some runtime exception which has not been properly handled by Spark and exceptions are non serializable
Mr.Gordon
@GofferdoXu
Hey @pomadchin How to convert RDD to Tile? I know stitch at the moment.
Grigory
@pomadchin

@GofferdoXu depends on what you need

stitch stitches everything into a single raster on a driver => a pretty expensive operation

Mr.Gordon
@GofferdoXu
Is there any other way? @pomadchin
Grigory
@pomadchin
@GofferdoXu what way you need?
you can save them as chips without stitching
Mr.Gordon
@GofferdoXu
This is my code
val tile = value
      .where(Intersects(extent.center))
      .where(Between(ZonedDateTime.of(2017, 12, 27, 7, 0, 0, 0, ZoneOffset.UTC), ZonedDateTime.of(2021, 12, 28, 9, 0, 0, 0, ZoneOffset.UTC)))
      .result.sortByKey(false).map { case (key, tile) => (key.getComponent[SpatialKey], tile) }.reduceByKey(_.merge(_)).stitch()
    val png = tile.renderPng()
    png
@pomadchin I want to generate png in real time and display it on the map
Grigory
@pomadchin
so usually; you need to use Value reader in this case
and if you have the TMS map than each key would correspond to a tile on a map
Mr.Gordon
@GofferdoXu
But I want to perform a query operation😅
Grigory
@pomadchin
in this case that is the only way