Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Jun 15 22:29
    pomadchin unassigned #3388
  • Jun 15 07:10
    qw845602 commented #3465
  • Jun 15 06:49
    qw845602 commented #3465
  • Jun 15 06:47
    qw845602 commented #3465
  • Jun 02 01:04

    pomadchin on npm_and_yarn

    (compare)

  • Jun 02 01:04

    pomadchin on master

    Bump eventsource from 1.0.7 to … (compare)

  • Jun 02 01:04
    pomadchin closed #3466
  • Jun 01 20:52
    dependabot[bot] labeled #3466
  • Jun 01 20:52
    dependabot[bot] opened #3466
  • Jun 01 20:52

    dependabot[bot] on npm_and_yarn

    Bump eventsource from 1.0.7 to … (compare)

  • May 27 12:47
    pomadchin synchronize #3435
  • May 22 01:34
    pomadchin commented #3465
  • May 22 01:33
    pomadchin labeled #3465
  • May 21 16:17
    pomadchin commented #3465
  • May 21 16:10
    qw845602 commented #3465
  • May 21 16:07
    qw845602 commented #3465
  • May 21 15:59
    qw845602 commented #3465
  • May 21 15:57
    qw845602 commented #3465
  • May 21 15:56
    qw845602 commented #3465
  • May 21 15:56
    pomadchin commented #3465
Grigory
@pomadchin
ava.util.NoSuchElementException: Either.right.get on Left
    at scala.util.Either$RightProjection.get(Either.scala:640)
    at geotrellis.server.example.ndvi.NdviService.$anonfun$redQueryParamDecoder$1(NdviService.scala:55)
    at cats.data.Validated.map(Validated.scala:559)
    at org.http4s.QueryParamDecoder$$anon$7.decode(QueryParam.scala:205)
    at org.http4s.dsl.impl.QueryParamDecoderMatcher.$anonfun$unapply$3(Path.scala:327)
    at scala.Option.flatMap(Option.scala:271)
    at org.http4s.dsl.impl.QueryParamDecoderMatcher.unapply(Path.scala:327)
    at geotrellis.server.example.ndvi.NdviService$$anonfun$routes$1.applyOrElse(NdviService.scala:75)
    at geotrellis.server.example.ndvi.NdviService$$anonfun$routes$1.applyOrElse(NdviService.scala:73)
    at scala.PartialFunction$Lifted.apply(PartialFunction.scala:228)
    at scala.PartialFunction$Lifted.apply(PartialFunction.scala:224)
    at org.http4s.HttpRoutes$.$anonfun$of$2(HttpRoutes.scala:79)
    at $anonfun$combineK$1 @ org.http4s.syntax.KleisliResponseOps.$anonfun$orNotFound$1(KleisliSyntax.scala:49)
    at getOrElse @ org.http4s.syntax.KleisliResponseOps.$anonfun$orNotFound$1(KleisliSyntax.scala:49)
    at main$ @ geotrellis.server.example.ndvi.NdviServer$.main(NdviServer.scala:33)
btw aside of having an exception here; you have smth with your deps / jvm classpath:
KaTeX parse error: Can't use function '$' in math mode at position 5
that is not smth we bring into GT; that is smth on your host
It expects JSONs as inputs into arguments
not links to jsons
but be careful with percent ecndoing; passing jsons as arguments is not the best design decision :D
you can hijack the code and pass it inside manually for test purposes
Oh I see
Grigory
@pomadchin
have you resolved it?
yang162132
@yang162132
No,I'm trying
Grigory
@pomadchin
@yang162132 gotcha in the worst case hardcode it in the code - it is a demo app anyway
Jeroen Dries
@jdries
FYI, last week at Phi week in Rome, we released an early adopter release of a new ESA platform for EO data processing. One of the backends is built entirely with GeoTrellis (https://github.com/Open-EO/openeo-geotrellis-extensions/).
We're also giving away 3-month accounts to interested beta testers: https://openeo.cloud/early-adopters/
Grigory
@pomadchin
@jdries :tada:
Mr.Gordon
@GofferdoXu
Hey @pomadchin I want to run the ogc-example project, https://github.com/geotrellis/geotrellis-server/blob/main/ogc-example,How to write url?
Max
@Max-AR
@pomadchin Using graalvm in JIT mode (not a native binary) gave about a 7-10% boost in perf, but I wasn't able to figure out how to get a native compile working to really test the differences between java byte code and graalvms binary. Do you think it would be possible to do a native-image compile of geotrellis? (not using spark that is)
Grigory
@pomadchin
Hey @Max-AR I think I mentioned somehwere above, I don’t think it is possible right now to build native image: spark has complicated dependencies and java reflection calls, uses SPI etc
the same is applicable to some GeoTrellis dependencies (like geotools) and almost every GeoTrellis package: cassandra backend, raster package - uses SPI, etc
but what errors do you have?
I think i’ve seen before a docker image with Spark
Max
@Max-AR
Yeah do you did, that is all good. Basically just having problems finding libraries at runtime. I didn't dig too much into it cause it felt like a complete rabbit hole of complications haha, I'd need to be across the geotrellis codebase a lot better to resolve those problems, i think.
Grigory
@pomadchin
Yea; I think it is impossible
the whole point of the native image is to preload classes at the build time
Spark relies on the dynamic classload
Max
@Max-AR
Yeah exactly. I was hoping that since spark/geotrellis-spark isn't a dependency in the project I am working on right now i coud skirt the issue :D
But all good
Grigory
@pomadchin
well you can try the raster and vector packages only; and without SPI usage
:+1: good plan
btw if you will endup looking into it more dont heistate to drop stacktraces
I’m also interested in the native image so mb we can craft smth together :rocket:
hey @GofferdoXu
$ sbt
$ project ogc-example
$ run http://localhost:port/
@Max-AR oooooh one thing
an issue can be related to jts
Max
@Max-AR

raster and vector packages only; and without SPI usage

Sounds interesting, maybe I can cut down the geotrellis deps a bit more and do a minimal install

Grigory
@pomadchin
:shrug: still worth trying you don’t really need any reflection hopefully

raster and vector packages only; and without SPI usage

Sounds interesting, maybe I can cut down the geotrellis deps a bit more and do a minimal install

makes sense +

nice nested quotes
:D
Max
@Max-AR
:D
Grigory
@pomadchin
how deep we can go?
Max
@Max-AR

raster and vector packages only; and without SPI usage

Sounds interesting, maybe I can cut down the geotrellis deps a bit more and do a minimal install

makes sense +
nice nested quotes
:D

A long way haha

Thanks again for the gudience, I'll let you know how i go
Grigory
@pomadchin
leo.jpg

Thanks again for the gudience, I'll let you know how i go

:+1: :rocket:

James Hughes
@jnh5y
@pomadchin what issues do you think JTS would cause with GraalVM?
Grigory
@pomadchin
@jnh5y not with GraalVM but with its Native Image https://www.graalvm.org/reference-manual/native-image/Limitations/
I think JTS uses some reflection; but mb it will 'just work'
Mr.Gordon
@GofferdoXu

Hi I want to read the RDD by time. This is my code

val attributeStore = FileAttributeStore("/Users/xugaofeng/tif/LC08TIME/123")
    val layerReader = FileLayerReader(attributeStore)
    val queryResult: TileLayerRDD[SpaceTimeKey] = layerReader.query[SpaceTimeKey, Tile, TileLayerMetadata[SpaceTimeKey]](LayerId("TESTTIME", 2))
      .where(Between(ZonedDateTime.of(2018, 12, 27, 7, 0, 0, 0, ZoneOffset.UTC), ZonedDateTime.of(2018, 12, 28, 9, 0, 0, 0, ZoneOffset.UTC)))
      .result
    val maximumTemperature: RDD[(SpatialKey, Tile)] = queryResult.map { case (key, tile) => (key.getComponent[SpatialKey], tile) }.reduceByKey(_.localMax(_))
    val tile = maximumTemperature.stitch()
    val png = tile.renderPng(colorMapPm25)
    png.write("/Users/xugaofeng/tif/LC08TIME/123/time.png")

Error message

Exception in thread "dispatcher-event-loop-1" java.lang.StackOverflowError
    at com.esotericsoftware.kryo.util.ObjectMap.get(ObjectMap.java:351)
    at com.esotericsoftware.kryo.util.DefaultClassResolver.getRegistration(DefaultClassResolver.java:79)
    at com.esotericsoftware.kryo.Kryo.getRegistration(Kryo.java:488)
    at com.esotericsoftware.kryo.util.DefaultClassResolver.writeClass(DefaultClassResolver.java:97)
    at com.esotericsoftware.kryo.Kryo.writeClass(Kryo.java:540)
    at com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:75)
    at com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:508)
    at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:651)
    at com.twitter.chill.Tuple2Serializer.write(TupleSerializers.scala:36)
    at com.twitter.chill.Tuple2Serializer.write(TupleSerializers.scala:33)
    at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:575)
Grigory
@pomadchin
hey @GofferdoXu what GT version is that and what spark version?
is it the full stack trace?
Mr.Gordon
@GofferdoXu
@pomadchin It seems to be a problem with the temporal_resolution parameter. The parameter value is changed to 86400000, and the error will no longer occur.
Grigory
@pomadchin
Oh mb that was some runtime exception which has not been properly handled by Spark and exceptions are non serializable