geotrellis/geotrellis

GeoTrellis is a geographic data processing engine for high performance applications.

Max

@Max-AR

Has anyone sucessfully used graalvm with geotrellis?

Grigory

@pomadchin

Hey @Max I actually haven’t tried but may work; what issues do you have when using it as a runtime?

* not talking about native image

Max

@Max-AR

I haven'

t tried at the moment, I was thinking of ways to eek out more performance. In the project I am working right now I have some libs using reflection based DI that need to be removed first

If I end up getting around to it I'll post back here for sure

Grigory

@pomadchin

@Max you’re probably talking about native image - yea this one would not work; but you may try graalvm; there are articles in the wild that demonstrate some benefits of Spark + Graal

yang162132

@yang162132

Hey@pomadchin I'm trying the example in https://github.com/geotrellis/geotrellis-server,my url is
http://0.0.0.0:9000/0/1/1.png?red=http://127.0.0.1:1234/tiff/1.json&nir=http://127.0.0.1:1234/tiff/2.json and the json is {"uri":"http://127.0.0.1:1234/tiff/1/LC08_L1TP_015032_20180521_20180605_01_T1_B4.tif","band": 4,"celltype":null}
but it has a wrong -java.util.NoSuchElementException: Either.right.get on Left

klmc66666

@klmc66666

@pomadchin geotrellis is support read and write webp format?

Grigory

@pomadchin

Hey @yang162132 could you drop here the stack trace?

hello @klmc66666 only reading and only via GDAL

yang162132

@yang162132

java.util.NoSuchElementException: Either.right.get on Left
at scala.util.Either$RightProjection.get(Either.scala:640)
at geotrellis.server.example.ndvi.NdviService.$anonfun$redQueryParamDecoder$1(NdviService.scala:55)
at cats.data.Validated.map(Validated.scala:559)
at org.http4s.QueryParamDecoder

KaTeX parse error: Can't use function '$' in math mode at position 5: anon$̲7.decode(QueryP…: anon$7.decode(QueryParam.scala:205)
    at org.http4s.dsl.impl.QueryParamDecoderMatcher.$anonfun$unapply$3(Path.scala:327)
    at scala.Option.flatMap(Option.scala:271)
    at org.http4s.dsl.impl.QueryParamDecoderMatcher.unapply(Path.scala:327)
    at geotrellis.server.example.ndvi.NdviService

anonfun$routes$1.applyOrElse(NdviService.scala:75)
at geotrellis.server.example.ndvi.NdviService$$anonfun$routes$1.applyOrElse(NdviService.scala:73)
at scala.PartialFunction$Lifted.apply(PartialFunction.scala:228)
at scala.PartialFunction$Lifted.apply(PartialFunction.scala:224)
at org.http4s.HttpRoutes$.$anonfun$of$2(HttpRoutes.scala:79)
at $anonfun$combineK$1 @ org.http4s.syntax.KleisliResponseOps.$anonfun$orNotFound$1(KleisliSyntax.scala:49)
at getOrElse @ org.http4s.syntax.KleisliResponseOps.$anonfun$orNotFound$1(KleisliSyntax.scala:49)
at main$ @ geotrellis.server.example.ndvi.NdviServer$.main(NdviServer.scala:33)

@pomadchin

Grigory

@pomadchin

@yang162132 oh I see, the incorrect format of inputs

yang162132

@yang162132

Oh, the uri or json ?

Grigory

@pomadchin

I think it expects JSONS as an input

yea indeed also getting it

ava.util.NoSuchElementException: Either.right.get on Left
    at scala.util.Either$RightProjection.get(Either.scala:640)
    at geotrellis.server.example.ndvi.NdviService.$anonfun$redQueryParamDecoder$1(NdviService.scala:55)
    at cats.data.Validated.map(Validated.scala:559)
    at org.http4s.QueryParamDecoder$$anon$7.decode(QueryParam.scala:205)
    at org.http4s.dsl.impl.QueryParamDecoderMatcher.$anonfun$unapply$3(Path.scala:327)
    at scala.Option.flatMap(Option.scala:271)
    at org.http4s.dsl.impl.QueryParamDecoderMatcher.unapply(Path.scala:327)
    at geotrellis.server.example.ndvi.NdviService$$anonfun$routes$1.applyOrElse(NdviService.scala:75)
    at geotrellis.server.example.ndvi.NdviService$$anonfun$routes$1.applyOrElse(NdviService.scala:73)
    at scala.PartialFunction$Lifted.apply(PartialFunction.scala:228)
    at scala.PartialFunction$Lifted.apply(PartialFunction.scala:224)
    at org.http4s.HttpRoutes$.$anonfun$of$2(HttpRoutes.scala:79)
    at $anonfun$combineK$1 @ org.http4s.syntax.KleisliResponseOps.$anonfun$orNotFound$1(KleisliSyntax.scala:49)
    at getOrElse @ org.http4s.syntax.KleisliResponseOps.$anonfun$orNotFound$1(KleisliSyntax.scala:49)
    at main$ @ geotrellis.server.example.ndvi.NdviServer$.main(NdviServer.scala:33)

btw aside of having an exception here; you have smth with your deps / jvm classpath:

KaTeX parse error: Can't use function '$' in math mode at position 5

that is not smth we bring into GT; that is smth on your host

It expects JSONs as inputs into arguments

not links to jsons

Grigory

@pomadchin

but be careful with percent ecndoing; passing jsons as arguments is not the best design decision :D

you can hijack the code and pass it inside manually for test purposes

yang162132

@yang162132

but there nothing happende when i change the uri to http://0.0.0.0:9000/0/1/1.png?red={'uri':'http://127.0.0.1:1234/tiff/1/LC08_L1TP_015032_20180521_20180605_01_T1_B4.tif','band':4,'celltype':null}&nir={'uri':'http://127.0.0.1:1234/tiff/1/LC08_L1TP_015032_20180521_20180605_01_T1_B5.tif','band':4,'celltype':null} @pomadchin

Oh I see

Grigory

@pomadchin

have you resolved it?

yang162132

@yang162132

No，I'm trying

Grigory

@pomadchin

@yang162132 gotcha in the worst case hardcode it in the code - it is a demo app anyway

Jeroen Dries

@jdries

FYI, last week at Phi week in Rome, we released an early adopter release of a new ESA platform for EO data processing. One of the backends is built entirely with GeoTrellis (https://github.com/Open-EO/openeo-geotrellis-extensions/).
We're also giving away 3-month accounts to interested beta testers: https://openeo.cloud/early-adopters/

Grigory

@pomadchin

@jdries :tada:

Mr.Gordon

@GofferdoXu

Hey @pomadchin I want to run the ogc-example project, https://github.com/geotrellis/geotrellis-server/blob/main/ogc-example，How to write url？

Max

@Max-AR

@pomadchin Using graalvm in JIT mode (not a native binary) gave about a 7-10% boost in perf, but I wasn't able to figure out how to get a native compile working to really test the differences between java byte code and graalvms binary. Do you think it would be possible to do a native-image compile of geotrellis? (not using spark that is)

Grigory

@pomadchin

Hey @Max-AR I think I mentioned somehwere above, I don’t think it is possible right now to build native image: spark has complicated dependencies and java reflection calls, uses SPI etc
the same is applicable to some GeoTrellis dependencies (like geotools) and almost every GeoTrellis package: cassandra backend, raster package - uses SPI, etc

but what errors do you have?

I think i’ve seen before a docker image with Spark

Max

@Max-AR

Yeah do you did, that is all good. Basically just having problems finding libraries at runtime. I didn't dig too much into it cause it felt like a complete rabbit hole of complications haha, I'd need to be across the geotrellis codebase a lot better to resolve those problems, i think.

Grigory

@pomadchin

Yea; I think it is impossible
the whole point of the native image is to preload classes at the build time

Spark relies on the dynamic classload

Max

@Max-AR

Yeah exactly. I was hoping that since spark/geotrellis-spark isn't a dependency in the project I am working on right now i coud skirt the issue :D

But all good

Grigory

@pomadchin

well you can try the raster and vector packages only; and without SPI usage

:+1: good plan

btw if you will endup looking into it more dont heistate to drop stacktraces

I’m also interested in the native image so mb we can craft smth together :rocket:

hey @GofferdoXu

$ sbt
$ project ogc-example
$ run http://localhost:port/

@Max-AR oooooh one thing

an issue can be related to jts

Max

@Max-AR

raster and vector packages only; and without SPI usage

Sounds interesting, maybe I can cut down the geotrellis deps a bit more and do a minimal install

Grigory

@pomadchin

:shrug: still worth trying you don’t really need any reflection hopefully

Where communities thrive

People

Repo info

Activity