GeoTrellis is a geographic data processing engine for high performance applications.
Hi I want to read the RDD by time. This is my code
val attributeStore = FileAttributeStore("/Users/xugaofeng/tif/LC08TIME/123")
val layerReader = FileLayerReader(attributeStore)
val queryResult: TileLayerRDD[SpaceTimeKey] = layerReader.query[SpaceTimeKey, Tile, TileLayerMetadata[SpaceTimeKey]](LayerId("TESTTIME", 2))
.where(Between(ZonedDateTime.of(2018, 12, 27, 7, 0, 0, 0, ZoneOffset.UTC), ZonedDateTime.of(2018, 12, 28, 9, 0, 0, 0, ZoneOffset.UTC)))
.result
val maximumTemperature: RDD[(SpatialKey, Tile)] = queryResult.map { case (key, tile) => (key.getComponent[SpatialKey], tile) }.reduceByKey(_.localMax(_))
val tile = maximumTemperature.stitch()
val png = tile.renderPng(colorMapPm25)
png.write("/Users/xugaofeng/tif/LC08TIME/123/time.png")Error message
Exception in thread "dispatcher-event-loop-1" java.lang.StackOverflowError
at com.esotericsoftware.kryo.util.ObjectMap.get(ObjectMap.java:351)
at com.esotericsoftware.kryo.util.DefaultClassResolver.getRegistration(DefaultClassResolver.java:79)
at com.esotericsoftware.kryo.Kryo.getRegistration(Kryo.java:488)
at com.esotericsoftware.kryo.util.DefaultClassResolver.writeClass(DefaultClassResolver.java:97)
at com.esotericsoftware.kryo.Kryo.writeClass(Kryo.java:540)
at com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:75)
at com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:508)
at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:651)
at com.twitter.chill.Tuple2Serializer.write(TupleSerializers.scala:36)
at com.twitter.chill.Tuple2Serializer.write(TupleSerializers.scala:33)
at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:575)
is it the full stack trace?
you can save them as chips without stitching
This is my code
val tile = value
.where(Intersects(extent.center))
.where(Between(ZonedDateTime.of(2017, 12, 27, 7, 0, 0, 0, ZoneOffset.UTC), ZonedDateTime.of(2021, 12, 28, 9, 0, 0, 0, ZoneOffset.UTC)))
.result.sortByKey(false).map { case (key, tile) => (key.getComponent[SpatialKey], tile) }.reduceByKey(_.merge(_)).stitch()
val png = tile.renderPng()
png
@pomadchin I want to generate png in real time and display it on the map
and if you have the TMS map than each key would correspond to a tile on a map
if you’re sure that the result will always be small you can use the
CollectionLayerReader for the temporal query to make it faster; the API would be the same but you wont need Spark
😅
😅
looks pretty much normal to me
I guess to get some support from me you’d need to share 1. the source, 2. the set of operations you’re doing and 3. the result you’re gettings vs expecting
Hey! @pomadchin. How to use the Branch in https://github.com/geotrellis/maml/blob/develop/shared/src/main/scala/ast/Expression.scala#L165. I think it hard to understand
what example do you need btw? and example of a json or the case classes usage?
and what do you want to do there? mb I can help basing on your question
Hello! How could I
reproject a MultibandTileLayerRDD? When trying geotrellis-landsat-tutorial, I simply migrate to a newer "org.locationtech.geotrellis" %% "geotrellis-spark" % "3.5.2", however it may be incompatible to reproject a RasterRDD.
And how to read from a
pyramid type datasat using "pipeline json string"?
Here is my ingest code, it is for a multiband geotiff. It works, but how to
reproject a MultibandTileLayerRDD?def run(implicit sc: SparkContext): Unit = {
val inputRdd: RDD[RasterSource] = {
sc.parallelize(inputPath, inputPath.length)
.map(uri => RasterSource(uri))
.cache()
}
val layoutScheme = ZoomedLayoutScheme(WebMercator, tileSize = 256)
val summary = RasterSummary.fromRDD(inputRdd)
val LayoutLevel(zoom, layout) = summary.levelFor(layoutScheme)
val tiled:MultibandTileLayerRDD[SpatialKey] = RasterSourceRDD
.tiledLayerRDD(inputRdd, layout)
// We need to reproject the tiles to WebMercator
// TileRDDReproject(tiled, WebMercator, layoutScheme, Some(Bilinear))
val attributeStore = FileAttributeStore(outputPath)
val writer = FileLayerWriter(attributeStore)
Pyramid.fromLayerRDD(tiled, Some(zoom), Some(0)).write("landsat",
writer,
ZCurveKeyIndexMethod)
}Hey! Didn’t have a chance to look into your question but will do so shortly
@pomadchin Thx! Would you like to conclude all the ways to ingest/output geotiffs and do ETL things? According to my exploring on the internet, there may have three ways like 1) RasterLayerRDD func 2) MAML 3) pipeline DSL. Where could I find a new DEMO with GT3.x on some web attributes?
hey @stjimreal;
we wanted to avoid terms ETL and to provide a good syntax so it is easy to build own pipelines.
RasterSources syntax and DSL is some sort of a nice replacement to the ETL.
However, Pipeline project is the closes to the old ETL one.
MAML is about lazy raster and rdds transformations
GT server uses GT 3.6 https://github.com/geotrellis/geotrellis-server/tree/main/stac-example
good afternoon my friends
I'm instantiating Geotiff*RasterSources with an s3:// URI; that used to work fine but now I have to support reading geotiffs in buckets in different regions at the same time. How do I tell the underlying S3Client with which region it should communicate?
I'm instantiating Geotiff*RasterSources with an s3:// URI; that used to work fine but now I have to support reading geotiffs in buckets in different regions at the same time. How do I tell the underlying S3Client with which region it should communicate?