geotrellis/geotrellis

GeoTrellis is a geographic data processing engine for high performance applications.

Grigory

@pomadchin

in this case that is the only way

if you’re sure that the result will always be small you can use the CollectionLayerReader for the temporal query to make it faster; the API would be the same but you wont need Spark

Mr.Gordon

@GofferdoXu

@pomadchin Thank you.

Grigory

@pomadchin

@GofferdoXu :+1:

Mr.Gordon

@GofferdoXu

Hey @pomadchin I still have a question. I use the pipeline, and the picture that comes out looks like this, and it doesn’t look normal.

😅

Grigory

@pomadchin

hey @GofferdoXu what do you mean by “doesnt look normal” ?

how should it look like?

Mr.Gordon

@GofferdoXu

@pomadchin I think it should be a remote sensing image, not such a dotted image

😅

Grigory

@pomadchin

not a ‘dotted' image?

looks pretty much normal to me

I guess to get some support from me you’d need to share 1. the source, 2. the set of operations you’re doing and 3. the result you’re gettings vs expecting

Mr.Gordon

@GofferdoXu

@pomadchin Thank you, let me study it first😅😅😅

wongjimsan

@wongjimsan_gitlab

Add ColorInterp support into GeoTiffOptions and GeoTiffWriter / GeoTiffReader #3020

Grigory

@pomadchin

hey @wongjimsan_gitlab ! that is not on our TODOs list for now; but I will help you in case you’d like to create a PR; and also - have you tried the color space param?

yang162132

@yang162132

Hey! @pomadchin. How to use the Branch in https://github.com/geotrellis/maml/blob/develop/shared/src/main/scala/ast/Expression.scala#L165. I think it hard to understand

Grigory

@pomadchin

hey @yang162132 the first expression in the list should be the condition (expression that returns boolean)

what example do you need btw? and example of a json or the case classes usage?

and what do you want to do there? mb I can help basing on your question

yang162132

@yang162132

@pomadchin Oh Thank you I want to an example of a json to know how it work such as If I want to do addition or multiplication by giving a condition could it achieve？

yang162132

@yang162132

And by the way is there a function to make two layers with a band to one layer with two bands

林海听枫

@stjimreal

Hello! How could I reproject a MultibandTileLayerRDD? When trying geotrellis-landsat-tutorial, I simply migrate to a newer "org.locationtech.geotrellis" %% "geotrellis-spark" % "3.5.2", however it may be incompatible to reproject a RasterRDD.

And how to read from a pyramid type datasat using "pipeline json string"?

Grigory

@pomadchin

hey @stjimreal this tutorial is just very outdated :/ probably imports are messed up

林海听枫

@stjimreal

@pomadchin I did make it to update the APIs to GT3.x. Well, I just connot fetch out the way to reproject a MultibandTileLayerRDD. Is it because the lack of some other packages?

林海听枫

@stjimreal

Here is my ingest code, it is for a multiband geotiff. It works, but how to reproject a MultibandTileLayerRDD?

def run(implicit sc: SparkContext): Unit = {
    val inputRdd: RDD[RasterSource] = {
    sc.parallelize(inputPath, inputPath.length)
      .map(uri => RasterSource(uri))
      .cache()
    }
    val layoutScheme = ZoomedLayoutScheme(WebMercator, tileSize = 256)
    val summary = RasterSummary.fromRDD(inputRdd)
    val LayoutLevel(zoom, layout) = summary.levelFor(layoutScheme)
    val tiled:MultibandTileLayerRDD[SpatialKey] = RasterSourceRDD
        .tiledLayerRDD(inputRdd, layout)
// We need to reproject the tiles to WebMercator
//    TileRDDReproject(tiled, WebMercator, layoutScheme, Some(Bilinear))
    val attributeStore = FileAttributeStore(outputPath)
    val writer = FileLayerWriter(attributeStore)
    Pyramid.fromLayerRDD(tiled, Some(zoom), Some(0)).write("landsat",
      writer,
      ZCurveKeyIndexMethod)
  }

Grigory

@pomadchin

Hey! Didn’t have a chance to look into your question but will do so shortly

林海听枫

@stjimreal

Hey! Didn’t have a chance to look into your question but will do so shortly

@pomadchin Thx! Would you like to conclude all the ways to ingest/output geotiffs and do ETL things? According to my exploring on the internet, there may have three ways like 1) RasterLayerRDD func 2) MAML 3) pipeline DSL. Where could I find a new DEMO with GT3.x on some web attributes?

Grigory

@pomadchin

hey @stjimreal;

we wanted to avoid terms ETL and to provide a good syntax so it is easy to build own pipelines.
RasterSources syntax and DSL is some sort of a nice replacement to the ETL.

However, Pipeline project is the closes to the old ETL one.

MAML is about lazy raster and rdds transformations

GT server uses GT 3.6 https://github.com/geotrellis/geotrellis-server/tree/main/stac-example

Jan Van den bosch

@bossie

good afternoon my friends
I'm instantiating Geotiff*RasterSources with an s3:// URI; that used to work fine but now I have to support reading geotiffs in buckets in different regions at the same time. How do I tell the underlying S3Client with which region it should communicate?

Grigory

@pomadchin

hey @bossie you can configure client via the S3ClientProducer.set method

Jan Van den bosch

@bossie

yes but that singleton is not going to work in this multithreaded application

Grigory

@pomadchin

@bossie oh fair enough hm

well i’d say that you could have different prupose clients in this case for different regions

allocating a client per thread is certainly an option but very slow one

could you mock up some code to demonstrate the API usage and what you want (ideally) to achieve?

Grigory

@pomadchin

hey @chdsb could you clear it up: what value and when is different? you mean after the ingestion you’re reading a chip via the colleciton reader and the min value is different than in the source? If I understood it correctly than it is due to reprojection + resampling + regridding. Since the ingestion was done in your case into the GeoTrellis layers. That is a precision issue

@chdsb I think it just happened so (: it could also not match

Grigory

@pomadchin

if you want a complete match to the source - I would recommend to consider operating with unchanged rasters via the RasterSource API

@chdsb answering the second question: how do you read sources into the RDD at the moment?

chdsb

@chdsb

@pomadchin hadoopGeoTiffRDD

Grigory

@pomadchin

@chdsb okay, and you would like to use smth like sc.hadoopTemporalGeoTiffRDD but to pass time somehow different rather than setting TIFF tags.

You could use HadoopGeoTiffRDD.singleband directly:

Where I = ProjectedExtent and K = TemporalProjectedExtent; the function you’re interested in is uriToKey: (URI, I) => Kyou can get the temporal information from the path (that used to be one of the ways temporal metadata could be encoded) of the tiff and add it to the SpatialComponent

林海听枫

@stjimreal

@pomadchin great! I will check it.

yang162132

@yang162132

Hey!@pomadchin I guess it's a bug in https://github.com/geotrellis/maml/blob/ce9296f678f1699849f344a4807775b8eca54300/shared/src/main/scala/ast/codec/MamlCodecInstances.scala#L61 it should be classifications not classifcations

Grigory

@pomadchin

@yang162132 yea you’re right that’s a bug

a typo

Where communities thrive

People

Repo info

Activity