Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Jan 27 04:52

    dependabot[bot] on npm_and_yarn

    Bump ua-parser-js from 0.7.28 t… (compare)

  • Jan 27 04:52
    dependabot[bot] labeled #3497
  • Jan 27 04:52
    dependabot[bot] opened #3497
  • Jan 25 15:26
    pomadchin commented #3496
  • Jan 25 15:26
    pomadchin labeled #3496
  • Jan 25 15:23
    jdries opened #3496
  • Jan 10 19:31

    pomadchin on master

    SBT Version up (compare)

  • Jan 10 18:09

    pomadchin on master

    fix the ci-release (compare)

  • Jan 10 17:41

    pomadchin on master

    ci-release test (compare)

  • Jan 10 17:34

    pomadchin on master

    skip s3-spark docs (compare)

  • Jan 10 17:18

    pomadchin on master

    rm the traget flag (compare)

  • Jan 10 17:03

    pomadchin on master

    replace the target flag (compare)

  • Jan 10 16:52

    pomadchin on master

    try compile before doc (compare)

  • Jan 10 16:23

    pomadchin on master

    sbt version downgrade (compare)

  • Jan 10 16:16

    pomadchin on master

    revert scala versions back (compare)

  • Jan 10 16:05

    pomadchin on master

    use sbt-extras in a containeriz… (compare)

  • Jan 10 16:02

    pomadchin on master

    use the tests image for the rel… (compare)

  • Jan 10 15:59

    pomadchin on master

    use docker conatiner to publish… (compare)

  • Jan 10 15:47

    pomadchin on master

    try scala 2.13 docs only (compare)

  • Jan 10 15:27

    pomadchin on master

    prebuild sequential s3-spark/do… (compare)

Grigory
@pomadchin
oh no that is a broken gitter mobile app; these were answers to @caoguangshun about issue #3334 :facepalm:
santocp94
@santocp94
Hi everyone! I'm working with GeoTrellis and I need to mosaic different clipped rasters together (atm two, but it could be any number of them) with extents that can be totally disjoint or partially overlapping. I tried the merge function to mosaic two rasters but the result isn't correct because the extent of the output raster is that of the former. I see that there is a utility to compute the smallest extent that contains two extents, but I'm not sure it could be helpful for my use case.
Grigory
@pomadchin

hey @santocp94 I think right now the way would be to allocate the tile that would be of a size of a raster after merge and merge things into it

the merge would be mosaic

I don’t think we have any fancy API around it :/ if you have a keyed collection of raster you could stitch them
santocp94
@santocp94

Hi @pomadchin, I need to merge clipped Tiles so I'm not sure I can use stitch. I tried to write the first solution:

  val firstTile =  // my first tile
  val secondTile = // my second tile
  val firstExtent = // envelope of first tile
  val secondExtent = // envelope of second tile
  val combinedExtent = firstExtent.expandToInclude(secondExtent)
  val newRaster = new Raster(???, combinedExtent) 
  val outputRaster = newRaster.merge(new Raster(firstTile, firstExtent))
.merge(new Raster(secondTile, secondExtent))

Is this what you were proposing? Should I create an empty tile initially for the temporary raster (or working just with tiles without creating rasters)? How can I do it?

Grigory
@pomadchin
@santocp94 yea that’s what im roughly talking about
you got the idea right
merge merges the right raster into the left if they are not set
santocp94
@santocp94
ok, I think I will do it this way. I know it's a bit trivial question but I have some difficulty understanding how to instantiate an empty tile to create the raster with the desired extent
1 reply
Grigory
@pomadchin

hmmm yea I see; the Tile creation indeed could be tricky

so the API is ArrayTile.alloc(cellType, cols, rows)

santocp94
@santocp94
ok, can the number of columns and rows be defined starting from the extent?
Grigory
@pomadchin

@santocp94 I think the easiest thing here is to create a RasterExtent(combinedExtent, firstTile.cellSize) // all tiles should be of the same cellSize in the same resolution

rasterExtent has cols and rows (well you can derive it from the cellSize and the extent)

so you will have smth like:

val combinedExtent = firstExtent.expandToInclude(secondExtent)
val re = RaterExtent(combinedExtent, firstTile.cellSize)

val newRaster = Raster(ArrayTile.alloc(first.cellType, re.cols, re.rows), re.extent) 
val outputRaster = 
  newRaster
    .merge(Raster(firstTile, firstExtent))
    .merge(Raster(secondTile, secondExtent))
Mr.Gordon
@GofferdoXu
How to implement time series?
def createSinglebandTileJson(input: String, output: String, layerName: String): Unit = { val maskJson: String = """ |[ | { | "uri" : "{input}", | "time_tag" : "TIFFTAG_DATETIME", | "time_format" : "yyyy:MM:dd HH:mm:ss", | "type" : "singleband.spatial.read.hadoop" | }, | { | "resample_method" : "nearest-neighbor", | "type" : "singleband.spatial.transform.tile-to-layout" | }, | { | "crs" : "EPSG:3857", | "scheme" : { | "crs" : "epsg:3857", | "tileSize" : 256, | "resolutionThreshold" : 0.1 | }, | "resample_method" : "nearest-neighbor", | "type" : "singleband.spatial.transform.buffered-reproject" | }, | { | "end_zoom" : 0, | "resample_method" : "nearest-neighbor", | "type" : "singleband.spatial.transform.pyramid" | }, | { | "name" : "{layerName}", | "uri" : "{output}", | "key_index_method" : { | "type" : "zorder", | "temporal_resolution": 1 | }, | "scheme" : { | "crs" : "epsg:3857", | "tileSize" : 256, | "resolutionThreshold" : 0.1 | }, | "type" : "singleband.temporal.write" | } |] """.stripMargin val maskJsonStr = maskJson.replace("{input}", input).replace("{output}", output).replace("{layerName}", layerName) val list: Option[Node[Stream[(Int, TileLayerRDD[SpaceTimeKey])]]] = maskJsonStr.node list match { case None => println("Couldn't parse the JSON") case Some(node) => { node.eval.foreach { case (zoom, rdd) => println(s"ZOOM: ${zoom}") println(s"COUNT: ${rdd.count}") } } }
def createSinglebandTileJson(input: String, output: String, layerName: String): Unit = { val maskJson: String = """ |[ | { | "uri" : "{input}", | "time_tag" : "TIFFTAG_DATETIME", | "time_format" : "yyyy:MM:dd HH:mm:ss", | "type" : "singleband.spatial.read.hadoop" | }, | { | "resample_method" : "nearest-neighbor", | "type" : "singleband.spatial.transform.tile-to-layout" | }, | { | "crs" : "EPSG:3857", | "scheme" : { | "crs" : "epsg:3857", | "tileSize" : 256, | "resolutionThreshold" : 0.1 | }, | "resample_method" : "nearest-neighbor", | "type" : "singleband.spatial.transform.buffered-reproject" | }, | { | "end_zoom" : 0, | "resample_method" : "nearest-neighbor", | "type" : "singleband.spatial.transform.pyramid" | }, | { | "name" : "{layerName}", | "uri" : "{output}", | "key_index_method" : { | "type" : "zorder", | "temporal_resolution": 1 | }, | "scheme" : { | "crs" : "epsg:3857", | "tileSize" : 256, | "resolutionThreshold" : 0.1 | }, | "type" : "singleband.temporal.write" | } |] """.stripMargin val maskJsonStr = maskJson.replace("{input}", input).replace("{output}", output).replace("{layerName}", layerName) val list: Option[Node[Stream[(Int, TileLayerRDD[SpaceTimeKey])]]] = maskJsonStr.node list match { case None => println("Couldn't parse the JSON") case Some(node) => { node.eval.foreach { case (zoom, rdd) => println(s"ZOOM: ${zoom}") println(s"COUNT: ${rdd.count}") } } }
Mr.Gordon
@GofferdoXu
def createSinglebandTileJson(input: String, output: String, layerName: String): Unit = {
    val maskJson: String =
      """
        |[
        |  {
        |    "uri" : "{input}",
        |    "time_tag" : "TIFFTAG_DATETIME",
        |    "time_format" : "yyyy:MM:dd HH:mm:ss",
        |    "type" : "singleband.spatial.read.hadoop"
        |  },
        |  {
        |    "resample_method" : "nearest-neighbor",
        |    "type" : "singleband.spatial.transform.tile-to-layout"
        |  },
        |  {
        |    "crs" : "EPSG:3857",
        |    "scheme" : {
        |      "crs" : "epsg:3857",
        |      "tileSize" : 256,
        |      "resolutionThreshold" : 0.1
        |    },
        |    "resample_method" : "nearest-neighbor",
        |    "type" : "singleband.spatial.transform.buffered-reproject"
        |  },
        |  {
        |    "end_zoom" : 0,
        |    "resample_method" : "nearest-neighbor",
        |    "type" : "singleband.spatial.transform.pyramid"
        |  },
        |  {
        |    "name" : "{layerName}",
        |    "uri" : "{output}",
        |    "key_index_method" : {
        |      "type" : "zorder",
        |      "temporal_resolution": 1
        |    },
        |    "scheme" : {
        |      "crs" : "epsg:3857",
        |      "tileSize" : 256,
        |      "resolutionThreshold" : 0.1
        |    },
        |    "type" : "singleband.temporal.write"
        |  }
        |]
      """.stripMargin
    val maskJsonStr = maskJson.replace("{input}", input).replace("{output}", output).replace("{layerName}", layerName)
    val list: Option[Node[Stream[(Int, TileLayerRDD[SpaceTimeKey])]]] = maskJsonStr.node
    list match {
      case None => println("Couldn't parse the JSON")
      case Some(node) => {
        node.eval.foreach { case (zoom, rdd) =>
          println(s"ZOOM: ${zoom}")
          println(s"COUNT: ${rdd.count}")
        }
      }
    }
  }
How to implement time series?
Grigory
@pomadchin
hey @GofferdoXu what time series do you talk about?
also on the read stage you have a typo: singleband.temporal.read.hadoop
just change all spatial prefixes to temporal
Mr.Gordon
@GofferdoXu
thank you
Mr.Gordon
@GofferdoXu
I wonder how to update layers in geotrellis3
Mr.Gordon
@GofferdoXu
hey@pomadchin How to implement layer update?When there are new images
Grigory
@pomadchin
@GofferdoXu I don’t think it is implemented in terms of the Pipeline DSL
but you can implement it yourself via GT API we have LayerUpdater
Eetu Pursiainen
@epursiai

Hello! I am currently trying to get GeoTrellis to work in Databricks, but I am facing some issue(s). I have installed geotrellis-spark and geotrellis-raster (versions 3.6.0) from the Databrick's UI through Maven. I have the following code:
import geotrellis.raster._
import geotrellis.spark._
import geotrellis.raster.io.geotiff._

val rasterFilePath = "path/to/my.tif"
val geoTiff = GeoTiffReader.readSingleband(rasterFilePath)

But I'm getting java.lang.NoClassDefFoundError: Could not initialize class geotrellis.raster.io.geotiff.TiffType$ error. I noticed that someone else had this problem here before, but unfortunately the solution was never posted here. Any clues what I'm doing wrong here?

Grigory
@pomadchin
hey @epursiai I think it can be the circe library version mismatch
try to shade it in your artifacts
Grigory
@pomadchin
We can also roll a GT release with the fresh circe version
do you know which version is needed for the Databricks runtime?
Eetu Pursiainen
@epursiai
I'll try to exclude circe from the dependencies and use what's provided in the runtime, and I'll let you know if it works. Unfortunately I have no clue what the version there is. I can try to find out. :)
Grigory
@pomadchin
@epursiai yes please! that would be very much welcome
I think shading can be a better approach
Eetu Pursiainen
@epursiai
Probably will need to do that for circe-core, circe-generic and circe-parser all?
Grigory
@pomadchin
try to shade the entire io.circe package
Eetu Pursiainen
@epursiai
Yeah, I'll try to do that. I'll update this issue tomorrow.
Grigory
@pomadchin
:+1: thank you
santocp94
@santocp94

hey @pomadchin I'm sorry but I'm still having some trouble with my mosaic task :( . I wrote this starting from your suggestion:

  val firstTiff = GeoTiffReader
    .readSingleband("...")
  val combinedExtent = firstExtent.expandToInclude(secondExtent)
  val re = RasterExtent(combinedExtent, firstTiff.cellSize)

  val newRaster = Raster(ArrayTile.alloc(firstTiff.cellType, re.cols, re.rows), re.extent)
  val mergedRaster = newRaster.merge(Raster(firstRaster.tile, firstExtent))

I can't understand why but the merge function with that configuration is not resolved. I tried with other two rasters read form files and the function is correctly resolved. The only difference I noticed between the created raster and the read ones is that the geotrellis.raster.DoubleUserDefinedNoDataArrayTile produces a merge while the read raster have a inner geotrellis.raster.io.geotiff.Float64GeoTiffTile. Did I mess up something with imports and packages?
Thanks in advance

Grigory
@pomadchin
hey @santocp94 what to you mean by not correctly resolved?
caoguangshun
@caoguangshun
ok @pomadchin ,i have sovled the problem referring to your test code in the github. and i add a configuration file in resource folder .as following
Grigory
@pomadchin
@caoguangshun great!
santocp94
@santocp94

hey @santocp94 what to you mean by not correctly resolved?

I tried with different configurations but an error occurred:

value merge is not a member of geotrellis.raster.Raster[geotrellis.raster.MutableArrayTile]

or

value merge is not a member of geotrellis.raster.Raster[geotrellis.raster.ArrayTile]
Grigory
@pomadchin
@santocp94 try to cast internals of the Raster to MultibandTile
if that would not work, check if withMultibandRasterMergeMethod(raster).merge works
but I bet this is due to types mismatch
MutableArrayTile is kind of a dangerous type (:
I’m surprised that it didn’t work with ArrayTile
looks like some imports hell.
santocp94
@santocp94
Looks like I can't just cast it: Exception in thread "main" java.lang.ClassCastException: class geotrellis.raster.DoubleUserDefinedNoDataArrayTile cannot be cast to class geotrellis.raster.MultibandTile
and I'm actually failing to import withMultibandRasterMergeMethod. I also tried to re-import everything from scratch but the problem persists