GeoTrellis is a geographic data processing engine for high performance applications.
dependabot[bot] on npm_and_yarn
Bump http-cache-semantics from … (compare)
dependabot[bot] on npm_and_yarn
Bump ua-parser-js from 0.7.28 t… (compare)
pomadchin on master
SBT Version up (compare)
pomadchin on master
fix the ci-release (compare)
pomadchin on master
ci-release test (compare)
pomadchin on master
skip s3-spark docs (compare)
pomadchin on master
rm the traget flag (compare)
pomadchin on master
replace the target flag (compare)
pomadchin on master
try compile before doc (compare)
pomadchin on master
sbt version downgrade (compare)
pomadchin on master
revert scala versions back (compare)
pomadchin on master
use sbt-extras in a containeriz… (compare)
pomadchin on master
use the tests image for the rel… (compare)
@pomadchin Hi, Grigory. Do you know if there is a workaround to get the same read
performance in RasterFrames
?
From my test, GT raster reading using partitionBytes
is several times faster than RF using .withSpatialIndex(partitionnum)
?
val options =
HadoopGeoTiffRDD.Options(
numPartitions = Option(1000),
partitionBytes = Option(64l * 1024 * 1024)
)
val geoTiffRDD: RDD[(ProjectedExtent, MultibandTile)] = HadoopGeoTiffRDD
.spatialMultiband(new Path(inputPath), options)(spark.sparkContext)
val df_init: DataFrame = spark.read.raster
.withSpatialIndex(partitionnum)
.withTileDimensions(256, 256)
.withBandIndexes(bandIndexInt: _*)
.fromCSV(cat).load()
.withColumn("tile_dimensions", rf_dimensions(col(firstBandTile)))
Raster(MbTile.resample(rasterExtent, Math.round(rasterExtent.width / 30).toInt, Math.round(rasterExtent.height / 30).toInt, NearestNeighbor), rasterExtent)
to get a 30m/pixel tiff
stitch
works well. While using foreach
to write out, the image got distorted. val floatingLayout = FloatingLayoutScheme(512, 512)
val tlm: TileLayerMetadata[SpatialKey] = inputRDD.collectMetadata[SpatialKey](floatingLayout)._2
val tiled: RDD[(SpatialKey, MultibandTile)] with Metadata[TileLayerMetadata[SpatialKey]] = inputRDD.tileToLayout(tlm)
val cropRDD: RDD[(SpatialKey, MultibandTile)] with Metadata[TileLayerMetadata[SpatialKey]] = {
tiled.crop(targetExtent)
}
// it's okay output-1
val stitchedTile: Raster[MultibandTile] = cropRDD.crop(targetExtent).stitch()
val croppedTile = stitchedTile //.crop(totalCols, totalRows)
MultibandGeoTiff(croppedTile.tile, croppedTile.extent,
tlm.crs, GeoTiffOptions.DEFAULT.copy(compression = DeflateCompression))
.withOverviews(NearestNeighbor)
.write(new Path("xxx"), hconf.value)
// the image distorted output-2
cropRDD.regrid(512, 512).toGeoTiffs()
.filter{
case (_: SpatialKey, null) ⇒ false // remove any null Tiles
case _ ⇒ true
}
.foreach { case (sk: SpatialKey, gt: MultibandGeoTiff) ⇒
val path = new Path(new Path("xxx"), s"${sk.col}_${sk.row}.tif")
MultibandGeoTiff(gt.tile, gt.extent,
tlm.crs, GeoTiffOptions.DEFAULT.copy(compression = DeflateCompression))
.withOverviews(NearestNeighbor)
.write(path, hconf.value)
}
I am trying to convert landscan raster cell to polygon using the following code
def rasterToPolygonsT <: Tile: Seq[Polygon] = {
raster.polygonalize().map { case (polygon, tile) => polygon }
}
but not able to find the polygonalize() method, I believe this polygonalize method is defined in the Raster trait, which is the base trait for all raster types in GeoTrellis. but on using in code it is not able to find the definition. I am using import Geotrellis.raster.Raster. Any other import that I need to use, or any other function that can be used.
So this is probably a dumb question, but I can't seem to find the answer (likely don't know how to google correctly).
If I were to have a a list of lat lon with an associated Int value. How would I know where to place those values in the IntArrayTile? I know how to calculate the extents, find out what tile they go into to generate the png, but I have no clue how to place those values in the correct positions in the ArrayTile.