Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Feb 03 15:33
    dependabot[bot] labeled #3498
  • Feb 03 15:33
    dependabot[bot] opened #3498
  • Feb 03 15:33

    dependabot[bot] on npm_and_yarn

    Bump http-cache-semantics from … (compare)

  • Jan 27 04:52

    dependabot[bot] on npm_and_yarn

    Bump ua-parser-js from 0.7.28 t… (compare)

  • Jan 27 04:52
    dependabot[bot] labeled #3497
  • Jan 27 04:52
    dependabot[bot] opened #3497
  • Jan 25 15:26
    pomadchin commented #3496
  • Jan 25 15:26
    pomadchin labeled #3496
  • Jan 25 15:23
    jdries opened #3496
  • Jan 10 19:31

    pomadchin on master

    SBT Version up (compare)

  • Jan 10 18:09

    pomadchin on master

    fix the ci-release (compare)

  • Jan 10 17:41

    pomadchin on master

    ci-release test (compare)

  • Jan 10 17:34

    pomadchin on master

    skip s3-spark docs (compare)

  • Jan 10 17:18

    pomadchin on master

    rm the traget flag (compare)

  • Jan 10 17:03

    pomadchin on master

    replace the target flag (compare)

  • Jan 10 16:52

    pomadchin on master

    try compile before doc (compare)

  • Jan 10 16:23

    pomadchin on master

    sbt version downgrade (compare)

  • Jan 10 16:16

    pomadchin on master

    revert scala versions back (compare)

  • Jan 10 16:05

    pomadchin on master

    use sbt-extras in a containeriz… (compare)

  • Jan 10 16:02

    pomadchin on master

    use the tests image for the rel… (compare)

Grigory
@pomadchin
val attributeStore = HBaseAttributeStore.apply(hbaseInstance,"catalogLatLng") => replace to val attributeStore = HBaseAttributeStore.apply(hbaseInstance)
goodjobtry
@goodjobtry
I'll give it a try
How do I know to read which table in Hbase?
goodjobtry
@goodjobtry
image.png
Where should I specify the table name
goodjobtry
@goodjobtry
@pomadchin
Grigory
@pomadchin
@goodjobtry you don’t need that; the idea is to create a value reader with an attribute store that points to the hbase metadata table and that’s it
by inspecting the metadata table contents it’s possible to notice that it contins info about the tile table location
goodjobtry
@goodjobtry
but when i relplaced ,there also have a error
image.png
@pomadchin
Grigory
@pomadchin
@goodjobtry that’s a bit truncated error (: I’d tell smth shrinks the stacktrace, could you get a better one? But I doubt that is related to GeoTrellis
goodjobtry
@goodjobtry
It means that I can not get tile in Hbase and published as a TMS service?
@pomadchin
Grigory
@pomadchin
yes, some error happened and soome npe exception has been thrown
goodjobtry
@goodjobtry
The corresponding tile data must be found through the attribute table, rather than reading the tile directly from the table where the tile is stored
4 replies
Is that so
@pomadchin
tosen1990
@tosen1990

@pomadchin Hi, Grigory. Do you know if there is a workaround to get the same read performance in RasterFrames ?

From my test, GT raster reading using partitionBytes is several times faster than RF using .withSpatialIndex(partitionnum) ?

    val options =
      HadoopGeoTiffRDD.Options(
        numPartitions = Option(1000),
        partitionBytes = Option(64l * 1024 * 1024)
      )
    val geoTiffRDD: RDD[(ProjectedExtent, MultibandTile)] = HadoopGeoTiffRDD
      .spatialMultiband(new Path(inputPath), options)(spark.sparkContext)
    val df_init: DataFrame = spark.read.raster
      .withSpatialIndex(partitionnum)
      .withTileDimensions(256, 256)
      .withBandIndexes(bandIndexInt: _*)
      .fromCSV(cat).load()
      .withColumn("tile_dimensions", rf_dimensions(col(firstBandTile)))
3 replies
goodjobtry
@goodjobtry
@pomadchin Here's a new question about get tile from Hbase , I can't resolve it,Can you give me a suggestion?
org.apache.avro.AvroTypeException: Found geotrellis.raster.ArrayMultibandTile, expecting union. This can be caused by using a type parameter which doesn't match the object being deserialized.
I didn't change the code, I just reran the code
4 replies
yang162132
@yang162132
@pomadchin Hey! how could I get a tiff in fixed pixel scale. I had try
Raster(MbTile.resample(rasterExtent, Math.round(rasterExtent.width / 30).toInt, Math.round(rasterExtent.height / 30).toInt, NearestNeighbor), rasterExtent)
to get a 30m/pixel tiff
But finally I just get 26m/pixel
2 replies
yang162132
@yang162132
And it seems that the number of pixels has increased by about 30% from zoom =12(about 38.21meter) to 30 meter
mwhei
@mwhei
Hello, I store data into Hbase via " val layerId = LayerId("testid", z) writer.write(layerId, rdd, keyIndex)".But when fetch data via
" reader.readSpaceTimeKey, Tile, TileLayerMetadata[SpaceTimeKey])" .It has error. Exception in thread "main" geotrellis.store.package$LayerNotFoundError: Layer Layer(name = "testid", zoom = 0) not found in the catalog
2 replies
Is there some whole examples about how to store and fetch tif to/from hbase?
yang162132
@yang162132
Hey!@pomadchin
I've been thinking about a question recently.
which is the best choice for "tiff -> Map_algebra ->tiff" in GT for some big tif?
when I try readMultiband to operate directly on tiff it's really slow(it took about 15 minutes for reproject without other calculate).
and if create gt cataloge in advance,it will quickly in calculat.But it hard to load the result from the memory to a geotiff, both stitch from TileLayerRDD and meger to collect all tiles for GeoTrellisRasterSource expensive(both them are unsupport multi thread I guess).They also faced difficulty to get the pixel scale we want because the layer in zoom has fixed resolution.
So how can we make the most of our machines.
1 reply
tosen1990
@tosen1990
Using crop on RDD to output the image, I got two different results.
Output-1 using stitch works well. While using foreach to write out, the image got distorted.
   val floatingLayout = FloatingLayoutScheme(512, 512)
    val tlm: TileLayerMetadata[SpatialKey] = inputRDD.collectMetadata[SpatialKey](floatingLayout)._2

    val tiled: RDD[(SpatialKey, MultibandTile)] with Metadata[TileLayerMetadata[SpatialKey]] = inputRDD.tileToLayout(tlm)

    val cropRDD: RDD[(SpatialKey, MultibandTile)] with Metadata[TileLayerMetadata[SpatialKey]] = {
      tiled.crop(targetExtent)
    }

    // it's okay       output-1
    val stitchedTile: Raster[MultibandTile] = cropRDD.crop(targetExtent).stitch()

    val croppedTile = stitchedTile  //.crop(totalCols, totalRows)
    MultibandGeoTiff(croppedTile.tile, croppedTile.extent,
      tlm.crs, GeoTiffOptions.DEFAULT.copy(compression = DeflateCompression))
      .withOverviews(NearestNeighbor)
      .write(new Path("xxx"), hconf.value)

    // the image distorted  output-2
    cropRDD.regrid(512, 512).toGeoTiffs()
      .filter{
        case (_: SpatialKey, null) ⇒ false  // remove any null Tiles
        case _ ⇒ true
      }
      .foreach { case (sk: SpatialKey, gt: MultibandGeoTiff) ⇒
        val path = new Path(new Path("xxx"), s"${sk.col}_${sk.row}.tif")
        MultibandGeoTiff(gt.tile, gt.extent,
          tlm.crs, GeoTiffOptions.DEFAULT.copy(compression = DeflateCompression))
          .withOverviews(NearestNeighbor)
          .write(path, hconf.value)
      }
2 replies
output-1
WeChat1d8930ec7b50645eb156f78e8eb229e1.png
output2
WeChat9e76e8d97456c8c79eb6d3caedd435ff.png
1 reply
MuzDing
@MuzDing
java.lang.IllegalArgumentException: Encountered NaN during a refinement step: (NaN / NaN). Input Extent(-180.0, -90.00999999999934, 180.00999999999868, 90.0) is likely not in source projection.
What does this NaN/NaN error mean
MuzDing
@MuzDing
java.lang.IllegalArgumentException: Encountered NaN during a refinement step: (NaN / NaN). Input Extent(-179.125, -166.875, -51.125, -38.875) is likely not in source projection.
1 reply
MuzDing
@MuzDing
The image is low resolution and global
MuzDing
@MuzDing
@pomadchin
imhsagar
@imhsagar
Hi, Anyone worked with landscan tiff files with geotrellis? Need to understand on how to get population value cell wise from that.
2 replies
loubenhao
@loubenhao
Hello, I want render a png like this:
The result can be correctly output, but the process cannot be shut down.How can I stop it?
2 replies
image.png
loubenhao
@loubenhao
Hello, I use COGLayerWriter to write a COGLayer on S3, then I read the COGLayer and stitch it to a geotiff, and then write it on file system.
Singleband geotiff can be loaded with openlayers, but the MultibandTile geotiff cannot.
when I use gdal(validate_cloud_optimized_geotiff.py) to check it, the following warning like this:
Warning 1: TIFFReadDirectory:Sum of Photometric type-related color channels and ExtraSamples doesn't match SamplesPerPixel. Defining non-color channels as ExtraSamples.
How can I fix it?
4 replies
image.png
imhsagar
@imhsagar

I am trying to convert landscan raster cell to polygon using the following code
def rasterToPolygonsT <: Tile: Seq[Polygon] = {
raster.polygonalize().map { case (polygon, tile) => polygon }
}

but not able to find the polygonalize() method, I believe this polygonalize method is defined in the Raster trait, which is the base trait for all raster types in GeoTrellis. but on using in code it is not able to find the definition. I am using import Geotrellis.raster.Raster. Any other import that I need to use, or any other function that can be used.

3 replies
loubenhao
@loubenhao
Hi, it seems that geotrellis has no LZW compression algorithm.
However, geotiff.js--openlayers using library can not load the COG which compressed with DeflateCompression or NoCompression in geotrellis.
4 replies
Bryant Baltes
@Bryology

So this is probably a dumb question, but I can't seem to find the answer (likely don't know how to google correctly).

If I were to have a a list of lat lon with an associated Int value. How would I know where to place those values in the IntArrayTile? I know how to calculate the extents, find out what tile they go into to generate the png, but I have no clue how to place those values in the correct positions in the ArrayTile.

5 replies
Bryant Baltes
@Bryology
If I have a GeoTiff and a Polygon and I wanted to only render the data that falls within that polygon from the GeoTiff, what is the best way to do that?
5 replies