Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Dec 05 18:08

    echeipesh on master

    Fixes fallback implementation o… (compare)

  • Dec 05 18:08
    echeipesh closed #3156
  • Dec 05 18:08
    echeipesh closed #3153
  • Dec 05 17:50

    echeipesh on master

    Update hbase 2.1.8 for tests (compare)

  • Dec 05 15:36

    echeipesh on master

    remove unnecessary calculations… (compare)

  • Dec 05 15:36
    echeipesh closed #3147
  • Dec 05 15:36
    echeipesh edited #3147
  • Dec 05 15:36
    echeipesh commented #3147
  • Dec 04 19:19
    moradology edited #3160
  • Dec 04 19:19
    moradology opened #3160
  • Dec 04 17:41
    CloudNiner closed #3159
  • Dec 04 17:41
    CloudNiner commented #3159
  • Dec 04 16:36
    pomadchin commented #3159
  • Dec 04 16:35
    pomadchin commented #3159
  • Dec 04 16:34
    pomadchin labeled #3159
  • Dec 04 16:33
    pomadchin commented #3159
  • Dec 04 16:32
    pomadchin commented #3159
  • Dec 04 16:27
    CloudNiner labeled #3159
  • Dec 04 16:27
    CloudNiner labeled #3159
  • Dec 04 16:27
    CloudNiner opened #3159
Grigory
@pomadchin
will you file this issue? It is really sad that we havent noticed that
15952026052
@15952026052
image.png
How can I fix this?
val inputRdd: RDD[(ProjectedExtent, MultibandTile)] =
sc.hadoopMultibandGeoTiffRDD(inputPath).mapValues(m => m.withNoData(Option(0)))
val his: Array[Histogram[Double]] = inputRdd.histogram
val (_, rasterMetaData) =
  TileLayerMetadata.fromRDD(inputRdd, FloatingLayoutScheme(512))


val tiled: RDD[(SpatialKey, MultibandTile)] =
  inputRdd
    .tileToLayout(rasterMetaData.cellType, rasterMetaData.layout, Bilinear)
    .repartition(200)
Grigory
@pomadchin
@15952026052
import geotrellis.spark._
import geotrellis.spark.store._
import geotrellis.spark.store.hadoop._
15952026052
@15952026052
It works and fixes an error but how do I get the parameter rasterMetaData Geotrellis-3.0.0 without this method.
image.png
Grigory
@pomadchin
CollectTileLayerMetadata.fromRDD
15952026052
@15952026052
hello I am running an error about jackson
image.png
image.png
15952026052
@15952026052
image.png
I encountered this error when I used Geotrellis-3.0.0
image.png
This is my tif information. There are a few more TiffTags that can't be recognized by Geotrellis-2.1.0. So I used Geotrellis-3.0.0
15952026052
@15952026052
But when I processed another image, there was no error. This is the only difference between two tifs. And the following is the information of the tif without error.
image.png
15952026052
@15952026052
geotrellis.raster.io.geotiff.reader.MalformedGeoTiffException: No StripByteCount information.
    at geotrellis.raster.io.geotiff.tags.TiffTags.segmentCount(TiffTags.scala:556)
    at geotrellis.raster.io.geotiff.compression.Decompressor$.apply(Decompressor.scala:96)
    at 
geotrellis.raster.io.geotiff.reader.GeoTiffInfo$.getGeoTiffInfo$1(GeoTiffInfo.scala:179)
Grigory
@pomadchin
Hey, this is probably a bug, is it a public data? I can create a unit test and we’ll fix it in our next release
Grigory
@pomadchin
the difference is not in these tags you pointed to
it looks like the issue is that we cant read tiffs with the stripe size infeormation (in case it is a striped tiff)
Grigory
@pomadchin
Without** (typo, wrote from phone)
Simeon H.K. Fitch
@metasim
@pomadchin @echeipesh Let me know if my writupe of #3134 is unclear, and if you want more out-of-band discussion of it and what our options are around it.
Grigory
@pomadchin
@metasim :100:
Eugene Cheipesh
@echeipesh
@metasim that's super clear. Seems like there are only two options? either make abstract class Tile or push Grid and CellGrid back into trait.
Grigory
@pomadchin
the second option will definitely work
lets try the first one and see would it break smth?
Grigory
@pomadchin
Tile, ArrayTile, MultibandTile to abstract classes
ArrayMultibandTile is already a class
I’m voting for this change and it looks like it fixes everything
@echeipesh I can push some code really quick so you and @metasim can look into it
Eugene Cheipesh
@echeipesh
:+1: - lets check it out
Grigory
@pomadchin
@echeipesh @metasim locationtech/geotrellis#3136
Simeon H.K. Fitch
@metasim
:eyes: ing
I like the Tile as abstract class approach.
Grigory
@pomadchin
@metasim can you doublechek that Spark codecs would be derived?
Simeon H.K. Fitch
@metasim
Yeh, might take me a few to get everything built, etc.
Simeon H.K. Fitch
@metasim
@pomadchin I keep getting this (even after a clean):
[error] Symbol 'type geotrellis.store.hadoop.Implicits' is missing from the classpath.
[error] This symbol is required by 'package geotrellis.store.hadoop.package'.
[error] Make sure that type Implicits is in your classpath and check for conflicting dependencies with `-Ylog-classpath`.
[error] A full rebuild may help if 'package.class' was compiled against an incompatible version of geotrellis.store.hadoop.
(On branch pomadchin/fix/tile-hierarchy)
Eugene Cheipesh
@echeipesh
@metasim are you doing clean in away that it would get picked up by store project ?
Simeon H.K. Fitch
@metasim
:shrug:?
sbt clean
Eugene Cheipesh
@echeipesh
nope ... woops, store should be aggreagted from the root project
Simeon H.K. Fitch
@metasim
:+1:
Grigory
@pomadchin
@metasim I usually clean targets mannually rm -r ./*/target
Simeon H.K. Fitch
@metasim
gotta trust the sbt, man
Add layer to the aggregate list too
Grigory
@pomadchin
:+1:
will add it in terms of this PR