by

Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • May 21 20:23
    pomadchin closed #3245
  • May 21 19:55
    pomadchin synchronize #3245
  • May 21 19:54
    pomadchin opened #3245
  • May 21 12:59

    pomadchin on master

    WKTParser extension support (#3… (compare)

  • May 21 12:59
    pomadchin closed #3244
  • May 21 12:59
    pomadchin closed #3241
  • May 20 23:52
    pomadchin synchronize #3244
  • May 20 23:50
    pomadchin synchronize #3244
  • May 20 19:34
    CloudNiner assigned #3244
  • May 20 19:34
    CloudNiner review_requested #3244
  • May 16 13:20
    pomadchin synchronize #3244
  • May 16 13:09
    pomadchin edited #3244
  • May 15 17:14
    pomadchin synchronize #3244
  • May 15 17:11
    pomadchin synchronize #3244
  • May 15 16:46
    pomadchin edited #3244
  • May 15 16:46
    pomadchin opened #3244
  • May 14 22:15

    pomadchin on master

    Fix bad ArrayTile equals (#3243) (compare)

  • May 14 22:15
    pomadchin closed #3243
  • May 14 22:15
    pomadchin closed #3242
  • May 14 20:31
    pomadchin assigned #3241
Michał Gołębiewski
@mjgolebiewski
oh yes, sorry for tag then.
Grigory
@pomadchin
@mjgolebiewski nope feel free to tag me :D im happy to help whatever way I can
Michał Gołębiewski
@mjgolebiewski
@pomadchin thanks :). @jamesmcclain im not sure which channel is more appropriate (geotrellis or rasterframes) so id be really grateful if you could choose where should i continue asking these questions...
Grigory
@pomadchin
@mjgolebiewski I think here is good :100:
James McClain
@jamesmcclain
Indeed
caoguangshun
@caoguangshun
@pomadchin hi, i want to ask the gt is supported hyperspectrum image or not ?
Grigory
@pomadchin
Hi @caoguangshun it depends on what do you mean by “support"?
but yes we can read it into memory; these are just pictures with lots of bands
caoguangshun
@caoguangshun
yes , i want to know whether the gt can read hyperspectrum image into "MultibandTile" object or not,because the hyperspectrum image has hundreds of bands @pomadchin
Grigory
@pomadchin

@caoguangshun yes, while you have enough of memory you can do that. the number of bands is limited with Int.Max

However, AFAIK, to work with hyperspecral data you may need only the bands that correspond to some specific wavelengths (i.e. to perform some specific spectrum analysys to detect smth); it is also possible to do it on read and only to read only bands you’re interested in

Michał Gołębiewski
@mjgolebiewski
@pomadchin hello again :). could you provide me with parameters that gt is using to build gdal?
Grigory
@pomadchin
hi @mjgolebiewski what parameters do you mean?
Michał Gołębiewski
@mjgolebiewski
let me rephrase that: is --with-python required when compilling?
Grigory
@pomadchin
@mjgolebiewski no, we deliver JNI bindings with jar dependencies
So in theory it doesn’t matter how you build GDAL
Michał Gołębiewski
@mjgolebiewski
thats helpful, thanks!
Grigory
@pomadchin
yep; just to doublecheck that we are on the same page - we use our own JNI bindings that are thread safe and easier to manage in a multithreaded environment; we don’t depend on official GDAL jar deps
John
@Canadianboy122_twitter
Hey guys. I am interested into using geotrellis, but cant see does it have any web interface. For example can i render tiff file. I saw quickstart with some kind of template on github and that works ok.
Grigory
@pomadchin
@Canadianboy122_twitter Hi! I think most of our demos are out of date, but if you’re interested in a tile server, check out the example module of GeoTrellis server https://github.com/geotrellis/geotrellis-server
or what do you mean by the web interface?
John
@Canadianboy122_twitter
Ok i will check that server. Thanks for fast reply. I want to render tiff file but guess cant do that ? Or i miss something?
Also for example if i want to use geotrellis in docker cant expose it outside if i get it. I am new to this
Grigory
@pomadchin

Ahhhhh, so I see what is your question about (hopefully :D)

So GT is a Scala library; you don’t need to ‘install it’ or smth like that; you can add is a dependnecy into the project;

we have an outdated demo https://github.com/geotrellis/geotrellis-landsat-tutorial but it can give you the sense of what is the API and how you may use it

you can clone this repo and hack on it, the only thing you need is an installed Java 8+
we also have a bit more up to date template of a clean project with all deps, which is here: https://github.com/geotrellis/geotrellis-spark-job.g8
John
@Canadianboy122_twitter
Oh nice. I will see all the links soon. Thanks a lot for help. Hope i will find answer to my question
iceland1906
@iceland1906

Hi @pomadchin , I have a simple testing script of bufferTiles:

def test(inputRdd: RDD[(SpatialKey, Tile)]: Unit = {
    val bufferLength = 12
    val bufferedRdd = inputRdd.bufferTiles(bufferLength)
    val outRdd = bufferedRdd.mapValues(
      rddTile => {
        val bufferedTile = rddTile.tile
        val targetArea = rddTile.targetArea
        bufferedTile.crop(targetArea)
      })
    print(outRdd.count())

but got an error "geotrellis.raster.GeoAttrsError: Grid bounds do not intersect: GridBounds(12,0,36,24) crop GridBounds(0,0,11,24)", cannot figure out why, do you have any suggestion? Thanks

Grigory
@pomadchin
hey @iceland1906 it looks like your gridbounds don’t intersect
the signature of gridBounds is the following: GridBounds(colMin, rowMin, colMax, rowMax)
so the GridBounds(12,0,36,24) crop GridBounds(0,0,11,24)” indeed has no intersection since the first gridbounds starts at the 12 column and the second one stops at the 11th column
iceland1906
@iceland1906
@pomadchin thanks for the response, I don't know what tiles those two GridBounds are related, here are the inputRDD with only two tiles, the size of the tile is 25 pixels by 25 pixels, looks like the first GridBounds is the green rectangle and the second one is the yellow rect
image.png
but why the GridBounds need to intersect? and in this case why they do not intersect?
Grigory
@pomadchin
@iceland1906 hm, what is the inputRdd, what is the target region and what result do you expect from this operation?
iceland1906
@iceland1906
sorry, not sure whether I get your questions right, the input is a big tif image, but I added a filter to keep only two tiles for a quick test. what I want to do is for each tile, I want to join it with its neighbouring tiles to get some extra information, and do some operations, the final output Rdd would have the same tile size as the inputRdd, that is, the target area is the original tile before buffering
Grigory
@pomadchin
hm, and what this crop operation is supposed to do?
iceland1906
@iceland1906
I thought after this step "val bufferedRdd = inputRdd.bufferTiles(bufferLength)", the bufferedTile would be bigger than the original tile size (i.e., buffered by 12), that's why need to crop using the targetArea to get back to the original size
Grigory
@pomadchin
ah so you are buffering and after that doing crop so the result is if the same area right?
iceland1906
@iceland1906
yes
just did another quick test, found something strange, the actual inputRdd is an output from another buffering operation, if I remove the buffering operation in the previous step, then no error
I will upload a code snippet to explain what I mean shortly
Grigory
@pomadchin
yea, if you would provide some complete example it would be much easier to help :+1:
iceland1906
@iceland1906
def main(inRdd1: RDD[(SpatialKey, Tile)]): Unit = {
    // Given the input rdd, call funcOne and funcTwo sequentially to do some operations
    // inRdd1, rdd2, and rdd3 are expected to have same tile size
    // funcOne and funcTwo actually do similar work: do buffering -> some operation -> crop back to original size

    val rdd2: RDD[(SpatialKey, Tile)] = funcOne(rdd1) // works

    val rdd3: RDD[(SpatialKey, Tile)] = funcTwo(rdd2) // error happened inside
}

def funcOne(inRdd1: RDD[(SpatialKey, Tile)]): RDD[(SpatialKey, Tile)] = {
    val bufferedRdd1 = inRdd1.bufferTiles(12)
    val outRdd1 = bufferedRdd1.mapValues(
      rddTile => {
        val bufferedTile = rddTile.tile
        val targetArea = rddTile.targetArea

        val editedBufferedTile = func1(bufferedTile)

        editedBufferedTile.crop(targetArea)
      })
    outRdd1
}

def funcTwo(rdd2: RDD[(SpatialKey, Tile)]: Unit = {
    val bufferedRdd2 = rdd2.bufferTiles(12)
    val outRdd2 = bufferedRdd2.mapValues(
      rddTile => {
        val bufferedTile = rddTile.tile
        val targetArea = rddTile.targetArea

        val editedBufferedTile = func2(bufferedTile)

        editedBufferedTile.crop(targetArea)
      })
    println(outRdd2.count())
}
here is a more complete script
iceland1906
@iceland1906
seems to me that do bufferTiles() to an Rdd which was an output from another bufferTiles() operation does not work
iceland1906
@iceland1906
hope I explained it clear
gispathfinder
@zyxgis
I want to set dfs.replication parameter when I save vector tile
but the geotrellis.spark.store.hadoop.SaveToHadoop class
  def setup[K](
    rdd: RDD[(K, Array[Byte])],
    keyToUri: K => String
  ): RDD[(K, Array[Byte])] = {
    rdd.mapPartitions { partition =>
      saveIterator(partition, keyToUri, new Configuration){ (k, v) => v }
   }
  }
the Configuration parameter is not exposed