by

Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Jun 25 19:54

    CloudNiner on release-files-modified

    (compare)

  • Jun 25 19:54

    CloudNiner on master

    gitignore sensitive files used … (compare)

  • Jun 25 19:54
    CloudNiner closed #3266
  • Jun 25 14:14
    CloudNiner review_requested #3266
  • Jun 25 14:14
    CloudNiner assigned #3266
  • Jun 25 14:14
    CloudNiner review_requested #3266
  • Jun 25 14:14
    CloudNiner opened #3266
  • Jun 25 14:11

    CloudNiner on release-files-modified

    gitignore sensitive files used … (compare)

  • Jun 24 23:46
    pomadchin assigned #3265
  • Jun 24 18:02

    CloudNiner on awf

    (compare)

  • Jun 24 18:02

    CloudNiner on master

    Add GitHub Issue Templates (#32… (compare)

  • Jun 24 18:02
    CloudNiner closed #3264
  • Jun 24 17:44
    CloudNiner commented #3264
  • Jun 24 17:43
    CloudNiner synchronize #3264
  • Jun 24 17:43

    CloudNiner on awf

    Add github issue config.yml wit… (compare)

  • Jun 24 14:18
    pomadchin commented #3264
  • Jun 24 14:18
    pomadchin commented #3264
  • Jun 24 14:18
    pomadchin commented #3264
  • Jun 24 13:24
    CloudNiner synchronize #3264
  • Jun 24 13:24

    CloudNiner on awf

    Cleanup from PR feedback (compare)

Grigory
@pomadchin
but at this point I don’t have time to look into it; it is a pretty serious issue - so I would be happy to assist you if you would come up with some really good solution
Frederic Guiet
@fguiet
hummm I see...ok I am gonna split my las file into chunks via pdal split command
Check Iqumuls library as it seems to be able to load big las file into Hive
worth a look at the implementation at the implementation
Grigory
@pomadchin
Yes, i know about this library. It is completely different and much more restrictive
Frederic Guiet
@fguiet
moreover it depracated and works only with spark 1.6...that a shame
thank to point me this known limition
limitation
Grigory
@pomadchin
well, it is restrctive in the sense of operations and file formats it supports
pointclouds are not only las / laz files
Frederic Guiet
@fguiet
yeah of course
Grigory
@pomadchin
and the implemenataion details are completely different; in our case we’re hitting a problem of trying to allocate a single array for the entire las file
Frederic Guiet
@fguiet
but las file can be very big...so gt-pointcloud must handle this otherwise it will be useless
Grigory
@pomadchin
not really. usually you don’t need all the dimensions to be loaded into memory and not everything at once
Frederic Guiet
@fguiet
single array for the entire las file!!!
Grigory
@pomadchin
it is doable and we would be happy to work on it once it would be required for someone
Frederic Guiet
@fguiet
u usre?
Grigory
@pomadchin
Well yes, Im sure - because I was the one who worked on all the PDAL / jni interfaces and geotrellis-pointcloud itself
Frederic Guiet
@fguiet
yeah but are you sure it is the good implementation
as a las file can contain billion of point
Grigory
@pomadchin
em; I didnt tell that it is a good implementation
it is a naive implementation
Frederic Guiet
@fguiet
:)
anyway, will try to chunk my las file
Grigory
@pomadchin
or you can create a PR and fix the way spark loads pointclouds into memory ;)
Frederic Guiet
@fguiet
so I can load it X,Y,Z values into a big hive table
Grigory
@pomadchin
Also, if you have only x,y,z you can filter the file by dimensions
Frederic Guiet
@fguiet
I will make a PR if I was smarter enough for sure
no the cas thought
case
not
:(
Frederic Guiet
@fguiet
gtg seeya @pomadchin
thank for ya help
Grigory
@pomadchin
np, you’re welcome
rexuexiangbei
@rexuexiangbei
@pomadchin hi,which hbase version is required in gt3.0.0;hbase 1.3.1 is ok?
Grigory
@pomadchin
Hey @rexuexiangbei nope, we are on hbase 2.2 but you can try, API can be compatible
John Smith
@GintokiYs
image.png
image.png
hello @pomadchin What is the use of this class?
Grigory
@pomadchin
Hey @15952026052 it is a zoomed layout scheme (as I already mentioned according to tms tiling scheme) - allows to derive zoom level based not the raster extent and the cellSize; also allows to derive layoutDefinjtion and derive it for some specific zoom level
Frank Dekervel
@kervel
hello, maybe stupid question, i'm trying to create the trivial PR suggested by @pomadchin and i'd like to test it on my project. now i have geotrellis/ and myproject/ but myproject refers to the released version of geotrellis in its build.sbt. how can i make my project using the geotrellis i built myself ?
Frank Dekervel
@kervel
i now did it with "sbt publishLocal" and then i changed version of geotrellis to 3.1.0-snapshot in my build.sbt but there must be a better way ?
Grigory
@pomadchin
Hey @kervel ; nope, you can use https://github.com/locationtech/geotrellis/blob/master/scripts/publish-local.sh if you have difficulties. The good part of the story that you usually need to publish everything once and after that you can just rebuild the project you modified
Frank Dekervel
@kervel
ah, the push creates symlinks ?
Grigory
@pomadchin
symlinks? ah sowe had difficulties with publishing everything through a single sbt commands (memory / etc). sowe created a seaparate script that publishes everything into a local maven repository
James Cosford
@jamescosford
I've just come back to using geotrellis/vector after not touching it since 2.10, and I thought I would bring everything up to date with 3.1.0. Is that a good idea? Is there a good summary of changes somewhere? Feels like everything is miles from where it was.
Grigory
@pomadchin

hey @jamescosford you can get a high order summary from here: https://www.azavea.com/blog/2019/10/29/geotrellis-3-0-release/ also we have some release notes https://github.com/locationtech/geotrellis/releases/tag/v3.0.0 and we don’t have updated docs

the difference between 3.0 and 3.1 in a broken types hierarchy that was fixed in 3.1 and you probably won’t even notice it

James Cosford
@jamescosford
Thanks! I was stupidly looking at the master changelog and not the 3.0 branch changelog
I think I'm running into mostly locationtech/geotrellis#2932 stuff because I am using lots of JTS functionality