Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Jan 31 2019 17:36
    schnerd starred locationtech/geowave
  • Jan 30 2019 11:01
    hsg77 commented #1474
  • Jan 30 2019 10:58
    hsg77 commented #1474
  • Jan 30 2019 10:57
    hsg77 commented #1474
  • Jan 30 2019 10:53
    hsg77 commented #1474
  • Jan 30 2019 10:53
    hsg77 commented #1474
  • Jan 30 2019 10:51
    hsg77 commented #1474
  • Jan 29 2019 16:30
    JWileczek commented #1474
  • Jan 29 2019 16:30
    JWileczek commented #1474
  • Jan 29 2019 16:12
    rfecher commented #1474
  • Jan 29 2019 10:44
    hsg77 commented #1474
  • Jan 28 2019 22:47
    sunapi386 starred locationtech/geowave
  • Jan 28 2019 21:12

    rfecher on gh-pages

    Lastest javadoc on successful t… (compare)

  • Jan 28 2019 20:47

    rfecher on master

    fixing coveralls (#1488) (compare)

  • Jan 28 2019 20:47
    rfecher closed #1488
  • Jan 28 2019 20:47
    rfecher opened #1488
  • Jan 28 2019 17:02

    rfecher on master

    Update README.md (compare)

  • Jan 28 2019 16:53

    rfecher on master

    updated readme.md (#1486) (compare)

  • Jan 28 2019 16:53
    rfecher closed #1486
Robin Darby
@robinjamesdarby_twitter
Peeps; looking for a solution that is able to store voxels in HDFS (Hadoop) - am I in the right place??
Muhammed Kalkan
@Nymria
Any ideas how to set units ? I have set spatial index to EPSG:32636 which uses meters as unit, but GeoWave api tries to ingest them as lat / lon
Oct 24, 2020 8:57:26 PM org.geotools.referencing.operation.projection.TransverseMercator transform
WARNING: Possible use of "Transverse_Mercator" projection outside its valid area.
Longitude 309764°25.8'E is out of range (±180°).
Latitude 4355003°38.8'N is out of range90°).
Muhammed Kalkan
@Nymria
And if I dont set index crs same as shp file crs, ingestion is done in a wrong way.
image.png
Above is only one feature and clearly wrong
rfecher
@rfecher
@Nymria does your shp have a prj file as well? That's how any shapefile reader would pick up the correct CRS. The index CRS shouldn't need to be set to the same as the source data, typically you want to set the index CRS to the CRS that you most commonly want to use on query.
@robinjamesdarby_twitter yeah it's a good option if you want to index those voxels for efficient 3D queries.
Muhammed Kalkan
@Nymria
@rfecher yes shp file has prj file. I read feature and build my own type, not directly ingesting shp file, write geometry into my own type and ingest it. Thats the same code I ve sent. Simply I get geometry and insert it to my newly created type. If I dont need to set any CRS info ingesting via new type, what should I do ?
Leaving as it is gives the result as above, kind of confused
Muhammed Kalkan
@Nymria
Ok fixed it after many tries and solutions. Need to point out some details in examples tho.
@rfecher index error however continues, can you reproduce it ? 1.2.0-SNAPSHOT has the same behavior.
        SpatialIndexBuilder spIdxBldr = new SpatialIndexBuilder();
        spIdxBldr.setCrs("EPSG:32636");
        spatialIndex = spIdxBldr.createIndex();

        TemporalIndexBuilder tmpIdxBldr = new TemporalIndexBuilder();
        tmpIdxBldr.setName("TEMPORAL_IDX");
        tmpIdxBldr.setPeriodicity(TemporalBinningStrategy.Unit.HOUR);
        temporalIndex = tmpIdxBldr.createIndex();

       dataStore.addIndex(spatialIndex);
       dataStore.addIndex(temporalIndex);

       dataStore.addType(sfAdapter, spatialIndex,temporalIndex);
25 Oct 12:33:01 ERROR [vector.FeatureDataAdapter] - Multiple indices with different CRS is not supported
Exception in thread "main" java.lang.RuntimeException: Multiple indices with different CRS is not supported
    at org.locationtech.geowave.adapter.vector.FeatureDataAdapter.init(FeatureDataAdapter.java:241)
    at org.locationtech.geowave.core.store.base.BaseDataStore.internalAddIndices(BaseDataStore.java:1142)
    at org.locationtech.geowave.core.store.base.BaseDataStore.addType(BaseDataStore.java:1167)
    at com.uasis.geowaveapi.Geowave.ingestFromFile(Geowave.java:95)
    at com.uasis.geowaveapi.Geowave.main(Geowave.java:82)
Muhammed Kalkan
@Nymria
Seems there is something wrong in FeatureDataAdapter.java:241
public boolean init(final Index... indices) throws RuntimeException {
    String indexCrsCode =
        reprojectedFeatureType == null ? null
            : GeometryUtils.getCrsCode(reprojectedFeatureType.getCoordinateReferenceSystem());
    for (final Index primaryindx : indices) {
      // for first iteration
      if (indexCrsCode == null) {
        if (primaryindx.getIndexModel() instanceof CustomCrsIndexModel) {
          indexCrsCode = ((CustomCrsIndexModel) primaryindx.getIndexModel()).getCrsCode();
        } else {
          indexCrsCode = GeometryUtils.DEFAULT_CRS_STR;
        }
      } else {
        if (primaryindx.getIndexModel() instanceof CustomCrsIndexModel) {
          // check if indexes have different CRS
          if (!indexCrsCode.equals(
              ((CustomCrsIndexModel) primaryindx.getIndexModel()).getCrsCode())) {
            LOGGER.error("Multiple indices with different CRS is not supported");
            throw new RuntimeException("Multiple indices with different CRS is not supported");
          }
        } else {
          if (!indexCrsCode.equals(GeometryUtils.DEFAULT_CRS_STR)) {
            LOGGER.error("Multiple indices with different CRS is not supported");
            throw new RuntimeException("Multiple indices with different CRS is not supported");
          }
        }
      }
    }
First index is Spatial. First iteration gets EPSG:32636 as indexCrsCode successfully. Second iteration it gets Temporal Index, which is not a CustomCrsIndexModel . Falls directly to DEAFULT_CRS_STR check and errors out.
There it checks first index, which is spatial, has the default CRS.
Maybe I got something wrong ?
npv114
@npv114

Java docs for ingest:

<T> void ingest(String inputPath,
Index... index)
Ingest from path. If this is a directory, this method will recursively search for valid files to ingest in the directory. This will iterate through registered IngestFormatPlugins to find one that works for a given file. The applicable ingest format plugin will choose the DataTypeAdapter and may even use additional indices than the one provided.
Parameters:
inputPath - The path for data to read and ingest into this data store
index - The indexing approach to use.

Does the inputPath accept absolute path?
rfecher
@rfecher
@npv114 yes, GeoWave ingest() should be able to work with paths such as s3://<bucket>/path... or hdfs://...
RejiniSP
@RejiniSP
Integrated the geowave plugin in geoserver, but when creating a dynamodb store getting 'java.lang.NullPointerException: Could not acquire data access'. Any one update me how we can connect to dynamodb since we only providing the dynamo endpoint only ?
rfecher
@rfecher
I think "Could not acquire data access" is a generic geoserver error message. Did you create the GeoServer DataStore and/or layer through GeoWave CLI or the GeoServer admin console? do you have a stack trace associated with that error?
RejiniSP
@RejiniSP
image.png
After adding the geowave plugin to the geoserver, tried to create the datastore from Geoserver admin console.
image.png
image.png
While creating the store only the dynamodb endpoint is specified, then how can we connect to the required tables in our aws account?
rfecher
@rfecher
in the geoserver admin console instead of defining "endpoint" define "region" as us-east-1
it uses your AWS credentials resolved through GeoServer to connect to dynamodb ... also it may be helpful to use geowave CLI instead of geoserver admin console which should setup the geoserver datastore exactly the same as the CLI
RejiniSP
@RejiniSP
In our webapplication the geometrical layers are rendered with geoserver.Our requirement is to take geometries from dynamodb and create layers in geoserver by using geowave
image.png
I tried to use region but getting this error
rfecher
@rfecher
do you have any relevant logs in the geoserver log from that?
also, have you tried adding the dynamodb store using geowave's CLI?
RejiniSP
@RejiniSP
No logs for that error
I haven't tried it from cli
RejiniSP
@RejiniSP

@rfecher I tried to create the store using geowave cli, it worked and not getting any error like with geoserver admin console.

But I am not clear how the store can be linked with a dynamodb table in the aws?

Our requirement is to create a geoserver layer from the geometrical data in dynamodb. Is that possible with geowave?

When trying from geoserver console, only the endpoint required and the parameter list not mentioning anything related to the aws credentials to access the dynamodb. So how the credentials are resolved by geoserver?

rfecher
@rfecher
we use the DefaultAWSCrednetialsProviderChain for which there are a series of approaches for providing credentials. Here's some more docs on it.
Nathan Zimmerman
@moradology
I'm curious if anyone could help me understand the implications of this passage from 'theory': At some point, with high precision, high dimensionality curves, the number of possible unit cells can become too large to deal with. In such a case, GeoWave optimizes this by treating the curve as a “lower cardinality” curve than it actually is. So the unit cell size might not be 1, but instead 64, 128, 1024, etc. This allows the user to still achieve high precision when selection windows are small but not spend an inordinate amount of time fully decomposing for large selection windows.
when constructing an index, are there knobs that can be turned to alter this behavior to avoid generating what are sometimes prohibitively large queries?
James Hughes
@jnh5y
@moradology the space filling curves typically used for indexing typically have a "nesting" property which means that one can truncate a key and have a representation of a coarser grid
Nathan Zimmerman
@moradology
do you have a link to some materials/source i could read to learn more about nesting and truncation of these keys (implementation rather than theory) in geowave?
James Hughes
@jnh5y
GeoHashes somewhat clearly explain the big idea around gridding. The wiki page may get to some of the ideas I'm talking about: https://en.wikipedia.org/wiki/Geohash
rfecher
@rfecher
@moradology perhaps this code gives a good reference within geowave...the "knob" this description of "theory" is referring to is the constant "UNIT_CELL_SIZE"
rfecher
@rfecher
basically unit cell size is a cap on the worst case amount of cells produced in decomposition (practically speaking it does not hit this worst case decomposition but it at least provides a ceiling) ... generally speaking when in multi-dimensional space the max range per dimension from the query range is sufficiently small enough to imply that decomposition could not exceed that unit cell size, you get a cell size of one and full decomposition, if you don't cap it though it could lead to exponentially extreme decomposition on large query windows with a high precision curve
Chamin Nalinda
@0xchamin
I am aware of GeoWave's capability of storing and managing point clouds. But I haven't seen work that shares the performance benchmark of using GeoWave for point cloud data management. Are there any while paper or research paper in this. My objective is to compare the performance in terms of storage, loading and querying with other solutions such as PostgreSQL/PostGIS and Oracle.
Brad Hards
@bradh
Unlikely to see Oracle. Their standard EULA prohibits benchmarking. Assume its worse and move on.
Chamin Nalinda
@0xchamin
are there any white paper that publishes results of point cloud data management with in GeoWave??
Brad Hards
@bradh
I just searched Google and it found a couple from 2018.
Chamin Nalinda
@0xchamin
could you please share it ? I found papers that say GeoWave provide facility to manage Point cloud data. But didn't find any work that has done experimentation with point cloud in GeoWave. The materials refer GeoWave as an example system. But I was unable to find a paper that use GeoWave for point cloud data management. could you please share it with me. many thanks!
Brad Hards
@bradh
Maybe it mentioned it, but didn't benchmark it.