Bringing the scalability of distributed computing to modern geospatial software.
@rfecher yes shp file has prj file. I read feature and build my own type, not directly ingesting shp file, write geometry into my own type and ingest it. Thats the same code I ve sent. Simply I get geometry and insert it to my newly created type. If I dont need to set any CRS info ingesting via new type, what should I do ?
Leaving as it is gives the result as above, kind of confused
@rfecher index error however continues, can you reproduce it ? 1.2.0-SNAPSHOT has the same behavior.
SpatialIndexBuilder spIdxBldr = new SpatialIndexBuilder();
spIdxBldr.setCrs("EPSG:32636");
spatialIndex = spIdxBldr.createIndex();
TemporalIndexBuilder tmpIdxBldr = new TemporalIndexBuilder();
tmpIdxBldr.setName("TEMPORAL_IDX");
tmpIdxBldr.setPeriodicity(TemporalBinningStrategy.Unit.HOUR);
temporalIndex = tmpIdxBldr.createIndex();
dataStore.addIndex(spatialIndex);
dataStore.addIndex(temporalIndex);
dataStore.addType(sfAdapter, spatialIndex,temporalIndex);25 Oct 12:33:01 ERROR [vector.FeatureDataAdapter] - Multiple indices with different CRS is not supported
Exception in thread "main" java.lang.RuntimeException: Multiple indices with different CRS is not supported
at org.locationtech.geowave.adapter.vector.FeatureDataAdapter.init(FeatureDataAdapter.java:241)
at org.locationtech.geowave.core.store.base.BaseDataStore.internalAddIndices(BaseDataStore.java:1142)
at org.locationtech.geowave.core.store.base.BaseDataStore.addType(BaseDataStore.java:1167)
at com.uasis.geowaveapi.Geowave.ingestFromFile(Geowave.java:95)
at com.uasis.geowaveapi.Geowave.main(Geowave.java:82)public boolean init(final Index... indices) throws RuntimeException {
String indexCrsCode =
reprojectedFeatureType == null ? null
: GeometryUtils.getCrsCode(reprojectedFeatureType.getCoordinateReferenceSystem());
for (final Index primaryindx : indices) {
// for first iteration
if (indexCrsCode == null) {
if (primaryindx.getIndexModel() instanceof CustomCrsIndexModel) {
indexCrsCode = ((CustomCrsIndexModel) primaryindx.getIndexModel()).getCrsCode();
} else {
indexCrsCode = GeometryUtils.DEFAULT_CRS_STR;
}
} else {
if (primaryindx.getIndexModel() instanceof CustomCrsIndexModel) {
// check if indexes have different CRS
if (!indexCrsCode.equals(
((CustomCrsIndexModel) primaryindx.getIndexModel()).getCrsCode())) {
LOGGER.error("Multiple indices with different CRS is not supported");
throw new RuntimeException("Multiple indices with different CRS is not supported");
}
} else {
if (!indexCrsCode.equals(GeometryUtils.DEFAULT_CRS_STR)) {
LOGGER.error("Multiple indices with different CRS is not supported");
throw new RuntimeException("Multiple indices with different CRS is not supported");
}
}
}
}
First index is Spatial. First iteration gets EPSG:32636 as indexCrsCode successfully. Second iteration it gets Temporal Index, which is not a CustomCrsIndexModel . Falls directly to DEAFULT_CRS_STR check and errors out.
There it checks first index, which is spatial, has the default CRS.
Maybe I got something wrong ?
There it checks first index, which is spatial, has the default CRS.
Maybe I got something wrong ?
Hi, I am trying to use https://locationtech.github.io/geowave/latest/apidocs/org/locationtech/geowave/core/store/api/DataStore.html ingest with s3 file path. How can I do that
Java docs for ingest:
<T> void ingest(String inputPath,
Index... index)
Ingest from path. If this is a directory, this method will recursively search for valid files to ingest in the directory. This will iterate through registered IngestFormatPlugins to find one that works for a given file. The applicable ingest format plugin will choose the DataTypeAdapter and may even use additional indices than the one provided.
Parameters:
inputPath - The path for data to read and ingest into this data store
index - The indexing approach to use.
Does the inputPath accept absolute path?
After adding the geowave plugin to the geoserver, tried to create the datastore from Geoserver admin console.
While creating the store only the dynamodb endpoint is specified, then how can we connect to the required tables in our aws account?
it uses your AWS credentials resolved through GeoServer to connect to dynamodb ... also it may be helpful to use geowave CLI instead of geoserver admin console which should setup the geoserver datastore exactly the same as the CLI
I tried to use region but getting this error
also, have you tried adding the dynamodb store using geowave's CLI?
I haven't tried it from cli
@rfecher I tried to create the store using geowave cli, it worked and not getting any error like with geoserver admin console.
But I am not clear how the store can be linked with a dynamodb table in the aws?
Our requirement is to create a geoserver layer from the geometrical data in dynamodb. Is that possible with geowave?
When trying from geoserver console, only the endpoint required and the parameter list not mentioning anything related to the aws credentials to access the dynamodb. So how the credentials are resolved by geoserver?
we use the DefaultAWSCrednetialsProviderChain for which there are a series of approaches for providing credentials. Here's some more docs on it.
I'm curious if anyone could help me understand the implications of this passage from 'theory':
At some point, with high precision, high dimensionality curves, the number of possible unit cells can become too large to deal with. In such a case, GeoWave optimizes this by treating the curve as a “lower cardinality” curve than it actually is. So the unit cell size might not be 1, but instead 64, 128, 1024, etc. This allows the user to still achieve high precision when selection windows are small but not spend an inordinate amount of time fully decomposing for large selection windows.
when constructing an index, are there knobs that can be turned to alter this behavior to avoid generating what are sometimes prohibitively large queries?
GeoHashes somewhat clearly explain the big idea around gridding. The wiki page may get to some of the ideas I'm talking about: https://en.wikipedia.org/wiki/Geohash
@moradology perhaps this code gives a good reference within geowave...the "knob" this description of "theory" is referring to is the constant "UNIT_CELL_SIZE"
basically unit cell size is a cap on the worst case amount of cells produced in decomposition (practically speaking it does not hit this worst case decomposition but it at least provides a ceiling) ... generally speaking when in multi-dimensional space the max range per dimension from the query range is sufficiently small enough to imply that decomposition could not exceed that unit cell size, you get a cell size of one and full decomposition, if you don't cap it though it could lead to exponentially extreme decomposition on large query windows with a high precision curve
I am aware of GeoWave's capability of storing and managing point clouds. But I haven't seen work that shares the performance benchmark of using GeoWave for point cloud data management. Are there any while paper or research paper in this. My objective is to compare the performance in terms of storage, loading and querying with other solutions such as PostgreSQL/PostGIS and Oracle.
could you please share it ? I found papers that say GeoWave provide facility to manage Point cloud data. But didn't find any work that has done experimentation with point cloud in GeoWave. The materials refer GeoWave as an example system. But I was unable to find a paper that use GeoWave for point cloud data management. could you please share it with me. many thanks!
@rfecher :tada:
interesting that the netCDF reader is giving you a GridCoverage2D for each band in the image ... so for term clairification a "Coverage name" refers to a homogenous gridded dataset (the same sample model, which can be multiple types, where each band is of a given type, but every grid position is going to have the same bands/types) and from my experience with multi-dimensional data using net CDF each dimension is a band, not a separate coverage ... basically, my understanding of your statement "using the geotools netCDF reader, i got a GridCoverage2D for each band in the image" is that for
n bands you got n single-band coverages rather than one n-band coverage. If thats the case, and you want a single multi-band coverage I've used geotools' BandMerge operation successfully before (just make sure you do it before ingesting into geowave so geowave gets a consistent representation of the coverage). Actually this is exactly one of the options available within geowave's landsat8 ingest so here's code referencing how to do exactly this