locationtech/geowave

Bringing the scalability of distributed computing to modern geospatial software.

Denis Rykov

@drnextgis

Hello. I installed GeoWave on Linux according to Installation Guide. But when I run geowave help command I get these warnings:

$ geowave help
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by javassist.ClassPool (file:/home/denis/geowave/lib/core/javassist-3.20.0-GA.jar) to method java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int)
WARNING: Please consider reporting this to the maintainers of javassist.ClassPool
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release

Denis Rykov

@drnextgis

seems like it is something related to Java 11, with Java 8 I don't see it anymore

Johnathan Garrett

@jdgarrett

@drnextgis Thank you for bringing this to our attention. Currently GeoWave is built and tested on Java 8.

Grigory

@pomadchin

@jdgarrett @drnextgis I actually tried to build a library with geowave as a dep using Java 11 (tbh by an accident) - it fails indeed (almost all the reflective stuff throws exceptions)

Nick Levandoski

@metric-chicken

Good morning, I'm using GeoWave-1.1.0 and trying to ingest data with a Mapreduce job. When I run the job I get an error:

org.opengis.referencing.NoSuchAuthorityCodeException: No code "EPSG:4326" from authority "European Petroleum Survey Group" found for object of type "IdentifiedOb
ject".

I'm stumped. Has anyone received this error before?

rfecher

@rfecher

@metric-chicken are you using a jar file that geowave supplies or are you building your own? My understanding is that a dependency org.geotools:gt-epsg-wkt is critical to have on the classpath for discovering these EPSG coordinate reference systems (using SPI), and should be in included in any of the geowave shaded jars that are distributed. The only other thing I can think of if it is on your classpath is that SPI is being hosed somehow in the context of your hadoop workers, not exactly sure how.

Nick Levandoski

@metric-chicken

@rfecher I've been using org.geotools:gt-epsg-hsql. I'll try the gt-epsg-wkt

rfecher

@rfecher

ahh, hsql writes out a DB to a tmp directory and I believe that in fact does cause some trouble in the context of a mapreduce job while gt-epsg-wkt just has it as a resource on the classpath and is fine in the MR worker node context - I think and hope that this will do the trick for you

Nick Levandoski

@metric-chicken

20/04/20 17:07:53 ERROR vector.FeatureDataAdapter: Unable to decode 'EPSG:4326' CRS
org.opengis.referencing.NoSuchAuthorityCodeException: No code "EPSG:4326" from authority "European Petroleum Survey Group" found for object of type "IdentifiedObject".

I wished it worked

rfecher

@rfecher

damn

Nick Levandoski

@metric-chicken

Ive worked with a co-worker who's really GeoWave savvy but were stumped

its like the Index never gets set in the GeoWaveOutputFormat

I've even tried making my own CUSTOM:4326 CRS using WKT

rfecher

@rfecher

did you take org.geotools:gt-epsg-hsql out of the classpath when you added org.geotools:gt-epsg-wkt?

Nick Levandoski

@metric-chicken

yes. I commented it out

rfecher

@rfecher

I'm not entirely sure what you mean by "its like the Index never gets set in the GeoWaveOutputFormat"

well, in other words, I'm not sure how that relates to the problem

Nick Levandoski

@metric-chicken

It might not. Im new to GeoWave. I assumed that I had to specify the CRS in the SpatialTemporalIndex.

rfecher

@rfecher

you can optionally specify it, although the default CRS is EPSG:4326

Nick Levandoski

@metric-chicken

I just noticed that in the source, sorry

Maybe adding the gt-epsg-wkt to the distributed class will work. let me try that

rfecher

@rfecher

thats ok, but then to do any coordinate transforms geotools needs to look up the crs in the classpath (geotools is a popular underlying library that geowave uses for many GIS operations such as this)

yeah, it has to be on the worker nodes classpath because they are the ones really doing the ingest

Nick Levandoski

@metric-chicken

getting the same error but I'm seeing this as well

WARNING: Extension lookup 'ExtensionFilter', but ApplicationContext is unset.                                                                                                                                             
20/04/20 17:30:42 ERROR util.GeometryUtils: Unable to decode EPSG:4326 CRS

rfecher

@rfecher

i take it this is not using the geowave CLI, right? If so, I wonder if you can try using the CLI: geowave-$VERSION-$VENDOR-tools as documented here: https://locationtech.github.io/geowave/downloads.html

I believe it assumes HADOOP_HOME is set so that it can launch the MR ingest job

Nick Levandoski

@metric-chicken

no its using yarn to deploy the job

rfecher

@rfecher

at least that way if that part works we could back off and see what is different or it doesn't work with the same problem then we'd know it isn't necessarily related to how you launch it

Nick Levandoski

@metric-chicken

ok ill give it a shot and let you know

Michael Fazio

@michaeljfazio

Any advise for building fat jar with geowave-datastore-hbase and geowave-adapter-vector for Spark? Seems like a never ending assembly merge conflict :(

Michael Fazio

@michaeljfazio

at least with geowave-adapter-vector, it would seem there are many conflicting transitive dependencies.

Michael Fazio

@michaeljfazio

Nevermind! Gave up on sbt. Went for maven instead ;)

Michael Fazio

@michaeljfazio

Hi @rfecher

I've developed a plugin you might be interested in for addition to geowave core.

Let me know if you have some time to discuss

rfecher

@rfecher

sorry, I just caught this message now @michaeljfazio ... is this the Glue MetaStore ingest format discussed in issue #1705 ?

Zachary Whitley

@zacharywhitley

I know this is a stupid question but can someone let me know what artifact provides SpatialIndexBuilder?

rfecher

@rfecher

@zacharywhitley it's in org.locationtech.geowave:geowave-core-geotime

Kirin Patel

@ajchili

Hi all, I am just getting started with Geowave and DynamoDB for spatial-temporal queries and I not sure if the latency I am experiencing with queries should be expected. Currently I have some GDELT data (roughly 4k rows) and I am querying on actor name but my query is taking roughly 6 seconds. I am using the Java SDK. Does anyone have any insight on why this may be the case or if this is the expected behavior?

James Hughes

@jnh5y

Did you create an index on actor name? (If not, your query may end up scanning everything in DynamoDB)

Kirin Patel

@ajchili

Hey @jnh5y, I have not and that likely would be why I am experiencing the latency. Would I just create a secondary index for the actor name via Geowave?

James Hughes

@jnh5y

@ajchili to be honest, I don't know how GeoWave handles secondary indexes, so I'm just answering the first question.;)

Kirin Patel

@ajchili

@jnh5y no worries, you've helped point me in the right direction. Thanks so much!

James Hughes

@jnh5y

loridigia

@loridigia

Hello guys, does Geowave currently support hbase 2.0.x ?

rfecher

@rfecher

@loridigia not currently but it is on the short-list for future enhancements ... we have locationtech/geowave#1609 for hbase 2.0 support but it could use some updating and testing

yaseminakyuz

@YaseminAkyuz

Helloi I'm just getting started with Geowave and Cassandra. The cassandra runs locally. I'm trying to add a data store using following cli command: geowave store add --gwNamespace geowave.gdelt -t cassandra --contactPoints 127.0.0.1:9042. I'm getting error and could not a solution. Error: java.lang.ExceptionInInitializerError
at org.locationtech.geowave.core.store.cli.store.DataStorePluginOptions.selectPlugin(DataStorePluginOptions.java:72)
at org.locationtech.geowave.core.store.cli.store.AddStoreCommand.prepare(AddStoreCommand.java:60)
at org.locationtech.geowave.core.cli.parser.OperationParser.parseInternal(OperationParser.java:94)
at org.locationtech.geowave.core.cli.parser.OperationParser.parse(OperationParser.java:65)
at org.locationtech.geowave.core.cli.GeoWaveMain.main(GeoWaveMain.java:34)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:567)
at com.exe4j.runtime.LauncherEngine.launch(LauncherEngine.java:85)
at com.exe4j.runtime.WinLauncher.main(WinLauncher.java:94)
at com.install4j.runtime.launcher.WinLauncher.main(WinLauncher.java:25)
Caused by: java.lang.IllegalArgumentException: org.locationtech.geowave.core.store.GeoWaveStoreFinder is not an ImageIO SPI class
at java.desktop/javax.imageio.spi.ServiceRegistry.checkClassAllowed(ServiceRegistry.java:722)
at java.desktop/javax.imageio.spi.ServiceRegistry.<init>(ServiceRegistry.java:117)
at org.locationtech.geowave.core.index.SPIServiceRegistry.<init>(SPIServiceRegistry.java:42)
at org.locationtech.geowave.core.store.GeoWaveStoreFinder.getRegisteredFactories(GeoWaveStoreFinder.java:326)
at org.locationtech.geowave.core.store.GeoWaveStoreFinder.getRegisteredStoreFactoryFamilies(GeoWaveStoreFinder.java:281)
at org.locationtech.geowave.core.store.GeoWaveStoreFinder.getStoreNames(GeoWaveStoreFinder.java:229)
at org.locationtech.geowave.core.store.GeoWaveStoreFinder.<clinit>(GeoWaveStoreFinder.java:40)

Do you have any idea or solution above error?

@rfecher

rfecher

@rfecher

@YaseminAkyuz my guess is that you are using a newer version of java than java8 ... geowave currently requires java8

Where communities thrive

People

Repo info

Activity