locationtech/geowave

Bringing the scalability of distributed computing to modern geospatial software.

rfecher

@rfecher

the data will go into HBase and if you setup EMR to use S3 that means the hbase.rootdir is configured to the S3 bucket and key prefix that you provide EMR. So the HBase tables will be in S3 and you will be able to terminate the cluster, start another one back up with the data fully in tact as long as you use that same S3 location

HuiWang

@scially

I configured many geoservers by ngnix using direction agents, and how to addlayer to this geoserver? is it releated to Lock Management?

kullaibigdata

@kullaibigdata

Hi Please try below one
layer name is your store name
geowave gs addlayer layername

did you configure geowave with geoserver

HuiWang

@scially

but i have many geoserver, example

http://10.66.150.1:8080/geoserver
http://10.66.150.2:8080/geoserver
...
http://10.66.150.10:8080/geoserver

and i used a reverse proxy：http://10.66.150.1:8900/geoserver

does need to reconfigure the geowave.properties each time when execute a command gs addlayer name?

kullaibigdata

@kullaibigdata

which one do u need to add layer

HuiWang

@scially

e... all

kullaibigdata

@kullaibigdata

ok did u create the store and name space

HuiWang

@scially

ok...thanks... and i hava an other problem

Can you explain how to set this parameter?

kullaibigdata

@kullaibigdata

sure

HuiWang

@scially

rfecher

@rfecher

I don't think its very likely you need to be concerned with lock management - its only for maintaining transactions within WFS-T

I think your best/easiest option is to just loop through the geoservers using a script

for gs_url in $(GEOSERVERS); do
  geowave config geoserver $gs_url
  geowave gs addlayer <datastore>
done

HuiWang

@scially

ok , i know ,thans ...

thanks...

kullaibigdata

@kullaibigdata

Hi All,

I am ingesting Table data in hbase using geowave but it's showing not supported raster data.

Please advise anyone.

kullaibigdata

@kullaibigdata

I ingested shape file data in hbase using geowave very first time . Now I am planning to append some more data in my shape file and those are also shape files.

Please advise anyone

What are the steps I need to follow

rfecher

@rfecher

if its the same feature type, it will just append to the existing data by default

kullaibigdata

@kullaibigdata

Yes same type of data we are planning to append .
Can you please share any document related to fro the same or else please share any command on this scenario.

kullaibigdata

@kullaibigdata

@rfecher

1) I appended my data in shape file and I check that count is increasing but when I see in geoserver it's showing my previous shape line same as my last data ?

2) With in same feature type data is ok now . Let's say one example I want add few more columns in feature with in my previous shape file ? Is it creating on problem feature ?

How to handle such type of situation.

rfecher

@rfecher

for (1) its going to include the previous shapes but should also show the new shape. If you configured the layer previously you may need to recalculate the bounds in geoserver to include the new shapes. for (2) if you want to add attribute types to a feature type, that should become a new feature type per geotools/geoservers definition of what a feature type is. There may be some tricks that could be applied to maximize efficiencies of translating the previous data within geowave but that would require familiarity with the geowave baseline internals - really the straightforward thing to do that I recommend for you would be to just take the old features and migrate them to the new feature type.

kullaibigdata

@kullaibigdata

@rfecher

For (1) Thanks it's working fine.

kullaibigdata

@kullaibigdata

@rfecher

Hello Sir,

Is there any possible to view on hdfs data in geoserver. As I checked In geoserver there is no option on hdfs path. We have option to access local directory.

Please advise sir.

rfecher

@rfecher

you're asking about geoserver's handling of URL protocols, assuming you have a file format that is already consumable by geoserver's local file readers

to register an HDFS protocol handler you'd do something like this

if instead you want to use geowave directly, technically HBase is already on HDFS, so I can only assume you're looking for something lighter weight than hbase in which case you can just use geowave-rocksdb on hdfs

githubuser7000

@githubuser7000

I have a build/test failure on Sentinel2 Operations. What I've done so far:

Cloned repo
Updated auth_theia.txt with my credentials
Built successfully by skipping tests

When I build with tests, the failure occurs.

I've debugged it to the extent that I've determined org.locationtech.geowave.format.sentinel2.Tests#jp2ecwPluginIsWorking() is returing false because
the JP2_TEST_FILE ("sentinel2_band_example.jp2") does not exist as part of the project on my local machine.

Build environment is Ubuntu 16 / OpenJDK 8.

I'm literally brand new to this project so any help/guidance is greatly appreciated.

rfecher

@rfecher

the sentinel2 module is something @ahuarte47 contributed so he may be able to further help for anything specific to theia if you are truly interested in that module. If instead you are just looking to get tests to pass I noticed that for some JVM's that have the default timezone far from GMT, some sentinel2 tests fail fails. I had asked someone to test it on their JVM and submit a PR, but I guess that got forgotten. The fix should be to set the timezone to GMT explicitly within a method annotated with @BeforeClass in VectorIngestRunnerTest. You can explicitly set timezone with something like: TimeZone.setDefault(TimeZone.getTimeZone("GMT"))
and just leave auth_theia.txt untouched with the default credentials

...I went ahead and added the timezone change to that test, let me know if it works for you

Alvaro Huarte

@ahuarte47

Hi @githubuser7000, I have verified this "sentinel2_band_example.jp2" file is missing in current master branch. Sometime it was moved. I am going to refresh my geowave version and I will try to fix any issue.

Alvaro Huarte

@ahuarte47

@githubuser7000 I added a PR to fix this missing file: locationtech/geowave#1530

rfecher

@rfecher

Thanks @ahuarte47!

John Wileczek

@JWileczek

Hello @rfecher, and rest of GeoWave team!
Hope all is well! Just wanted to reach out because I was trying to pull down the latest sandbox image to show someone GeoWave, and ran into an access denied issue.
https://locationtech.github.io/geowave/sandbox-env.html
https://s3.amazonaws.com/geowave-vm/1.0.0-SNAPSHOT/VirtualBox_HDP_Geowave_Sandbox_1.0.0-SNAPSHOT.ova
I'm assuming it's just a simple permissions issue that wasn't on the radar. If someone could provide me a valid sandbox download link, or a ping once the permissions are fixed on this one it would be greatly appreciated! Thanks!

githubuser7000

@githubuser7000

Thanks @ahuarte47 and @rfecher !

rfecher

@rfecher

hey @JWileczek, hope all is well! this comment speaks to it a bit: https://github.com/locationtech/geowave/issues/1520#issuecomment-463676899

basically the docs are versioned with the VMs which is a challenge for what we want to do with VMs (we couldn't make it part of the build process because there was a snag in automating it), so snapshot VMs aren't really a thing and the 0.9.8 VM isn't just going to work in all cases with 1.0, but may in most...

so you can navigate to any release docs such as 0.9.8 and the link to the VM is good: http://s3.amazonaws.com/geowave/0.9.8/docs/quickstart.html

rfecher

@rfecher

or honestly with rocksdb and redis support it is infinitely easier to have a quickstart guide use those data stores locally without the need for basically the hadoop ecosystem kitchen sink coming from that VM which was based off hortonworks data platform ... which is what we will move towards with HDP going away ... there was always the nicety of being able to experiment with the couple dozen different services/apps available in HDP, but a simpler local setup is probably more compelling

kullaibigdata

@kullaibigdata

Hi ,

We are processing spatial data in hive and we are converting result into the shape files using python script.

When we are trying to ingest those shape files in Hbase getting error and please go through below one.

Store Name: test_shape
Index: test_shape_spatial

Data:

test_shape.dbf
test_shape.shp
test_shape.shx

ERROR: Unable to ingest data source for feature name 'test_shape'

IOException: Unsupported shape type:Null

Please advise for the same.

rfecher

@rfecher

I'd assume you're hitting this in geotools when parsing the shapefile: https://github.com/geotools/geotools/blob/master/modules/plugin/shapefile/src/main/java/org/geotools/data/shapefile/shp/ShapefileReader.java#L350

which seems to indicate the shapefile is not to spec (or at least not compatible with geotools parsing)? Can you successfully load and visualize that test_shape.shp into QGIS?

kullaibigdata

@kullaibigdata

Thanks rfecher,
We did some mistakes in python script, Now we are modifying that script now it's working fine .
please find the reference URL future purpose.

https://gis.stackexchange.com/questions/248461/shapely-multiline-string-to-shapefile-as-single-line

Where communities thrive

People

Repo info

Activity