Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Jul 14 2017 09:41

    pomadchin on master

    Update README.md (compare)

  • Feb 03 2017 20:24
    pomadchin closed #4
  • Feb 03 2017 20:24
    pomadchin commented #4
  • Feb 03 2017 20:23

    pomadchin on master

    Fix quay.io link Merge pull request #5 from jwal… (compare)

  • Feb 03 2017 20:23
    pomadchin closed #5
  • Feb 03 2017 20:23
    pomadchin commented #5
  • Feb 03 2017 20:10
    jwalgran opened #5
  • Dec 10 2016 21:50
    pomadchin commented #4
  • Dec 10 2016 13:42
    Plippe opened #4
Emilio
@elahrvivaz
i bumped the version to 1.9.2 and switched the link to archive.apache.org
Grigory
@pomadchin
:+1:
Rakesh
@rakeshkr00
@elahrvivaz If I run the latest docker image with EMR having hadoop 2.8.4, will that work ?
I think the docker images uses the host's hadoop config and binaries
Austin Heyne
@aheyne
@rakeshkr00 That should be fine, I've been running the gm 2.x line on hadoop 2.8.4 for a while now.
namely emr 15-19
Rakesh
@rakeshkr00
thanks @aheyne
Rakesh
@rakeshkr00
After deployment of the docker images with Accumulo 1.9.2, I am encountering below error. Any help would be appreciated.
docker logs accumulo-master
Template config: /etc/hadoop/conf/core-site.xml from /etc/hadoop/conf/core-site.xml.template
/sbin/hdfs-lib.sh: line 11: HADOOP_MASTER_ADDRESS: unbound variable
Found config: /etc/hadoop/conf/core-site.xml
/etc/hadoop/conf/core-site.xml:1: parser error : Document is empty
Grigory
@pomadchin
hey @rakeshkr00 so you don’t have HADOP_MASTER_ADDRESS var set?
Rakesh
@rakeshkr00
@pomadchin where to set that env variable? On EMR I bootstrap using the bootstrap script given in the geomesa-accumulo.
Grigory
@pomadchin
Anywhere; this var has to be passed on the container startup
^ just a docker-compose example but can give you clues about how to start it
Rakesh
@rakeshkr00
In EMR, it asks for the bootstrap script and then it prepare and start the cluster.
Grigory
@pomadchin
just pass it via -e VARNAME=VARVALUE
Rakesh
@rakeshkr00
@pomadchin only after the cluster instances are created, I could know the HADOOP_MASTER_ADDRESS. The bootstrap script is given as part of request for an EMR cluster. So, I don't see how I can provide the -e VARNAME=VARVALUE
Grigory
@pomadchin
hm it is possible
i just don’t remember how
i’ll get back to you soon
MASTER_IP=$(xmllint --xpath "//property[name='yarn.resourcemanager.hostname']/value/text()"  /etc/hadoop/conf/yarn-site.xml)
^ you can do smth like that in boostrap script
Rakesh
@rakeshkr00
Sure. Thanks
Rakesh
@rakeshkr00
thanks @pomadchin I got past that error using your suggestion.
Grigory
@pomadchin
:+1:
Rakesh
@rakeshkr00

When I am connecting to accumulo shell it gives

ERROR: Problem reading instance id out of hdfs at hdfs://<hostname>:8020/gis/instance_id

why it is not picking up from https://github.com/geodocker/geodocker-geomesa/blob/master/geodocker-accumulo-geomesa/docker-compose.yml. Pls help.

the port in the error msg is 8020 and in docker-compose it is 8121
Grigory
@pomadchin
I think you can specify shell port
Are you are trying to log into the shell from your accumulo master container?
Rakesh
@rakeshkr00
@pomadchin to me it seems like for some reason docker-compose.yml is not getting executed/picked up. Previously I had issue with HADOOP_MASTER_ADDRESS (which I got past after using -e ENVVAR=VARVALUE), and now the zookeeper port issue.
Grigory
@pomadchin
Accumulo servers are up?
It’s UI is available?
Rakesh
@rakeshkr00
the full error was
2018-11-20 20:56:47,718 [zookeeper.ZooUtil] ERROR: Problem reading instance id out of hdfs at hdfs://<hostName>:8020/gis/instance_id
org.apache.hadoop.security.AccessControlException: Permission denied: user=root, access=EXECUTE, inode="/gis":hadoop:hadoop:drwx------
Grigory
@pomadchin
ha indeed
looks like smth is wrong with zoo
ah w8
i think we need geomesa people there to help
smth with the --hostname of containers there
looks like some hadoop stack security thing
or probably smth is not up
or tries to be up but in a wrong moment
@elahrvivaz
Emilio
@elahrvivaz
is it just the shell that's failing?
seems to be hdfs user permissions
Austin Heyne
@aheyne
probably need to do sudo -u hadoop stuff
Rakesh
@rakeshkr00
it seem permission as geomesa is not installed. geomesa says command not found.
Emilio
@elahrvivaz
the command is geomesa-accumulo now
Rakesh
@rakeshkr00

@elahrvivaz

geomesa-accumulo
Error: Could not find or load main class .usr.lib.zookeeper.zookeeper.jar

Thanks

Emilio
@elahrvivaz
seems like something is screwed up in that script - that shouldn't be the main class being executed
Rakesh
@rakeshkr00
oh
Rakesh
@rakeshkr00
fyi: after giving hadoop hdfs chmod 777 /gis, the accumulo shell came up fine, though 777 is not an advisable permission