big-data-europe/Lobby

Anton Kulaga

@antonkulaga

@earthquakesan is it possible for you to publish docker with spark 2.3.1 ? I have some minor dependency clashes with 2.3.0 container using SANSA-RDF dependencies

Anton Kulaga

@antonkulaga

Any plands for alluxio docker-swarm configs?

comboo

@wings-xue

I can't open http://<dockerhadoop_IP_address>:8088/ , and i see run.sh , there is not run commod about yarn ?

Peter Viskovics

@jr.visko_gitlab

Hi,

when running docker pull bde2020/hive I face this problem:
hive_1 | mkdir: Call From 57c424f72c21/172.22.0.2 to 57c424f72c21:8020 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused
hive_1 | mkdir: Call From 57c424f72c21/172.22.0.2 to 57c424f72c21:8020 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused
hive_1 | chmod: Call From 57c424f72c21/172.22.0.2 to 57c424f72c21:8020 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused
hive_1 | chmod: Call From 57c424f72c21/172.22.0.2 to 57c424f72c21:8020 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused
and beeline cannot join to localhost:10000
➜ epam-qa-metrics git:(master) ✗ docker-compose exec hive bash
root@57c424f72c21:/opt# /opt/hive/bin/beeline -u jdbc:hive2://localhost:10000
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/hive/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/hadoop-2.7.4/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Connecting to jdbc:hive2://localhost:10000
18/11/14 13:11:30 [main]: WARN jdbc.HiveConnection: Failed to connect to localhost:10000
Could not open connection to the HS2 server. Please check the server URI and if the URI is correct, then ask the administrator to check the server status.
Error: Could not open client transport with JDBC Uri: jdbc:hive2://localhost:10000: java.net.ConnectException: Connection refused (Connection refused) (state=08S01,code=0)
Beeline version 2.3.2 by Apache Hive
beeline> CREATE TABLE pokes (foo INT, bar STRING);
No current connection

this is my docker version:
➜ ~ docker -v
Docker version 18.06.1-ce, build e68fc7a

can anyone help please?

Peter Viskovics

@jr.visko_gitlab

The issue is resolved, but I think it may be worth to mention on bde2020/hive that this is just part of the whole project, which can be found on https://github.com/big-data-europe/docker-hive

Andrii Gakhov

@gakhov

Hi guys! For anyone interested in learning space-efficient data structures and fast algorithms that are extremely useful in modern Big Data applications, take a look at my recently published book "Probabilistic Data Structures and Algorithms for Big Data Applications" (ISBN: 978-3748190486). In this book, you can find algorithms and data structures for Membership querying (Bloom filter, Counting Bloom filter, Quotient filter, Cuckoo filter), Cardinality (Linear counting, probabilistic counting, LogLog, HyperLogLog, HyperLogLog++), Frequency (Majority algorithm, Frequent, Count Sketch, Count-Min Sketch), Rank (Random sampling, q-digest, t-digest), and Similarity (LSH, MinHash, SimHash). Check at Amazon or the book's webpage

Zhoodar

@zhoodar

Hello there, does someone familiar with this isue big-data-europe/docker-hadoop#38

It appeared when I tried to write data into remote hdfs.

purbanow

@purbanow

Hi guys, When running a job with 1 spark worker + hadoop setup everything is going well by when im trying to run with 2 workers im getting : JvmPauseMonitor: Detected pause in JVM or host machine (eg GC)

any ideas?

Juan Santillana

@ratasxy_twitter

Hi I have a cuestiones in big-data-europe/hbase-docker for use the port 9090

just should I expose the port?

Xining Li

@xiningli

Hello, I am new here.

Diego Quintana

@diegoquintanav

just should I expose the port?
to connect a client to hdfs from the docker host, yes

@ratasxy_twitter ^

I'm also running into problems with that repo. How should I connect a client using the java API?

                Configuration config = HBaseConfiguration.create();
                config.set("hbase.zookeeper.quorum", "localhost");
                config.set("hbase.zookeeper.property.clientPort", "2181");
                HBaseAdmin.checkHBaseAvailable(config);

Returns org.apache.hadoop.hbase.MasterNotRunningException: java.net.UnknownHostException: can not resolve hbase-master,16000,1592487871967

Diego Quintana

@diegoquintanav

I'm getting that error

Diego Quintana

@diegoquintanav

any ideas?

billsteve

@billsteve

could you "ping hbase-master " or "telnet hbase-master 16000"?

luigi-asprino

@luigi-asprino

Hi all, anyone can help me with this issue big-data-europe/docker-hadoop#79

Anatoly Danilov

@anatolyD

Hey guys, the 3.0.0 is announced in README.md although i fail to find it in the docker registry, anyone had it working?

billsteve

@billsteve

Which image?

yunkai

@JanYunkai

java.net.ConnectException: Call to 5830b8c484ea/127.0.0.1:42667 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: 5830b8c484ea/127.0.0.1:42667

docker-compose-standalone.yml

yunkai

@JanYunkai

skip

ktpktr0

@ktpktr0

Hey guys, I installed big-data-europe/docker-hadoop on swarm and they run successfully. But I found a problem. The datanode ip in hdfs dfsadmin -report does not match the actual datanode. This error prevents me from installing hbase correctly. I don’t know if anyone has encountered such a problem

Marcel-Jan Krijgsman

@Marcel-Jan

Hi everyone,
I created a shared Hadoop-Spark-Hive docker-compose based on your great Dockerized Hadoop, Spark and Hive versions.
https://github.com/Marcel-Jan/docker-hadoop-spark
And I blogged about it how I got it working here:
https://marcel-jan.eu/datablog/2020/10/25/i-built-a-working-hadoop-spark-hive-cluster-on-docker-here-is-how/

Ben Baysinger

@BennyBaysinger_twitter

@Marcel-Jan Have you been able to get the Namenode UI to allow upload and download of data?

Marcel-Jan Krijgsman

@Marcel-Jan

@BennyBaysinger_twitter I've just tried. I can create a directory, but I can't upload a file. I'll look into that.

4 replies

Where communities thrive

People

Repo info

Activity