Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Antoine Cotten
    @antoineco
    @JackyW83 if you don't clean up old data with curator, Elasticsearch keeps your entire history forever.
    @weichea can you try the default-paid-trial branch instead? We're currently merging it to x-pack.
    weichea
    @weichea
    @antoineco thanks for the response. I tried the default-paid-trial branch, and still the same error. I havent not make any changes to the configuration. just switch branch and compose up.
    Antoine Cotten
    @antoineco
    @weichea you forgot docker-compose build I think
    weichea
    @weichea
    @antoineco thank you! thats right.
    etriti
    @etriti

    hey guys, since I upgraded from v6.7.0 to 7.0.1 i have issues with the master node. When starting services, i get this message from Elasticsearch:
    "message": "master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes [docker-cluster] to bootstrap a cluster: have discovered []; discovery will continue using [] from hosts providers and [{635a3b09de09}{uLT2f9gqShOyinDUrK_QdQ}{Q_59sQbRSdW9U9n1hmSktg}{172.25.0.2}{172.25.0.2:9300}{ml.machine_memory=2096164864, xpack.installed=true, ml.max_open_jobs=20}] from last-known cluster state; node term 0, last-accepted version 0 in term 0".

    elasticsearch.yml:

    cluster.name: "docker-cluster"
    network.host: 0.0.0.0
    cluster.initial_master_nodes: "docker-cluster"

    Someone had the same issue?

    Antoine Cotten
    @antoineco
    @etriti cluster.initial_master_nodes: "docker-cluster" there is no node with that name, node names are randomly generated unless you set them explicitly.
    etriti
    @etriti
    @antoineco so in order to scale out my cluster, I need to set a "cluster.initial_master_nodes", that means that the only way to do it is:
    docker-compose scale elasticsearch=2
    set "cluster.initial_master_nodes" to a node name randomly generated and restart that node.
    Am I right?
    Thanks for your help
    Antoine Cotten
    @antoineco
    @etriti you don't even need to fiddle with names (which can become complex in such a dynamic environment). We have detailed instructions on how to scale out Elasticsearch in the README, did you check them out?
    The trick is to use the dynamic docker DNS to form your cluster. Zen supports it out of the box, it's really simple.
    Nuno Ferreira
    @NunoCardia
    Hi guys,
    This message was deleted

    When i executed "docker-compose up" on your project i got the following error:
    Starting docker-elk-master_elasticsearch_1 ... error

    ERROR: for docker-elk-master_elasticsearch_1 Cannot start service elasticsearch: b'OCI runtime create failed: container_linux.go:344: starting container process caused "process_linux.go:424: container init caused \"rootfs_linux.go:58: mounting \\\"/host_mnt/d/NB24801/Downloads/docker-elk-master/elasticsearch/config/elasticsearch.yml\\\" to rootfs \\\"/var/lib/docker/overlay2/cf157cb8af7079f9aeb00a6af47cd9f0c51e161cc696388d28483dadd0704b3a/merged\\\" at \\\"/var/lib/docker/overlay2/cf157cb8af7079f9aeb00a6af47cd9f0c51e161cc696388d28483dadd0704b3a/merged/usr/share/elasticsearch/config/elasticsearch.yml\\\" caused \\\"not a directory\\\"\"": unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type'

    ERROR: for elasticsearch Cannot start service elasticsearch: b'OCI runtime create failed: container_linux.go:344: starting container process caused "process_linux.go:424: container init caused \"rootfs_linux.go:58: mounting \\\"/host_mnt/d/NB24801/Downloads/docker-elk-master/elasticsearch/config/elasticsearch.yml\\\" to rootfs \\\"/var/lib/docker/overlay2/cf157cb8af7079f9aeb00a6af47cd9f0c51e161cc696388d28483dadd0704b3a/merged\\\" at \\\"/var/lib/docker/overlay2/cf157cb8af7079f9aeb00a6af47cd9f0c51e161cc696388d28483dadd0704b3a/merged/usr/share/elasticsearch/config/elasticsearch.yml\\\" caused \\\"not a directory\\\"\"": unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type'
    ERROR: Encountered errors while bringing up the project.

    Can you help? thanks in advance
    nicks1993
    @nicks1993
    Hey what would be the best way to parse this time field out of "message: words::words::words (time=517, words)" to get the time=517 field out of this message block and create a new field with this "time: 517" field?
    Herpiko Dwi Aguno
    @herpiko
    Hi, is there any elk docker image that fit on armhf architecture? I'm readjusting and building the image from scratch (based on spujadas/elk-docker). In case there is already such image for armhf, I'll drop the effort.
    iDmple
    @iDmple
    hi everyone, thanks for the really nice work on docker-elk. Do you have any plans to integrate ES 7.1.x? The security features are now free from that version and I'd like to use them. Thanks!
    Antoine Cotten
    @antoineco
    @iDmple definitely. If you want to test it already, all you have to do is bump the version number inside the .env file as indicated in the README. PR and feedback welcome!
    @herpiko (sorry for the very late answer) unfortunately Elastic only releases Docker images for x86_64.
    iDmple
    @iDmple
    @antoineco thank you for your answer. Do you have any timeline for version change to 7.1.x? I'm in the process of switching my install to the master branch so I can try and use it. I'll let you know if I have any issues
    Antoine Cotten
    @antoineco
    @iDmple I just opened an issue for it (#401), tests are running: https://travis-ci.org/deviantony/docker-elk/builds/547187821. As soon as they pass, I'll merge to master.
    iDmple
    @iDmple
    @antoineco thank you you're awesome!
    iDmple
    @iDmple
    @antoineco another question, do you have any plans to add support for the keystore?
    iDmple
    @iDmple
    for elasticsearch and logstash the keystores are created automatically, i only had to bind the files
    for kibana i had to create it too
    iDmple
    @iDmple
    ok for logstash also doesn't exist
    Antoine Cotten
    @antoineco
    @iDmple are you referring to #402?
    iDmple
    @iDmple
    @antoineco not at all
    sorry for the late answer
    Antoine Cotten
    @antoineco
    OK. TLS features also use keystores so I was confused.
    What do you mean by "adding support"? Anything stopping you from using them with the provided config?
    iDmple
    @iDmple
    @antoineco it was complicated for elasticsearch because i had to go inside the docker container to cp the already created keystore and add it to the docker-compose.yml, for logstash and kibana it was even worse as i had to create them and figure out the right paths
    i just think if this was already done it could save people a lot of trouble
    as it took me a while to figure things and make them work
    to me this is a step everyone will have to do so that passwords aren't stored in config files
    yeah i also then had to modify the config files and the logstash one was different so it was also something
    Maxime Lebastard
    @maximelebastard
    Hi guys ! :wave: How do you configure multiple logstash pipelines ? I tried to create one configuration file per pipeline in logstash/pipeline - but it results on Elasticsearch mixing the indexes (some data have the bad index despite of my configuration).
    Is it the right way to configure multiple pipelines ? Thx, have a nice weekend !
    Antoine Cotten
    @antoineco
    @maximelebastard you have to remove the main pipeline first. Take a look at https://github.com/deviantony/docker-elk/issues/343#issuecomment-449646117
    muzam
    @muzam
    I am developing a kibana plugin when I choose the packages for it should I consciously check whether kibana or search guard is using the same version of libraries or can I blindly go forward and choose my own preferred version. If so will there be any conflict with it or will any of the kibana functionality get impacted?
    Christine Campo
    @ccampo55
    With this latest version, has anyone been able to connect to MySQL? I have the MySQL connector working with an older elastic stack, but I can't seem to get it working with this one. AFAIK all my settings are the same. All I get is :
    logstash_1 | [2019-10-07T17:32:33,962][ERROR][logstash.inputs.jdbc ] Failed to load /usr/share/mysql-connector-java-8.0.17/mysql-connector-java-8.0.17.jar {:exception=>#<TypeError: failed to coerce jdk.internal.loader.ClassLoaders$AppClassLoader to java.net.URLClassLoader>}
    logstash_1 | [2019-10-07T17:32:33,967][ERROR][logstash.javapipeline ] A plugin had an unrecoverable error. Will restart this plugin.
    Antoine Cotten
    @antoineco
    Try copying the MySQL connector Jar inside the docker-elk/logstash directory, then add the following line to docker-elk/logstash/Dockerfile:
    COPY mysql-connector-java-8.0.17.jar logstash-core/lib/jars/
    Antoine Cotten
    @antoineco
    This issue will supposedly be fixed in the next version of the plugin, which doesn't seem to be included in Logstash yet. See logstash-plugins/logstash-input-jdbc#343
    Keep in mind that this is just a temporary workaround. It also requires setting the following option in your Logstash input: jdbc_driver_library => "" (for now)
    Christine Campo
    @ccampo55

    @antoineco ] Yes! that was it. New to Docker so was trying to figure out where the jar file needed to exist for the Dockerfile to find it!

    I ended up leaving both blank...
    jdbc_driver_class => ""
    jdbc_driver_library => ""

    But it is now all working! So... how will I know when the fix is included?

    Antoine Cotten
    @antoineco
    @ccampo55 You can keep an eye on the plugin’s documentation page. The current version number should be updated in a future release.
    You should expect v v4.3.14 or above.
    Deniz Parlak
    @DenizParlak

    hi guys, there is a problem about search guard when I try to initialize:

    root@elk3:/docker-elk# docker-compose exec -T elasticsearch bin/init_sg.sh OCI runtime exec failed: exec failed: container_linux.go:345: starting container process caused "exec: \"bin/init_sg.sh\": stat bin/init_sg.sh: no such file or directory": unknown

    Antoine Cotten
    @antoineco
    @DenizParlak sounds like another case of outdated images. Try docker-compose build.