These are chat archives for hypriot/talk

2nd
Jun 2016
Sign in to start talking
Min Zhou
@coderplay
Jun 02 2016 05:13 UTC
I ran docker run -d -p 8500:8500 --name=consul hypriot/rpi-consul agent -data-dir=/data
under a HypriotOS on a raspberry pi

but got error 

$ docker logs 22a3b4e3fdcf
==> Starting Consul agent...
==> Starting Consul agent RPC...
==> Consul agent running!
         Node name: '22a3b4e3fdcf'
        Datacenter: 'dc1'
            Server: false (bootstrap: false)
       Client Addr: 127.0.0.1 (HTTP: 8500, HTTPS: -1, DNS: 8600, RPC: 8400)
      Cluster Addr: 10.1.98.2 (LAN: 8301, WAN: 8302)
    Gossip encrypt: false, RPC-TLS: false, TLS-Incoming: false
             Atlas: <disabled>

==> Log data will now stream in as it occurs:

    2016/06/02 03:45:54 [INFO] serf: EventMemberJoin: 22a3b4e3fdcf 10.1.98.2
    2016/06/02 03:45:54 [ERR] agent: failed to sync remote state: No known Consul servers
==> Failed to check for updates: Get https://checkpoint-api.hashicorp.com/v1/check/consul?arch=arm&os=linux&signature=670da1bd-90bd-dd9a-6aa7-98581bae29b3&version=0.6.4: x509: failed to load system roots and no roots provided
    2016/06/02 03:46:19 [ERR] agent: failed to sync remote state: No known Consul servers
    2016/06/02 03:46:37 [ERR] agent: failed to sync remote state: No known Consul servers
    2016/06/02 03:46:54 [ERR] agent: failed to sync remote state: No known Consul servers
    2016/06/02 03:47:22 [ERR] agent: failed to sync remote state: No known Consul servers
    2016/06/02 03:47:45 [ERR] agent: failed to sync remote state: No known Consul servers
    2016/06/02 03:48:14 [ERR] agent: failed to sync remote state: No known Consul servers
    2016/06/02 03:48:41 [ERR] agent: failed to sync remote state: No known Consul servers
    2016/06/02 03:48:59 [ERR] agent: failed to sync remote state: No known Consul servers
    2016/06/02 03:49:28 [ERR] agent: failed to sync remote state: No known Consul servers
    2016/06/02 03:49:47 [ERR] agent: failed to sync remote state: No known Consul servers

Anyone tell me the reason?

firecyberice
@firecyberice
Jun 02 2016 06:15 UTC
@coderplay please read the consul documentation or use our cluster lab
ufud.org
@ufud-org
Jun 02 2016 15:01 UTC
I'm facing some weird behaviour with the current 0.8.0 release on a Raspberry Pi3 - after a few minutes IPv4 network connectivity is lost. SSH connections freeze and the machine no longer replies to ping requests. Downloaded the image again to make sure it isn't corrupted. Same problem with the re-downloaded 0.8.0 image. What can I do to provide you with more detailed information regarding this issue (send logifles, etc.)?
Ohm to clarify: the Pi3 is connected through LAN, not Wireless.
ufud.org
@ufud-org
Jun 02 2016 15:06 UTC
Logging on locally works fine and the system is running. It only losing network connectivity
Govinda Fichtner
@Govinda-Fichtner
Jun 02 2016 15:11 UTC
@ufud-org What is "ip a" showing? Does eth0 have an IP v4 address?
ufud.org
@ufud-org
Jun 02 2016 15:11 UTC
yes it gets one IP address from dhcp and the ip is "fixed" to the mac address.
Govinda Fichtner
@Govinda-Fichtner
Jun 02 2016 15:12 UTC
So it doesn't lose the IP?
ufud.org
@ufud-org
Jun 02 2016 15:13 UTC
just checking to make 100% sure now. got to login locally. just a second ...
ufud.org
@ufud-org
Jun 02 2016 15:19 UTC
okay, so the ip is set and not lost. but the connection drops
ip a outputs the correct ip and stuff, but the ping to the machine says dest. host unreachable
ping to other hosts on the network are ok
Govinda Fichtner
@Govinda-Fichtner
Jun 02 2016 15:23 UTC
@ufud-org I guess you can not ping other hosts from the Pi, too?
ufud.org
@ufud-org
Jun 02 2016 15:24 UTC
whoa, that's working. i can ping other hosts on the same subnet from hyproit os
Govinda Fichtner
@Govinda-Fichtner
Jun 02 2016 15:24 UTC
@ufud-org And what does the output of "ip a" show for eth0? Can you paste this? Is the interface up or down?
ufud.org
@ufud-org
Jun 02 2016 15:24 UTC
can't paste as im logged on locally and so i can't copy/paste.
Govinda Fichtner
@Govinda-Fichtner
Jun 02 2016 15:25 UTC
@ufud-org So it is not totally broken...
ufud.org
@ufud-org
Jun 02 2016 15:25 UTC
ip a say: eth0 state UP
no, doesn't look like it
let me check something else ...
Govinda Fichtner
@Govinda-Fichtner
Jun 02 2016 15:26 UTC
@ufud-org That's good. Can you ping the IP of eth0 from the outside?
ufud.org
@ufud-org
Jun 02 2016 15:27 UTC
aaaahhh .... yes, i can ping from another Pi on the same subnet ... hmmm
Govinda Fichtner
@Govinda-Fichtner
Jun 02 2016 15:27 UTC
And how did you try to connect via SSH? Via hostname like "ssh pirate@black-pearl.local"?
ufud.org
@ufud-org
Jun 02 2016 15:27 UTC
nope - ssh pirate@192.168.100.128
maybe it's related to something else on the network (taking a glimpse at my openwrt router)
Govinda Fichtner
@Govinda-Fichtner
Jun 02 2016 15:29 UTC
@ufud-org It looks like this not really an issue of HypriotOS but like an issue with your network...
ufud.org
@ufud-org
Jun 02 2016 15:29 UTC
yup, I think you're right. will dig into this. but thank you very much for your help so far
ufud.org
@ufud-org
Jun 02 2016 16:02 UTC
@Govinda-Fichtner Sorry to disturb once more, but I switched routers and the issue persists. But I am able to reproduce the behaviour of dropping connections via eth0 on a Pi3
Boot up, log in and run 'docker info' or just do a 'service docker status' and the connection drops a few seconds after that
ufud.org
@ufud-org
Jun 02 2016 16:07 UTC
Just to give some background: i habe a 0.7.0(beta) image that works just fine on the same device
ufud.org
@ufud-org
Jun 02 2016 16:50 UTC
Did some more checking: access from another host on the same subnet is OK. access from another host in a different subnet causes the connection drops. at first I thought it was something with my openwrt setup, but then i used an older tp-link router and the issue persists. So to sum it up: network connection drops only appear to happen when the Pi3 with hypriot os is accessed from another subnet. access on the same subnet is no problem at all.
Govinda Fichtner
@Govinda-Fichtner
Jun 02 2016 16:52 UTC
@firecyberice What do you think regarding @ufud-org problem?
Edward Plitt
@eplitt
Jun 02 2016 17:55 UTC
this sounds like a networking problem. the symptoms point to a gateway not allowinf traffic through, or the possibly lost or changed what the default gateway is
can you run "netstat -rn" and then see if you can ping the gateway (the entry in the gateway column, alongside the destination 0.0.0.0 entry)
ufud.org
@ufud-org
Jun 02 2016 17:58 UTC
Sure. Just a moment. Must switch SD cards first.
Edward Plitt
@eplitt
Jun 02 2016 17:59 UTC
based on your ip-address, I am guessing that your gateway is 192.168.100.1
ufud.org
@ufud-org
Jun 02 2016 18:00 UTC
that should be the case indeed
it's 192.168.200.1 since i switched back to my openwrt router. but the problem remains. anyway, here's the output
root@service01:~# netstat -rn
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
0.0.0.0 192.168.200.1 0.0.0.0 UG 0 0 0 eth0
172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0
192.168.200.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
192.168.200.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0.200
ip a says:
root@service01:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether <MAC> brd ff:ff:ff:ff:ff:ff
inet 192.168.200.128/24 brd 192.168.200.255 scope global eth0
valid_lft forever preferred_lft forever
3: wlan0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether <MAC> brd ff:ff:ff:ff:ff:ff
4: docker0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
link/ether <MAC> brd ff:ff:ff:ff:ff:ff
inet 172.17.0.1/16 scope global docker0
valid_lft forever preferred_lft forever
9: eth0.200@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether <MAC> brd ff:ff:ff:ff:ff:ff
inet 192.168.200.1/24 scope global eth0.200
valid_lft forever preferred_lft forever
11: vethf92ef32@if10: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker0 state UP group default
link/ether <MAC> brd ff:ff:ff:ff:ff:ff
13: vethf8520a8@if12: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker0 state UP group default
link/ether <MAC> brd ff:ff:ff:ff:ff:ff
Edward Plitt
@eplitt
Jun 02 2016 18:03 UTC
ok, so your gateway address is "wrong" right now. your pi doesn't know how to talk to 192.168.200.anything
when you reboot, is the gateway correct at 192.168.100.1?
wait, your ip address is 192.168.200.128 now
you are on a new subnet - 192.168.200.x
can you ping the gateway at 192.168.200.1 ?
hmmm - device eth0.200 is assigned 192.168.200.1 - which is your gateway
where is eth0.200 being assigned?
it looks like this is the culprit
Govinda Fichtner
@Govinda-Fichtner
Jun 02 2016 18:16 UTC
@ufud-org Sound like did start the Cluster Lab on this node. The Cluster Lab creates a eth0.200 interface. Maybe thats conflicting with your rest of your network config. You can stop the Cluster Lab with cluster-lab stop
Afterwards the eth0.200 interface should be gone.
ufud.org
@ufud-org
Jun 02 2016 18:21 UTC
on the console, when connection is lost, it reads: 'docker0: port 2(veth...) entered forwarding state
ufud.org
@ufud-org
Jun 02 2016 18:42 UTC
wait a minute ... in /boot/device-init.yml there's rpi-consul and rpi-swarm enabled (not commented out). so these two are pulled and started on first boot, right? now, I've re-flashed my sd card, commented out those two images in device-init.yml and then booted up the Pi3 for the first time. and now there's no more problems accessing the device from either the same or a different subnet. I also noticed, that the hostname defined in /boot/device-init.yml gets appended to /etc/hosts after every reboot. So issue solved (at least for me) since I don't run swarm or consul on my machine. Thanks a lot to all of you for your help and support!
regarding the continuous appending of the hostname to /etc/hosts, that can be fixed by commenting out the 'hostname: black-pearl' stanza after first boot. at least it works as a workaround for me.
Edward Plitt
@eplitt
Jun 02 2016 18:58 UTC
It sounds like you adjusted the DHCP network on your router to the 192.168.200.x network - which conflicts with the Hypriot Cluster Lab address space
ufud.org
@ufud-org
Jun 02 2016 19:22 UTC
I see. that explains it all, of course! Like I said, I've not yet worked with Cluster Lab since I do run custom made images on that device. But it's good to know about the address space, so I won't tap into that trap if I set up a cluster in the future.
Govinda Fichtner
@Govinda-Fichtner
Jun 02 2016 19:37 UTC
@ufud-org the swarm and consul images are only imported on boot but not started. Also the Cluster Lab only starts on boot if it is explicitky enabled.
firecyberice
@firecyberice
Jun 02 2016 20:25 UTC
@eplitt @ufud-org you cal also change the cluster-lab ip range in /etc/cluster-lab/cluster.conf
Mathias Renner
@MathiasRenner
Jun 02 2016 22:33 UTC

@MingTheMerciful Did you meanwhile succeed installing our docker-hypriot? If not, run 

wget https://raw.githubusercontent.com/docker/docker/master/contrib/check-config.sh
chmod +x check-config.sh
./check-config

Check if everything in section "Generally Necessary" is green.