These are chat archives for nlf/dlite

7th
Mar 2016
Onni Hakala
@onnimonni
Mar 07 2016 11:59

Hello!

I'm trying to install and start dlite in ansible script for OS-X. Documents show that $ dlite install and $ dlite start should be enough but dlite doesn't automatically start. How can I immediately start dlite? Reboot fixes the problem and after reboot dlite is already started.

Johann Zelger
@zelgerj
Mar 07 2016 12:52
Hi there
Is there a good way to see what's going on inside when i call dlite start. Log files etc?
Bruno
@bpinto
Mar 07 2016 17:41
Hey there
Great work @nlf
Nathan LaFreniere
@nlf
Mar 07 2016 17:41
howdy
thanks! still shaking out some bugs, but we'll get there
Bruno
@bpinto
Mar 07 2016 17:42
I just noticed the gitter call on the readme
Nathan LaFreniere
@nlf
Mar 07 2016 17:42
@zelgerj not really. there's a small amount of logging in ~/Library/Logs/dlite-out.log and dlite-err.log in the same directory, but it doesn't give you much
Bruno
@bpinto
Mar 07 2016 17:42
dhyve (now dlite) is the sole reason why I can use docker on OSX, so thank you very much
Nathan LaFreniere
@nlf
Mar 07 2016 17:43
you're very welcome! i'm glad to hear you like it
i might already have a fix for the file permissions thing, just verified that i can run your example to show the bug several times without issues
building a new rootfs right now
Bruno
@bpinto
Mar 07 2016 17:47
:+1: hope the example was useful
Nathan LaFreniere
@nlf
Mar 07 2016 17:47
it definitely was
Bruno
@bpinto
Mar 07 2016 17:50
I have read the issue and the comments on dhyve-os regarding docker download
Do you think it makes sense to come with latest docker pre-installed and only download in case an argument is specified?
Nathan LaFreniere
@nlf
Mar 07 2016 17:52
i considered that, but "latest" is relative to when i last built the image. for example, latest right now is 1.10.2 but tomorrow 1.10.3 could come out and anyone installing then would end up with the script having to download it anyway
it didn't seem worth it to me. removing it cut the size of the rootfs archive roughly in half, and lends way more flexibility
what i would like to do, however, is figure out a way to intelligently discover what the vm is doing so that i can give a better error message when the user tries to run docker commands before the virtual machine has booted all the way as well as before the docker daemon has been downloaded and started
Bruno
@bpinto
Mar 07 2016 17:55
I see
I had thought about supporting ARGS on the dockerfile and coming up with a link that always points to latest docker release
https://get.docker.io/builds/Linux/x86_64/docker-latest perhaps
Nathan LaFreniere
@nlf
Mar 07 2016 17:59
but again, that relies on the rootfs being rebuilt on a regular basis to keep up with latest versions or there's very little gain
just uploaded a new test build of the os, if you do dlite stop && dlite update -v 3.0.0 && dlite start it'll update you. hopefully that helps resolve the file permission issues
Bruno
@bpinto
Mar 07 2016 18:02
What about linked docker images? Is that something you would like as a solution?
Oh... nvm
I don't think there is a docker image on docker hub we could use as a hook?
Nathan LaFreniere
@nlf
Mar 07 2016 18:03
i don't think there is, no
i think i'd have to convince them to set one up for me in github
unless there's some feed somewhere else that gets notified of new releases
Bruno
@bpinto
Mar 07 2016 18:06
New build worked for the test scenario, checking on real application now
Nathan LaFreniere
@nlf
Mar 07 2016 18:08
fingers crossed
Bruno
@bpinto
Mar 07 2016 18:09
All good! :heart:
Nathan LaFreniere
@nlf
Mar 07 2016 18:09
awesome! glad to hear it
Bruno
@bpinto
Mar 07 2016 18:10
~8MB did this come from 9p change?
Nathan LaFreniere
@nlf
Mar 07 2016 18:10
it does seem to be creating files on the host as root, so it would be nice to fix that.. but at least the containers aren't failing now
the 9p change and removing docker from the rootfs
the docker binary is huge, so even heavily compressed it adds a lot
Bruno
@bpinto
Mar 07 2016 18:11
but if you install it later, doesn't that mean the space is the same?
Nathan LaFreniere
@nlf
Mar 07 2016 18:12
the initial download is ~8MB
that's all i mean in the readme
Bruno
@bpinto
Mar 07 2016 18:12
:+1:
Nathan LaFreniere
@nlf
Mar 07 2016 18:12
i can clarify that
Bruno
@bpinto
Mar 07 2016 18:12
is a small ~8MB download
I think it's clear, just me missing the important keywords :P
Nathan LaFreniere
@nlf
Mar 07 2016 18:13
fair enough :)
Ian Taylor
@itaylor
Mar 07 2016 18:15
Hey nlf... loving D-lite, and the 2.0 version seems like it should solve the NFS pain-point I had with previous versions. I was naively hoping that migrating to plan9 would allow for FSEvents on OS X to get translated to inotify events on linux, which would help a lot with node.js development workflows. Is that something you'd given any thought to?
Nathan LaFreniere
@nlf
Mar 07 2016 18:15
i had the same naive hope :) i haven't actually tested it yet, have you?
Ian Taylor
@itaylor
Mar 07 2016 18:15
I just tried it... they don't come though
Bruno
@bpinto
Mar 07 2016 18:17
Should I be experiencing a slower docker container with this change to 9P?
Nathan LaFreniere
@nlf
Mar 07 2016 18:17
dang, that's a bummer. i wonder if there's a way to make that work..
@bpinto you shouldn't, no... is it noticeably slower?
Bruno
@bpinto
Mar 07 2016 18:18
I am struggling to log into the rails application
Nathan LaFreniere
@nlf
Mar 07 2016 18:18
oh that's interesting
i'm using it to build the rootfs image and it's been plenty fast for me
Bruno
@bpinto
Mar 07 2016 18:19
app_1    |   Rendered message_comments/_form.html.erb (4561.5ms)
app_1    |   Rendered messages/_message.html.erb (10437.2ms)
app_1    |   Rendered messages/_index.html.erb (15612.2ms)
Ian Taylor
@itaylor
Mar 07 2016 18:19
The approach that https://github.com/codekitchen/dinghy/ takes is to set a file watcher on /Users/<username> that uses Mac FS events and then calls to something inside of the docker host that calls touch. It's not perfect, but it does work for most cases.
Nathan LaFreniere
@nlf
Mar 07 2016 18:20
hmm, interesting
doable. i'd have to call touch through the ssh connection, but i could make something work i think
@bpinto wow! that's really really slow
i'm definitely not seeing that on my end, i/o to the 9p shares seems totally fine
Bruno
@bpinto
Mar 07 2016 18:25
It is similar to virtual box share speeds
Do you know if and how I can track this?
Ian Taylor
@itaylor
Mar 07 2016 18:25
It looks like https://github.com/codekitchen/fsevents_to_vm is the util dinghy uses proxy those events. It's just using ssh to do the touch, like you suggested.
Nathan LaFreniere
@nlf
Mar 07 2016 18:26
@bpinto i'm not sure, honestly
how much ram do you have allocated to the vm?
@itaylor good to know my hunch is the same, i'll see if i can implement something
Bruno
@bpinto
Mar 07 2016 18:32
I've got 2GB, standard
my only settings is 50gb disk
Nathan LaFreniere
@nlf
Mar 07 2016 18:32
i'm curious if you raise it if you see better performance
Bruno
@bpinto
Mar 07 2016 18:33
I'll do it
Running on 4gb, no changes
at least noticeable changes
app_1    |   Rendered shared/_icon.html.slim (696.5ms)
app_1    |   Rendered message_comments/_form.html.erb (996.3ms)
app_1    |   Rendered messages/_message.html.erb (2129.2ms)
app_1    |   Rendered messages/_index.html.erb (3362.3ms)
Nathan LaFreniere
@nlf
Mar 07 2016 18:39
weren't you seeing significantly slower times before? you were showing near 4 seconds for _form.html.erb in your last snippet
Bruno
@bpinto
Mar 07 2016 18:40
The times were higher, I agree
but my perception is the same :P
Nathan LaFreniere
@nlf
Mar 07 2016 18:40
sure
it's better, but still not good
Bruno
@bpinto
Mar 07 2016 18:55
I'm trying to find a rails application i could use as example
Bruno
@bpinto
Mar 07 2016 19:36
/shrug can't find anything
I'll need to downgrade to previous version otherwise I can't work :P
Nathan LaFreniere
@nlf
Mar 07 2016 19:37
that's awful. i'll have to see if i can come up with anything else
Bruno
@bpinto
Mar 07 2016 19:41
I am trying another open source project... :pray:
Nathan LaFreniere
@nlf
Mar 07 2016 19:42
as soon as you find one, send it my way and i'll see if i can figure out something better. we might end up not using 9p if it's going to be that slow
Bruno
@bpinto
Mar 07 2016 20:31
are you familiar with rails development?
Nathan LaFreniere
@nlf
Mar 07 2016 20:32
nope, i'm not
Bruno
@bpinto
Mar 07 2016 20:37
:+1: Still haven't been able to find a good open source project :P
Bruno
@bpinto
Mar 07 2016 21:13
Tried the n-th project, no luck
either it does not work or the project is too simple
it needs to be a project with lots of files
I suppose a disk write/read performance test would likely give us a better overview of the problem
it would be nice if there was a docker container made for this purpose
Johann Zelger
@zelgerj
Mar 07 2016 21:24
@nlf ok thanks for the information
Bruno
@bpinto
Mar 07 2016 21:24
I have something that might be useful
### HOST ###
(log) master ◦ $ time sh -c "dd if=/dev/zero of=ddfile bs=8k count=25000 && sync"; rm ddfile
25000+0 records in
25000+0 records out
204800000 bytes transferred in 0.357641 secs (572641309 bytes/sec)
        0.41 real         0.01 user         0.24 sys

### GUEST (non-shared folder) ###
(log) master ◦ $ docker run -it -v $PWD:/log ruby:2.1.6 /bin/bash
root@1751a52d0b87:/# time sh -c "dd if=/dev/zero of=ddfile bs=8k count=25000 && sync"; rm ddfile
25000+0 records in
25000+0 records out
204800000 bytes (205 MB) copied, 0.08 s, 2.6 GB/s

real    0m1.540s
user    0m0.000s
sys    0m0.120s

### GUEST (shared folder) ###
root@1751a52d0b87:/# cd log/
root@1751a52d0b87:/log# time sh -c "dd if=/dev/zero of=ddfile bs=8k count=25000 && sync"; rm ddfile
25000+0 records in
25000+0 records out
204800000 bytes (205 MB) copied, 3.4 s, 60.2 MB/s

real    0m3.480s
user    0m0.020s
sys    0m1.420s
now I need to run it on previous dlite versions :D
Bruno
@bpinto
Mar 07 2016 21:41
almost the same... perhaps the problem is on read?
I'm out of ideas
root@db138225b9b7:/log# time sh -c "dd if=/dev/zero of=ddfile bs=8k count=25000 && sync"; rm ddfile
25000+0 records in
25000+0 records out
204800000 bytes (205 MB) copied, 3.1161 s, 65.7 MB/s

real    0m3.126s
user    0m0.000s
sys    0m0.190s
sys time was much lower though
Nathan LaFreniere
@nlf
Mar 07 2016 22:31
entirely possible it's just that xhyve isn't optimized enough for your use case