These are chat archives for rust-lang/rust

16th
Sep 2017
Restioson
@Restioson
Sep 16 2017 07:36
I've built my binary for armv7, but when running it on my pi3 (32 bit kernel), it just hangs
(gdb) run rover3_pi
The program being debugged has been started already.
Start it from the beginning? (y or n) y

Starting program: /home/pi/rover3_pi rover3_pi
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/arm-linux-gnueabihf/libthread_db.so.1".
just hangs forever after the last line
It's debug profile
Running normally also produces infinite hang, or so it seems
Restioson
@Restioson
Sep 16 2017 07:49
Same with release profile
Restioson
@Restioson
Sep 16 2017 08:00
Hm... even happens to rustup -- when not root!
how bizzare! Root - fine, user - hang!
image.png
Very strange
Ok... su root and then ./rover3_pi also hangs
Seems like armv7 stuff only works in sudo
root@Rover3:/home/pi# file rover3_pi
rover3_pi: ELF 32-bit LSB shared object, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-armhf.so.3, for GNU/Linux 2.6.32, BuildID[sha1]=50e2ea78df2f46b6e030a8dfa13db6f09ba885bf, stripped

root@Rover3:/home/pi# file /bin/ls
/bin/ls: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-armhf.so.3, for GNU/Linux 2.6.32, BuildID[sha1]=5e052e8de057d379ab51d4af510ad9318fe77b46, stripped
Exact same info, yet ls works in non sudo, and rover3_pi doesn't
The same goes for rustup
Denis Lisov
@tanriol
Sep 16 2017 09:21
Does it consume 100% CPU or not? What does the backtrace look like?
Restioson
@Restioson
Sep 16 2017 09:21
I'll see in a sec
0.0% cpu usage
running with RUST_BACKTRACE=1 makes no difference - how else would I get backtrace?
Theres no error - just infinite hang
Denis Lisov
@tanriol
Sep 16 2017 09:29
gdb ./rover3_pi, Ctrl+C, bt
Restioson
@Restioson
Sep 16 2017 09:32
(gdb) run rover3_debug

Starting program: /home/pi/rover3_debug rover3_debug
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/arm-linux-gnueabihf/libthread_db.so.1".
^C
Program received signal SIGINT, Interrupt.
0x76e11780 in __lll_lock_wait (futex=futex@entry=0x54c365bc <init_lock>,
    private=<optimized out>)
    at ../ports/sysdeps/unix/sysv/linux/arm/nptl/lowlevellock.c:46
46      ../ports/sysdeps/unix/sysv/linux/arm/nptl/lowlevellock.c: No such file or directory.

(gdb) bt

#0  0x76e11780 in __lll_lock_wait (futex=futex@entry=0x54c365bc <init_lock>,
    private=<optimized out>)
    at ../ports/sysdeps/unix/sysv/linux/arm/nptl/lowlevellock.c:46
#1  0x76e0c340 in __GI___pthread_mutex_lock (mutex=0x54c365bc <init_lock>)
    at pthread_mutex_lock.c:134
#2  0x76f1cb88 in pthread_mutex_lock (mutex=mutex@entry=0x54c365bc <init_lock>)
    at forward.c:192
#3  0x54bdad04 in je_malloc_mutex_lock (mutex=0x54c365bc <init_lock>)
    at /checkout/src/liballoc_jemalloc/../jemalloc/include/jemalloc/internal/mutex.h:85
#4  malloc_init_hard ()
    at /checkout/src/liballoc_jemalloc/../jemalloc/src/jemalloc.c:1366
#5  0x54bdc99a in malloc_init ()
    at /checkout/src/liballoc_jemalloc/../jemalloc/src/jemalloc.c:302
#6  calloc (num=num@entry=1, size=size@entry=20)
    at /checkout/src/liballoc_jemalloc/../jemalloc/src/jemalloc.c:1699
#7  0x76e2c300 in _dlerror_run (operate=0x76e2bd10 <dlsym_doit>,
    args=args@entry=0x7efff2f8) at dlerror.c:141
#8  0x76e2bd84 in __dlsym (Cannot access memory at address 0x0
handle=<optimized out>, name=<optimized out>)
    at dlsym.c:70
#9  0x76fb4574 in ?? () from /usr/lib/uv4l/uv4lext/armv6l/libuv4lext.so
Cannot access memory at address 0x0
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
That backtrace might have something to do with the ^C though
(i'm new to gdb, thanks!)
Denis Lisov
@tanriol
Sep 16 2017 09:37
Try running it with
LD_PRELOAD="" ./rover3_pi
Restioson
@Restioson
Sep 16 2017 09:38
Works!
pi@Rover3:~ $ echo $LD_PRELOAD
:/usr/lib/uv4l/uv4lext/armv6l/libuv4lext.so
Denis Lisov
@tanriol
Sep 16 2017 09:39
According to RPi forums, this is likely a bug in libuv4lext.so
Restioson
@Restioson
Sep 16 2017 09:40
I wonder why uv4l set that...
Thanks so much!
It's strange how it only affects rust programs
It says armv6l... maybe i'll try compile for armv6 and see what happens
Denis Lisov
@tanriol
Sep 16 2017 09:41
It may affect, for example, everything using jemalloc instead of the default allocator.
Restioson
@Restioson
Sep 16 2017 09:41
Ah, I see
If you don't mind asking, what keywords did you google? :smiley:
Denis Lisov
@tanriol
Sep 16 2017 09:45
Initially I asked about jemalloc hang malloc_init_hard, in one of the links saw a mention of LD_PRELOAD problems, looked into the backtrace again and asked for libuv4lext.so hang
Restioson
@Restioson
Sep 16 2017 09:46
Ah, this is a (jem)alloc bug
tysm!
Denis Lisov
@tanriol
Sep 16 2017 09:47
Most likely this is a bug in libuv4lext.so which uses memory allocation before jemalloc can provide it (while the default allocator is already funtcional at that point).
Restioson
@Restioson
Sep 16 2017 09:50
So, I can either comment the line in /etc/environment, or not use jemalloc?
Denis Lisov
@tanriol
Sep 16 2017 09:51
Yes, looks like that.
I'd guess that the library intercepts mmap calls and uses malloc in the handler, assuming that small malloc uses the brk heap, while jemalloc's malloc tries using mmap and gets into a deadlock.
Restioson
@Restioson
Sep 16 2017 09:53
I could also run it with LD_PRELOAD=""
I'll probably do this
(or may as well run with sudo)
Thanks!