These are chat archives for atomix/atomix

5th
Aug 2018
Junbo Ruan
@aruanruan
Aug 05 2018 07:39
today i met one crash

--------------- T H R E A D ---------------

Current thread (0x0000000020507000): JavaThread "raft-server-system-partition-1" [_thread_in_Java, id=10524, stack(0x000000003ac90000,0x000000003ad90000)]

siginfo: ExceptionCode=0xc0000005, reading address 0x00000000024111f9

Registers:
RAX=0x00000006c190e3e8, RBX=0x00000006c1c30090, RCX=0x000000000000ffc0, RDX=0x00000006c1c25918
RSP=0x000000003ad8f0e0, RBP=0x000000003ad8f330, RSI=0x00000006c1c30060, RDI=0x000000004d285620
R8 =0x00000000000011f9, R9 =0x00000000f8008cd7, R10=0x0000000000000000, R11=0x0000000002410000
R12=0x0000000000000000, R13=0x000000003ad8f2e8, R14=0x000000004d290570, R15=0x0000000020507000
RIP=0x0000000003194c99, EFLAGS=0x0000000000010206

Top of Stack: (sp=0x000000003ad8f0e0)
0x000000003ad8f0e0: 0000000000000001 0000000003504828
0x000000003ad8f0f0: 00000006c1c30060 000000000000005f
0x000000003ad8f100: 00002c31e093a76d 000000000369c064
0x000000003ad8f110: 0000000020507000 000000003ad8f180
0x000000003ad8f120: 0000000020507000 00000000778f0bd2
0x000000003ad8f130: 000000003ad8f330 00000000036f0b5c
0x000000003ad8f140: 00000006c1c33cd8 0000000077a0421c
0x000000003ad8f150: 0000000000000000 0000000002bab7f5
0x000000003ad8f160: 0000000600000261 00000006c1c30060
0x000000003ad8f170: 00000006c1c33ca8 000000000341c0e5
0x000000003ad8f180: 000000003ad8f330 00000000032022fc
0x000000003ad8f190: 000000003ad8f1f0 0000000002957ced
0x000000003ad8f1a0: 00000006c1c33cb8 0000000003292a20
0x000000003ad8f1b0: 000000003ad8f1b0 000000003a5a7629
0x000000003ad8f1c0: 00000006c1c33ea8 0000000003098934
0x000000003ad8f1d0: 0000000000000000 000000003a5a7638

Instructions: (pc=0x0000000003194c99)
0x0000000003194c79: 02 00 00 44 8b 52 1c 45 2b d0 41 83 fa 04 0f 8c
0x0000000003194c89: dc 02 00 00 4c 8b 5a 10 44 0f b6 52 2a 4d 63 c0
0x0000000003194c99: 43 8b 04 03 45 85 d2 0f 85 eb 02 00 00 0f c8 85
0x0000000003194ca9: c0 0f 85 d1 01 00 00 4c 8b 54 24 10 45 8b 5a 0c

Register to memory mapping:

RAX=0x00000006c190e3e8 is an oop
java.lang.Long

  • klass: 'java/lang/Long'
    RBX=0x00000006c1c30090 is an oop
    io.atomix.storage.buffer.SlicedBuffer
  • klass: 'io/atomix/storage/buffer/SlicedBuffer'
    RCX=0x000000000000ffc0 is an unknown value
    RDX=0x00000006c1c25918 is an oop
    java.nio.DirectByteBuffer
  • klass: 'java/nio/DirectByteBuffer'
    RSP=0x000000003ad8f0e0 is pointing into the stack for thread: 0x0000000020507000
    RBP=0x000000003ad8f330 is pointing into the stack for thread: 0x0000000020507000
    RSI=0x00000006c1c30060 is an oop
    io.atomix.storage.journal.JournalSegmentReader
  • klass: 'io/atomix/storage/journal/JournalSegmentReader'
    RDI=0x000000004d285620 is pointing into metadata
    R8 =0x00000000000011f9 is an unknown value
    R9 =0x00000000f8008cd7 is an unknown value
    R10=0x0000000000000000 is an unknown value
    R11=0x0000000002410000 is an unknown value
    R12=0x0000000000000000 is an unknown value
    R13=0x000000003ad8f2e8 is pointing into the stack for thread: 0x0000000020507000
    R14=0x000000004d290570 is pointing into metadata
    R15=0x0000000020507000 is a thread

Stack: [0x000000003ac90000,0x000000003ad90000], sp=0x000000003ad8f0e0, free space=1020k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
C 0x0000000003194c99

--------------- P R O C E S S ---------------

Java Threads: ( => current thread )
0x0000000022fbd800 JavaThread "ForkJoinPool.commonPool-worker-7" daemon [_thread_blocked, id=10336, stack(0x000000003dd80000,0x000000003de80000)]
0x0000000022f47800 JavaThread "ForkJoinPool.commonPool-worker-6" daemon [_thread_blocked, id=3660, stack(0x000000003da00000,0x000000003db00000)]
0x0000000022f49000 JavaThread "ForkJoinPool.commonPool-worker-5" daemon [_thread_blocked, id=7984, stack(0x000000003d680000,0x000000003d780000)]
0x0000000022f44800 JavaThread "ForkJoinPool.commonPool-worker-4" daemon [_thread_blocked, id=7572, stack(0x000000003d550000,0x000000003d650000)]
0x0000000022f4b800 JavaThread "ForkJoinPool.commonPool-worker-3" daemon [_thread_blocked, id=10460, stack(0x000000003cfa0000,0x000000003d0a0000)]
0x000000001ff20800 JavaThread "globalEventExecutor-1-2" [_thread_blocked, id=7296, stack(0x00000000481e0000,0x00000000482e0000)]
0x000000004549e800 JavaThread "ForkJoinPool.commonPool-worker-2" daemon [_thread_blocked, id=11312, stack(0x0000000049570000,0x0000000049670000)]
0x000000001ff1f800 JavaThread "ForkJoinPool.commonPool-worker-1" daemon [_thread_blocked, id=6716, stack(0x00000000464d0000,0x00000000465d0000)]
0x0000000022fcc800 JavaThread "atomix-data-15" [_thread_blocked, id=5252, stack(0x000000004f750000,0x000000004f850000)]
0x0000000022fcb800 JavaThread "atomix-data-14" [_thread_blocked, id=10964, stack(0x000000004f640000,0x000000004f740000)]
0x0000000022fcb000 JavaThread "atomix-data-13" [_thread_blocked, id=8736, stack(0x000000004f4d0000,0x000000004f5d0000)]
0x0000000022fca000 JavaThread "atomix-data-12" [_thread_blocked, id=12028, stack(0x000000004f220000,0x000000004f320000)]
0x0000000022fc9800 JavaThread "atomix-data-11" [_thread_blocked, id=11852, stack(0x000000004f360000,0x000000004f460000)]
0x0000000022fc8800 JavaThread "atomix-data-10" [_thread_blocked, id=7152, stack(0x000000004f050000,0x000000004f150000)]
0x0000000022fc8000 JavaThread "atomix-data-9" [_thread_blocked, id=3984, stack(0x000000004e1e0000,0x000000004e2e0000)]
0x0000000022fc7000 JavaThread "atomix-data-8" [_thread_blocked, id=3100, stack(0x000000004e9d0000,0x000000004ead0000)]
0x0000000022fc6800 JavaThread "atomix-data-7" [_thread_blocked, id=8108, stack(0x000000004eb60000,0x000000004ec60000)]
0x0000000022fc5800 JavaThread "atomix-data-6" [_thread_blocked, id=11184, stack(0x000000004ee80000,0x000000004ef80000)]
0x0000000022fc5000 JavaThread "atomix-data-5" [_thread_blocked, id=11560, stack(0x000000004ec70000,0x000000004ed70000)]
0x0000000022fc3800 JavaThread "atomix-data-4" [_thread_blocked, id=7184, stack(0x000000004e6f0000,0x000000004e7f0000)]
0x00000000454a3800 JavaThread "atomix-data-3" [_thread_blocked, id=11408, stack(0x000000004cfc0000,0x000000004d0c0000)]
0x00000000454a0800 JavaThread "atomix-data-2" [_thread_blocked, id=10328, stack(0x000000004ca00000,0x000000004cb00000)]
0x00000000454a0000 JavaThread "atomix-data-1" [_thread_blocked, id=2416, stack(0x000000004b7a0000,0x000000004b8a0000)]
0x000000004549f000 JavaThread "atomix-data-0" [_thread_blocked, id=11480, stack(0x000000004b9b0000,0x000000004bab0000)]

0x000000001ff1f000 JavaThread "raft-client-raft-partition-3-0" [_thread_blocked, id=11672, stack(0x0000000044e60000,0x0000000044f60000)]
0x000000001ff1e000 JavaThread "raft-client-raft-partition-3-0" [_thread_blocked, id=10032, stack(0x0000000045190000,0x0000000045290000)]
0x000000001ff1c800 JavaThread "raft-client-raft-partition-2-0" [_thread_blocked, id=10764, stack(0x0000000044d40000,0x0000000044e40000)]
0x000000001ff1c000 JavaThread "raft-client-raft-partition-2-0" [_thread_blocked, id=11052, stack(0x0000000044bd0000,0x0000000044cd0000)]
0x000000001ff1a800 JavaThread "raft-client-raft-partition-1-0" [_thread_blocked, id=10924, stack(0x0000000044aa0000,0x0000000044ba0000)]
0x000000001ff19000 JavaThread "raft-client-raft-partition-1-0" [_thread_blocked, id=4368, stack(0x0000000044800000,0x0000000044900000)]
0x000000001ff18000 JavaThread "raft-server-raft-partition-3-compaction" [_thread_blocked, id=4552, stack(0x0000000042d40000,0x0000000042e40000)]
0x000000001ff17800 JavaThread "raft-server-raft-partition-3-state" [_thread_blocked, id=9436, stack(0x0000000044700000,0x0000000044800000)]
0x000000001ff16800 JavaThread "raft-server-raft-partition-3-load" [_thread_blocked, id=856, stack(0x00000000444f0000,0x00000000445f0000)]
0x00000000229c7000 JavaThread "raft-server-raft-partition-3" [_thread_blocked, id=10052, stack(0x00000000442c0000,0x00000000443c0000)]
0x00000000229c6800 JavaThread "raft-server-raft-partition-3-compaction" [_thread_blocked, id=8552, stack(0x00000000441b0000,0x00000000442b0000)]
0x00000000229c5800 JavaThread "raft-server-raft-partition-3-state" [_thread_blocked, id=9348, stack(0x0000000044080000,0x0000000044180000)]
0x00000000229c5000 JavaThread "raft-server-raft-partition-3-load" [_thread_blocked, id=448, stack(0x0000000043f10000,0x0000000044010000)]
0x00000000229c4000 JavaThread "raft-server-raft-partition-3" [_thread_blocked, id=7160, stack(0x0000000043d20000,0x0000000043e20000)]
0x00000000229c0800 JavaThread "raft-server-raft-partition-2-compaction" [_thread_blocked, id=6280, stack(0x0000000043580000,0x0000000043680000)]
0x000000001ff3a800 JavaThread "raft-server-raft-partition-2-state" [_thread_blocked, id=3260, stack(0x00000000431a0000,0x00000000432a0000)]
0x000000001ff3a000 JavaThread "raft-server-raft-partition-2-load" [_thread_blocked, id=11380, stack(0x0000000043410000,0x0000000043510000)]
0x000000001ff39000 JavaThread "raft-server-raft-partition-2" [_thread_blocked, id=7588, stack(0x00000000432e0000,0x00000000433e0000)]
0x000000001ff38800 JavaThread "raft-server-raft-partition-2-compaction" [_thread_blocked, id=716, stack(0x0000000042f70000,0x0000000043070000)]
0x000000001ff37800 JavaThread "raft-server-raft-partition-2-state" [_thread_blocked, id=11632, stack(0x0000000040200000,0x0000000040300000)]
0x000000001ff37000 JavaThread "raft-server-raft-partition-2-load" [_thread_blocked, id=8952, stack(0x00000000425c0000,0x00000000426c0000)]
0x000000001ff36000 JavaThread "raft-server-raft-partition-2" [_thread_blocked, id=10632, stack(0x0000000043070000,0x0000000043170000)]
0x000000001ff35800 JavaThread "raft-server-raft-partition-2-compaction" [_thread_blocked, id=8928, stack(0x0000000042e60000,0x0000000042f60000)]
0x000000001ff34800 JavaThread "raft-server-raft-partition-2-state" [_thread_blocked, id=10412, stack(0x00000000419a0000,0x0000000041aa0000)]
0x000000001ff34000 JavaThread "raft-server-raft-partition-2-load" [_thread_blocked, id=8064, stack(0x0000000042c30000,0x0000000042d30000)]
0x00000000229b7000 JavaThread "raft-server-raft-partition-2" [_thread_blocked, id=11824, stack(0x0000000042a00000,0x0000000042b00000)]
0x00000000229b5800 JavaThread "raft-server-system-partition-1-6" [_thread_blocked, id=4192, stack(0x00000000426e0000,0x00000000427e0000)]
0x00000000229b5000 JavaThread "raft-server-system-partition-1-5" [_thread_blocked, id=12140, stack(0x00000000424b0000,0x00000000425b0000)]
0x00000000229b4000 JavaThread "raft-server-system-partition-1-4" [_thread_blocked, id=4100, stack(0x00000000422c0000,0x00000000423c0000)]
0x00000000229b3800 JavaThread "raft-server-system-partition-1-3" [_thread_blocked, id=760, stack(0x000000003fbf0000,0x000000003fcf0000)]
0x00000000229b2800 JavaThread "raft-server-system-partition-1-2" [_thread_blocked, id=10728, stack(0x0000000042180000,0x0000000042280000)]
0x00000000229b2000 JavaThread "raft-server-system-partition-1-1" [_thread_blocked, id=11288, stack(0x0000000042080000,0x0000000042180000)]
0x00000000229b1000 JavaThread "raft-server-system-partition-1-0" [_thread_blocked, id=10952, stack(0x0000000041bc0000,0x0000000041cc0000)]
0x00000000229b0800 JavaThread "raft-server-raft-partition-1-compaction" [_thread_blocked, id=10648, stack(0x0000000041d20000,0x0000000041e20000)]
0x0000000023b9a000 JavaThread "raft-server-raft-partition-1-state" [_thread_blocked, id=12040, stack(0x0000000041aa0000,0x0000000041ba0000)]
0x0000000023b99000 JavaThread "raft-server-raft-partition-1-load" [_thread_blocked, id=3776, stack(0x0000000041890000,0x0000000041990000)]
0x0000000023b98800 JavaThread "raft-server-raft-partition-1" [_thread_blocked, id=11392, stack(0x0000000041710000,0x0000000041810000)]
0x0000000023b97800 JavaThread "raft-server-raft-partition-1-compaction" [_thread_blocked, id=11540, stack(0x000000003bcb0000,0x000000003bdb0000)]
0x0000000023b97000 JavaThread "raft-server-raft-partition-1-state" [_thread_blocked, id=12128, stack(0x0000000040710000,0x0000000040810000)]
0x0000000023b96000 JavaThread "raft-server-raft-partition-1-load" [_thread_blocked, id=6272, stack(0x0000000041270000,0x0000000041370000)]
0x0000000023b95800 JavaThread "raft-server-raft-partition-1" [_thread_blocked, id=8176, stack(0x0000000041390000,0x0000000041490000)]
0x0000000023b94800 JavaThread "raft-server-raft-partition-1-compaction" [_thread_blocked, id=10036, stack(0x0000000040920000,0x0000000040a20000)]
0x0000000023b94000 JavaThread "raft-server-raft-partition-1-state" [_thread_blocked, id=9800, stack(0x0000000040ca0000,0x0000000040da0000)]
0x0000000023b93000 JavaThread "raft-server-raft-partition-1-load" [_thread_blocked, id=7252, stack(0x0000000041120000,0x0000000041220000)]
0x00000000229d2800 JavaThread "raft-server-raft-partition-1" [_thread_blocked, id=1964, stack(0x0000000040fb0000,0x00000000410b0000)]
0x00000000229d2000 JavaThread "raft-client-system-partition-1-15" [_thread_blocked, id=3108, stack(0x0000000040dc0000,0x0000000040ec0000)]
0x00000000229d0800 JavaThread "raft-client-system-partition-1-15" [_thread_blocked, id=7036, stack(0x0000000040a20000,0x0000000040b20000)]
0x00000000229cf800 JavaThread "raft-client-system-partition-1-14" [_thread_blocked, id=11704, stack(0x000000003fe40000,0x000000003ff40000)]
0x00000000229cf000 JavaThread "raft-client-system-partition-1-14" [_thread_blocked, id=11232, stack(0x0000000040810000,0x0000000040910000)]
0x00000000229cd800 JavaThread "raft-client-system-partition-1-13" [_thread_blocked, id=3588, stack(0x0000000040410000,0x0000000040510000)]
0x00000000229cc800 JavaThread "raft-client-system-partition-1-13" [_thread_blocked, id=9052, stack(0x0000000040300000,0x0000000040400000)]
0x00000000229cc000 JavaThread "raft-client-system-partition-1-12" [_thread_blocked, id=10192, stack(0x00000000400f0000,0x00000000401f0000)]
0x0000000022f4a800 JavaThread "raft-client-system-partition-1-12" [_thread_blocked, id=1632, stack(0x000000003fcf0000,0x000000003fdf0000)]
0x0000000022f4a000 JavaThread "raft-client-system-partition-1-11" [_thread_blocked, id=8532, stack(0x000000003f9f0000,0x000000003faf0000)]
0x0000000022f48800 JavaThread "raft-client-system-partition-1-11" [_thread_blocked, id=6252, stack(0x000000003f8c0000,0x000000003f9c0000)]
0x0000000022f47000 JavaThread "raft-client-system-partition-1-10" [_thread_blocked, id=12020, stack(0x000000003f7c0000,0x000000003f8c0000)]
0x0000000022f46000 JavaThread "raft-client-system-partition-1-9" [_thread_blocked, id=7828, stack(0x000000003f540000,0x000000003f640000)]
0x0000000022f45800 JavaThread "raft-client-system-partition-1-8" [_thread_blocked, id=10712, stack(0x000000003f3a0000,0x000000003f4a0000)]
0x00000000224ac000 JavaThread "raft-client-system-partition-1-10" [_thread_blocked, id=11128, stack(0x000000003e3c0000,0x000000003e4c0000)]
0x00000000224ab800 JavaThread "raft-client-system-partition-1-9" [_thread_blocked, id=936, stack(0x000000003f200000,0x000000003f300000)]
0x00000000224aa800 JavaThread "raft-client-system-partition-1-8" [_thread_blocked, id=6628, stack(0x000000003e1c0000,0x000000003e2c0000)]
0x00000000224a8800 JavaThread "raft-client-system-partition-1-7" [_thread_blocked, id=7464, stack(0x000000003ec10000,0x000000003ed10000)]
0x00000000224a7800 JavaThread "raft-client-system-partition-1-6" [_thread_blocked, id=11332, stack(0x000000003eb10000,0x000000003ec10000)]
0x00000000224a7000 JavaThread "raft-client-system-partition-1-5" [_thread_blocked, id=6660, stack(0x000000003e750000,0x000000003e850000)]
0x00000000224a6000 JavaThread "raft-client-system-partition-1-4" [_thread_blocked, id=1168, stack(0x000000003ea10000,0x000000003eb10000)]
0x00000000224a5800 JavaThread "raft-client-system-partition-1-3" [_thread_blocked, id=10864, stack(0x000000003e860000,0x000000003e960000)]
0x00000000224a0800 JavaThread "raft-client-system-partition-1-2" [_thread_blocked, id=2084, stack(0x000000003e620000,0x000000003e720000)]
0x000000002249f000 JavaThread "raft-client-system-partition-1-7" [_thread_blocked, id=11164, stack(0x000000003e2c0000,0x000000003e3c0000)]
0x000000002249e000 JavaThread "raft-client-system-partition-1-6" [_thread_blocked, id=12120, stack(0x000000003dba0000,0x000000003dca0000)]
0x000000002249c800 JavaThread "raft-client-system-partition-1-5" [_thread_blocked, id=11724, stack(0x000000003dee0000,0x000000003dfe0000)]
0x000000002249a800 JavaThread "raft-client-system-partition-1-4" [_thread_blocked, id=11036, stack(0x000000003d8c0000,0x000000003d9c0000)]
0x0000000022499800 JavaThread "raft-client-system-partition-1-3" [_thread_blocked, id=12096, stack(0x000000003d790000,0x000000003d890000)]
0x0000000020528000 JavaThread "raft-client-system-partition-1-2" [_thread_blocked, id=6556, stack(0x000000003d200000,0x000000003d300000)]
0x0000000020526800 JavaThread "raft-client-system-partition-1-1" [_thread_blocked, id=11964, stack(0x000000003d350000,0x000000003d450000)]
0x0000000020525000 JavaThread "raft-client-system-partition-1-0" [_thread_blocked, id=4856, stack(0x000000003d0d0000,0x000000003d1d0000)]
0x0000000020523800 JavaThread "raft-client-system-partition-1-1" [_thread_blocked, id=8264, stack(0x000000003c7a0000,0x000000003c8a0000)]
0x0000000020523000 JavaThread "raft-client-system-partition-1-0" [_thread_blocked, id=11008, stack(0x000000003ca20000,0x000000003cb20000)]
0x0000000020523800 JavaThread "raft-client-system-partition-1-1" [_thread_blocked, id=8264, stack(0x000000003c7a0000,0x000000003c8a0000)]
0x0000000020523000 JavaThread "raft-client-system-partition-1-0" [_thread_blocked, id=11008, stack(0x000000003ca20000,0x000000003cb20000)]
0x0000000020522000 JavaThread "netty-messaging-event-nio-client-15" [_thread_in_native, id=9156, stack(0x000000003cd60000,0x000000003ce60000)]
0x000000001fd30000 JavaThread "netty-messaging-event-nio-client-14" [_thread_in_native, id=9240, stack(0x000000003cb70000,0x000000003cc70000)]
0x000000001fd2f800 JavaThread "netty-messaging-event-nio-client-15" [_thread_in_native, id=11844, stack(0x000000003c8a0000,0x000000003c9a0000)]
0x000000001fd2e800 JavaThread "netty-messaging-event-nio-client-14" [_thread_in_native, id=8584, stack(0x000000003c6a0000,0x000000003c7a0000)]
0x000000001fd2e000 JavaThread "netty-messaging-event-nio-client-13" [_thread_in_native, id=9552, stack(0x000000003c540000,0x000000003c640000)]
0x000000001fd2d000 JavaThread "netty-messaging-event-nio-client-15" [_thread_in_native, id=11416, stack(0x000000003c370000,0x000000003c470000)]
0x000000001fd2c800 JavaThread "netty-messaging-event-nio-client-14" [_thread_in_native, id=7220, stack(0x000000003c220000,0x000000003c320000)]
0x000000001fd2b800 JavaThread "netty-messaging-event-nio-client-12" [_thread_in_native, id=10508, stack(0x000000003c0d0000,0x000000003c1d0000)]
0x000000001fd2b000 JavaThread "netty-messaging-event-nio-client-13" [_thread_in_native, id=6552, stack(0x000000003bfa0000,0x000000003c0a0000)]
0x000000001fd2a000 JavaThread "netty-messaging-event-nio-client-13" [_thread_in_native, id=10104, stack(0x000000003bdb0000,0x000000003beb0000)]
0x000000001fd29800 JavaThread "netty-messaging-event-nio-client-12" [_thread_in_native, id=1656, stack(0x000000003bbb0000,0x000000003bcb0000)]
0x000000002050d000 JavaThread "netty-messaging-event-nio-client-12" [_thread_in_native, id=9292, stack(0x000000003b100000,0x000000003b200000)]
0x000000002050c000 JavaThread "raft-server-system-partition-1-compaction" [_thread_blocked, id=6860, stack(0x000000003b7d0000,0x000000003b8d0000)]

Other Threads:
0x000000001c736800 VMThread [stack: 0x000000001e720000,0x000000001e820000] [id=11168]
0x000000001d866000 WatcherThread [stack: 0x000000001f670000,0x000000001f770000] [id=12156]

VM state:synchronizing (normal execution)

VM Mutex/Monitor currently owned by a thread: ([mutex/lock_event])
[0x000000000249c820] Safepoint_lock - owner thread: 0x000000001c736800
[0x000000000249c8a0] Threads_lock - owner thread: 0x000000001c736800

Heap:
PSYoungGen total 1132544K, used 302997K [0x000000076b300000, 0x00000007bec80000, 0x00000007c0000000)
eden space 894464K, 15% used [0x000000076b300000,0x00000007736357f8,0x00000007a1c80000)
from space 238080K, 70% used [0x00000007b0400000,0x00000007ba8b0000,0x00000007bec80000)
to space 237056K, 0% used [0x00000007a1c80000,0x00000007a1c80000,0x00000007b0400000)
ParOldGen total 535552K, used 354986K [0x00000006c1800000, 0x00000006e2300000, 0x000000076b300000)
object space 535552K, 66% used [0x00000006c1800000,0x00000006d72aa850,0x00000006e2300000)
Metaspace used 32342K, capacity 35552K, committed 35712K, reserved 1081344K
class space used 4158K, capacity 4658K, committed 4736K, reserved 1048576K

Card table byte_map: [0x0000000011d10000,0x0000000012510000] byte_map_base: 0x000000000e704000

Marking Bits: (ParMarkBitMap*) 0x0000000077f80600
Begin Bits: [0x0000000013610000, 0x00000000175b0000)
End Bits: [0x00000000175b0000, 0x000000001b550000)

Polling page: 0x0000000000170000

CodeCache: size=245760Kb used=17645Kb max_used=18923Kb free=228114Kb
bounds [0x0000000002950000, 0x0000000003c00000, 0x0000000011950000]
total_blobs=5979 nmethods=5411 adapters=479
compilation: enabled

Compilation events (10 events):
Event: 182.690 Thread 0x000000001d75f800 7176 3 jdk.internal.org.objectweb.asm.MethodWriter::visitFrame (233 bytes)
Event: 182.691 Thread 0x000000001d75f800 nmethod 7176 0x0000000003131450 code [0x0000000003131620, 0x0000000003131e18]
Event: 182.692 Thread 0x000000001d75f800 7177 3 jdk.internal.org.objectweb.asm.Frame::pop (53 bytes)
Event: 182.692 Thread 0x000000001d75f800 nmethod 7177 0x00000000031c5390 code [0x00000000031c5560, 0x00000000031c5a48]
Event: 182.692 Thread 0x000000001d75f800 7180 1 java.util.concurrent.atomic.AtomicInteger::set (6 bytes)
Event: 182.692 Thread 0x000000001d75f800 nmethod 7180 0x0000000003490550 code [0x00000000034906a0, 0x00000000034907b0]
Event: 182.692 Thread 0x000000001d75f800 7178 ! 3 java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue::remove (120 bytes)
Event: 182.693 Thread 0x000000001d75f800 nmethod 7178 0x0000000002fcfad0 code [0x0000000002fcfd00, 0x0000000002fd0b68]
Event: 182.693 Thread 0x000000001d75f800 7179 3 java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue::indexOf (79 bytes)
Event: 182.693 Thread 0x000000001d75f800 nmethod 7179 0x00000000030f9350 code [0x00000000030f9500, 0x00000000030f9bc8]

GC Heap History (10 events):
Event: 155.275 GC heap before
{Heap before GC invocations=46 (full 4):
PSYoungGen total 1086464K, used 943568K [0x000000076b300000, 0x00000007bf300000, 0x00000007c0000000)
eden space 806400K, 100% used [0x000000076b300000,0x000000079c680000,0x000000079c680000)
from space 280064K, 48% used [0x000000079c680000,0x00000007a4c74268,0x00000007ad800000)
to space 274432K, 0% used [0x00000007ae700000,0x00000007ae700000,0x00000007bf300000)
ParOldGen total 535552K, used 318250K [0x00000006c1800000, 0x00000006e2300000, 0x000000076b300000)
object space 535552K, 59% used [0x00000006c1800000,0x00000006d4eca850,0x00000006e2300000)
Metaspace used 31905K, capacity 34832K, committed 35328K, reserved 1079296K
class space used 4048K, capacity 4514K, committed 4608K, reserved 1048576K
Event: 155.367 GC heap after
Heap after GC invocations=46 (full 4):
PSYoungGen total 1102848K, used 143457K [0x000000076b300000, 0x00000007bed80000, 0x00000007c0000000)
eden space 834048K, 0% used [0x000000076b300000,0x000000076b300000,0x000000079e180000)
from space 268800K, 53% used [0x00000007ae700000,0x00000007b73184d0,0x00000007bed80000)
to space 267776K, 0% used [0x000000079e180000,0x000000079e180000,0x00000007ae700000)
ParOldGen total 535552K, used 325930K [0x00000006c1800000, 0x00000006e2300000, 0x000000076b300000)
object space 535552K, 60% used [0x00000006c1800000,0x00000006d564a850,0x00000006e2300000)
Metaspace used 31905K, capacity 34832K, committed 35328K, reserved 1079296K
class space used 4048K, capacity 4514K, committed 4608K, reserved 1048576K
}

Event: 164.078 GC heap before
{Heap before GC invocations=48 (full 4):
PSYoungGen total 1101824K, used 982544K [0x000000076b300000, 0x00000007bf000000, 0x00000007c0000000)
eden space 834048K, 100% used [0x000000076b300000,0x000000079e180000,0x000000079e180000)
from space 267776K, 55% used [0x000000079e180000,0x00000007a72842c8,0x00000007ae700000)
to space 260096K, 0% used [0x00000007af200000,0x00000007af200000,0x00000007bf000000)
ParOldGen total 535552K, used 333746K [0x00000006c1800000, 0x00000006e2300000, 0x000000076b300000)
object space 535552K, 62% used [0x00000006c1800000,0x00000006d5dec850,0x00000006e2300000)
Metaspace used 31905K, capacity 34832K, committed 35328K, reserved 1079296K
class space used 4048K, capacity 4514K, committed 4608K, reserved 1048576K
Event: 164.183 GC heap after
Heap after GC invocations=48 (full 4):
PSYoungGen total 1112064K, used 162320K [0x000000076b300000, 0x00000007be680000, 0x00000007c0000000)
eden space 861696K, 0% used [0x000000076b300000,0x000000076b300000,0x000000079fc80000)
from space 250368K, 64% used [0x00000007af200000,0x00000007b90842c8,0x00000007be680000)
to space 250880K, 0% used [0x000000079fc80000,0x000000079fc80000,0x00000007af180000)
ParOldGen total 535552K, used 333746K [0x00000006c1800000, 0x00000006e2300000, 0x000000076b300000)
object space 535552K, 62% used [0x00000006c1800000,0x00000006d5dec850,0x00000006e2300000)
Metaspace used 31905K, capacity 34832K, committed 35328K, reserved 1079296K
class space used 4048K, capacity 4514K, committed 4608K, reserved 1048576K
}
Event: 178.894 GC heap before
{Heap before GC invocations=50 (full 4):
PSYoungGen total 1112576K, used 1028288K [0x000000076b300000, 0x00000007bf180000, 0x00000007c0000000)
eden space 861696K, 100% used [0x000000076b300000,0x000000079fc80000,0x000000079fc80000)
from space 250880K, 66% used [0x000000079fc80000,0x00000007a9f30000,0x00000007af180000)
to space 243200K, 0% used [0x00000007b0400000,0x00000007b0400000,0x00000007bf180000)
ParOldGen total 535552K, used 343506K [0x00000006c1800000, 0x00000006e2300000, 0x000000076b300000)
object space 535552K, 64% used [0x00000006c1800000,0x00000006d6774850,0x00000006e2300000)
Metaspace used 31905K, capacity 34832K, committed 35328K, reserved 1079296K
class space used 4048K, capacity 4514K, committed 4608K, reserved 1048576K
Event: 178.983 GC heap after
Heap after GC invocations=50 (full 4):
PSYoungGen total 1132544K, used 168640K [0x000000076b300000, 0x00000007bec80000, 0x00000007c0000000)
eden space 894464K, 0% used [0x000000076b300000,0x000000076b300000,0x00000007a1c80000)
from space 238080K, 70% used [0x00000007b0400000,0x00000007ba8b0000,0x00000007bec80000)
to space 237056K, 0% used [0x00000007a1c80000,0x00000007a1c80000,0x00000007b0400000)
ParOldGen total 535552K, used 354986K [0x00000006c1800000, 0x00000006e2300000, 0x000000076b300000)
object space 535552K, 66% used [0x00000006c1800000,0x00000006d72aa850,0x00000006e2300000)
Metaspace used 31905K, capacity 34832K, committed 35328K, reserved 1079296K
class space used 4048K, capacity 4514K, committed 4608K, reserved 1048576K
}
Deoptimization events (10 events):
Event: 182.674 Thread 0x00000000224aa000 Uncommon trap: reason=unstable_if action=reinterpret pc=0x000000000350b758 method=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(J)J @ 89
Event: 182.674 Thread 0x0000000022f47800 Uncommon trap: reason=unstable_if action=reinterpret pc=0x000000000350b758 method=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(J)J @ 89
Event: 182.674 Thread 0x0000000022f4b800 Uncommon trap: reason=unstable_if action=reinterpret pc=0x000000000350b758 method=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(J)J @ 89
Event: 182.679 Thread 0x0000000022fbd800 Uncommon trap: reason=unstable_if action=reinterpret pc=0x000000000385a290 method=java.util.concurrent.ThreadPoolExecutor$Worker.tryAcquire(I)Z @ 6
Event: 182.679 Thread 0x00000000229b7000 Uncommon trap: reason=unstable_if action=reinterpret pc=0x0000000003997b7c method=java.util.HashMap$Values.forEach(Ljava/util/function/Consumer;)V @ 19
Event: 182.680 Thread 0x00000000229d2800 Uncommon trap: reason=null_check action=make_not_entrant pc=0x000000000342f664 method=io.atomix.cluster.NodeId.equals(Ljava/lang/Object;)Z @ 1
Event: 182.680 Thread 0x00000000229b7000 Uncommon trap: reason=null_check action=make_not_entrant pc=0x000000000342f664 method=io.atomix.cluster.NodeId.equals(Ljava/lang/Object;)Z @ 1
Event: 182.688 Thread 0x0000000022fbd800 Uncommon trap: reason=unstable_if action=reinterpret pc=0x00000000030923b0 method=java.lang.Class.getSimpleName()Ljava/lang/String; @ 39
Event: 182.709 Thread 0x0000000020507000 Uncommon trap: reason=class_check action=maybe_recompile pc=0x0000000003445bb4 method=io.atomix.protocols.raft.impl.RaftContext.lambda$null$23(Lio/atomix/protocols/raft/protocol/AppendRequest;)Ljava/util/concurrent/CompletableFuture; @ 5
Event: 182.710 Thread 0x00000000221c8800 Uncommon trap: reason=unstable_if action=reinterpret pc=0x00000000034c52f0 method=io.atomix.protocols.raft.roles.AbstractAppender.handleAppendResponse(Lio/atomix/protocols/raft/cluster/impl/RaftMemberContext;Lio/atomix/protocols/raft/protocol/AppendRequestU5被幷f@
Internal exceptions (10 events):
Event: 25.578 Thread 0x0000000022fbf800 Implicit null exception at 0x00000000034da414 to 0x00000000034da565
Event: 180.224 Thread 0x000000000242f000 Implicit null exception at 0x00000000039acf8f to 0x00000000039aefad
Event: 180.253 Thread 0x000000004548d800 Implicit null exception at 0x0000000002b447e3 to 0x0000000002b44ee5
Event: 180.294 Thread 0x000000001ff22000 Exception <a 'java/lang/IncompatibleClassChangeError': Found class java.lang.Object, but interface was expected> (0x0000000771393ee0) thrown at [C:\re\workspace\8-2-build-windows-amd64-cygwin\jdk8u74\6087\hotspot\src\share\vm\interpreter\linkResolver.cpp, n]笼琭@
Event: 181.406 Thread 0x0000000021155800 Exception <a 'java/net/ConnectException': Connection refused: no further information> (0x0000000771b2e420) thrown at [C:\re\workspace\8-2-build-windows-amd64-cygwin\jdk8u74\6087\hotspot\src\share\vm\prims\jni.cpp, line 735]
Event: 181.407 Thread 0x0000000021155800 Implicit null exception at 0x0000000003b4705a to 0x0000000003b47136
Event: 181.546 Thread 0x000000001fd99000 Exception <a 'java/net/ConnectException': Connection refused: no further information> (0x0000000771dd7378) thrown at [C:\re\workspace\8-2-build-windows-amd64-cygwin\jdk8u74\6087\hotspot\src\share\vm\prims\jni.cpp, line 735]
Event: 182.680 Thread 0x00000000229d2800 Implicit null exception at 0x000000000342f4cf to 0x000000000342f655
Event: 182.680 Thread 0x00000000229b7000 Implicit null exception at 0x000000000342f4cf to 0x000000000342f655
Event: 182.687 Thread 0x0000000022fbd800 Exception <a 'java/lang/NoSuchMethodError': sun.misc.Unsafe.invokeCleaner(Ljava/nio/ByteBuffer;)V> (0x00000007734cb230) thrown at [C:\re\workspace\8-2-build-windows-amd64-cygwin\jdk8u74\6087\hotspot\src\share\vm\interpreter\linkResolver.cpp, line 582]
Junbo Ruan
@aruanruan
Aug 05 2018 07:44

M Arguments:
jvm_args: -Dfile.encoding=UTF-8
java_command: org.eclipse.jdt.internal.junit.runner.RemoteTestRunner -version 3 -port 59594 -testLoaderClass org.eclipse.jdt.internal.junit4.runner.JUnit4TestLoader -loaderpluginname org.eclipse.jdt.junit4.runtime -test com.xxx.dcp.registry2.graph.UdoGraphTest:testdNotificationsOfManyUnhealthyTimeout
java_class_path (initial): D:\src\xxx-dcp\java-dcp-registry2\target\test-classes;D:\src\xxx-dcp\java-dcp-registry2\target\classes;D:\apache-maven-3.3.9\repo\javax\servlet\javax.servlet-api\3.1.0\javax.servlet-api-3.1.0.jar;D:\apache-maven-3.3.9\repo\javax\persistence\persistence-api\1.0.2\persistence-api-1.0.2.jar;D:\apache-maven-3.3.9\repo\javax\activation\activation\1.1.1\activation-1.1.1.jar;D:\apache-maven-3.3.9\repo\javax\xml\bind\jaxb-api\2.2.12\jaxb-api-2.2.12.jar;D:\apache-maven-3.3.9\repo\com\alibaba\hessian-lite\3.2.3\hessian-lite-3.2.3.jar;D:\apache-maven-3.3.9\repo\commons-cli\commons-cli\1.3\commons-cli-1.3.jar;D:\apache-maven-3.3.9\repo\org\rocksdb\rocksdbjni\5.9.2\rocksdbjni-5.9.2.jar;D:\apache-maven-3.3.9\repo\com\google\code\gsonplus\gsonplus\0.1.0\gsonplus-0.1.0.jar;D:\apache-maven-3.3.9\repo\org\slf4j\slf4j-api\1.7.9\slf4j-api-1.7.9.jar;D:\apache-maven-3.3.9\repo\org\slf4j\jcl-over-slf4j\1.7.9\jcl-over-slf4j-1.7.9.jar;D:\apache-maven-3.3.9\repo\ch\qos\logback\logback-core\1.2.3\logback-core-1.2.3.jar;D:\apache-maven-3.3.9\repo\org\antlr\antlr4\4.7\antlr4-4.7.jar;D:\apache-maven-3.3.9\repo\org\antlr\antlr4-runtime\4.7\antlr4-runtime-4.7.jar;D:\apache-maven-3.3.9\repo\org\antlr\antlr-runtime\3.5.2\antlr-runtime-3.5.2.jar;D:\apache-maven-3.3.9\repo\org\antlr\ST4\4.0.8\ST4-4.0.8.jar;D:\apache-maven-3.3.9\repo\org\abego\treelayout\org.abego.treelayout.core\1.0.3\org.abego.treelayout.core-1.0.3.jar;D:\apache-maven-3.3.9\repo\org\glassfish\javax.json\1.0.4\javax.json-1.0.4.jar;D:\apache-maven-3.3.9\repo\com\ibm\icu\icu4j\58.2\icu4j-58.2.jar;D:\apache-maven-3.3.9\repo\com\xxx\dcp\xxx-dcp-common\0.1.0\xxx-dcp-common-0.1.0.jar;D:\apache-maven-3.3.9\repo\com\xxx\dcp\xxx-dcp-common8\0.1.0\xxx-dcp-common8-0.1.0.jar;D:\apache-maven-3.3.9\repo\org\apache\commons\commons-lang3\3.7\commons-lang3-3.7.jar;D:\apache-maven-3.3.9\repo\com\alibaba\fastjson\1.2.39\fastjson-1.2.39.jar;D:\apache-maven-3.3.9\repo\org\springframework\spring-context\
Launcher Type: SUN_STANDARD

Environment Variables:
JAVA_HOME=D:\Java\jdk1.8.0_131
CLASSPATH=d:\antlr\antlr-4.5.3-complete.jar;d:\antlr\antlr-4.5.3-complete.jar;d:\antlr\antlr-4.5.3-complete.jar;%CLASSPATH%;d:\opencv\;;d:\opencv\;;d:\opencv\;
PATH=C:/Program Files/Java/jre1.8.0_144/bin/server;C:/Program Files/Java/jre1.8.0_144/bin;C:/Program Files/Java/jre1.8.0_144/lib/amd64;C:\Program Files (x86)\Common Files\Oracle\Java\javapath;D:\python\python27\Scripts\;D:\python\python27\;D:\Java\jdk1.8.0_131\bin;C:\ProgramData\Oracle\Java\javapath;D:\apache-maven-3.3.9\bin;D:\apache-ant-1.9.7\bin;C:\Program Files (x86)\Git\bin;C:\Program Files (x86)\PuTTY;c:\windows;c:\windows\system32;D:\nodejs\;d:\nodejs;D:\nodejs\node_global;d:\gradle-2.14\bin;C:\Program Files\TortoiseSVN\bin;d:\antlr;C:\Program Files (x86)\GtkSharp\2.12\bin;C:\Program Files\dotnet\;C:\Program Files\CMake\bin;C:\Strawberry\c\bin;C:\Strawberry\perl\site\bin;C:\Strawberry\perl\bin;C:\ZeroC\Ice-3.7.0\bin;C:\Program Files\Microsoft SQL Server\110\Tools\Binn\;d:\cmder;d:\clang\llvm\bin;D:\Go\bin;D:\Program Files (x86)\Graphviz2.38\bin;d:\Program Files\Git\cmd;d:\antlr;d:\curl\bin;d:\gradle-4.6\bin;;C:\Program Files\Microsoft VS Code\bin;D:\Java\jdk1.8.0_131\bin;D:\rust\rust-1.18\bin;C:\Users\r00362327\AppData\Local\atom\bin;C:\Users\r00362327\AppData\Roaming\npm;d:\antlr;d:\python\python27;C:\Program Files\Microsoft VS Code\bin;d:\curl\bin;;D:\eclipse;
USERNAME=r00362327
OS=Windows_NT
PROCESSOR_IDENTIFIER=Intel64 Family 6 Model 94 Stepping 3, GenuineIntel

--------------- S Y S T E M ---------------

OS: Windows 7 , 64 bit Build 7601 (6.1.7601.24150)

CPU:total 8 (4 cores per cpu, 2 threads per core) family 6 model 94 stepping 3, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ssse3, sse4.1, sse4.2, popcnt, avx, avx2, aes, clmul, erms, rtm, 3dnowpref, lzcnt, ht, tsc, tscinvbit, bmi1, bmi2, adx

Memory: 4k page, physical 16675768k(9625964k free), swap 33349660k(23850860k free)

vm_info: Java HotSpot(TM) 64-Bit Server VM (25.74-b02) for windows-amd64 JRE (1.8.0_74-b02), built on Jan 29 2016 17:42:36 by "java_re" with MS VC++ 10.0 (VS2010)

time: Sun Aug 05 11:57:46 2018
elapsed time: 182 seconds (0d 0h 3m 2s)

Jordan Halterman
@kuujo
Aug 05 2018 08:27
Memory mapped files have issues on Windows. Use StorageLevel.DISK in Raft partitions. That should actually be the default
Johno Crawford
@johnou
Aug 05 2018 08:35
@kuujo but we fixed that on windows
Maybe we used the buffer after cleaning it
That would result in a crash
Jordan Halterman
@kuujo
Aug 05 2018 08:44
Shouldn’t even be deleting files after 3 minutes, and we’d see exceptions with closed files when using StorageLevel.DISK
I’d still suggest not using it
Jordan Halterman
@kuujo
Aug 05 2018 09:14
Any logs from prior to the crash?