These are chat archives for atomix/atomix

1st
Mar 2018
Johno Crawford
@johnou
Mar 01 2018 18:30
@kuujo is there a list of tests which need porting or something
Jordan Halterman
@kuujo
Mar 01 2018 20:07

I think we gotta start from scratch here. I’ll take parts of our fault injection/verification tests, but everything else would probably be easier to just write new.

The test framework is done enough now to start using it. It does node crashes, various partitions, memory/cpu/disk stress tests, etc.

I think the thing to do is just start by writing primitive tests. The Python client is very incomplete, and writing primitive tests will force the client to be completed and cleaned up. Also should get the fault injection test done since it’s good for finding general problems.

Then I’ll start adding stress tests, which I’m actually doing now. My current work priority is profiling, so I’m writing tests to verify that the cluster stabilizes after periods of high CPU/memory/disk/network usage and timeouts don’t lead to memory leaks when retrying operations.

FYI here’s the fault injection/verification test I wrote for ONOS:
https://github.com/opennetworkinglab/onos/blob/master/tools/test/bin/onos-dist-verify

It just randomly kills/partitions nodes and attempts writes/reads on a value, then records the register and runs it through the Knossos linearizability checker

Johno Crawford
@johnou
Mar 01 2018 20:27
@kuujo sgtm you might want to tune Netty too eg. -Dio.netty.leakDetectionLevel=disabled
Johno Crawford
@johnou
Mar 01 2018 20:45
Maybe a bad example, I think in the latest netty they vastly improved the performance impact of the leak detector but it would still be nice to have a place to specify tuning opts
Jordan Halterman
@kuujo
Mar 01 2018 21:45
Ultimately should just use a configuration file for that