These are chat archives for CommBank/maestro

16th
Apr 2015
Stephan Hoermann
@stephanh
Apr 16 2015 01:09
@kfish here are my JVM_OPTS:
export JVM_OPTS="-Dsbt.override.build.repos=true -Dsbt.repository.config=$HOME/workspace/tooling.repositories/repositories-dev -Dfile.encoding=UTF8 -XX:MaxPermSize=2G -Xms512m -Xmx3g -XX:+CMSClassUnloadingEnabled -XX:+UseConcMarkSweepGC"
Conrad Parker
@kfish
Apr 16 2015 01:11
thanks
Sam Roberts
@SamRoberts
Apr 16 2015 01:11
only with sbt ...
would the max perm size be nearly as big as the max heap size
Stephan Hoermann
@stephanh
Apr 16 2015 04:48
Can someone please review #367.
Conrad Parker
@kfish
Apr 16 2015 05:54
CommBank/ebenezer#100 is ready for review (unsquashed)
Vineeth Varghese
@vineethvarghese
Apr 16 2015 07:21
Can someone please take a look at CommBank/ci#8
Sam Roberts
@SamRoberts
Apr 16 2015 13:59
can someone please review #368 , which:
  • updates the benchmark project to work on modern maestro versions
  • adds a check that the benchmark code compiles to travis (but that's all, doesn't run the benchmarks)
  • adds a benchmark for the delimited splitter
@stephanh interestingly, if the splitter benchmarks can be compared to the codec benchmarks (which is a big if, although I have made some preliminary attempt to keep them operating on "comparable" datasets), the delimited splitter is about twice as slow to split rows as the corresponding codec is to turn them into a thrift struct
however, the impact of wrapping the splitter results in a Result as per #364 is negligible, according to the benchmark (although that doesn't prove that the impact of other load pipeline changes is negligible)
Sam Roberts
@SamRoberts
Apr 16 2015 14:09
to give credit where credit is due, this made me take a second look at the DecodeMacro implementation, and I had forgotten how efficient it is
Stephan Hoermann
@stephanh
Apr 16 2015 22:06
@samroberts nice result.
I might be a little bit late again today
Rowan Davies
@rowandavies
Apr 16 2015 23:32
I’m conducting an interview at 10.