These are chat archives for freeman-lab/discussion

8th
Mar 2016
Jeremy Freeman
@freeman-lab
Mar 08 2016 04:13
@jwittenbach that benchmark is pretty impressive!
cc @d-v-b @sofroniewn
Jason Wittenbach
@jwittenbach
Mar 08 2016 04:15
yeah, it’s looking good!
bumped up to 40 nodes
using map_as_series it cut the time in half…I’ll take a linear speed-up :)
Jeremy Freeman
@freeman-lab
Mar 08 2016 04:16
ah, so that’s now measuring performance as number of nodes increase, not the comparison to the old approach
also really good to know!
Jason Wittenbach
@jwittenbach
Mar 08 2016 04:16
yeah
Jeremy Freeman
@freeman-lab
Mar 08 2016 04:16
the boost over old seems at least 20x!
Jason Wittenbach
@jwittenbach
Mar 08 2016 04:16
also trying to see if more nodes will let it complete with the old approach
Jeremy Freeman
@freeman-lab
Mar 08 2016 04:16
yeah definitely
Jason Wittenbach
@jwittenbach
Mar 08 2016 04:17
if you watch the job, you can see where the old approach gets hung up: I think it eventually runs out of scratch space, because eventually you see the GC time for one task just shoot through the roof
and it never recovers
Jeremy Freeman
@freeman-lab
Mar 08 2016 04:18
yeah that’s consistent with @d-v-b’s experience
this is huge
might be the most valuable performance enhancement since, well, we started all this!
Jason Wittenbach
@jwittenbach
Mar 08 2016 04:19
it’s good to see all of the work with the swapping and chunking finally pay some dividends :)
Jeremy Freeman
@freeman-lab
Mar 08 2016 04:51
yeah!
Nicholas Sofroniew
@sofroniewn
Mar 08 2016 05:07
do you think something like this could help the localCorr
that's failing quite a lot too
Nicholas Sofroniew
@sofroniewn
Mar 08 2016 06:18
@freeman-lab initial labjack testing over the USB is quite disappointing 1 - 10 ms times for most DAQ operations
We can also talk to the labjack over a network cable using TCP https://labjack.com/support/software/examples/modbus/nodejs
Not sure yet if that will be faster, but probably give if I can't improve the USB communication
Nicholas Sofroniew
@sofroniewn
Mar 08 2016 06:24
  • give it a try if
Nicholas Sofroniew
@sofroniewn
Mar 08 2016 22:30