These are chat archives for thunder-project/thunder

27th
Nov 2015
Jeremy Freeman
@freeman-lab
Nov 27 2015 22:16
@kkcthans currently when loading most image formats the data is by default parallelized based on the number of images, or the number of cores, whichever is smaller. you can check explicitly after loading by calling data.rdd.getNumPartitions(), and you can choose the level of parallelism with the npartitions argument
Jeremy Freeman
@freeman-lab
Nov 27 2015 23:37
@/all a fairly major refactoring is nearly done, so would appreciate any feedback! Once it's merged I'll also put together a more thorough explanation of the changes. The PR is at thunder-project/thunder#237
and would appreciate feedback in particular on this naming decision https://github.com/thunder-project/thunder/pull/237#issuecomment-160198372