These are chat archives for thunder-project/thunder

Jun 2017
Jun 30 2017 15:23
@d-v-b So I went back to running it on Spark again and it seems to still be slightly faster to load the multi-page tiffs than the individual tiffs. I'm still loading the files from a standard file system since Thunder can't load tiffs from an hdfs. I imagine reading from the hdfs it might be quicker, but it seems like the multipage tiffs are still quicker.
Davis Bennett
Jun 30 2017 15:31
i don't understand how reading from a single file can be faster than reading from multiple files in a parallel context, but whatever works!
Jun 30 2017 15:33
It's a difference of 3-7 seconds or so from my testing, so not much, but it is faster in both cases on Spark than it is in local mode.