These are chat archives for thunder-project/thunder

5th
Mar 2015
Jeremy Freeman
@freeman-lab
Mar 05 2015 01:10
@wolfbill maybe tell me more specific questions you have, or aspects you are interested in?
mixtou
@mixtou
Mar 05 2015 11:38
Is Thender capable of using realtime streams??
Noah Young
@npyoung
Mar 05 2015 20:31
I noticed thunder-project/thunder#125 because it was getting my whole cluster into a strange state and I was curious if anyone's seen something similar. If only about 5% of tasks fail, the stage is salvaged by spark.speculation and the job completes. If more than about 5% of tasks fail, the stage fails and keeps retrying with different (arbitrary?) partitioning indefinitely. Ideas on what Spark is trying to do there?