These are chat archives for CommBank/maestro
Ok, execution experts, can any of you think of a non-hacky (or not-completely-hacky) way of achieving the following with the Execution framework?
Currently in realtime we run a scalding Job inside a cluster-wide lock (the lock is managed via zookeeper). If, while running the Job, our connection to zookeeper drops out, we assume we have lost the lock and cancel the Job's Flow. We can cancel the Flow because the Job api allows us to add our own Flow listeners, and the listener callbacks give us access to the Flow instance.