These are chat archives for CommBank/maestro

8th
Jun 2015
Sam Roberts
@SamRoberts
Jun 08 2015 13:29

Ok, execution experts, can any of you think of a non-hacky (or not-completely-hacky) way of achieving the following with the Execution framework?

Currently in realtime we run a scalding Job inside a cluster-wide lock (the lock is managed via zookeeper). If, while running the Job, our connection to zookeeper drops out, we assume we have lost the lock and cancel the Job's Flow. We can cancel the Flow because the Job api allows us to add our own Flow listeners, and the listener callbacks give us access to the Flow instance.

In order to do the same thing with Execution, it seems I'll be needing to build and run the Flow myself (maybe with some help from some "discouraged" Execution methods, and maybe copying some of Execution's internal code?). And I'll need to handle the possibility that the Flow may or may not exist at the time we lose the zookeeper connection.
Stephan Hoermann
@stephanh
Jun 08 2015 13:39
Do you just need to get access to the flow def that used to run the job?
Sam Roberts
@SamRoberts
Jun 08 2015 14:23
Ultimately, I need to be able to cancel the Flow. AFAIK that means I need access to the Flow instance. I don't think I can use the FlowDef to get access to the Flow instance.
Vineeth Varghese
@vineethvarghese
Jun 08 2015 23:38
Will be working from home today. Please call me for the planning meeting
Stephan Hoermann
@stephanh
Jun 08 2015 23:43
@SamRoberts I'm not sure of a nice way to do that.
Conrad Parker
@kfish
Jun 08 2015 23:44
Running some stats on the ingest log, we seem to be getting around 29GB per business day incoming, does that sound about right?
Stephan Hoermann
@stephanh
Jun 08 2015 23:46
Ask in Datahub. They will have more of an idea.
Gavin Whyte
@gavinwhyte
Jun 08 2015 23:47
please note no access to dn01 or dsa2 this morning.
Sam Roberts
@SamRoberts
Jun 08 2015 23:49
@stephanh alright. We're going to lose the lock for now, and just ensure we don't run the program concurrently instead.
Vineeth Varghese
@vineethvarghese
Jun 08 2015 23:53
Can someone please take a look at CommBank/etl-controller#28
Rowan Davies
@rowandavies
Jun 08 2015 23:55
@vineethvarghese Looking now.