These are chat archives for CommBank/maestro

Oct 2014
Sam Roberts
Oct 13 2014 02:01
Could someone please review #205? Not urgent.
Vineeth Varghese
Oct 13 2014 02:20
For the Sqoop "custom import" we are not doing archival. Shouldn't we enforce this? Last friday I believe someone was asking whether upload is available without archival. We get no indication about the life of the source data, so I think we shouldn't let users opt out of it. What do you guys think?
Sam Roberts
Oct 13 2014 02:42
@vineethvarghese :+1:
Sam Roberts
Oct 13 2014 06:27
can someone please review #207 ? Not urgent ...
Sam Roberts
Oct 13 2014 22:17

I won't be available for the standup this morning (should be available from next week onwards though). My status:

  • Submitted two minor PRs: encourage users to use existing hadoop Config and change upload var names.
  • Helping @afsalthaj with his deltafinder commit. This marks a few times now that I've seen confusion around the validation stage throwing errors when people forget to use MaestroCascade.
  • Will start looking at some of my old TCS code reviews (or new code review requests if any come) to see how we may convert existing bash scripts to scala.
  • Will start talking with @jamindaw about log4j configuration.

(An interesting factoid: to add one 5 line function to maestro, the corresponding commit introduced 78 new lines, and 8 modified lines. That's two copyright notices, one set of specs boilerplate, two unit tests (could have been much more), adding the function to the maestro API, and changes to example code. Will still need more code changes to the downsteam example and util projects too ...)