Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
Seth Yates
@sethyates
Is this thing on?
Kexin Xie
@realstraw
Which thing?
Kexin Xie
@realstraw
@/all code review please krux/hyperion#395
Joseph Price
@joprice
is there a way to update a pipeline’s definition without deleting and recreating?
Kexin Xie
@realstraw
@joprice at the moment, no, it does not support that
Joseph Price
@joprice
thanks for the quick response. checking out the code, it seems like it wouldn’t be too tricky. might just need to skip creation here on update https://github.com/krux/hyperion/blob/ee1291dce9d24ee0d354a2e270c64de78d9aa241/core/src/main/scala/com/krux/hyperion/client/UploadPipelineObjectsTrans.scala#L31
Kexin Xie
@realstraw
actually, if you use the CLI and do create --no-check it will perform an update to the pipeline that has the same name. However, because we generate the id for activities. It's going to create new objects in the pipeline rather than update the existing ones.
Joseph Price
@joprice
i see. i’m mostly concerned with retaining history
Bryce Cottam
@bcottam
it looks like the version available on github is behind that of the one leveraged in DMP.
I'm trying to find the api for specifying that an EmrCluster should use spot instances, but it's not standing out to me (in newer versions it's just spot(instanceType))