Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Simon Kissane
    @skissane
    @kishorebanala @flavioschuindt I created a PR to allow config setting to control whether it is mandatory or optional – Netflix/conductor#1586 – hopefully this solution satisfies both those use cases which want it to be mandatory and also those use cases which don't want to use it
    Kishore
    @kishorebanala

    @chiragarora17

    Hi, qq : I want to orchestrate multiple libraries / Functions within a service. Is conductor a right solution for this kind of use case? Is there an example? I thought maybe system tasks (sub - workflow could fulfill) but would like to check it with an SME here.

    Please check https://netflix.github.io/conductor/labs/beginner/ if you haven't already to get a feel of Conductor. I'm not sure I understood what you meant by orchestrating multiple libraries / functions, but if you can split your workloads into multiple units of tasks and would want to orchestrate them repeatedly, Conductor would be great candidate. Feel free to ask if you have any further questions.

    @kishorebanala @flavioschuindt I created a PR to allow config setting to control whether it is mandatory or optional – Netflix/conductor#1586 – hopefully this solution satisfies both those use cases which want it to be mandatory and also those use cases which don't want to use it

    Thank you!

    Simon Kissane
    @skissane
    Hi. Can we have a discussion about Copyright notices. If I contribute a new file, is the Copyright notice in the license header my own copyright (which would be my employer's name), or Netflix Copyright notice, or both? Generally speaking, if someone writes code, the copyright belongs either to themselves or to their employer (although if they base it on existing Netflix code, it belongs to Netflix too). In my past experience, to make all files copyright by same copyright holder, requires a Contributor License Agreement (CLA) to be signed.
    Carlton Schuyler
    @cschuyle
    @sfeitler-oracle I finally got a branch up with code to reindex into embedded Elasticsearch at startup time. It's a work in progress:
    https://github.com/platform-acceleration-lab/conductor/tree/reindex-to-es
    sfeitler-oracle
    @sfeitler-oracle
    @cschuyle Cool. I'll take a look at it tomorrow (US time)
    ritu-p
    @ritu-p
    are all the cassandra test cases passing in dev branch?
    ritu-p
    @ritu-p
    Netflix/conductor#1280 this issue will it get merged?
    If not what is the work around?
    Carlton Schuyler
    @cschuyle

    @ritu-p I posted a workaround a little bit up the page on this forum. That workaround is essentially the same as the PR on Netflix/conductor#1280, with one difference: If you've already tried to run the migration and failed, the PR by itself would still require manual intervention (either by manipulating the database manually, or running a flyway repair). I'm wondering if the PR could be slightly tweaked by renaming the migration file ... that would require no manual intervention.

    I commented to the same effect on the issue, yesterday. No response yet ...

    Carlton Schuyler
    @cschuyle

    I would like some help on getting the tests on the dev branch to run.

    The section on 'Building' in the README at https://github.com/Netflix/conductor simply says "To build the server, use the following dependencies in your classpath", but honestly I don't know what that means. What I want is to be able to run the tests that I care about ( in my case, MySQL backend, and ES6 for the index data store).

    I am using a Mac locally and our pipeline does the build on Ubuntu. Both suites fail a plain ./gradlew clean build.

    These are the failing tests:

    MySQLExecutionDAOTest. complexExecutionTest
    MySQLExecutionDAOTest. testCreateTaskException
    MySQLExecutionDAOTest. testCreateTaskException2
    MySQLExecutionDAOTest. testPending
    MySQLExecutionDAOTest. testPendingByCorrelationId
    MySQLExecutionDAOTest. testTaskCreateDups
    MySQLExecutionDAOTest. testTaskExceedsLimit
    MySQLExecutionDAOTest. testTaskOps
    MySQLMetadataDAOTest. testDuplicate
    MySQLMetadataDAOTest. testEventHandlers
    MySQLMetadataDAOTest. testRemoveTaskDef
    MySQLMetadataDAOTest. testTaskDefOperations
    MySQLMetadataDAOTest. testWorkflowDefOperations
    MySQLQueueDAOTest. complexQueueTest
    MySQLQueueDAOTest. pollDeferredMessagesTest
    MySQLQueueDAOTest. pollMessagesTest
    MySQLQueueDAOTest. processUnacksTest
    charan986
    @charan986
    Hi I'm working on a conductor workflow definition and I need to send the current time as an input in the body, how can I achieve this?
    My body currently looks like
    "body": {
    "Appointment": "${workflow.input.Appointment}",
    "GracePeriodMinutes": "${workflow.input.GracePeriodMinutes}",
    "RouteRequestList": "${workflow.input.RouteRequestList}",
    "TrafficBufferMinutes": "${workflow.input.TrafficBufferMinutes}"
    "TimeStamp" :
    },
    how can I add the current system time for TimeStamp?
    ritu-p
    @ritu-p
    @cschuyle Thanks ..I did fix it locally with manula intervention..But was wondering if this can get fixed in the master itself
    Is anyone getting cassandra test failure in dev
    Task :conductor-cassandra-persistence:test
    05:59:21.733 [pool-7-thread-1] ERROR c.n.c.d.c.CassandraMetadataDAO - Failed to get all task defs
    com.datastax.driver.core.exceptions.DriverInternalError: Unexpected exception thrown
    Kartik Arora
    @akartik80
    Hi all, I'm new to conductor so apologies if this seems naive or is already answered. In development the server startup time is upwards of 1 min. Is there any way I can reduce it? It impacts developer productivity.
    ritu-p
    @ritu-p
    Please review Netflix/conductor#1589 this is fix for the issue Netflix/conductor#1538 @kishorebanala
    Arnaud-Francois-Fausse
    @Arnaud-Francois-Fausse
    Hi. Could you advice on how to debug a workflow. Where are execution logs ? Syslog catches some "technical logs" but I don't see any workflow execution log. Can I set breakpoints ? Thanks a lot.
    Carlton Schuyler
    @cschuyle
    Hi @mdepak , I have a question about the code to startup embedded Elasticsearch in conductor server. I think your name is on a ommit from January 20. Are you available for a brief chat?
    Kartik Arora
    @akartik80
    @Arnaud-Francois-Fausse yes you can import the project in intellij idea and set breakpoints while running in debug mode
    varchaswi gandham
    @varchaswi
    hi, anyone has vue or react UI example for metadata workflow registration api endpoin based on swqgger API?
    Anoop Panicker
    @apanicker-nflx

    @skissane After discussing this with other OSS projects and the legal team, your observation is correct.

    If I contribute a new file, is the Copyright notice in the license header my own copyright (which would be my employer's name).

    To add to this, any new contributions to the file would append their name/employer to the copyright header.

    Arnaud-Francois-Fausse
    @Arnaud-Francois-Fausse
    @akartik80 Thank you, how can I import the project, it's made of json files ?
    Kartik Arora
    @akartik80
    @Arnaud-Francois-Fausse apologies, I did'nt understand. I meant cloning https://github.com/Netflix/conductor on local and importing into idea. Post that you can set breakpoints.
    evgvain
    @evgvain
    Hi. I am trying to set a priority to the workflow when sending POST request to /workflow. However all workflows get priory 0. Any idea what can I check? I am using conductor with mysql .
    Simon Kissane
    @skissane
    @apanicker-nflx Thanks for the clarification Anoop. And thanks for merging those two PRs
    快刀青衣
    @kuaidaoqingyi1
    @mdepak Hi, how is the WorkflowStatusListener of the conductor used in the project, is there a simple case?
    ritu-p
    @ritu-p
    @apanicker-nflx Netflix/conductor#1589 .Please do take a look
    Kishore
    @kishorebanala

    I would like some help on getting the tests on the dev branch to run.

    The section on 'Building' in the README at https://github.com/Netflix/conductor simply says "To build the server, use the following dependencies in your classpath", but honestly I don't know what that means. What I want is to be able to run the tests that I care about ( in my case, MySQL backend, and ES6 for the index data store).

    I am using a Mac locally and our pipeline does the build on Ubuntu. Both suites fail a plain ./gradlew clean build.

    These are the failing tests:

    MySQLExecutionDAOTest. complexExecutionTest
    MySQLExecutionDAOTest. testCreateTaskException
    MySQLExecutionDAOTest. testCreateTaskException2
    MySQLExecutionDAOTest. testPending
    MySQLExecutionDAOTest. testPendingByCorrelationId
    MySQLExecutionDAOTest. testTaskCreateDups
    MySQLExecutionDAOTest. testTaskExceedsLimit
    MySQLExecutionDAOTest. testTaskOps
    MySQLMetadataDAOTest. testDuplicate
    MySQLMetadataDAOTest. testEventHandlers
    MySQLMetadataDAOTest. testRemoveTaskDef
    MySQLMetadataDAOTest. testTaskDefOperations
    MySQLMetadataDAOTest. testWorkflowDefOperations
    MySQLQueueDAOTest. complexQueueTest
    MySQLQueueDAOTest. pollDeferredMessagesTest
    MySQLQueueDAOTest. pollMessagesTest
    MySQLQueueDAOTest. processUnacksTest

    @cschuyle Consider checking https://netflix.github.io/conductor/server/ for more clear instructions about building and running locally. Basically, server module is the entry point that installs all other modules like mysql, ES5 or ES6 etc. You can tweak server module as required to install / remove Guice modules. You should still be able to run tests for each module separately.

    Hi I'm working on a conductor workflow definition and I need to send the current time as an input in the body, how can I achieve this?
    @charan986 Conductor doesn't have a way to pass in current timestamp to variables. Your start workflow input or task workers can supply this at workflow runtime.
    Kishore
    @kishorebanala
    Hey @akartik80, Conductor server on startup does a lot of things like setting up modules, applying configuration, ensuring ES indices are available etc. You should be able to remove some of these and bring down the startup time. We'll consider providing a "lite" version of Conductor server with quick start times.
    Carlton Schuyler
    @cschuyle
    @sfeitler-oracle I noticed that the branch I pushed in fact had an error. I pushed a correction; best to delete and re-checkout the branch if you've already checked it out locally.
    Carlton Schuyler
    @cschuyle
    Thanks @kishorebanala , I'll take a deeper look.
    sfeitler-oracle
    @sfeitler-oracle
    @cschuyle Thanks for the heads-up. As is the nature of things, I have been delayed by other works. So I haven't checked it out yet.
    neokyle1
    @neokyle1
    anyone know if priority works?
    I just submitted three jobs sequentially with varying priority (1,99,50) and they appear to run in the order submitted not in order of priority
    curl -X POST --header 'Content-Type: application/json' --header 'Accept: text/plain' -d '{}' 'http://conductor-api:9138/api/workflow/priority-workflow?priority=1'
    curl -X POST --header 'Content-Type: application/json' --header 'Accept: text/plain' -d '{}' 'http://conductor-api:9138/api/workflow/priority-workflow?priority=99'
    curl -X POST --header 'Content-Type: application/json' --header 'Accept: text/plain' -d '{}' 'http://conductor-api:9138/api/workflow/priority-workflow?priority=50'
    the priority shows correctly in the ui so it seems to be set correctly
    i am also only running one worker and starting it after all jobs are submitted
    neokyle1
    @neokyle1
    I am wondering if anyone has used the priority feature successfully
    ritu-p
    @ritu-p
    @kishorebanala Netflix/conductor#1589 ,I have addresses the comments
    ritu-p
    @ritu-p
    Leads Netflix/conductor#1589 Please do take a look at your convenience
    Kishore
    @kishorebanala
    @ritu-p Done.
    Bohdan
    @BohdanPr
    anyone know how to connect to the conductor via grpc? Link on some documentation will be helpful.
    maheshyaddanapudi
    @maheshyaddanapudi
    @BohdanPr please have a look this have gRPC clients available in GitHub https://github.com/Netflix/conductor/tree/master/grpc-client/src/main/java/com/netflix/conductor/client/grpc
    May be it will help in creating the gRPC client you were looking for
    skumar-GL
    @skumar-GL
    For our use case implementation we are trying to use Dynamic fork join to start n number of sub workflow and after completion of all sub workflow need to call join and execute further the main work flow, Not getting proper details on documentation how i can proceed with that https://netflix.github.io/conductor/configuration/systask/#dynamic-fork While using fork join i can only start fixed e.g. 2 or 3 or 4 etc number of sub workflow which not full fill our requirement, we want to start number of sub work flow which will very dynamic value of n
    Flávio Schuindt
    @flavioschuindt
    @skumar-GL we had the same use case. What we did was to create a SIMPLE task in the workflow followed by a dynamic fork. Then, we created a worker that pulls this task and change on the fly the inputData for the dynamic fork with N executions. This makes the dynamic fork starts N forks and execute each of them in parallel.
    skumar-GL
    @skumar-GL
    @flavioschuindt could you please share if you can for sample code base for the same thanks in advance