Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Kristian Mørch
    @kritollm
    vocabulary, so it makes no immediate sense without interpreting the code .
    Patricio M. Ros
    @pmros
    thank you, @kritollm
    you are right, we need more and better documentation
    specially at Graflow project
    Muxerand Demuxer are concepts borrowed from TSERS project.
    and, I agree with you, they need to be explained
    David Boyer
    @mangr3n
    You could very simply turn your tests into examples using the different components.
    Muxer, Demuxer, Accumulator are all simple if you look at the tests.
    Kristian Mørch
    @kritollm
    Thanks, I will look at the tests.
    Patricio M. Ros
    @pmros
    yes, tests are some kind of documentation. Anyway, I'll try to improve docs
    Patricio M. Ros
    @pmros
    a new sample: Ajax requests
    Patricio M. Ros
    @pmros
    I've added e2e test for Cyclow samples
    Chris Russell
    @ChrisRus
    just popping in to acknowledge good effort @pmros. I wrote https://github.com/Encapsule/jsgraph and have more recently been building a declaratively-configured plug-in app server for Node.js based on another lib called filter https://encapsule.io/docs/ARCcore/filter I'll need to dig into the docs and try a few examples to understand your model in depth.
    Patricio M. Ros
    @pmros
    @ChrisRus thank you for trying Cyclow
    David Boyer
    @mangr3n
    @pmros Are you still around?
    I want to move forward in a project using cyclow, AND I'm having some difficulties with it, and I'd like to check with you.
    I want a system of independent components that can be easily embedded in other applications. There is no simple way to do that currently.
    run(componentFunction) leaves me with no reference to the actual thing itself, so I can't do a comp.send(), or comp.on()
    So I wrote my own run function that returns a reference to the instantiated component, but now I'm struggling to understand how to send a signal into the component.
    I'd love to work with you to understand how to do this, and to write some wiki documentation to show some of the use cases, I'm interested in.
    David Boyer
    @mangr3n
    @pmros, given a cyclow Component, how do I call myComponent.on() to receive an output signal to some arbitrary function?
    David Boyer
    @mangr3n
    or how do I wire the component internally to output a value that I can listen for in the "on";
    David Boyer
    @mangr3n
    This is getting really frustrating. I don't understand how to send a message through the bus, that can be consumed through the Component's on function.
    David Boyer
    @mangr3n
    So it looks like I finally have a way of getting a message out of the component...
    component.outputs.default.on works, but all messages go to all handlers... and the handlers are simple functions. I'm looking at putting in a demuxer for outputs so I can register to listen on a specific port.
    David Boyer
    @mangr3n
    ok, so I have forked the codebase and started to work on making changes that simplify my use cases.
    David Boyer
    @mangr3n
    @pmros Really hoping I could get some help from you. I'm reworking this api to allow for some more complex usage scenarios. Cyclow is a great proof of concept, but it's lacking when it comes to real world use in an application. In particular, I'm trying to rework the emitting of events from a component, and I'd like to get access to the internal state of the component when I'm sending an output signal.
    David Boyer
    @mangr3n
    I've forked the codebases https://github.com/mangr3n/cyclow https://github.com/mangr3n/graflow, and am starting to make some changes to support a richer set of concerns in particular building an ecosystem of reusable embeddable components.
    David Boyer
    @mangr3n
    The current cyclow component model
    • Doesn't return a reference to the component for direct interaction (send/on)
    • The component defined doesn't provide a means of defining output ports for the component instance to expose it's output streams to the external application.
    David Boyer
    @mangr3n
    So I've made a couple of changes to my fork of the repository to allow a new function runComponent which will return a reference to the component. Pass it a reference to a function that returns a cyclow Block and it will return a reference to the instantiated graflow Component.
    David Boyer
    @mangr3n
    The second change is to add a demuxer to the output coming from the Block component, it now recognizes 'out.signals' and you can listen to runComponent(MyBlock).on(myListener), and will receive every value sent to out.signals
    David Boyer
    @mangr3n
    I'm here, supporting graflow, and cyclow, though I am working on a replacement to both, adding debugging, and some other features.
    And by supporting I mean I'm using it in a production application and migrating more and more of the application to graflow/cyclow
    David Boyer
    @mangr3n
    I'm really personally excited about the flow model of structuring an application, it's pretty amazing how simply the solutions to basic application problems flow out of the model, as you become fluent in the programming style.
    David Boyer
    @mangr3n
    So for everyone here, I've been using this for the past 7 months and have expanded it extensively. I'll list some of the modifications I've made to graflow and to cyclow, and more importantly additional capabilities I've built that expand on and use the graflow Component base.
    First thing I built was what I call a breadboard. It has the same API on/send as the basic component. It also supports "register()" which takes a data structure that is identical to a component. But it allows me to dynamically add and remove components and connections, adds a debug command for logging values as they are emitted from an output port. This allows me to dynamically construct a component, and to store that as an externalized graph of components and connections.
    I added a Graph function, which takes another data structure that is a list of graphs, and a list of connections. Each graph is a breadboard. and connections cross breadboards. It's all the same component API.
    Next I added the ability in the graph definitions or breadboard definitions to wrap connections with the equivalent of AOP's advice to allow for orthogonal concerns to be described in the component, breadboard ,or application graph.
    David Boyer
    @mangr3n
    Some examples of connection wrappers are, Counter, History, Assertion, CircuitBreaker, Debouncer. These could have been added as Components, but due to the complexity of some components, it's easier to wrap the orthogonal behavior around the data flowing through the connection, than to try to wrap the entire componet.
    Now, I've rebuilt a single page application bit by bit using this new set of capabilities. The debugging characteristics are unbelievable, I can debug at will, override connections, watch them, make assertions about the values flowing between components, track the history for any single connection, or track a large set of connections as a queryable sequence. Turn debugging and history on and off at will on any port in the graph.
    David Boyer
    @mangr3n
    Now due to the fact that it is all a series of graphs described in data, I am now ready to work on a Debugging UI that shows the running application as an interactive graph of Breadboards and Components. A breadboard is just a grouping of components, with their attendant connections, and an Application graph is a set of Breadboards and with their attendant connections, and an initialization block to kick of the graph.
    With a visual representation, and in particular the ability to track history per connection, I will be able to inspect the running system.
    David Boyer
    @mangr3n
    For me philosophically representing the entire running system as a dynamic graph of flowing data fits the reality of programming at this point in history. Performance demands and multi-core, multi-system applications are pushing us into creating small functional units of computation that maintain their own state and have a very limited view of the world. Components and systems built in this way can be rapidly distributed across cores, across system, and can be expanded to incredibly large scale. This same model can be implemented on the JVM through clojure (see core.async) and in other systems. To introspect the application and the values OVER TIME, and to graph the performance characteristics allows us to rapidly diagnose all of the common problems of asynchronous programs.
    David Boyer
    @mangr3n
    As an interesting footnote, here is a list of some interesting components I've built as I've expanded this system.
    • EventSourceComponent(object, eventName) --> uses addEventListener in the browser to listen for change events that the browser produces and introduces the values produced as signals into the application graph.
    • Throttle(count) => limits a unique resource. I use it for connections. It receives "reserve" and "release" signals, and a unique key for each signal. It emits a value when it is reserved. I use this to throttle the number of connections to the . server.
    • ThrottledValueEmitter => Connects to a Throttle, and queues values to emit. It sends a request to reserve a key associate with each value it receives. When it receives the key back from the Throttle, it emits the value, and at some point when it is done using the limited resource it calls release. I use this to emit values to an HttpComponent. that are transformed into server calls, which when they come back (success/error/timeout) release the key in the Throttle.
    • Assertion => wraps a Component or a connection (currently only single input/output) and asserts rules the value must conform to in order to be valid for the port (input or output).
      For example: ['existsChecker', 'unresolvedQueue', { assert: { 'pending?': [{ id: String, docId: String, status: String }], 'resolved?': [Object] } }] or Assertion('responseError', { in: { error: KeyedErrorSpec, args: HyLightsByDocSpec }, out: KeyedErrorSpec }, Prop('error'))
    David Boyer
    @mangr3n
    In February, I expect to start building a set of public repositories on github that contain the sources for this new set of capabilities. In particular I'd like to build out and expand the visualization as a project that can be embedded into a running Browser application. In addition I'd like to:
    • Rewrite graflow (maybe under a new name) to support a richer component level API, and to support dynamic addition of inputs, components, and connections, thus merging the Breadboard capabilities and the Component capabilities into a single entity. )
    • Add support to the graflow API for instantiation of nested graphs of components, thus merging in the Graph functionality described above.
    • Add support for metadata that is used by capabilities that are inserted into the graph at runtime, but are not part of the core graflow(or other name) API. Auto-wiring for pub/sub semantics is my first use.
    • Formalize the connection AOP function wrappers so that they can coded independently. I imagine there are a large set of behaviors that can be implemented this way.
    • Explore graphs rewriting graphs at runtime.
    • Use auto-wiring to rewrite the cyclow API to enable it to more easily embed in existing applications.
    • Formalize the component API, so that other types of components can be written.
    • Formalize a communication bus for wrapping around a websocket and dispatching into the running application, either through auto-wiring, or through a dispatch graph.
    • Explore use of graflow to build nodejs services and perhaps a service ecosystem (microservices?)
    • Produce a set of Browser Components, that wrap http calls, local storage, cookies, browser object (title, url, etc.) both as sources and sinks for the graph.
    David Boyer
    @mangr3n
    The debugging and visualization possibilities of this programming model are unbelievable. As developers we have trouble using our tools to visualize the running system. It's especially difficult for specific types of performance and synchronization bugs. I've used ramda extensively in order to avoid modifying a data value owned elsewhere. That's the number one issue. The second major issue is runaway feedback loops. Debouncers, History, and the CircuitBreaker allow me to quickly identify and debug these cases. Given an interactive visualization, this would be an order of magnitude simpler and faster. I see this system as a logical integration of the idea of microservices with the recent influx of reactive programming models like cyclejs.
    David Boyer
    @mangr3n
    I grew up in a house with an electrical engineer and had 4 years of electronics in high school. I have that model deeply ingrained in my head. This started for me as an epiphany that with a debug capability on any port, I had the equivalent of an oscilloscope that could poke and prod a running graph for information. The Breadboard grew out of that, into the idea of putting components together into a circuit on a breadboard. Like one does in electronics. Finally, I realized that this component graph model is simply a different isomorphism for all of the standard programming memes. The value is in what the graph/component topography allows and at what cost. The function call overhead is present, due to large chains of functions, in Javascript this seems to be negligible impact, I don't see much. The system I have now is far faster than the redux based on that I was building. And my data structures are now isolated to what is actually needed to know to do the specific data transformation at this location exactly when it's needed. I don't have to filter huge atomic structures to find the values that I'm interested in. So in exchange for this "cost" I have the ability to view my running application as a cyclic directed graph. Making this interactive will allow me to see the flow semantics visually and to diagnose specific types of errors extremely quickly. Assertions give me type checking at runtime, obviously allowing me to visualize type mismatches in the visual graph. History takes it even further, allowing me to examine the sequence and timing of the firing of signals, to graph sets of information over time, and to graph multiple values over time into the same visual representation. I have timing information in history, etc. The possibilities are endless, and I'm quite frankly beside myself in love with this programming paradigm.
    David Boyer
    @mangr3n
    Inside my components, I still follow certain rules about using pure functions as much as possible, and isolating state to a single component. There is no shared state, I have stores that handle mutation of the data they own. Everything else is passed as a value as much as possible. I have a copy and an immutable one that won't change underneath me. As I've become more and more fluent in this paradigm, I continually am surprised at the simplicity and the elegance of the solutions that emerge. More importantly, I'm ending up with reusable components that are the equivalent of integrated chips, or mechanical subsystems in a vehicle. There's a tremendous amount of reuse. Can we finally build programming components that can be assembled rapidly into running systems?
    David Boyer
    @mangr3n
    No one's been in here in quite a while. This is disappointing.