Where communities thrive

  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
Repo info
    Jon Wood
    That would be really interesting @lucasvfventura!
    Eirik Tsarpalis
    Hi @lucasvfventura, that'd be awesome! There's no MBrace.ML library as of yet, but we'd be keen to get it started. I've worked on a couple of simple ML applications in MBrace together with @brandewinder, so we could obviously help in that direction.
    /cc @mathias-brandewinder
    Nick Palladinos
    Hi @lucasvfventura it would be great to have something like Spark.ML
    If you have mbrace related questions feel free to ask us.
    Lucas V. F. Ventura
    Hey guys have you hear of Prajna? Also I'm just starting with MBrace, do you have something close to a distributed data frame?
    Nick Palladinos
    Yes we are aware of Prajna. I think the main difference is that our main primitive is the cloud { ... } builder
    and we build everything on top of it.
    About the dataframe... we don't have something similar but I know that Tomas did some similar work with deedle and mbrace https://github.com/BlueMountainCapital/Deedle.BigDemo
    but I'm not familiar with the details of his work.
    Eirik Tsarpalis
    I should add that the MBrace.Flow library defines distributed data structures similar to data frames. But they are part of the library, not baked into the runtime.
    Similarly, a distributed data frame could be defined as part of an ML library
    or perhaps deedle could be integrated with MBrace
    Lucas V. F. Ventura
    For sure... I'm just taking the time to look deeper into F#, Scala and Spark. I'l try to make a demo combining the R TypeProvider and MBrace to build multiple machine learning models in a distributed way. I think doing this I'll get a sense of the MBrace.
    Eirik Tsarpalis
    Ok, great
    let us know if you have any questions
    afaik some work on mbrace/deedle has already been done by Tomas Petricek, see https://github.com/BlueMountainCapital/Deedle.BigDemo
    Lucas V. F. Ventura
    Unfortunately I have limited time to work on this =/.. only after work
    I'll take a look at that
    Eirik Tsarpalis
    Sure, no worries :-)
    Gauthier Segay
    I'd like to know what this would look like if I had to implement this in C#, just want a rough idea
      let SetupEverything () =
        let uplanFolder = ""
        let clusterPort = 11111
        let networkTimeout = TimeSpan.FromSeconds(5.0)
        cloud {
          do! Cloud.Logf "making archive from folder %s" uplanFolder
          let! archive = makePowerstackCloudArchive uplanFolder
          do! Cloud.Logf "installing archive on machines"
          do! installPowerstack archive
          let! machines = getMachineNames ()
          let clusterMachine = machines |> Seq.head
          do! Cloud.Logf "start cluster on %s:%i" clusterMachine clusterPort
          do! startClusterProcess clusterMachine
          let clusterReachable = NetworkChecker.CheckHost networkTimeout { HostName = clusterMachine ; Port = clusterPort }
          if not clusterReachable.ListensToSocket then
            do! Cloud.Logf "cluster %s:%i not reachable" clusterMachine clusterPort
          let! serversOutput = startServerProcesses clusterPort clusterMachine
          return clusterMachine, serversOutput
    Eirik Tsarpalis
    you mean using MBrace.CSharp?
    it can be done, but obviously it will be a lot more cumbersome
    using .OnSuccess(x => .. ) all the time
    Avi Avni
    @eiriktsarpalis please open new channel for mbrace.aws
    Kostas Rontogiannis
    @AviAvni done
    Avi Avni
    Avi Avni
    Hi please look on my new PR started adding graph processing to mbrace.core