Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Jun 24 14:18

    nahsi on aws_ci

    Cancel in progress (compare)

  • Jun 24 14:04

    nahsi on aws_ci

    Run publish_branch on builder (compare)

  • Jun 24 14:02

    nahsi on aws_ci

    Run on builder (compare)

  • Jun 24 13:48

    folex on aws_ci

    publish branch (compare)

  • Jun 24 13:00

    folex on aws_ci

    runs-on: xlarge (compare)

  • Jun 23 12:45
    nahsi synchronize #1282
  • Jun 23 12:45

    nahsi on detect_anomalies

    Trigger (compare)

  • Jun 23 12:43
    folex synchronize #1282
  • Jun 23 12:43

    folex on detect_anomalies

    avm-server master (compare)

  • Jun 23 12:42
    nahsi synchronize #1282
  • Jun 23 12:42

    nahsi on detect_anomalies

    Format FluenceCLA (compare)

  • Jun 23 12:23
    nahsi synchronize #1282
  • Jun 23 12:23

    nahsi on detect_anomalies

    Format CONTRIBUTING (compare)

  • Jun 23 12:21
    folex review_requested #1282
  • Jun 23 12:21
    folex review_requested #1282
  • Jun 23 12:21
    folex opened #1282
  • Jun 23 12:16

    folex on detect_anomalies

    write anomaly data to a single … (compare)

  • Jun 23 10:50

    nahsi on detect_anomalies

    Format README (compare)

  • Jun 23 10:40

    folex on detect_anomalies

    detect anomalies and write thei… (compare)

  • Jun 22 13:50

    github-actions[bot] on gh-pages

    deploy: 8141fb0aa03501bdb6a9e29… (compare)

Dmitry Kurinskiy
@alari
Hi @osarrouy , at least I am, nice to meet you
Olivier Sarrouy
@osarrouy
Greaaat! :)
Hi everyone. I'm the main dev of the pando protocol which is a distributed VCS - whose dev is actually funded by Aragon - and we are working on database integration right now. I would have a few technical questions here. Mostly about the ability to store data on IPFS and optionnaly have them indexed through Fluence 🙂
I'm not sure to understand how fluency works, actually. But I've seen your presentation at Dappcon and was really excited about that :)
Dmitry Kurinskiy
@alari
sounds quite relevant to Fluence :)
do you need to encrypt the index? how are you going to update it? who should be able to query? what are expected read and write patterns?
do you have whitepaper?
Olivier Sarrouy
@osarrouy
Hey. Don’t have a white paper for now. An old one actually but things have move pretty fast since then.
Right now we are focusing on allowing people to deploy and use standalone repositories
All datas - such as the history tree of snapshots - are stored on IPFS through linked data
You can find more informations about that right here : https://medium.com/ryhope-network/pando-b5e1a2af3152
At one point we also intend to build a network of such repositories. Let's imagine that the pando protocol is git and we wanna build the GitHub for pando.
So we would like to optionally allow for a fluency database to index these IPLD data to make them searchable and all
So we don't need the index to be encrypted - for now, as it mostly aims at open source projects. The update would be made by allowed developer and the last commit would be stored on chain.
Dmitry Kurinskiy
@alari
on which chain?
Olivier Sarrouy
@osarrouy
Ethereum
Dmitry Kurinskiy
@alari
so you will have a smart contract for each repo, list of committers in that contract, and the last commit hash, right?
Olivier Sarrouy
@osarrouy
So basically we would need Fluence to listen for events on the Ethereum blockchain, then rebuild the IPLD data tree and index it to make it easily searchable and traversable.
Exactly
Dmitry Kurinskiy
@alari
yeah, that's the use case we're digging in right now: listening for Ethereum events and updating Fluence index based on them
Olivier Sarrouy
@osarrouy
That would be really cool for us. Because basically that would allow people to deploy their own repository smart contract without being indexed - if they don't want to. And to opt-in to our network to gain indexability.
Would there be any constraints or requirements about the IPLD data structure ?
(if there is so, would like to conform our data structures ASAP to avoid refactoring all the code once we integrate with Fluence :)
Dmitry Kurinskiy
@alari
let us study it a little? we'll discuss the use case internally and ping you back on the beginning of next week, wdyt?
Olivier Sarrouy
@osarrouy
That would be perfect.
Dmitry Kurinskiy
@alari
great :)
Olivier Sarrouy
@osarrouy
We're still at the early stage of the development so there's also some space on our side to adapt to Fluence requirements.
Also; here is our deck: http://slides.com/osarrouy/deck-3#/
Dmitry Kurinskiy
@alari
thanks!
Olivier Sarrouy
@osarrouy
We still have a few thing to finish but it can give you a good idea of what we're trying to build :)
Dmitry Kurinskiy
@alari

Hey @osarrouy
So, for Pando Protocol case.

The main idea that Fluence is going to update index from Ethereum (not only, but it's an important feature we're working on), and then you can implement logic to organize your data with any Webassembly-supported language. Then do whatever you want with it :)

The workflow with Fluence might be the following:

  • As I understand, on the first step a contract is created on Ethereum blockchain, and some files are added to IPFS. Once files are added, there's a commit transaction to that Ethereum contract
  • We're planning to listen for Ethereum events, fetch a part of Ethereum state on event, and build a verifiable write transaction to Fluence based on it
  • That transaction will come to Webassembly code you write, launched with Fluence, having access to persistent mutable state. On this step, you may organize links between data and get ready to output ipld
  • A client will be able to read data from Webassembly via some API
  • We are not planning to implement external writes, so Fluence is not going to be able to modify ipld data. But you may implement some service that, for example, launches a periodical query to Fluence and updates ipld accordingly. Or you may query Fluence directly from client's browser

So there's no special constrainst on ipld structure :)

Olivier Sarrouy
@osarrouy
Hey @alari . That's really cool. Just two questions: 1. to make it clear, it's gonna be possible to fetch the whole IPLD tree with the webassembly code to index it just with the hash of the root of the tree ? 2. When will it be possible to start writing the working on that feature and play with Fluence :D
Really exciting to start working with fluence anyhow :)
And also: thanks a lot for the update.
Dmitry Kurinskiy
@alari
@osarrouy , for point 1: probably yes. depends on the size of the tree, and on ipld hashing logic -- you'll need to implement the same hashing to get addresses from the webassembly; and for 2: I hope, we will have something to play with in September or October -- yet you'll need to launch your own Fluence nodes to do it
Florian Lenz
@florianlenz
Hi. Which query language will you support?
I read the last comments in the channel, will fluence "only" support building indices or will it take care of data storage and replication as well?
Dmitry Kurinskiy
@alari
Hi @florianlenz ! For the query language: it depends on your actual needs. Fluence is a bottom layer for launching data-local computations, and you may express them the way you like with use of Webassembly. We're preparing a demo with a subset of SQL
About "only" building indices. Fluence builds, store, and replicate data in a small cluster
Florian Lenz
@florianlenz
Cool. What is the definition of a small cluster?
Dmitry Kurinskiy
@alari
It's a subset of Fluence network, approx. 4-25 nodes that replicate the data and operate with it in near real-time
Florian Lenz
@florianlenz
It's possible to build indices across cluster, right? What is the status of the project? I think fluence is the solution for a lot of problems we are facing in our applications. IPFS and there DHT is too low level. Will there be a technical specification?
Dmitry Kurinskiy
@alari
We are working on a new tech paper now, should be available this fall. Indices are to be built in each cluster, meaning that index is an efficient persistent data structure, optimized for particular access patterns
If you could describe us the problems you think Fluence is a good fit for, it would be very helpful :)
About status of the project, it's under active research and development, first demos should arrive soon, and we're aiming to release an alpha in Q1 2019 or maybe earlier
Florian Lenz
@florianlenz
Nice! Can't wait to read it. I can imagine when e.g. building a DEX this would be quite useful to build up a searchable orderbook. Were you can e.g. query for Where can select all offers with less than 3234 coins and were one coin costs less than 10ct (just an example).
What happens if the index gets too huge?
So that the nodes can't store them anymore.
Dmitry Kurinskiy
@alari
We are considering some kind of sharding but yet haven't found a way to guarantee performance in this case. So the memory limit is going to have a hard upper bound for now. The app developer should find a way to split the index into several parts according to application logic
Olivier Sarrouy
@osarrouy
Hey @alari Do you know if there’s anyone of your team at ethberlin ? That could be really cool to talk forward about the integration of fluence into ryhope / pando :h