do you need to encrypt the index? how are you going to update it? who should be able to query? what are expected read and write patterns?
do you have whitepaper?
Right now we are focusing on allowing people to deploy and use standalone repositories
All datas - such as the history tree of snapshots - are stored on IPFS through linked data
You can find more informations about that right here : https://medium.com/ryhope-network/pando-b5e1a2af3152
At one point we also intend to build a network of such repositories. Let's imagine that the pando protocol is git and we wanna build the GitHub for pando.
So we would like to optionally allow for a fluency database to index these IPLD data to make them searchable and all
So we don't need the index to be encrypted - for now, as it mostly aims at open source projects. The update would be made by allowed developer and the last commit would be stored on chain.
Exactly
Would there be any constraints or requirements about the IPLD data structure ?
(if there is so, would like to conform our data structures ASAP to avoid refactoring all the code once we integrate with Fluence :)
Also; here is our deck: http://slides.com/osarrouy/deck-3#/
Hey @osarrouy
So, for Pando Protocol case.
The main idea that Fluence is going to update index from Ethereum (not only, but it's an important feature we're working on), and then you can implement logic to organize your data with any Webassembly-supported language. Then do whatever you want with it :)
The workflow with Fluence might be the following:
- As I understand, on the first step a contract is created on Ethereum blockchain, and some files are added to IPFS. Once files are added, there's a commit transaction to that Ethereum contract
- We're planning to listen for Ethereum events, fetch a part of Ethereum state on event, and build a verifiable write transaction to Fluence based on it
- That transaction will come to Webassembly code you write, launched with Fluence, having access to persistent mutable state. On this step, you may organize links between data and get ready to output
ipld - A client will be able to read data from Webassembly via some API
- We are not planning to implement external writes, so Fluence is not going to be able to modify
iplddata. But you may implement some service that, for example, launches a periodical query to Fluence and updatesipldaccordingly. Or you may query Fluence directly from client's browser
So there's no special constrainst on ipld structure :)
Hey @alari . That's really cool. Just two questions: 1. to make it clear, it's gonna be possible to fetch the whole IPLD tree with the webassembly code to index it just with the hash of the root of the tree ? 2. When will it be possible to start writing the working on that feature and play with Fluence :D
Really exciting to start working with fluence anyhow :)
And also: thanks a lot for the update.
@osarrouy , for point 1: probably yes. depends on the size of the tree, and on ipld hashing logic -- you'll need to implement the same hashing to get addresses from the webassembly; and for 2: I hope, we will have something to play with in September or October -- yet you'll need to launch your own Fluence nodes to do it
I read the last comments in the channel, will fluence "only" support building indices or will it take care of data storage and replication as well?
About "only" building indices. Fluence builds, store, and replicate data in a small cluster
If you could describe us the problems you think Fluence is a good fit for, it would be very helpful :)
About status of the project, it's under active research and development, first demos should arrive soon, and we're aiming to release an alpha in Q1 2019 or maybe earlier
What happens if the index gets too huge?
So that the nodes can't store them anymore.
We are considering some kind of sharding but yet haven't found a way to guarantee performance in this case. So the memory limit is going to have a hard upper bound for now. The app developer should find a way to split the index into several parts according to application logic
