Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Jarrod Overson
    @jsoverson
    @flavio sorry for the short notice, I need to figure out gitter notifications
    Phil Kedy
    @pkedy
    @jsoverson I'll be on the call @ 2:30ish EST
    Jarrod Overson
    @jsoverson
    @pkedy np.
    Phil Kedy
    @pkedy
    almost there...
    Marius Ileana
    @dxps
    Hi everyone!
    Today I had an excellent development experience, while playing with three implementations of waPC host (Go, Node, and Rust) that use one waPC guest (Rust based). Beyond the natural Thank You!and excitement, since I'm pretty interested into WASM based capabilities on the server-side, I would really want to contribute to waPC!
    So, if you have any recommendations for me to warmup into this space, please share. Meanwhile, I'll study the code.
    Phil Kedy
    @pkedy
    Hi @dxps! Thanks for the feedback. My apologies for the delayed response. In terms of where the project needs contributions, waPC has two layers. The lowest layer are the host and guest libraries that basically handle passing and returning byte buffers. The next layer is code generation that creates the “glue code” to create modules that pass rich data structures. At both layers we need more language support. The codegen is missing support for Zig for example. And there is no support at either layer for Grain or Ruby. I’ll think about this a bit more but those are my initial thoughts.
    Daniel Pyrathon
    @PirosB3
    Hi All! I just found out about waPC and it's exactly what i was looking for: a thin wrapper around WASM that allows the passing of non-wasm datastructures through exported memory.
    I'm thinking of building a project that uses waPC, so before I venture into this I wanted to know how well maintained is waPC? is it being actively maintained and is there a roadmap of some kind that I can see?
    congrats on the project, I love how waPC is one of the few projects that doesn't make an assumption on my host environment (it's not JS) :)
    fawadasaurus
    @fawadasaurus
    Hey Daniel. This is definitely an actively supported project. In fact, @jsoverson and I are building a startup and are using waPC deeply embedded into the overall solution. We have no plan to move away from waPC and hope to continue to contribute to it for as long as we can.
    Daniel Pyrathon
    @PirosB3
    That's awesome! congrats

    One question for you, I'm using waPC and my code is using a library that supposedly requires wasm_bindgen even if my project is not using wasm_bindgen at all but only waPC.

    Because of this, now, my function imports have been completely polluted by wbindgen stuff

    wasmer inspect /tmp/hello_world_rust/target/wasm32-unknown-unknown/release/hello_world_rust.wasm | head -10
    Type: wasm
    Size: 2.6 MB
    Imports:
      Functions:
        "__wbindgen_placeholder__"."__wbindgen_describe": [I32] -> []
        "__wbindgen_placeholder__"."__wbg_instruction_new": [I32] -> [I32]
        "__wbindgen_placeholder__"."__wbindgen_object_drop_ref": [I32] -> []
        "__wbindgen_placeholder__"."__wbg_pubkey_new": [I32] -> [I32]
        "__wbindgen_placeholder__"."__wbindgen_string_get": [I32, I32] -> []
        "__wbindgen_placeholder__"."__wbindgen_is_undefined": [I32] -> [I32]

    Any thought on why this could be happening?

    fawadasaurus
    @fawadasaurus
    Can you share your code for hello_world_rust?
    Jarrod Overson
    @jsoverson
    @PirosB3 that's not too uncommon. Usually it requires a fork or a PR to move the wasm-bindgen stuff to a feature so it can be compiled out.
    Wasm-bindgen dominated the wasm space early and a glut of libraries now depend on it.
    What are you using if you don't mind me asking? I ran into this again this week so we may be overlapping.
    Daniel Pyrathon
    @PirosB3
    Of course I can share

    lib.rs

    mod generated;
    extern crate wapc_guest as guest;
    use std::collections::HashSet;
    
    use borsh::{BorshDeserialize, BorshSerialize};
    pub use generated::*;
    use guest::prelude::*;
    use serde::Serialize;
    use solana_sdk::transaction::Transaction;
    
    #[no_mangle]
    pub fn wapc_init() {
        Handlers::register_decode_transaction(decode_transaction);
    }
    
    #[derive(BorshSerialize, BorshDeserialize, PartialEq, Debug, Serialize)]
    struct SellInstruction {
        seller_state_bump: u8,
        program_as_signer_bump: u8,
        buyer_price: u64,
        token_size: u64,
        seller_state_expiry: [u8; 8],
    }
    
    pub fn decode_transaction(transaction_value: String) -> HandlerResult<Option<String>> {
        let transaction_bytes = base64::decode(transaction_value).unwrap();
        let decoded: Result<Transaction, _> = deserialize(&transaction_bytes);
        match decoded {
            Ok(val) => {
                let mut result: Option<String> = None;
                for ix in val.message.instructions.iter() {
                    if result.is_some() {
                        break;
                    }
                    let program_id = ix.program_id_index as usize;
                    let acct = val.message.account_keys.get(program_id).unwrap();
                    let encoded = bs58::encode(acct.to_bytes()).into_string();
                    if encoded == SMART_CONTRACT_ADDRESS {
                        let prefix = &ix.data[0..8];
                        match prefix {
                            &[51, 230, 133, 164, 1, 127, 131, 173] => {
                                let foo = SellInstruction::try_from_slice(&ix.data[8..]).unwrap();
                                let val: String = serde_json::to_string(&foo).unwrap();
                                result = Some(val);
                            }
                            _ => {}
                        }
                    }
                }
                Ok(result)
            }
            Err(_) => Ok(None),
        }
    }
    [dependencies]
    wapc-guest = "0.4.0"
    serde_derive = "1.0.115"
    serde_bytes = "0.11.5"
    rmp-serde = "0.14.4"
    lazy_static = "1.4.0"
    serde = { version = "1.0", features = ["derive"] }
    serde_json = "1.0.79"
    solana-sdk = "1.10.11"
    bincode = "1.3.3"
    bs58 = "0.4.0"
    hex = "0.4.3"
    base64 = "0.13.0"
    borsh = "0.9.3"
    solana-sdk in the depenency with wasm-bindgen in the tree
    however, as you can see from my code, I'm not using it
    Jarrod Overson
    @jsoverson
    @PirosB3 I'll have to look into those libraries. Is this in a public repo I can clone?
    Is this going to run on the browser or server?
    Daniel Pyrathon
    @PirosB3
    I'll be running this on the server\
    I'm basically trying to create some kind of backend plugin system where untrusted third parties can provide me a WASM module that adheres to a certain interface, and I can run classify certain data that I have through their WASM module
    Jarrod Overson
    @jsoverson
    If it's the server side then try compiling as wasm32-wasi.
    Daniel Pyrathon
    @PirosB3
    So, I thought about that, I am scared about wasi because I cannot guarantee the binary won't use networking or disk
    I want to really isolate/sandbox that WASM file from the rest of my app
    fawadasaurus
    @fawadasaurus
    you can use our framework which will run wasm as a GRPC service
    15 replies
    Jarrod Overson
    @jsoverson
    Yeah I get it. You can still restrain the wasm and limit what it can access.
    Daniel Pyrathon
    @PirosB3

    Yeah I get it. You can still restrain the wasm and limit what it can access.

    How can I do that? I would need to then bundle the WASI into a Docker container of some sort

    Jarrod Overson
    @jsoverson
    @PirosB3 we're in the middle of rebranding it "wasmflow" and opening it within the next month or so.
    Daniel Pyrathon
    @PirosB3
    Is Vino going to be wasmflow?
    fawadasaurus
    @fawadasaurus
    yes
    Daniel Pyrathon
    @PirosB3
    That's pretty cool
    Jarrod Overson
    @jsoverson
    @PirosB3 I can't vouch for the state of the public links. We cut out most direct references while we drop the vino name. We can get you access as soon as we finish the renaming work. If you're working with waPC then you're already working with the first part of the open source work.
    Daniel Pyrathon
    @PirosB3
    I see there being many use cases for wasmflow - especially for APIs that don't store state but rather perform business logic (like in my case - pure functions). In these cases, the API could ideally just run on your host as a wasm module.
    2 replies
    Jarrod Overson
    @jsoverson
    I have a lot of waPC changes in a private branch that add async support to rust. It's on hold because I want it to have consistent codegen across languages.
    Also, widl is being moved out and into apexlang as a generic codegen idl.
    So long story short, yeah wapc is maintained.
    Daniel Pyrathon
    @PirosB3
    If I was to use wasi, can I switch off certain features like networking?
    Jarrod Overson
    @jsoverson
    @PirosB3 you cant do much in the way of networking now as it is, and you can limit FS access to certain directories (or none). You can also pass through Stdio or isolate it, and can expose only a subset of environment variables.
    Phil Kedy
    @pkedy
    @jsoverson I made good progress on the "model" layer of @apexlang/core. I transitioned all my projects codegen to use it and its all working as expected so far. I had utilities in place before to handle dealing with all the type translation, but those certainly became simpler.
    Jarrod Overson
    @jsoverson
    @pkedy awesome
    David Schneider
    @dvob

    Hi everyone

    I try to call some code in a WASM module from Go and asked my self what are good ways to pass data between host and module. I found waPC and now I try to understand how it works under the hood.

    Would it be possible to pass an ID with __guest_call(id, op_len, msg_len) and __guest_request(id, op_ptr, op_len) to do multiple function calls concurrently with one instance?
    Or are the instance types of Wasmtime, Wasmer, and Wazero not safe for concurrent use? I'm not sure about this because I couldn't find something concerning concurrency in the documentation of Wasmtime, Wasmer and Wazero.

    Phil Kedy
    @pkedy
    Hi @dvob, great questions. Regarding concurrency, there is a WASM spec for threading/atomics but to my knowledge it is not ratified and certainly not supported by the non-wasmtime runtimes. As a work around, the Go and Rust host libraries let you create a poll of modules for concurrent usage. However, the invocations are still synchronous. Some of us have been chatting about an alternate invocation path that supports multiplexed async and streaming operations. Sounds like that what you might be interested in longer term. Is that why you are asking about an ID param?
    David Schneider
    @dvob

    @pkedy thank you for the response. Yes, that is why I'm asking. If I understand correctly, currently waPC relies on the fact that one instance is only being used by one call otherwise one would not know which initial __guest_call belongs to which __guest_request call. That's why I thought of the IDs.
    But even with the IDs it would only work if calls to the underlying instance implementations of Wasmtime, Wasmer and Wazero are safe for concurrent use which I'm not sure about. Do you know this?

    How would the alternate invocation path approximately look like you were chatting about?

    Phil Kedy
    @pkedy
    @dvob You are correct about __guest_call and __guest_request functions being part a "one at a time" call flow. Because we currently only have a single thread per module instance, something like a multiplexed reactive streaming model would need to be put in place. This type programming model is a bit trickier (e.g. no closures in AssemblyScript), but doable. We will likely put thoughts to paper on the protocol soon. The synchronous calls that exist probably solve the majority of WASM use cases (e.g. user-define functions), but there is certainly a need to async and streams. The challenge will be making it as developer-friendly as possible in all languages (not just Rust).