These are chat archives for rust-lang/rust

29th
Nov 2018
octave99
@octave99
Nov 29 2018 06:20 UTC
On similar lines, I have one code with dyn trait dispatch which works as below:
trait Animal {
    fn name(&self) -> &str;
}

struct Cat;
impl Animal for Cat {
    fn name(&self) -> &str {
        "cat"
    }
}

struct Dog;
impl Animal for Dog {
    fn name(&self) -> &str {
        "dog"
    }
}

struct Species {
    list: Vec<Box<Animal>>
}

impl Species {
    fn new() -> Self {
        Species { list: Vec::new() }
    }
    fn add(&mut self, animal: Box<dyn Animal>) {
        self.list.push(animal)
    }
}

fn main(){
    let mut s = Species::new();
    let d = Dog;
    let c = Cat;
    s.add(Box::new(d));
    s.add(Box::new(c));
}
But If I need to have serde serialization for all the structs, it throws an error since it will not be object safe. How do I achieve get a vector or trait objects and serialize the struct with vector at the same time, on same code pattern as above?
Below is the sample, implementation:
#[macro_use]
extern crate serde_derive;

trait Animal where Self: serde::Serialize {
    fn name(&self) -> &str;
}

#[derive(Serialize)]
struct Cat;
impl Animal for Cat {
    fn name(&self) -> &str {
        "cat"
    }
}

#[derive(Serialize)]
struct Dog;
impl Animal for Dog {
    fn name(&self) -> &str {
        "dog"
    }
}

#[derive(Serialize)]
struct Species {
    list: Vec<Box<Animal>>
}

impl Species {
    fn new() -> Self {
        Species { list: Vec::new() }
    }
    fn add(&mut self, animal: Box<dyn Animal>) {
        self.list.push(animal)
    }
}

fn main(){
    let mut s = Species::new();
    let d = Dog;
    let c = Cat;
    s.add(Box::new(d));
    s.add(Box::new(c));
}
Above is not object safe, how do I achieve both Vec<Box<Animal>> & Serialize in the Species struct
Tim Robinson
@1tgr
Nov 29 2018 06:57 UTC
Serialize is not object safe, but you could look at https://github.com/dtolnay/erased-serde
Normally I would do this with an enum Animal
Serde automates the writing of name: "cat" or name: "dog" depending on the variant
octave99
@octave99
Nov 29 2018 08:09 UTC
Could you share some sample code on enum?
octave99
@octave99
Nov 29 2018 08:20 UTC
@1tgr Did you mean something like this:
#[macro_use]
extern crate serde_derive;

#[derive(Serialize)]
enum Animal {
  Cat,
  Dog,
}

impl Animal {
  fn name(&self) -> &str {
    match self {
      Animal::Dog => "dog",
      Animal::Cat => "cat",
    }
  }
}

#[derive(Serialize)]
struct Species {
  list: Vec<Animal>,
}

impl Species {
  fn new() -> Self {
    Species { list: Vec::new() }
  }
  fn add(&mut self, animal: Animal) {
    self.list.push(animal)
  }
}

fn main() {
  let mut s = Species::new();
  let d = Animal::Dog;
  let c = Animal::Cat;
  s.add(d);
  s.add(c);
}
Tim Robinson
@1tgr
Nov 29 2018 08:35 UTC
This is a good explanation, with examples https://serde.rs/enum-representations.html
Serde knows the difference between a “cat” and a “dog”, you just have to tell it where to put that string in the data
Actually it will serialize this example as the string "Cat" or the string "Dog", because neither variant has any fields
But you can tell it to use internal tagging, which gives you {"name":"Cat"} and {"name":"Dog"}
And you can tell it to camelCase the Cat and Dog tags to give {"name":"cat"} and {"name":"dog"}
octave99
@octave99
Nov 29 2018 08:48 UTC
Sure. Enum worked in above case, where the items were simpler. I wonder how do we group complex structures in an enum. Consider Dog and Cat as structures with dozens of fields. How to group those in a vector and then serialize that group? erased-serde as you highlighted above is the only option or there is a way with enum as well?
Tim Robinson
@1tgr
Nov 29 2018 08:50 UTC
How do you want the output to look?
octave99
@octave99
Nov 29 2018 08:50 UTC
Or do we map structs into enum as types, something like this:
#[derive(Serialize)]
pub enum Animal {
  Cat { id: i32, name: String },
  Dog { id: i32, name: String, kind: i32 },
}
Tim Robinson
@1tgr
Nov 29 2018 08:51 UTC
Sure, you can serialize variants with fields inside
And the Cat and Dog variants can have different fields
octave99
@octave99
Nov 29 2018 08:51 UTC
Is this widely acceptable pattern?
Tim Robinson
@1tgr
Nov 29 2018 08:51 UTC
Yes
octave99
@octave99
Nov 29 2018 08:51 UTC
Cool. Let me try some experiments with this one. Hope this should solve my challenge
Tim Robinson
@1tgr
Nov 29 2018 08:52 UTC
There’s no established format for tagged unions on JSON, because most languages are OO and don’t have tagged unions
Which is why enums in Serde are so configurable
octave99
@octave99
Nov 29 2018 08:53 UTC
Sure. Thanks for sharing the inputs
octave99
@octave99
Nov 29 2018 09:03 UTC
@1tgr Does Enum items require preamble in memory? I see additional 4 bytes
I can strip those but is it always 4 bytes? Is it documented somewhere?
pub enum Animal {
  Dog { id: u8, key: u8 },
  Cat { id: u8, key: u8 },
}
Vector of Animal with both Dog and Cat looks like: [0, 0, 0, 0, 0, 0, 0, 2, 0, 0, 0, 0, 11, 21, 0, 0, 0, 1, 31, 41]
First 8 bytes are due to vector, but then 4 bytes before each u8,u8 item looks enum preamble, may be due to tagged union
element counter appears like
@polygdev how do u messure?
octave99
@octave99
Nov 29 2018 09:12 UTC
bincode::config()
    .big_endian()
    .serialize(&s)
    .and_then(|v| Ok(v));
this doesnt generate any output
octave99
@octave99
Nov 29 2018 09:14 UTC
Here is the complete code:
#[macro_use]
extern crate serde_derive;
extern crate bincode;
extern crate serde;

#[derive(Debug, Serialize)]
pub enum Animal {
  Dog { id: u8, key: u8 },
  Cat { id: u8, key: u8 },
}

#[derive(Debug, Serialize)]
struct Species {
  list: Vec<Animal>,
}

impl Species {
  fn new() -> Self {
    Species { list: Vec::new() }
  }
  fn add(&mut self, animal: Animal) {
    self.list.push(animal)
  }
}

fn main() {
  let mut s = Species::new();
  let d = Animal::Dog { id: 11, key: 21 };
  let c = Animal::Cat { id: 31, key: 41 };
  s.add(d);
  s.add(c);
  let bc = bincode::config()
    .big_endian()
    .serialize(&s)
    .and_then(|v| Ok(v));
  print!("{:?}", bc);
}
@polygdev check if release build doesn't create different results
octave99
@octave99
Nov 29 2018 09:15 UTC
@trsh Its solved with #[serde(untagged)] on the enum
Ok
octave99
@octave99
Nov 29 2018 09:15 UTC
So I had to untag the enum so that it doesn't preserve the extra memory space
@polygdev whats yout project about?
octave99
@octave99
Nov 29 2018 09:17 UTC
Protocol parser
There are many existing projects out there, but just been trying to figure out most efficient method by writing from scratch which gives good learning as well
Ok, so every byte counts? :)
octave99
@octave99
Nov 29 2018 09:18 UTC
yeah, true that. In fact I am aiming for zero-copy but everyday I am finding something new to learn
Cool
octave99
@octave99
Nov 29 2018 09:19 UTC
Already have zero-copy parsing with nom
Now writing packet crafting part for which need to insert parsed protocols into vector for easy iteration
Its fun :)
Anybody here has tried amethyst as well piston game engine, and can share some insights, versus versus? :)))
octave99
@octave99
Nov 29 2018 09:46 UTC
@1tgr erased_serde solves it all, like so:
#[macro_use]
extern crate serde_derive;
extern crate bincode;
extern crate serde;
extern crate serde_json;
#[macro_use]
extern crate erased_serde;

#[derive(Serialize)]
struct Dog {
  id: u8,
  key: u8,
}
#[derive(Serialize)]
struct Cat {
  id: u8,
  key: u8,
}

trait Animal: erased_serde::Serialize {}
impl Animal for Dog {}
impl Animal for Cat {}
serialize_trait_object!(Animal);

#[derive(Serialize)]
struct Species {
  list: Vec<Box<Animal>>,
}

impl Species {
  fn new() -> Self {
    Species { list: Vec::new() }
  }
  fn add(&mut self, animal: Box<dyn Animal>) {
    self.list.push(animal)
  }
}

fn main() {
  let d = Dog { id: 11, key: 21 };
  let c = Cat { id: 31, key: 41 };
  let mut s = Species::new();
  s.add(Box::new(d));
  s.add(Box::new(c));

  let bc = bincode::config()
    .big_endian()
    .serialize(&s)
    .and_then(|v| Ok(v));
  print!("{:?}", bc);
}
@1tgr Appreciate your inputs.
Tim Robinson
@1tgr
Nov 29 2018 09:51 UTC
@polygdev non-Rust (and non-coding) work has taken over for now, sorry!
(I'm hiring btw, in London or Hong Kong... not 100% Rust tho)
octave99
@octave99
Nov 29 2018 09:52 UTC
@1tgr I was thanking you for your input on erased-serde. It worked well
That was the working code, appreciating you in the end :). All the best for your future work and hirings
Tim Robinson
@1tgr
Nov 29 2018 09:53 UTC

That was the working code

Haha good, I read your message the other way at first

tsoernes
@tsoernes
Nov 29 2018 10:32 UTC
you have a big big string. you have 2 lists (same lenght) of words. efficiently replace every occurence in the string of every word in the first list with the corresponding (same index) of the second
how to do that efficiently?
Tim Robinson
@1tgr
Nov 29 2018 10:34 UTC
Search for some predefined (short) list of strings in an unknown (long) text
David O'Connor
@David-OConnor
Nov 29 2018 16:48 UTC

Hey dudes: Macro question. Do you know if there's a way to make a declarative macro that can take arguments of different types (0 or 1 of each), and parse them based on the type received? I'm running into issues with not being able to evaluate the type in the macro. I assume this is possible with proc macros, but haven't been able to figure out how to make those.

Eg:
mymac![A, B] and mymac![A, B, D] would both be valid calls, where A, B, C (not pictured) and D are all diff types, and the macro would do diff things depending on which types it received.

David O'Connor
@David-OConnor
Nov 29 2018 17:13 UTC
Example pseudocode:
macro_rules! example {
    { $($expr:expr),* } => {
        {
            let mut result = 0;
            $ (
                match type_of($expr) {
                    A => result += 1,
                    B => result += 2,
                    C => result += 4,
                    D => result += 8
                };
            )*
            result
        }
    };
}
Tim Robinson
@1tgr
Nov 29 2018 17:26 UTC
No, macros can't see types directly, because they are run by the compiler at a much lower level
However you could see how far you can go with things that can see types, such as traits
On that example you could have trait HowMuchDoIIncrementBy, with different implementations for A, B, C and D
But I get that this is probably a simplifed example
But in any case - Rust likes to do type-specific compile time behaviour via traits
David O'Connor
@David-OConnor
Nov 29 2018 17:31 UTC
I think you've answered my Q entirely; will report back once sorted! Another way of describing this: Something like keyword args in python funcs, but utilizing Rust's type system to avoid having to specify what kwarg you're passing; eg infer it by type
Tim Robinson
@1tgr
Nov 29 2018 17:34 UTC
Oh evil
Assuming fn f(x, y, z, ...) takes some number of A, B, C and D parameters, and at most one of each
David O'Connor
@David-OConnor
Nov 29 2018 19:15 UTC

@1tgr I read up on/set up some traits, and my naive approach is running into errors where the different match branches appear all to be evaluating, regardless of which branch is taken; this causes mismatched types errors:

macro_rules! mymacro {
    ( $($expr:expr),* ) => {
        {
            let mut result = Combined {a: A{val:0}, b: B{val2:0}};

            $ (
                match $expr.kind() {
                    Kind::A => result.a = $expr,
                    Kind::B => result.b = $expr,
                };
            )*
            result
        }
    };
}

Playground

Tim Robinson
@1tgr
Nov 29 2018 19:43 UTC
You could define a trait where types A and B are responsible for setting the relevant field on Combined
trait Update<T> {
    fn update(self, combined: &mut T);
}

impl Update<Combined> for A {
    fn update(self, combined: &mut Combined) {
        combined.a = self;
    }
}
Then the middle of your macro is $($expr.update(&mut result);)
David O'Connor
@David-OConnor
Nov 29 2018 20:35 UTC

@1tgr Bro. You've solved this entirely; I thought this was going to take proc_macros (Which AFAIK there is no guide or tutorial for). Love that soln.

Updated playground