by

Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Jan 31 2019 22:45
    eolivelli commented #4914
  • Jan 31 2019 22:07
    samsartor starred google/flatbuffers
  • Jan 31 2019 21:28
    marang starred google/flatbuffers
  • Jan 31 2019 20:51
    thyrlian starred google/flatbuffers
  • Jan 31 2019 19:19
    harshshah903 commented #5144
  • Jan 31 2019 19:19
    harshshah903 commented #5144
  • Jan 31 2019 18:56
    aardappel commented #4914
  • Jan 31 2019 18:54
    aardappel commented #5144
  • Jan 31 2019 18:51
    aardappel commented #5141
  • Jan 31 2019 18:51
    aardappel commented #5145
  • Jan 31 2019 18:51
    krojew commented #5142
  • Jan 31 2019 18:49
    krojew commented #5142
  • Jan 31 2019 18:48
    gabyx edited #5142
  • Jan 31 2019 18:48
    gabyx edited #5142
  • Jan 31 2019 18:47
    gabyx commented #5142
  • Jan 31 2019 18:47
    aardappel commented #5002
  • Jan 31 2019 18:43
    gabyx commented #5142
  • Jan 31 2019 18:43
    krojew commented #5142
  • Jan 31 2019 18:43
    aardappel commented #5143
  • Jan 31 2019 18:42
    gabyx commented #5142
MikkelFJ
@mikkelfj
I’m working on hiearchical state charts in SCXML. The action language can be anything, but there is a standard for Javascript case. Almost all test cases can be handled by a subset of Javascript which also happens to be an efficient expression language. So no prototype stuff etc. and only known classes with known members.
I did post some sample code here once - you saw it.
I also have a much more heavy duty GC for another project, but I avoid it for portability. That leaves some trade-offs. I can easily port to that when available. That GC has multiple generations with incremental page by page scans and is quite fast. I use it track distributed links between state machines or other actor objects.
Wouter van Oortmerssen
@aardappel
ahh ok
crazy :)
MikkelFJ
@mikkelfj
yes
MikkelFJ
@mikkelfj
I once had a convention of many thousands of dining philosphers seated a different tables to test concurrent concurrency and garbage collection. It was quite amazing to see it spit out a million chop stick passes without breaking a sweat.
Wouter van Oortmerssen
@aardappel
haha nice
classical test :)
Tsingson
@tsingson
@aardappel
this one ?
```
Namespace *Parser::UniqueNamespace(Namespace *ns) {
  for (auto it = namespaces_.begin(); it != namespaces_.end(); ++it) {
    if (ns->components == (*it)->components) {
      delete ns;
      return *it;
    }
  }
  namespaces_.push_back(ns);
  return ns;
}
this func clean up ?
Tsingson
@tsingson
memory leaks bug is fixed.
pavanbadugu
@pavanbadugu
can anyone help me on this
a,b are tables
c is a table in which a,b are fields
so i got the a flatbuffer data and b flatbuffer data which are loader seperately from file so now i need to construct a flatbuffer for c using .. is there any way just for copy raw bytes to flatbuffer and construct c table any api's for c++
MikkelFJ
@mikkelfj
No you cannot trivially do that in C++ as far as I know. You may be able to use the object API which is slower, but I am not familiar with that. For C, there is a clone method which can copy tables from other sources into a new buffer - it will do what you want, but it is still not simple block copy - each table field is copied recursively into the the new buffer.
pavanbadugu
@pavanbadugu
oh ok thanks
MikkelFJ
@mikkelfj
You can, however, store a and b as nested buffers if that is acceptable to you. That will allow storing each sub-buffer directly as a blob. It requires a schema change.
Normally I’d not recommend that, but it depends on the circumstances.
pavanbadugu
@pavanbadugu
yeah this was my alternative thanks for confirming i'll do this thanks @mikkelfj
Tsingson
@tsingson

In terms of the fbs specification, a table consists of two parts, a vtable , indicating all the fields in a table, and a binary array, containing all the actual field data. If you need to get a table as a field inside another table from a generated flatbuffer, just get the corresponding vtable and the actual data, then add a header to be a new flatbuffer.

In other words, it is theoretically possible to intercept the table fields from the generated flatbuffer and recreate a new flatbuffers.

In practice, there is no such API available. (If you are familiar with fbs, you can develop it yourself.)

pavanbadugu
@pavanbadugu
@tsingson will try that where can i find doc on how flatbuffer serelisation specification
Wouter van Oortmerssen
@aardappel
@tsingson and the table fields may point to more tables, strings and vectors, so this is not trivial
Maxim Zaks
@mzaks
Em .. not that simple @tsingson the values are stored together only if they are scalars (bool, int, uint, float). If properties in table are of type vector, string, union or other table, than there is a relative offset, which points to some region in the buffer
:)
Wouter van Oortmerssen
@aardappel
see, I typed a shorter response, so I got to go first :P
Maxim Zaks
@mzaks
Story of my life :)
pavanbadugu
@pavanbadugu
quick question when is latest grpc compatible code generation support for c++ happening
currently flatc generated code is supported till grpc 16.x
@aardappel @mzaks
Maxim Zaks
@mzaks
I have no idea, sorry.
MikkelFJ
@mikkelfj
@tsingson you can do this in flatcc for C - but it is a lot of work - you can bring your own vtable and you can get a pointer to a source table, but fields must still be processed recursively as Wouter says.

see, I typed a shorter response, so I got to go first :P

Missing a thumbs up here.

pavanbadugu
@pavanbadugu
quick question when is latest grpc compatible code generation support for c++ happening and javascript support
Wouter van Oortmerssen
@aardappel
@pavanbadugu some gRPC related fixes were merged just like.. a week ago. Are you working from latest?
pavanbadugu
@pavanbadugu
yes still facing couple of issues will push some example code to github and put it as a issue in repository please check it once @aardappel thank you
Wouter van Oortmerssen
@aardappel
Reminder, there is also a Discord FlatBuffers chat: discord.gg/6qgKs3R
Tsingson
@tsingson

@tsingson and the table fields may point to more tables, strings and vectors, so this is not trivial

yes. but, it's still can found the table field in a table. a table as a field , it's a vector.

Tsingson
@tsingson

@tsingson you can do this in flatcc for C - but it is a lot of work - you can bring your own vtable and you can get a pointer to a source table, but fields must still be processed recursively as Wouter says.

i can do it for go, i m not good for c / c++.

i m trying some job in fbs in go, make it's API clear and easy to use. at lease , in my business project.
Michael Ramos
@backnotprop

Hey all. Are there any examples of building nested flatbuffer in Javascript/Typescript.... I have an inter-service deployment working well with Golang...struggling on the client front specifically building the nested FB... using same approach as go, the JS lacks builder.createbytevector so have tried working a similar approach with asUint8Array ...

// 1. create nested FB with its own builder
... store finished buffer into asUint8Array

// 2. builder for root_table
... offset from generated table field createvector with nested FB asUint8Array
// 3. start root table
... add root table nested FB with above offset

the nested FB is not accessible on the server side when sent from the client....
maybe its my fetch request but I am sending raw bytes via fetch

Michael Ramos
@backnotprop
seems related: google/flatbuffers#4500
oh
that was it
asUInt8Array().slice(0) needed for nested and send data
William
@imWillX
Hello,
I am trying to get tables within a flatbuffer schema that is not a root type. Is this a feature?
MikkelFJ
@mikkelfj
It depends on the target language, but I believe most languages support having any table as the root table in the buffer, if that is what you mean. The root declaration in the schema is mostly for JSON parsing where the parser must know what table to parse as root. In FlatCC for C, you can also parse other tables as root, but there is more direct support for the declared root.
Konstantin Bläsi
@konstantinblaesi
I am trying to compare buffer/payload sizes of flatbuffers vs JSON, basically trying to see how much I save regarding wire transfer size. I came up with this code after reading some stackoverflow post on how to parse/serialize from/to JSON https://dpaste.org/BtXL According to that the ratio JSON / flatbuffer is 1.66237 . Does that make sense what I am doing ? I was expecting a bigger difference after seeing https://google.github.io/flatbuffers/flatbuffers_benchmarks.html, but what do I know :)
Wouter van Oortmerssen
@aardappel
@konstantinblaesi that really depends on your data.. the bulk of that data is two strings and a double. The strings are by themselves slightly larger in FlatBuffers, but you save not having to write the key or any of the formatting around it. The double is always 8 bytes in FlatBuffers, in your example it is sometimes shorter (0) and sometimes longer (58096.25652706856), and again you save the keys etc.
You could turn the double into a float, but thats not a big savings. The bigger one is to turn unit from a string into an enum, making it as small as 1 byte
Konstantin Bläsi
@konstantinblaesi
@aardappel thanks for the quick feedback and insights :)
MikkelFJ
@mikkelfj
If you have gzip compressed JSON, you can generally assume that JSON will be larger than GZIP compressed FlatBuffers by a factor 2 for smaller messages - say below 1K, but for messages above 100K GZIP compressed JSON will smaller than GZIP compressed flatbuffers by a factor 2. Alls this will vary on payload and schema. For uncompressed data, JSON tend to be large - I’m guessing a factor 4-10 for small messages. The reason is that JSON keywords take up space and flatbuffer pointers take up space. Keywords compress better when there are many of them.