Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Jan 31 2019 22:45
    eolivelli commented #4914
  • Jan 31 2019 22:07
    samsartor starred google/flatbuffers
  • Jan 31 2019 21:28
    marang starred google/flatbuffers
  • Jan 31 2019 20:51
    thyrlian starred google/flatbuffers
  • Jan 31 2019 19:19
    harshshah903 commented #5144
  • Jan 31 2019 19:19
    harshshah903 commented #5144
  • Jan 31 2019 18:56
    aardappel commented #4914
  • Jan 31 2019 18:54
    aardappel commented #5144
  • Jan 31 2019 18:51
    aardappel commented #5141
  • Jan 31 2019 18:51
    aardappel commented #5145
  • Jan 31 2019 18:51
    krojew commented #5142
  • Jan 31 2019 18:49
    krojew commented #5142
  • Jan 31 2019 18:48
    gabyx edited #5142
  • Jan 31 2019 18:48
    gabyx edited #5142
  • Jan 31 2019 18:47
    gabyx commented #5142
  • Jan 31 2019 18:47
    aardappel commented #5002
  • Jan 31 2019 18:43
    gabyx commented #5142
  • Jan 31 2019 18:43
    krojew commented #5142
  • Jan 31 2019 18:43
    aardappel commented #5143
  • Jan 31 2019 18:42
    gabyx commented #5142
MikkelFJ
@mikkelfj
so much for zero copy.
Lijie.Jiang
@lijie-jiang
If the reason is alignment issue, I should prepared a well aligned and copy the flat data to it, right?
MikkelFJ
@mikkelfj
yes, malloc will do that.
Lijie.Jiang
@lijie-jiang

table LidarSickInfo {
Time:ulong (id: 0);
object_number:uint (id: 1);
object_type:uint (id: 2);
detection_type:uint (id: 3);
object_dist_long:float (id: 4);
object_dist_lat:float (id: 5);
object_length:float (id: 6);
object_width:float (id: 7);
object_height:float (id: 8);
object_vrel_long:float (id: 10);
object_vrel_lat:float (id: 11);
object_min_long:float (id: 12);
}

table SenseResultInfo {
back_data:LidarSickInfo (id: 0);
lidar_front_data:Lidardis (id: 1);
steer_para:SteerPara (id: 2);
lidar_back_judge_result:bool = false (id: 3);
}

the root table is SenseResult, and make instance with c++ unique pointer. So it was filled with lots of zero.
MikkelFJ
@mikkelfj
Flatbuffers can have higher alignment requirements than malloc provides, but it is rarely the case (SIMD operations, GPU buffers …)
Just try copy the buffer to malloc’ed memory before unpack and see what happens. If it works, figure out if zmq can provide aligned memory on its own.
Lijie.Jiang
@lijie-jiang
Great suggestion. I will try it . appreciated it.
MikkelFJ
@mikkelfj
let me know if it works out
Lijie.Jiang
@lijie-jiang
Great catch, Mikkel! It work out, with the aligned memory.
MikkelFJ
@mikkelfj
good to know
Lijie.Jiang
@lijie-jiang
Much appreciated it!
MikkelFJ
@mikkelfj
you’re welcom
e
Lijie.Jiang
@lijie-jiang
I will figure out if the zmq could support align memory for message transfer.
MikkelFJ
@mikkelfj
I think you need to have zmq malloc memory, but it is probably better to preallocate a buffer and copy into that. zmq provides data as the they appear in low-level IO buffers. zeromq/libzmq#3448
Lijie.Jiang
@lijie-jiang
Indeed, there would introduced an extra memory copy if I pursue a aligned buffer.
Jay Narale
@carbylamine
Is there any other way of performing mutations using the Go API except creating the buffer again
and using mutation buffers?
MikkelFJ
@mikkelfj
You cannot mutate a buffer without mutations … but you can avoid rewriting the buffer if you have a scalar, or an array of scalars where the new length is not longer, and the same for strings. How to do that depends on the language interface.
However, you MUST trust the buffer before doing this because a malicious buffer could overlap seemingly harmless data with other buffer structures that then goes out of bounds even if the buffer has been verified.
Jay Narale
@carbylamine
I mean for mutating complex tables etc
MikkelFJ
@mikkelfj
You can only update existing table fields or truncated the length of existing arrays. Of course you can do more if there is somehow space in the buffer, but that amounts to writing a new buffer essentially.
I know see you mention Go, I’m not sure what Go permits, but if you can get the buffer offset to a field value, you can probably update it on the raw array, if you are on a little endian host.
Jay Narale
@carbylamine
But the C++ API supports mutations right? So I thought maybe there must be some tweak in go
MikkelFJ
@mikkelfj
See above, posted same time as your comment.
It is likely that you can get that offset if you look at Gos implementation, but it is probably not guaranteed to work for future releases.
I mean look at the generated code in detail.
More generally, I think it is better avoided to mutate buffers unless you have a specific highperformance case where you use mutations as a template for sending data at gigabit rates.
I do wish Go would make it easier to clone a part of a buffer, though.
Jay Narale
@carbylamine
Yes I think we can get The offset in golang, will try it. So you would suggest creating a new encoding again?
MikkelFJ
@mikkelfj
Yes. If your buffer is very large, and the changes small, better keep the changes in a separate buffer, and if the buffer is smallish, just rewrite it.
Jay Narale
@carbylamine
Okay will try, thanks a lot!
MikkelFJ
@mikkelfj
You are welcome
SSorbello
@SSorbello

Good Morning everybody, i just started to utilize flatbuffers on C# , inside a table i defined:
Value:[ubyte] (flexbuffer);

there is C# support for the flexbuffer in the original repo or i should rely on https://github.com/mzaks/FlexBuffers-CSharp ?

MikkelFJ
@mikkelfj
I don’t know, but it is rare that someone says goodmorning when it is actually morning where I live, so good morning to you too. My guess is you should use Max’ implementation you reference.
SSorbello
@SSorbello
Thanks, i will continue on this way.
Maxim Zaks
@mzaks
@SSorbello you are welcome to use my implementation. Let me know if you have any issues with it. AFAIK there is no C# FlexBuffers support on Googles repo and there is probably no extra support in flatc for C# code gen.
cyberquarks
@cyberquarks
Hi, can you use FlatBuffers to upload an image or short video?
MikkelFJ
@mikkelfj
@cyberquarks yes, you can have arbitrary binary data in flatbuffers, as long as it is below 2GB per buffer. FlatBuffers is perhaps best suited for containing metadata such as video title, copyright, image encoding etc., but raw frames can also be encoded. It is not suited for actually playing back video, at least not without a new dedicated video format.
Jon Lederman
@jon2718

Hi,

I am trying to use FlatBuffers in Swift.
After compiling the Monstes example with flatc --swift monsters.fbs, I then add the generated file into my Xcode project as a swift file.

When trying to build I get many errors of the type:

Cannot use mutating member on immutable value: 'fbb' is a 'let' constant, where fbb is a FlatBuffersBuilderType

I've tried the cocoapods as well as building from source. I am on Flatc 1.12.0
Not sure if the code is still being updated and whether Swift is even supported.

Any help would be appreciated.

Thank you.

MikkelFJ
@mikkelfj
@mzaks is the one you should ask, I can't help you here. But generally, building flatbuffers and reading flatbuffers are two separate things, so perhaps you are mixing types from the read api with the types from builder api.
Jon Lederman
@jon2718
perhaps I am making a mistake in my use of it. the docs weren't 100% clear
can you explain the diff between read api and builder api types
I just used flatc --swift monsters.fbs to generate the code and imported that into my Xcode project for ios
MikkelFJ
@mikkelfj
I’m not working with swift so I cannot be specific. The difference between build and read is that the buffers is first constructed once using various types that usually holds temporary data references, whereas the read types are designed to efficiently reference data in the constructed buffer.
Wouter van Oortmerssen
@aardappel
@jon2718 open an issue on github, and tag @mustiikhalil who is the swift maintainer
Jon Lederman
@jon2718
Ok. Can u provide instructions
Jon Lederman
@jon2718
Flatc seems to generate wrong code b/c it is trying to generate a struct for FlatBuffersBuilder and pass that to static method, which cannot be marked mutating. Something appears quite wrong.
cyberquarks
@cyberquarks
@mikkelfj thanks