Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Jan 31 22:45
    eolivelli commented #4914
  • Jan 31 22:07
    samsartor starred google/flatbuffers
  • Jan 31 21:28
    marang starred google/flatbuffers
  • Jan 31 20:51
    thyrlian starred google/flatbuffers
  • Jan 31 19:19
    harshshah903 commented #5144
  • Jan 31 19:19
    harshshah903 commented #5144
  • Jan 31 18:56
    aardappel commented #4914
  • Jan 31 18:54
    aardappel commented #5144
  • Jan 31 18:51
    aardappel commented #5141
  • Jan 31 18:51
    aardappel commented #5145
  • Jan 31 18:51
    krojew commented #5142
  • Jan 31 18:49
    krojew commented #5142
  • Jan 31 18:48
    gabyx edited #5142
  • Jan 31 18:48
    gabyx edited #5142
  • Jan 31 18:47
    gabyx commented #5142
  • Jan 31 18:47
    aardappel commented #5002
  • Jan 31 18:43
    gabyx commented #5142
  • Jan 31 18:43
    krojew commented #5142
  • Jan 31 18:43
    aardappel commented #5143
  • Jan 31 18:42
    gabyx commented #5142
Maxim Zaks
@mzaks
How do you want to manipulate FlexBuffers in memory?
Björn Harrtell
@bjornharrtell
Two cents: If size is more important than speed protobuf may be a better alternative.. or perhaps FBE final.
Paulo Pinheiro
@paulovap
I meant to say, usually, standard JSON data, on mobile, has the flow:
HTTP Response -> JSON Text -> JSON Adapter -> Generated Object
I want to investigate if moving to this will have any benefit:
HTTP Response -> JSON Text -> FlexBuffers Converter/Adapter -> Generated Object
But probably not worth it. Better to completely switch to a schema-based FlatBuffers
Maxim Zaks
@mzaks
Or as @bjornharrtell mentioned to protobuff if memory is the main concern. ProtoBuff is designed to minimise the payload, where FlatBuffers is designed to eliminate the read overhead.
MikkelFJ
@mikkelfj
FlatBuffers tend to compress better than JSON for small buffers but JSON compresses better than FlatBuffers for large buffers.
flatcc generates JSON parsers that convert to flatbuffers and printers the opposite direction. These are fairly fast.
Wouter van Oortmerssen
@aardappel
yes, relative offsets compress badly since they tend to be different every time.. but like @mzaks says, this data is wasteful in all sorts of ways that you wouldn't have if you made it for FlexBuffers natively. In addition to @mzaks points, there's floats as strings "36.166670" and unlike FlatBuffers, where it can know its supposed to be string, here it doesn't know.. so it is just storing a ton of strings
also, is this created with key and value sharing on? probably not..
Wouter van Oortmerssen
@aardappel
and the obvious: this data is pretty regular so would work well with FlatBuffers
I meant: where it can know its supposed to be a float
Björn Harrtell
@bjornharrtell
Have it been considered to add flatbuffers to https://github.com/google/oss-fuzz?
MikkelFJ
@mikkelfj
That sound like a huge amount of work, although it is awesome. Curl has an entire project dedicated to this: https://github.com/curl/curl-fuzzer.git
If you want to add flatcc, I'd give support, but I can't imagine taking this on on my own.
Wouter van Oortmerssen
@aardappel
@bjornharrtell that would be very cool, and we already have some fuzzer set up (to run manually) in tests/fuzzer.. but yeah, would need someone to hook them up to this infra
Diogo Castro
@dcastro
Just released my Haskell port of flatbuffers :D https://twitter.com/dfacastro/status/1175806400603312129
feedback is welcome!
MikkelFJ
@mikkelfj
cool!
Wouter van Oortmerssen
@aardappel
nice! retweeted :)
Diogo Castro
@dcastro
thanks for the support!
Wouter van Oortmerssen
@aardappel
no, thank YOU for the support :)
Neonit
@Neonit
Hello, I'm having a problem with structs in unions. I posted it on StackOverflow and would be very thankful if someone could have a look at it: https://stackoverflow.com/q/58112690/2477364
MikkelFJ
@mikkelfj
Can’t say what is going on, but one comment you made in stackoverflow is incorrect: flatbuffers does NOT store data compact and unaligned. It stores data aligned relative to buffer start, and always in little endian.
MikkelFJ
@mikkelfj
Also, to the comment strange, I'd think they'd like to have option to store data flipped. (e.g. to send it over to target with different\unknown endianness) That'd be very useful feature, actually one which is required in project at my work.: This is possible with flatcc for C, but then it is flatbuffers-be which is not an interoperable flatbuffers format anymore.
Neonit
@Neonit
@mikkelfj I'm sorry. I added a correction comment. Do you think I should file a bug report for this? Not sure if it should work that way or not. Couldn't find an example or test doing this either yet.
MikkelFJ
@mikkelfj
Just wait for Wouter to reply. I’m not that knowledgable about the C++ interface, only about the format itself.
Wouter van Oortmerssen
@aardappel
@Neonit I responded on stack overflow
Neonit
@Neonit
@aardappel Thank you very much.
MiheevN
@MiheevN
I Created in ProtoBuf
message MiheevMessage {
int32 B = 1;
string S = 2;
}
and flatbuffers
table Miheev {
Age: int;
Name: string;
}
The first empty Protobuf weighs 0bit,
the second flatbuffers empty weighs 28bit = 8 0 8 0 0 0 4 0 8 0 0 0 4 0 0 0 0 0 0 0 0 0 0 0 0
This is normal?
My goal is minimum size and maximum speed. I create a game.
MikkelFJ
@mikkelfj
Protobuf creates smaller buffers than FlatBuffers. FlatBuffers are faster than Protobuf. If you need to read large complex game data, FlatBuffers is best. If you need to send small messages, Protobuf may be best, but the data must be parsed into a structure before it can be read.
MiheevN
@MiheevN
OK. I study further, I realized that FlexBuffers is what I need. And dynamic, and brief, and faster Json.
But when assembling I get this error ...
1>FunctionLibrary.cpp.obj : error LNK2001: unresolved external symbol "private: static class flatbuffers::ClassicLocale flatbuffers::ClassicLocale::instance_" (?instance_@ClassicLocale@flatbuffers@@0V12@A)
1>Miheev.dll : fatal error LNK1120: 1 unresolved externals
MikkelFJ
@mikkelfj
Sorry, can’t help you with that. FlexBuffers is dynamic and more flexible as it requires no schema, but that can come back to bite you later. It is likely the slowest option. While it may be faster than JSON, that is not always the the case: flatcc (for C) generates a very fast JSON parser and printer from a FlatBuffers schema.
MiheevN
@MiheevN
It threw an error on this class on the 216 line.
https://github.com/google/flatbuffers/blob/32f47ad24727c7af17996aed5e48840e9139b2f5/include/flatbuffers/util.h#L207
I generally forbade him to compile. And it seems FlexBuffers works
Does anyone know why this class is needed? I can not find who refers to it.
Wouter van Oortmerssen
@aardappel
@MiheevN can you open an issue on github with your compile error, and mention @vglavnyy on it, he likely knows what's going on.
Björn Harrtell
@bjornharrtell
Isn't it time for a release soon? :)
Wouter van Oortmerssen
@aardappel
I guess it kinda is :)
let me see whats all still in flux
HarrievG
@HarrievG

Hi! I love flatbuffers, to start with.
But, i also love VCPKG, and before creating my own portfile to keep flatbuffers at head / recent revisions i want to check
I am a heavy UNION user, and i am in need of https://github.com/google/flatbuffers/commit/f9277e691dd10e42a999bba683db5df7d49226d6#diff-8f7f862c058e6e45a1abea4d04b48691 (Fixed GenerateText not handling vectors of unions. )

So my question is: will a new version be released soon? (creating a local portfile is not THAT big of a deal, but maintaining it is)
Thx!

Wouter van Oortmerssen
@aardappel
I don't make promises on when a new release will happen.. other than saying that I'm aware its overdue :)
does vcpkg not allow to pin to commits?
HarrievG
@HarrievG

If you need any help in fixing bugs to get to release , i am happy to pitch in if you want.

About pinning commits, Im not sure what you mean.
But looking at the history of https://github.com/microsoft/vcpkg/commits/5ef8bb9abc0f6a9551ac48006fc04b9d91cc5716/ports/flatbuffers
I dont see non release versions.
Having said that, i think updating the https://github.com/microsoft/vcpkg/blob/5ef8bb9abc0f6a9551ac48006fc04b9d91cc5716/ports/flatbuffers/portfile.cmake with a later commit should work?

But i dont think the port on vcpkg should host non release sourcetree versions ;) So i will create my own portfile.
Thx for the quick reply.

Wouter van Oortmerssen
@aardappel
There's no particular bug list for any release, we simply need some of the PR action to die down, and I have to coordinate with internal (Google) teams
that said, help is always very welcome in this project.. have a look at open issues, PRs and the "projects" tab on github to get an idea what is going on
I meant, being able to specify a commit as opposed to a version tag. versions simply point to a commit and are not in any way special
yup that would likely work
Katrina Ward
@Webneko675

@aardappel

If we have additional questions about flatbuffers, is it better to ask here or on your github page? You've been immensely helpful and I had another question.

Wouter van Oortmerssen
@aardappel
Quick / easy question here.. question that require code or explanation, on github :)
Björn Harrtell
@bjornharrtell
In a perhaps wierd edge case and with sanitizer tooling I can provoke this one : flatbuffers/base.h:386:23: runtime error: unsigned integer overflow: 18446744073709551615 + 1 cannot be represented in type 'unsigned long' - I'm not at all sure but could it be another place which should have __supress_ubsan__("unsigned-integer-overflow")?
Wouter van Oortmerssen
@aardappel
very likely
can you open an issue on github and mention @vglavnyy ?