Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • Dec 10 23:40

    GreyCat on master

    Fix whitespaces (compare)

  • Dec 10 23:38

    GreyCat on master

    utf8_string: fixed typos in doc (compare)

  • Dec 10 23:38
    GreyCat commented #649
  • Dec 10 21:28
    Travis kaitai-io/kaitai_struct (master) fixed (697)
  • Dec 10 21:21

    GreyCat on master

    Pulled https://github.com/kaita… (compare)

  • Dec 10 21:20

    GreyCat on master

    Fix enum generation for values … Merge pull request #186 from va… (compare)

  • Dec 10 21:20
    GreyCat closed #186
  • Dec 10 15:29
    Travis kaitai-io/kaitai_struct (master) broken (696)
  • Dec 10 15:26
    GreyCat commented #186
  • Dec 10 15:22

    GreyCat on master

    Pulled enum_long_* tests (compare)

  • Dec 10 15:22

    GreyCat on master

    Tests for enums with very big i… For u8 types as max value use o… Generate specs for `enum_(int|l… and 1 more (compare)

  • Dec 10 15:22
    GreyCat closed #70
  • Dec 10 15:22
    GreyCat commented #70
  • Dec 10 15:11
    valery1707 commented #70
  • Dec 10 15:10
    valery1707 opened #70
  • Dec 10 15:01
    lyubomyr-shaydariv commented #650
  • Dec 10 12:53
    valery1707 commented #186
  • Dec 10 11:00
    GreyCat labeled #652
  • Dec 10 10:47
    sheerluck commented #652
  • Dec 10 10:28
    GreyCat commented #186
Mikhail Yakshin
@GreyCat
What do you mean by "parity bit"?
eric-corumdigital
@eric-corumdigital
It just signals if the number of 1's in the rest of the message is even or odd
Mikhail Yakshin
@GreyCat
Sure, so you can just use type: b1 for a bit?
eric-corumdigital
@eric-corumdigital
Yes, but I also have to count the number of 1 bits
Mikhail Yakshin
@GreyCat
So, you want to add some layer of validation on top of parsing?
If that's the case, then likely what you're looking for can be implemented as part of kaitai-io/kaitai_struct#435
eric-corumdigital
@eric-corumdigital
Maybe? Ideally the parity would be checked before doing any other parsing
Mikhail Yakshin
@GreyCat
(although we don't have exactly that validation assertion yet)
eric-corumdigital
@eric-corumdigital
I am just wondering if it fits into Kaitai at all. If not, I can just do the check on the buffer myself
Mikhail Yakshin
@GreyCat
It fits, but we'll need to understand where exactly :)
eric-corumdigital
@eric-corumdigital
You know your section on magic signatures? It would be like that semantically. If the check does not pass, the whole result of parsing is an exception.
But to check a parity bit you have to parse the bit and then also count the 1's over the rest of the message
Mikhail Yakshin
@GreyCat
@eric-corumdigital So, what do you think of kaitai-io/kaitai_struct#435 ?
eric-corumdigital
@eric-corumdigital
If that passes, you then would want to parse that message
Mikhail Yakshin
@GreyCat
Magic signatures are implemented exactly using that at the moment
eric-corumdigital
@eric-corumdigital
I don't know if that proposal covers this scenario. Like I said, the valid value for the parity bit is determined by the count of 1's over the rest of the byte stream
I am not sure how to express that in Kaitai
Mikhail Yakshin
@GreyCat
Nobody knows ;) The very reason for that proposal is to invent some way to express that
So, the initial idea is legitimate — one needs to validate parity bit to match certain calculated value
eric-corumdigital
@eric-corumdigital
yes
Mikhail Yakshin
@GreyCat
the only thing we need to invent is how to express "grab that piece of stream and calculate some sort of checksum over it"
eric-corumdigital
@eric-corumdigital
that's right
Mikhail Yakshin
@GreyCat
this, from a very naïve standpoint, boils down to 2 things:
  • how to "grab that piece of stream" — i.e. can we reuse substreams for that?
  • how to "calculate some sort of checksum over it" — i.e. how can we define user-extensible functions that get a stream and output some value (like a number)
eric-corumdigital
@eric-corumdigital
There are many other error detecting and correcting codes. You could use those as reference for what such a feature may be.
With an error correcting code you want to just fix the error, probably, rather than failing
Mikhail Yakshin
@GreyCat
This way, again, a very naive proposal might be something like:
- id: buffer
  size: 123
  type: some_type
- id: parity_bit
  type: b1
  valid: calculate_parity(buffer)
eric-corumdigital
@eric-corumdigital
Perhaps but buffer is going to contain a useful message that you want to parse
Mikhail Yakshin
@GreyCat
Fixing an error looks like a job for custom process...
eric-corumdigital
@eric-corumdigital
Perhaps the way to do that is to run a second Kaitai parser over that. I don't know.
Mikhail Yakshin
@GreyCat
buffer is typed (type: some_type), so it will be parsed
eric-corumdigital
@eric-corumdigital
I see
Probably you will want calculate_parity to be an external definition, because there are many schemes for doing it
Mikhail Yakshin
@GreyCat
Exactly
Some kind of process-like interface
Take a look also at this one: kaitai-io/kaitai_struct#81
eric-corumdigital
@eric-corumdigital
Yes that is the same problem, different modulus
eric-corumdigital
@eric-corumdigital
Wait, Kaitai is a full codec right? Not just a decoder?
I am confused because the manual section on processors just talks about decoding
Looking through the generated code here… doesn't look like there are encoders? :o
James Elliott
@brunchboy
Not yet. It’s just a parser for now.
eric-corumdigital
@eric-corumdigital
From the home page… "Declarative: describe the very structure of the data, not how you read or write it", "Reading and writing binary formats is hard"… this gave me the impression it is a codec generator
James Elliott
@brunchboy
I think it’s a plan for the future, just not yet implemented.
That’s why I had to end up going a different route for my NFS implementation.
matrixbot
@matrixbot

oldbug > <@gitter_greycat:matrix.org> Fixing an error looks like a job for custom process...

Indeed! I am about to implement a Reed Solomon decoder used as process in kaitai struct

Mikhail Yakshin
@GreyCat

Wait, Kaitai is a full codec right? Not just a decoder?

"Kaitai" is a team name. "Kaitai Struct" is probably what you refer to, and it's neither a "codec" or "coder" or "decoder", but a generator of these.

There is a proof-of-concept branch that have some limited write support, only for Java:
Mikhail Yakshin
@GreyCat
@oldbug Cool ;)
James Elliott
@brunchboy
Thanks for the clarifications, @GreyCat! Talking about these things can get so meta, I should have been more precise.