kaitai_struct/Lobby

Mikhail Yakshin

@GreyCat

What do you mean by "parity bit"?

eric-corumdigital

@eric-corumdigital

It just signals if the number of 1's in the rest of the message is even or odd

Mikhail Yakshin

@GreyCat

Sure, so you can just use type: b1 for a bit?

eric-corumdigital

@eric-corumdigital

Yes, but I also have to count the number of 1 bits

Mikhail Yakshin

@GreyCat

So, you want to add some layer of validation on top of parsing?

If that's the case, then likely what you're looking for can be implemented as part of kaitai-io/kaitai_struct#435

eric-corumdigital

@eric-corumdigital

Maybe? Ideally the parity would be checked before doing any other parsing

Mikhail Yakshin

@GreyCat

(although we don't have exactly that validation assertion yet)

eric-corumdigital

@eric-corumdigital

I am just wondering if it fits into Kaitai at all. If not, I can just do the check on the buffer myself

Mikhail Yakshin

@GreyCat

It fits, but we'll need to understand where exactly :)

eric-corumdigital

@eric-corumdigital

You know your section on magic signatures? It would be like that semantically. If the check does not pass, the whole result of parsing is an exception.

But to check a parity bit you have to parse the bit and then also count the 1's over the rest of the message

Mikhail Yakshin

@GreyCat

@eric-corumdigital So, what do you think of kaitai-io/kaitai_struct#435 ?

eric-corumdigital

@eric-corumdigital

If that passes, you then would want to parse that message

Mikhail Yakshin

@GreyCat

Magic signatures are implemented exactly using that at the moment

eric-corumdigital

@eric-corumdigital

I don't know if that proposal covers this scenario. Like I said, the valid value for the parity bit is determined by the count of 1's over the rest of the byte stream

I am not sure how to express that in Kaitai

Mikhail Yakshin

@GreyCat

Nobody knows ;) The very reason for that proposal is to invent some way to express that

So, the initial idea is legitimate — one needs to validate parity bit to match certain calculated value

eric-corumdigital

@eric-corumdigital

yes

Mikhail Yakshin

@GreyCat

the only thing we need to invent is how to express "grab that piece of stream and calculate some sort of checksum over it"

eric-corumdigital

@eric-corumdigital

that's right

Mikhail Yakshin

@GreyCat

this, from a very naïve standpoint, boils down to 2 things:

how to "grab that piece of stream" — i.e. can we reuse substreams for that?
how to "calculate some sort of checksum over it" — i.e. how can we define user-extensible functions that get a stream and output some value (like a number)

eric-corumdigital

@eric-corumdigital

There are many other error detecting and correcting codes. You could use those as reference for what such a feature may be.

With an error correcting code you want to just fix the error, probably, rather than failing

Mikhail Yakshin

@GreyCat

This way, again, a very naive proposal might be something like:

- id: buffer
  size: 123
  type: some_type
- id: parity_bit
  type: b1
  valid: calculate_parity(buffer)

eric-corumdigital

@eric-corumdigital

Perhaps but buffer is going to contain a useful message that you want to parse

Mikhail Yakshin

@GreyCat

Fixing an error looks like a job for custom process...

eric-corumdigital

@eric-corumdigital

Perhaps the way to do that is to run a second Kaitai parser over that. I don't know.

Mikhail Yakshin

@GreyCat

buffer is typed (type: some_type), so it will be parsed

eric-corumdigital

@eric-corumdigital

I see

Probably you will want calculate_parity to be an external definition, because there are many schemes for doing it

Mikhail Yakshin

@GreyCat

Exactly

Some kind of process-like interface

Take a look also at this one: kaitai-io/kaitai_struct#81

eric-corumdigital

@eric-corumdigital

Yes that is the same problem, different modulus

eric-corumdigital

@eric-corumdigital

Wait, Kaitai is a full codec right? Not just a decoder?

I am confused because the manual section on processors just talks about decoding

Looking through the generated code here… doesn't look like there are encoders? :o

James Elliott

@brunchboy

Not yet. It’s just a parser for now.

eric-corumdigital

@eric-corumdigital

From the home page… "Declarative: describe the very structure of the data, not how you read or write it", "Reading and writing binary formats is hard"… this gave me the impression it is a codec generator

James Elliott

@brunchboy

I think it’s a plan for the future, just not yet implemented.

That’s why I had to end up going a different route for my NFS implementation.

matrixbot

@matrixbot

oldbug > <@gitter_greycat:matrix.org> Fixing an error looks like a job for custom process...

Indeed! I am about to implement a Reed Solomon decoder used as process in kaitai struct

Mikhail Yakshin

@GreyCat

Wait, Kaitai is a full codec right? Not just a decoder?

"Kaitai" is a team name. "Kaitai Struct" is probably what you refer to, and it's neither a "codec" or "coder" or "decoder", but a generator of these.

There is a proof-of-concept branch that have some limited write support, only for Java:

kaitai-io/kaitai_struct#27

namely, @generalmimon is working on this: https://github.com/kaitai-io/kaitai_struct/issues/27#issuecomment-554735671

Mikhail Yakshin

@GreyCat

@oldbug Cool ;)

James Elliott

@brunchboy

Thanks for the clarifications, @GreyCat! Talking about these things can get so meta, I should have been more precise.

Where communities thrive

People

Repo info

Activity