body.body.end_beat is impossibly high. In a correct parse, the end beat is 0x1ff, or 511. Here is a screen shot of the results using the version of the file I have in Git, which does not attempt to make unmasking conditional:
Maven plugin problem version (0.1.4) and kaitai( 0.9) on kaitai:generate goal
Failed to execute goal name.valery1707.kaitai:kaitai-maven-plugin:0.1.4:generate (default-cli) on project **: Fail to generate Java files
[INFO] /meta/bit-endian: unknown key found, expected: application, encoding, endian, file-extension, id, imports, ks-debug, ks-opaque-types, ks-version, license, title, xref
@KhalidAriche You need the Kaitai Struct compiler version 0.9 to be able to use the bit-endian key.
The kaitai-maven-plugin is apparently using the compiler bundled in the universal .zip from Bintray of version specified by the kaitai.version config option, so if you set it to 0.9, it should work.
toJsonString() methods everywhere that will generate JSON string for current class and all its members
@KhalidAriche
how to specify the --no-auto-read on kaitai compiler maven plugin ?
I assume that the https://github.com/valery1707/kaitai-maven-plugin does not support that, there is nothing like that in the documented plugin parameters or anywhere in the code (if I used the right needles).
You can contact @valery1707 if he doesn't want to add support for it, open an issue on that repo or even a pull request with the implementation (it can't be anything hard; register a new plugin parameter noAutoRead, then look at its value when invoking kaitai-struct-compiler and pass the --no-auto-read CLI arg if it's true; 6 added lines at most). Or you can resort to messing up with the command-line ksc directly without the Maven plugin intermediary - then you will have the most control.
Also note that the kaitai-maven-plugin is not managed by the Kaitai team - it even doesn't look like anybody else than @valery1707 actually has write access into that repo. It's not so important (we'll keep doing our best to help you anyway), but I'm just saying that we don't have any more control over that plugin than you do.
hi folks! does Kaitai support defining composite types of arbitrary Kaitai struct types? for example, given CustomArray<CustomType> entries in C++, I naively tried the following in Kaitai:
- id: entries
type: custom_array(custom_type)but that does not work as it understandably expects custom_type to be a field, rather than a Kaitai type. thoughts?
Is there any way to set up a type switch within a loop, being switched on the previous element in the loop?
I Tried this but was not having any luck due to this error "don't know how to call method 'name' of object type 'KaitaiStructType'"
types:
fields:
seq:
- id: fields
type:
switch-on: _.name
cases:
"XXXX": unknown_field_known_size(_.value)
_: unknown_field
repeat: until
repeat-until: _._io.pos == _parent.sizeBasically I am looping over a set of key/value pairs that follow the same format, the only outlier being that if the size of the value would require more than 2 bytes to store, the size is then stored in the previous pairs value, signified by the key of the previous value being "XXXX".
Is there another way to go about this that I missed?
serve_files.py file in the git, but there's no update since 2017 on it and no documentation anywhere
Hey. I'm really fond of the kaitai system. It looks great to reverse engineer, document and implement some protocol definitions. But is it possible to use the web ide with local filesystem storage? I've seen the
serve_files.pyfile in the git, but there's no update since 2017 on it and no documentation anywhere
Hi @Gansgar, as I can see in the commit that added serve_files.py, using local filesystem is a feature exclusive to version v2 of the Web IDE. The currently deployed version is v1. @koczkatamas was actively working on v2 about 4 years ago (in 2017), but nowadays, it's just an unfinished abandoned subproject (though a lot of work has been already done - see kaitai-io/kaitai_struct_webide#35). Hopefully someone will return to it and finish the work.
The current state of v2 is available at https://ide.kaitai.io/devel/v2.html, but it looks like there are some errors which prevent from using certain features (to the extent that I doubt whether it's even usable at this time). Although all file/folder operations seem to work fine, I haven't even figured out how to compile a .ksy spec (every time an error occurs) - maybe all that needs to be done is a 5-minute fix, maybe not. Also I'm not sure how to activate the local filesystem feature - apparently the serve_files.py script only allows requests that authenticate themselves with a 16-byte hex secret, which has to be passed in a query parameter secret to the Web IDE. Then you'd need to add a node for the "remote file system" to the directory tree (more specifically, to this list) to be able to access it with the UI, and just fix any errors/bugs as they come :)
So yeah, I'd say it is possible, just not without any work :smile: It's an unexplored area, so expect various pitfalls and bugs. However, if you feel brave enough to give it a try, I'd be happy to accept a PR with necessary changes and fixes, and possibly replace the v1 version with v2 once it's finished. Although it is not yet finished, I can see that v2 comes with a plenty of new features and improvements, so it would be great to get it working.
Unfortunately, it is not very high on my priority list right now - you might ask @luxaritas as he seemed interested in the exact same issue 5 months ago (see https://gitter.im/kaitai_struct/Lobby?at=5f6180d74f19dd093350c994). He wrote that he would be able to take a peek in the next couple weeks (written at September 17, 2020 in https://gitter.im/kaitai_struct/Lobby?at=5f63b81bce5bbc7ffdcfe553), so perhaps he already achieved some success.
Hi @dymaxionuk.
What about the timestamp, magic number packet based, micro/nanosecond switch, do you think that would be supported by the current code?
Yes, absolutely.
but for pcap files it doesn't work at the moment. There's a magicnumber per packet header rather than per file which indicates the endianess of the packet. This number however has 2 forms which signifies whether or not the packet has microsecond or nanosecond timestamps.
I'm not sure if we mean the same format, but the specification at https://wiki.wireshark.org/Development/LibpcapFileFormat says otherwise. The magic_number seems to be really only present in the Global Header, which is present only once at the beginning of the pcap file:
Global Header
This header starts the libpcap file and will be followed by the first packet header:
I can't see any other magic_number in the Record (Packet) Header.
Although they suggest that the wiki at wiki.wireshark.org is deprecated and the one correct is https://gitlab.com/wireshark/wireshark/-/wikis/Development/LibpcapFileFormat#global-header, I can't spot any difference in terms of information provided by the new page. It only treats the pcap format with nanosecond precision as a variant of pcap (not the core pcap itself, as the old wiki did), which is called Nanosecond pcap, but the conclusion is the same.
So if my interpretation is correct, I'd say the simplest solution is the following:
meta:
# ...
# "endian: le" should be removed from here because the spec no longer uses one predominant endianness
seq:
- id: magic_number
type: u4be
enum: magic_numbers
- id: body
type: pcap_body
instances:
has_nanosec_resolution: # this will be probably useful as well - you can
# access it from anywhere as `_root.has_nanosec_resolution`
value: |
magic_number == magic_numbers::nanosec_le
or magic_number == magic_numbers::nanosec_be
types:
pcap_body:
meta:
endian:
switch-on: _root.magic_number
cases:
magic_numbers::microsec_le: le
magic_numbers::nanosec_le: le
magic_numbers::microsec_be: be
magic_numbers::nanosec_be: be
seq:
- id: hdr
type: header
- id: packets
type: packet
repeat: eos
header:
doc-ref: 'https://wiki.wireshark.org/Development/LibpcapFileFormat#Global_Header'
seq:
# note: extracted `magic_number` from here to root `seq`
- id: version_major
type: u2
- id: version_minor
type: u2
enums:
magic_numbers:
0xa1b2_c3d4: microsec_be
0xd4c3_b2a1: microsec_le
0xa1b2_3c4d: nanosec_be
0x4d3c_b2a1: nanosec_leNote that if the parsed value of magic_number does not match anything in /types/pcap_body/meta/endian/cases, a KaitaiStream.UndecidedEndiannessError is thrown, so it's guaranteed that an invalid file is rejected as soon as possible.
However, if you are convinced that you need "per packet endianness" switch instead of the per file as it is now, it is still perfectly possible. There is no technical limitation telling that you can't do that: meta/endian/switch-on can be put on any subtype, and you can reference the parent struct by using _parent just as well you can access the top-level struct with _root. Moreover, if "pulling" the value of a field from an outer type using _root.{...} or _parent.{...} doesn't suit you or you need more flexibility, you can always "push" the values you need "from outside to the inside" using parameters.