Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Brendan Dolan-Gavitt
    @moyix
    Now to figure out what the new version of Tunnel() is...
    Brendan Dolan-Gavitt
    @moyix
    If anyone knows how to make Embedded work with the BitStruct there, it would be really helpful!
    wtc-peg
    @wtc-peg
    I've been trying Construct in v2.8 and making big use of conditional Embedded( If (xxx, Struct(yyy))), to get some conditional field in my output containers. With 2.9 i really don't find how to do that. The only thing i can do is getting None field in my resulting container , by using "FIELD" / if (xxx,Struct(yyy)). this is not really satisfying as I would prefer to have not the "FIELD" in my container if it is None.
    Alex Railean
    @ralienpp

    Hey, can someone help me with understanding how to use RawCopy with Checksum? I described the issue on Github: construct/construct#774

    The core of the matter is that it is not clear from the documentation how to use multiple fields when computing the checksum. The examples show how only one specific field is taken into account, and I cannot transpose that to my own case.

    dyial
    @dyial
    @arekbulski any chance you can merge half-precision floats PR? construct/construct#736
    sasumner
    @sasumner
    If I have some code like this:
    s = Struct(
        "header" / Struct(
            "seq" / Int32ul,
            "timestamp" / Struct(
                "secs" / Int32ul,
                "nsecs" / Int32ul,
            ),
            "frame_id_count" / Int32ul,
            "frame_id" / Bytes(this.frame_id_count),
        ),
        "target_number" / Int8ul,
        "f" / Float64l,
        "test_array" / Int8ul[3],
    )
    Is it possible for Construct to give me what I call an "example usage string" for the members? Maybe something like this:
    s.header.seq
    s.header.timestamp.secs
    s.header.timestamp.nsecs
    s.header.timestamp.frame_id_count
    s.header.timestamp.frame_id[]
    s.target_number
    s.f
    s.test_array[]
    Likely not as it is kind of an esoteric need, but I thought I'd ask anyway.
    Dan Gass
    @dangass_gitlab
    Thought I'd share a project I am getting off the ground that is very similar in purpose to construct, but being new is able to take advantage of features only available in Python 3.6+. My end goal is to approach performance of the struct module. Similar to construct, it supports arbitrary nesting and has a contextual protocol. It also has visual memory summaries to assist in debugging (especially when unpack/pack exceptions occur, a partial dump showing progress is included in the exception message). Hope to share ideas and perhaps see some of the developers of Construct at PyCon2019. Here is a link to the docs: plum-py
    MaruthiRathnakar
    @MaruthiRathnakar
    from construct.core import (Struct, StaticField) is not working, getting import error :ImportError: cannot import name StaticField
    wtc-peg
    @wtc-peg
    @dangass_gitlab Why not continuing work on Construct ? Many one may have start working with or discovering it.
    Dan Gass
    @dangass_gitlab
    @wtc-peg A while back I had discovered construct when searching for this kind of capability. But at the time it didn't appear it supported or could easily be made to support proxying behavior. By proxy I mean being able to create instances of a type that provided the behavior of the type but were backed by a memory object, remote or local. For example, at my employer we construct a memory interface to a remote device to get and set bytes. And we create instances of types that are a view into the memory. These instances act as proxies. The simplest example is an integer type. An instance of it created with a remote memory interface provides behavior of an integer, but when used (such as in a comparison), actually goes out to the remote memory, fetch the bytes and performs the action based on the memory bytes fetched. We ended up developing a system that solved both the easier pack/unpack problem and the proxy problem. I've been researching how to solve both problems in simpler, higher performance way, hence plum-py. Short of converging efforts, if I can help provide ideas for you and vice versa, I think that is a good thing. I'm very interesting in knowing what is on the typical user's wish list that isn't being satisfied today between the alternatives. What are the highest priority features being looked at by the Construct developers? Any thoughts on making it's performance better? Any thoughts on taking advantage of new features in Py3 for better syntactic experience?
    Bush-Lee
    @Bush-Lee
    Hey there, I'm having issues on building a struct with switches to other structs
    Mainly I have 0 ideea how to to do it
    Can anyone give a helping hand ?
    I'll leave code snippet if needed
    wtc-peg
    @wtc-peg

    @Bush-Lee, Here is a simple extract from a running project. With hope it will be helpful:

    Occurence = BitStruct(
        "Extended" / Flag,
        "Value"    / Switch(this.Extended,{ 
            False : BitsInteger(7),
            True  : Struct(
                "Reserved"       / Const(0x00,BitsInteger(7)),
                "OccurencesHigh" / BitsInteger(16),
                "OccurencesLow"  / BitsInteger(16),
                ),
        })
    )

    yess... succeded in putting formated code ;O) (for all rookies like me, see: Gitter Markdown basics)

    Bush-Lee
    @Bush-Lee
    @wtc-peg Thanks for the snippet, I already managed to make the struct i need but struct.build is throwing me off (it's throwing errors too in fact)
    CommandArgument = Enum(
        Byte, SetOutputs=1, SetPWM=2, SetConfiguration=3, SetConfigMode=5,
        GetNID=16, GetStatus=17, GetDidStauts=18, GetConfigMemory=19,
        GetCalibrationMemory=20, ForceResetRouter=21, GetPacketPath=22
    )
    
    ConfigRecord = Struct(
        'Adress' / Int8ub,
        'Value' / Int8ub
    )
    
    ConfigData = ConfigRecord[16]
    
    Command = Struct(
        'CommandArgument' / CommandArgument,
        'Data' / Switch(
            this.CommandArgument, {
                'SetConfiguration': ConfigData,
                'SetOutputs': ConfigRecord,
                'SetPWM': ConfigRecord,
            }, default=ConfigRecord
        )
    )
    # Works
    print(ConfigData.build([dict(Adress=i, Value=i) for i in range(16)]))
    
    # Doesen't work throws error
    # last 2 lines of the error log are
    #    subobj = obj[sc.name] # raises KeyError
    #    TypeError: list indices must be integers or slices, not str
    print(Command.build(
        dict(CommandArgument=3,
             Data=[dict(Adress=i, Value=i) for i in range(16)]
             )
        )
    )
    
    
    # Shouldn't work
    print(Command.build(
        dict(CommandArgument=3,
             Data=dict(Adress=1, Value=1)
             )
        )
    )
    Bush-Lee
    @Bush-Lee
    Ok so in case someone stumbles in this i'm getting the error because Switch isnt working so it goes to the default case automaticly
    wtc-peg
    @wtc-peg
    Oups, I remember a previous experience like this some time ago: Enum ID must be used when building, not enum Value .
    Maybe "Support of value OR ID of Enum" could be added in a wichlist ?
    print(Command.build(
        dict(
            CommandArg = 'SetConfiguration',
            Data=[dict(Adress=i, Value=i) for i in range(16)]
            )
        )
    )
    Bush-Lee
    @Bush-Lee
    @wtc-peg That was it. Thanks, saved me a lot of time with that.
    Alex Railean
    @ralienpp

    Is it possible to infer certain values based on the context? For example

    Tlv = Struct(
        'tag' / Int16ub,
        'length' / Int32ub,
        'value' / Array(this.length, Byte)
        )
    
    tlv_data = dict(tag=1, length=5, value=b'\xFF\xFF\xFF\xFF\xFF')
    Tlv.build(tlv_data)

    In this case I have to explicitly provide all the values when I call build, but is it possible to do tlv_data = dict(tag=1, value=b'\xFF\xFF\xFF\xFF\xFF') and have it figure out on its own what the adequate value of the length field should be?

    What is not clear to me is the duality... When serializing, I want to use a short-hand notation, but at the same time, the structure definition should also work when deserializing raw data.

    MaruthiRathnakar
    @MaruthiRathnakar
    I am getting an error while running nose test , AssertionError: MappingError not raised by parse..... Below is the snippet, can i know where i am going wrong

    mar_payload_type = "payload_type" / Enum(BitsInteger(7),
    RTP101=0x65)

    def test_payload_type_enum():
    test = "" / BitStruct(Padding(1), mar_payload_type)
    assert_equal(test.parse(b"\x65").payload_type, "RTP101")
    assert_raises(MappingError, test.parse, b"\xff")

    Stephan Gerhard
    @unidesigner
    Hi! Any suggestions on how to properly write the struct for a binary header sequence like: b'M1\n7\n17 17\n0 0\n0/0015/0015_0012_0_b align\n' ?
    I do have
    map_header = Struct(
        "type" / Const(b"M1"),
        "level" / Int32un,
        "width" / Int32un,
        "height" / Int32un,
        "xmin" / Int32un,
        "ymin" / Int32un,
        "imgName" / CString("ascii"),
        "refName" / CString("ascii"),
        "data" / Array(this.width * this.height * 3, Int32un),
    )
    But this does not seem to parse the spaces properly.
    Stephan Gerhard
    @unidesigner
    Thanks!
    Paul Moore
    @pfmoore

    Hi! I want to create a struct where one of the field types depends on another at runtime - so basically something like this

    def get_the_parser(id):
        if id == 1: return Byte
        if id == 2: return Int32un
    
    item = Struct(
        "id" / Byte,
        "data" / lambda ctx: get_the_parser(ctx.id),
    )

    Obviously the above doesn't work, because the "data" item is a function, not a Construct subclass. But LazyBound doesn't work, because the lambda it wraps doesn't have access to the context. I feel like this should be really easy, but I've been working on it most of the day, and it looks like I'm missing something, because I've no idea how to do it :-(

    Any suggestions?

    MaruthiRathnakar
    @MaruthiRathnakar

    repeat_enabled = "dispatch_controlled_repeat" / Struct("dispatch_controlled_repeat" / Flag,
    Padding(6),
    "repeat_enabled" / Flag)

    note = "note" / BitStruct(
    repeat_enabled
    )
    when i build this using below line

    note.build(dict(dispatch_controlled_repeat=True,repeat_enabled=True))

    I get this Error

    File "C:\Python37\lib\site-packages\construct\lib\containers.py", line 183, in update
    for k,v in seqordict:
    TypeError: 'bool' object is not iterable

    Where i am going Wrong, Any suggestions?

    I am quite new to construct, so please help me with this
    MaruthiRathnakar
    @MaruthiRathnakar
    @arekbulski , i m using construct 2.9.45 with python 3.7.. Using optional inside the Bitstruct raises Stream Error, I did find one similar raised in github, I also read your answer, tat If the struct is of Restreamed type, it wont work.. But i cant make my structure as Transformed and i want Optional to work inside Bitstruct, any way to do it.
    Any help would be highly appreciated
    Thanks
    Dan Gass
    @dangass_gitlab
    Is anyone from this or related projects going to be at PyCon? I am working on a similar project for flexible and fast memory packing/unpacking (https://plum-py.readthedocs.io/en/latest/index.html) that is leveraging features in Py3. I'm hoping to meet folks from this project to share ideas. I've also done a little benchmarking comparing basic pack/unpack speed performance for various packages (the standard library struct module, xnd/ndtypes, plum, construct, and a couple others). If anyone is interested in seeing what I have I'd be happy to share. My company (Rockwell Automation) is sponsoring a booth and hopefully you can find me there.
    Nguyễn Hồng Quân
    @hongquan
    Hi, how to use sizeof or other method to get the size of "root" struct?
    DanseMacabre
    @DanseMacabre
    @arekbulski Cheers for steering this library BTW, It's super useful in my life at the mo!
    reox
    @reox
    Hi! I have some wonderful file format which has an ID for some sections, but the ID is also a counter - that means there might be n sections of type A but then they have the IDs A, A+1, A+2, ... A+n-1. They define two values for each section ID, namely a valid start and valid end value. Is there some magic in construct to parse such stuff directly into a form where I get the section ID as well as the counter? Maybe someone has a hint how this could work?
    possibly I have to define my own type of Enum right?
    reox
    @reox
    So I put something together which make it work, but there might be a better way. If someone has one, please let me know!
    class StartEndEnum(Enum):
        def __init__(self, *args, **kwargs):
            new_args = dict()
            for k, v in kwargs.items():
                if k.endswith('_START'):
                    key = k[:-6]
                    end = key + "_END"
                    if end not in kwargs:
                        raise ValueError("{} without END item!".format(key))
                    for i in range(v, kwargs[end] + 1):
                        new_args[key + "_{}".format(i - v)] = i
                elif k.endswith('_END'):
                    pass
                else:
                    new_args[k] = v
            super().__init__(*args, **new_args)
    Edvard Pettersen
    @tiepettersen
    Hi, after parsing a construct and I get a container, are there any mechanisms to ensure fields in the construct are of appropriate length and such when setting new values based on what was defined in the construct?
    Something along these lines:
    definition = Struct(
        "name" / Bytes(8),
        "id" / Int32ul,
    )
    
    cont = definition.parse(b'ABCDEFGH\x00\x11\x22\x33')
    cont.name = b'ABCDEFGHX'  # I expect 8 bytes, not 9. Can I make this fail?
    st.build(cont)
    Adrien Guinet
    @aguinet

    Hello everyone!
    I have a construction that I try to parse w/ construct, but can't gigure out how with the documentation.
    Basically, in the stream, I first have a list of bytes that are mapped to an enum, whose count is already known:

    E0 E1 E2 E3

    And then, I have a list of structures whose type depends on E0,E1,...
    The easy part is to figure out E*, like this:

    "types" / Array(this.count, MyEnum)

    But then, I need to do a switch on top of the values that are in types. Is there an easy way to do this?
    Thanks for the help!

    Adrien Guinet
    @aguinet
    I managed to do it using a custom Construct object, but that looks like a hack to me :)
    Nguyễn Hồng Quân
    @hongquan
    👍