Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Chris
    @cmidgley
    Didn't catch xsuse before ... I'll play with it in a few days. Thx.
    Lawrence R Kincheloe
    @LokiMetaSmith

    just pulled the public branch and did the "make install" for linux, got hit with the following errors.
    make install -f simulator.mk
    make[1]: Entering directory '/home/loki/Projects/moddable/build/makefiles/lin'

    simulator debug : cc main.c

    /home/loki/Projects/moddable/build/simulator/lin/main.c: In function ‘onApplicationOpen’:
    /home/loki/Projects/moddable/build/simulator/lin/main.c:434:99: warning: ‘.xsa'’ directive output may be truncated writing 5 bytes into a region of size between 3 and 8193 [-Wformat-truncation=]
    434 | snprintf(cmd, sizeof(cmd), "/bin/cp -p \'%s\' \'%s.xsa\'", path, gxConfigPath);
    | ^~
    /home/loki/Projects/moddable/build/simulator/lin/main.c:434:49: note: ‘snprintf’ output between 21 and 8211 bytes into a destination of size 8208
    434 | snprintf(cmd, sizeof(cmd), "/bin/cp -p \'%s\' \'%s.xsa\'", path, gxConfigPath);
    | ^~~~~~~~~~~~~~~~~~
    /home/loki/Projects/moddable/build/simulator/lin/main.c:427:99: warning: ‘.so'’ directive output may be truncated writing 4 bytes into a region of size between 3 and 8193 [-Wformat-truncation=]
    427 | snprintf(cmd, sizeof(cmd), "/bin/cp -p \'%s\' \'%s.so\'", path, gxConfigPath);
    | ^~~
    /home/loki/Projects/moddable/build/simulator/lin/main.c:427:49: note: ‘snprintf’ output between 20 and 8210 bytes into a destination of size 8208
    427 | snprintf(cmd, sizeof(cmd), "/bin/cp -p \'%s\' \'%s.so\'", path, gxConfigPath);
    | ^
    ~~~~~~~~~~~~~~~~~~
    Assembler messages:
    Fatal error: can't create /home/loki/Projects/moddable/build/tmp/lin/debug/simulator/main.o: No such file or directory
    make[1]: [simulator.mk:59: /home/loki/Projects/moddable/build/tmp/lin/debug/simulator/main.o] Error 1
    make[1]: Leaving directory '/home/loki/Projects/moddable/build/makefiles/lin'
    make:
    [makefile:66: install] Error 2

    Michael Kellner
    @mkellner
    @LokiMetaSmith, please "make" before "make install -f simulator.mk"
    Peter Hoddie
    @phoddie

    @cmidgley – a fix the issue you ran into, being unable to create a Proxy for a host function primitive, will be pushed in a couple days. Meanwhile, if you want to give it a try before that, in xsProxy.c after line 780 add these lines:

    #ifdef mxHostFunctionPrimitive
        if ((mxArgc > 0) && (mxArgv(0)->kind == XS_HOST_FUNCTION_KIND))
            fxToInstance(the, mxArgv(0));
        if ((mxArgc > 1) && (mxArgv(1)->kind == XS_HOST_FUNCTION_KIND))
            fxToInstance(the, mxArgv(1));
    #endif

    There's a corresponding change for Proxy.revocable, but it doesn't sound like you are using that (yet!).

    Chris
    @cmidgley
    Thanks. I don't yet see a need for revocable, but good to know.
    Peter Hoddie
    @phoddie
    Just a matter of time. ;)
    Chris
    @cmidgley
    Feels that way!
    Lawrence R Kincheloe
    @LokiMetaSmith

    @LokiMetaSmith, please "make" before "make install -f simulator.mk"

    yep, that's my issue!

    Chris
    @cmidgley

    Using Instrumentation to get live info on slots - I'm finding the doc and the first example are wrong, as the numbers are not correct (for example, doc says 11 is SlotHeapSize yet it is actually 14 on ESP32). I see in xsuse it scans to find the correct number - that's how I got to 14. What's the story here?

    Also, the type definition for it is very difficult to use. The type InstrumentationOffset is not exported, and really only limits a numeric value and enforces it to be 1-16 (even though different numbers are used). And you can't use the actual names as they are only types and not exported constants. Same issue with WiFi (and perhaps others) on their type definitions. Just an FYI for now, but I think the fix (if instrumentation numbers are static and don't need to be scanned) is to export constants from the .js file and declare them in the .d.ts file (perhaps using a enum, not sure yet). If instrumentation numbers are dynamic, I've got no good ideas for typing ... maybe a dictionary lookup method in .js that does the scan and return the correct number for the instrumentation?

    Peter Hoddie
    @phoddie
    The offsets are different for each port. xsuse was working correctly on ESP32, ESP8266, and macOS very recently. Can take another look.
    Peter Hoddie
    @phoddie
    Just ran xsuse on macOS, ESP8266, and ESP32. All results look reasonable. Here's the start of output from ESP32:
    Boolean: 0 bytes
    Number: 0 bytes
    String `String in ROM`: 0 bytes
    String `fromCharCode(32)`: 12 chunk bytes = 12 bytes
    String `String in ` + 'RAM': 24 chunk bytes = 24 bytes
    Object {}: 1 slots = 16 bytes
    Object {x: 1}: 2 slots = 32 bytes
    Object {x: 1, y: 2}: 3 slots = 48 bytes
    Object {x: 1, y: 2, z: 3}: 4 slots = 64 bytes
    Date: 2 slots = 32 bytes
    BigInt 1n: 12 chunk bytes = 12 bytes
    BigInt 100000000001n: 16 chunk bytes = 16 bytes
    Function () => {}: 7 slots = 112 bytes
    ...
    Chris
    @cmidgley

    Sorry if I was unclear - xsuse works fine, and that's how I figured out the numbers. The documentation, TypeScript file and the other example are all wrong (or at least not portable, as they are wrong on ESP32).

    I do see how to find the base of instrumentation to get slots and instrumentation that follows (and I have that working). I found that before that, System Free Memory, was also offset wrong... Just wondering how I should approach accessing any of the instrumentation - for example, is the base of instrumentation (1) always correct until a particular index, and then we have to scan backwards from the end to find the rest? How would I find System Free Memory or Timers, for example?

    Chris
    @cmidgley

    @phoddie As you know, I've been working on out-of-memory issues, focused on getting preload working. I got much of Instrumentation working (still don't have all the indexes right, see prior comment) and found I needed to use another WeakMap.

    I'm now crashing in xsMapSet.js with an exception here. Spent the last two days trying to debug why and finally got a small fragment showing the crash - it's due to preload being used on a class that gets put into a WeakMap. See this gist for the files that recreate the failure. This is a blocking issue for me as I can't run without preload but crash with it. Hopefully there is a simple solution, but it's beyond me to figure out.

    Peter Hoddie
    @phoddie
    (Hey, @cmidgley. Busy day for me, so I won't get to this right away. Just acknowledging for now.)
    Chris
    @cmidgley
    Appreciate the ack. Understood on being busy. Want me to put in an issue?
    Peter Hoddie
    @phoddie
    Yes, please do. The example is small, which is great. It looks like it gets tripped up because the fully-inverted weak-map implementation assumes the WeakMap keys are mutable. Here the key is an immutable object in ROM. That object can never be collected (since it lives in ROM) but in principle the WeakMap should be able to contain that. We'll see. (Going back to my busy day now!)
    Lawrence R Kincheloe
    @LokiMetaSmith
    hmm, so I got everything to compile with the latest public branch, (there's a bug introduced in xs.h where the riscv compiler throws the size_t define for a loop)
    I think where life is giving me lemons is I'm not able to connect to the dev board over serial, so I can get some proper debug status. I suspect there's an issue in the sdkconfig.defaults
    Peter Hoddie
    @phoddie

    there's a bug introduced in xs.h where the riscv compiler throws the size_t define for a loop

    Is size_t already defined in the riscv build? That change is working on ESP32, ESP32-S2, ESP32-S3, and ESP8266.

    Lawrence R Kincheloe
    @LokiMetaSmith

    yes,

    In file included from /home/loki/.espressif/tools/riscv32-esp-elf/esp-2021r2-8.4.0/riscv32-esp-elf/riscv32-esp-elf/sys-include/sys/reent.h:14,
    from /home/loki/Projects/esp-idf/components/newlib/platform_include/sys/reent.h:17,
    from /home/loki/.espressif/tools/riscv32-esp-elf/esp-2021r2-8.4.0/riscv32-esp-elf/riscv32-esp-elf/sys-include/sys/errno.h:11,
    from /home/loki/.espressif/tools/riscv32-esp-elf/esp-2021r2-8.4.0/riscv32-esp-elf/riscv32-esp-elf/sys-include/errno.h:9,
    from /home/loki/Projects/esp-idf/components/newlib/platform_include/errno.h:18,
    from /home/loki/Projects/moddable/xs/includes/xs.h:130,
    from /home/loki/Projects/moddable/xs/includes/xsmc.h:41,
    from /home/loki/Projects/moddable/modules/pins/digital/digital.c:21:
    /home/loki/.espressif/tools/riscv32-esp-elf/esp-2021r2-8.4.0/riscv32-esp-elf/lib/gcc/riscv32-esp-elf/8.4.0/include/stddef.h:216:23: error: conflicting types for 'size_t'
    typedef SIZE_TYPE size_t;
    ^~
    In file included from /home/loki/Projects/moddable/xs/includes/xsmc.h:41,
    from /home/loki/Projects/moddable/modules/pins/digital/digital.c:21:
    /home/loki/Projects/moddable/xs/includes/xs.h:113:21: note: previous declaration of 'size_t' was here
    typedef uint32_t size_t;
    ^~
    make: * [/home/loki/Projects/moddable/build/tmp/esp32/esp32c3/debug/helloworld/makefile:760: /home/loki/Projects/moddable/build/tmp/esp32/esp32c3/debug/helloworld/digital.c.o] Error 1

    I just commented out

    // #if defined(ets)
    // typedef uint32_t size_t;
    // #endif

    I'm not sure where "ets" is defined
    '''// #if defined(ets)
    // typedef uint32_t size_t;
    // #endif'''
    Peter Hoddie
    @phoddie
    Sure, that will keep you moving. But it isn't a solution that can be merged since it will break other targets.
    Lawrence R Kincheloe
    @LokiMetaSmith
    hmm, my gitter formatting is not working, but I think you get the idea
    Peter Hoddie
    @phoddie
    __ets__ is defined in the make file for ESP32. It should not be removed as there is code that depends on it.
    Is there a way to check for RISC-V builds of ESP-IDF? That would be way to add a check here to cleanly eliminate the error.
    Lawrence R Kincheloe
    @LokiMetaSmith
    hmm, ok let me step back. usually size_t is defined in the standard library, I'm not sure why its being redefined in xs.h. it looks like that line of code was recently added as well.
    Peter Hoddie
    @phoddie
    Yes, it usually is. But not all platforms do. The ESP8266 builds, for example, don't.
    Maybe it is only required on ESP8266 builds. Updating the check to #if defined(__ets__) && !defined(ESP32) won't define it on ESP32 targets. Would that work on the RISC-V target?
    Lawrence R Kincheloe
    @LokiMetaSmith
    looks like the issue is that when size_t is defined, it needs a #define __size_t
    actually it's #define size_t 1
    Lawrence R Kincheloe
    @LokiMetaSmith
    or maybe not, anyways there's a catch in the riscv for if size_t is defined, just gotta get it to trigger
    Peter Hoddie
    @phoddie
    Does the suggested change above not work?
    Prateek Sharma
    @message2prateek
    Hello, just discovered Moddable SDK and I'm a beginner to the world of microcontrollers. Does anyone have an example to share for IR receiver and transmitter? I'll love to capture IR signals and control devices at home. Thanks.
    Peter Hoddie
    @phoddie
    Welcome @message2prateek! The Moddable SDK examples directory has IR transmitter and IR receiver projects. These use the RMT module of the ESP32. That looks like it might be a good starting point for you.
    Chris
    @cmidgley

    Once again a crazy question.... Is there any way to cause code executed during preload to execute after all modules have been preloaded?

    Background: I have unit test files (Jest-like) that if I include in the Manifest they are detected and executed. I used to use Modules to find them, but now I am preloading them - which is really cool because all sorts of overhead gets absorbed into flash. As each module is preloaded, when the Jest describe method is execute, I register the test suite. Once all modules are preloaded, I'd like to freeze that object - but I can't do it in main (it runs before they do) and I don't see a way to cause code to execute after they have been loaded. There are ways around this (I could have a module that is just for loading tests), but that has the side effect that you have to keep it and the manifest synchronized or it will fail to work.

    Also - can you confirm that slot/chunk allocation used during preload, even if they garbage collect out, will keep the "high water" mark of the slot and chunk usage when the code executes? What I think I see happening is after a big preload (the test stuff above) my slot and chunk numbers are decent (say 8K and 2K), but the allocated size is 80K and 20K. The problem is that when the code runs, it needs less slots and more chunk space available, which garbage collection won't help with.

    Peter Hoddie
    @phoddie
    Preload runs on your computer, not the MCU. During that time, there is no limit on how much memory it can allocate (beyond any limit enforced by your computer's OS). Before the VM state is captured for writing the firmware, a GC is performed to eliminate anything not in use (primitives, objects, code, etc). No state about slot of chunk use during preload is is put into the firmware image. When the image is used on the device, it is with a fresh heap.
    Main point: any allocations you see on the device are a result of code that ran on the device, not code that ran during preload.
    Peter Hoddie
    @phoddie
    Another possibility is that you have configured the initial slot and chunks in the creation section of the manifest to those sizes. We leave those small by default so that the balance between slots and chunks is found by runtime use.
    Chris
    @cmidgley
    Thanks - I'll dive deeper into it. I'm guessing the "execute preload code after modules have been loaded" question is a no... but can you confirm there is no trick to cause a module to be guaranteed to be last to preload or allow code to execute after all preloads are done (all during preload, so I can do a freeze)?
    Peter Hoddie
    @phoddie

    execute preload code after modules have been loaded

    Preload is based on modules and recursive. If you have code in Module A you want to run only after Module B is loaded, add an import statement to Module A for Module B to make that dependency visible to the language . Then, add Module A to the preload list. When Module A is preloaded, the dependency on Module B will be resolved first causing Module B to preload, then the top level of Module A will be executed. That's standard JavaScript module loading behavior.

    Chris
    @cmidgley

    Yeah - that's what I meant by "a module just for loading tests", but was hoping to avoid having to maintain both the manifest file and that separate include file anytime I change my tests. But it is what it is, and I'll work with it.

    FYI - I have the WebSocket working great now. I have a test suite with 15 tests that hit all sorts of tests (connection loss, early cancel, small messages, large multi-fragment messages, etc) and it's passing all after GB's of testing. Thanks for the hint about having to read all - was a pain to implement, but it's all good now. It's not a trivial change, so not sure you are going to want it ... but in the coming days I'll do a pull and let you decide.

    Peter Hoddie
    @phoddie
    I guess I don't understand what you are trying to do with the tests manifest to offer a helpful suggestion.
    Chris
    @cmidgley
    Not sure it's worth taking up more of your time - I can work with it.
    Peter Hoddie
    @phoddie
    Naive question: if you want a module to run after all other modules have preloaded, why not just put it at the end of the preload array in your top-level manifest?
    Chris
    @cmidgley
    Ummm.... because I didn't think of that? I'll give it a try!
    Chris
    @cmidgley

    Peter - I'm trying to reduce all the warnings I get from the linker on preload. Many I understand, but I'm struggling with a few. One in particular are typescript enums. I've been able to take an enum and freeze it (and will have a CDV update to support this), such as:

    enum MyEnum {
        someValue = 1
    }
    (<unknown>MyEnum) = Object.freeze(MyEnum);

    This appears to successfully freeze the enum, and the warnings for it go away.

    However when I use the enum, such as:

    if (myValue == MyEnum.someValue)
       ...

    I get the linker warning:

    MyEnum: no const

    It's not a big deal from a slot perspective, but I'm getting so many warnings that it's hard to find the important ones.

    Peter Hoddie
    @phoddie

    MyEnum is not const. It isn't the object that the warning is for, but the variable the object is assigned to. Recall that in compileDataView we allow the original variable the enum is assigned to (which is not const) get garbage collected and export a reference to it, so this C enum:

    #pragma language(typescript)
    
    enum Foo {
      one = 1,
      two = 2
    };

    outputs this TypeScript:

    enum Foo {
       one = 1,
       two = 2,
    }
    
    
    export { Foo };

    That was the second solution, as you didn't like the first one because it confused TypeScript in some way. It used an extra variable to get to const since there is no way to do that directly in TypeScript:

    enum Foo_ {...};
    const Foo = Object.freeze(Foo_);