Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Peter Hoddie
    @phoddie
    __ets__ is defined in the make file for ESP32. It should not be removed as there is code that depends on it.
    Is there a way to check for RISC-V builds of ESP-IDF? That would be way to add a check here to cleanly eliminate the error.
    Lawrence R Kincheloe
    @LokiMetaSmith
    hmm, ok let me step back. usually size_t is defined in the standard library, I'm not sure why its being redefined in xs.h. it looks like that line of code was recently added as well.
    Peter Hoddie
    @phoddie
    Yes, it usually is. But not all platforms do. The ESP8266 builds, for example, don't.
    Maybe it is only required on ESP8266 builds. Updating the check to #if defined(__ets__) && !defined(ESP32) won't define it on ESP32 targets. Would that work on the RISC-V target?
    Lawrence R Kincheloe
    @LokiMetaSmith
    looks like the issue is that when size_t is defined, it needs a #define __size_t
    actually it's #define size_t 1
    Lawrence R Kincheloe
    @LokiMetaSmith
    or maybe not, anyways there's a catch in the riscv for if size_t is defined, just gotta get it to trigger
    Peter Hoddie
    @phoddie
    Does the suggested change above not work?
    Prateek Sharma
    @message2prateek
    Hello, just discovered Moddable SDK and I'm a beginner to the world of microcontrollers. Does anyone have an example to share for IR receiver and transmitter? I'll love to capture IR signals and control devices at home. Thanks.
    Peter Hoddie
    @phoddie
    Welcome @message2prateek! The Moddable SDK examples directory has IR transmitter and IR receiver projects. These use the RMT module of the ESP32. That looks like it might be a good starting point for you.
    Chris
    @cmidgley

    Once again a crazy question.... Is there any way to cause code executed during preload to execute after all modules have been preloaded?

    Background: I have unit test files (Jest-like) that if I include in the Manifest they are detected and executed. I used to use Modules to find them, but now I am preloading them - which is really cool because all sorts of overhead gets absorbed into flash. As each module is preloaded, when the Jest describe method is execute, I register the test suite. Once all modules are preloaded, I'd like to freeze that object - but I can't do it in main (it runs before they do) and I don't see a way to cause code to execute after they have been loaded. There are ways around this (I could have a module that is just for loading tests), but that has the side effect that you have to keep it and the manifest synchronized or it will fail to work.

    Also - can you confirm that slot/chunk allocation used during preload, even if they garbage collect out, will keep the "high water" mark of the slot and chunk usage when the code executes? What I think I see happening is after a big preload (the test stuff above) my slot and chunk numbers are decent (say 8K and 2K), but the allocated size is 80K and 20K. The problem is that when the code runs, it needs less slots and more chunk space available, which garbage collection won't help with.

    Peter Hoddie
    @phoddie
    Preload runs on your computer, not the MCU. During that time, there is no limit on how much memory it can allocate (beyond any limit enforced by your computer's OS). Before the VM state is captured for writing the firmware, a GC is performed to eliminate anything not in use (primitives, objects, code, etc). No state about slot of chunk use during preload is is put into the firmware image. When the image is used on the device, it is with a fresh heap.
    Main point: any allocations you see on the device are a result of code that ran on the device, not code that ran during preload.
    Peter Hoddie
    @phoddie
    Another possibility is that you have configured the initial slot and chunks in the creation section of the manifest to those sizes. We leave those small by default so that the balance between slots and chunks is found by runtime use.
    Chris
    @cmidgley
    Thanks - I'll dive deeper into it. I'm guessing the "execute preload code after modules have been loaded" question is a no... but can you confirm there is no trick to cause a module to be guaranteed to be last to preload or allow code to execute after all preloads are done (all during preload, so I can do a freeze)?
    Peter Hoddie
    @phoddie

    execute preload code after modules have been loaded

    Preload is based on modules and recursive. If you have code in Module A you want to run only after Module B is loaded, add an import statement to Module A for Module B to make that dependency visible to the language . Then, add Module A to the preload list. When Module A is preloaded, the dependency on Module B will be resolved first causing Module B to preload, then the top level of Module A will be executed. That's standard JavaScript module loading behavior.

    Chris
    @cmidgley

    Yeah - that's what I meant by "a module just for loading tests", but was hoping to avoid having to maintain both the manifest file and that separate include file anytime I change my tests. But it is what it is, and I'll work with it.

    FYI - I have the WebSocket working great now. I have a test suite with 15 tests that hit all sorts of tests (connection loss, early cancel, small messages, large multi-fragment messages, etc) and it's passing all after GB's of testing. Thanks for the hint about having to read all - was a pain to implement, but it's all good now. It's not a trivial change, so not sure you are going to want it ... but in the coming days I'll do a pull and let you decide.

    Peter Hoddie
    @phoddie
    I guess I don't understand what you are trying to do with the tests manifest to offer a helpful suggestion.
    Chris
    @cmidgley
    Not sure it's worth taking up more of your time - I can work with it.
    Peter Hoddie
    @phoddie
    Naive question: if you want a module to run after all other modules have preloaded, why not just put it at the end of the preload array in your top-level manifest?
    Chris
    @cmidgley
    Ummm.... because I didn't think of that? I'll give it a try!
    Chris
    @cmidgley

    Peter - I'm trying to reduce all the warnings I get from the linker on preload. Many I understand, but I'm struggling with a few. One in particular are typescript enums. I've been able to take an enum and freeze it (and will have a CDV update to support this), such as:

    enum MyEnum {
        someValue = 1
    }
    (<unknown>MyEnum) = Object.freeze(MyEnum);

    This appears to successfully freeze the enum, and the warnings for it go away.

    However when I use the enum, such as:

    if (myValue == MyEnum.someValue)
       ...

    I get the linker warning:

    MyEnum: no const

    It's not a big deal from a slot perspective, but I'm getting so many warnings that it's hard to find the important ones.

    Peter Hoddie
    @phoddie

    MyEnum is not const. It isn't the object that the warning is for, but the variable the object is assigned to. Recall that in compileDataView we allow the original variable the enum is assigned to (which is not const) get garbage collected and export a reference to it, so this C enum:

    #pragma language(typescript)
    
    enum Foo {
      one = 1,
      two = 2
    };

    outputs this TypeScript:

    enum Foo {
       one = 1,
       two = 2,
    }
    
    
    export { Foo };

    That was the second solution, as you didn't like the first one because it confused TypeScript in some way. It used an extra variable to get to const since there is no way to do that directly in TypeScript:

    enum Foo_ {...};
    const Foo = Object.freeze(Foo_);
    Chris
    @cmidgley
    Ahhh.... I see. Thanks!
    Chris
    @cmidgley
    Sadly that doesn't work either (*&$# typescript). I can do as you recommend (create enum with altered name, and then create a new const for it), but the new element is a value and not a type, so usage of the enum fails with TypeScript errors (Foo.one reports 'Foo' refers to a value, but is being used as a type here. Did you mean 'typeof MessageType'?).
    Chris
    @cmidgley

    Good news - I've got preload now working on my unit tests so they now get the majority of their work done during preload! Reduced slots by around 20K bytes for a typical test suite. I separated all members that need to be writable at runtime into a writable object within each class, and then got the combination of preload array order and preload code execution to work so I then do a recursive scan on the objects (that define all test suites, test cases, etc) and freeze the non-writable objects.

    The side effect is I get hundreds of linker warnings, as all writable members warn (per test case) and tests themselves have unfrozen elements. Makes the enum issue seem trivial!

    Getting close on preload (wow it's been a ton of work). I've had to eliminate use of Mocking until that proxy WeakMap bug is resolved, but that's let me get closer to getting things working (even though many tests can't be done). Soon I should be able to see if I'm out of the woods on slots / chunks or not...!

    Peter Hoddie
    @phoddie
    Very nice. (Apart from TypeScript's little annoyance.)
    Michael Kellner
    @mkellner
    @LokiMetaSmith in xs/platforms/esp/xsHost.c, please try changing INSTRUMENT_CPULOAD to 0.
    Peter Hoddie
    @phoddie
    FYI – we had a long discussion today about WeakMap with keys in ROM. Obviously it shouldn't crash, and we'll fix that. Actually making it work is complicated because of how the weak map is represented internally. That representation gives amazing performance, even for massively huge weak maps, so we aren't anxious to undo that. But.... we have a couple of ideas on how to allow ROM keys to work. More in a day or so, with any luck.
    Chris
    @cmidgley
    Thanks, Peter. I appreciate the time and effort.
    Michael Kellner
    @mkellner
    @LokiMetaSmith - ignore the above. We found a better solution. With the change (and your branch), I was able to build and run "helloworld" connecting to xsbug with the esp32c3. I was also able to run some wifi examples.
    The change was pushed to Moddable-OpenSource this morning.
    Lawrence R Kincheloe
    @LokiMetaSmith
    o, my! ok, I updated the latest branch, created yet another new branch, merged the changes in debug into it and yes I do seem to have a working HelloWorld!
    alright, it probably needs a lot more testing, but it's looking promising! I'll now port the custom board I built and keep on keeping on.
    Peter Hoddie
    @phoddie
    🎉🎉🎉
    Michael Kellner
    @mkellner
    That's great to hear!
    Chris
    @cmidgley

    The Modules module currently does not work with preload. Is it possible (and not difficult) to enable Modules to work at preload and locate all modules (using host) and use Modules.importNow to import them (which would then pull them into preload)? Basically a runtime way to define preload.

    The problem I'm trying to solve for isn't so much the runtime control over preload (though I thought that was potentially an elegant solution to multiple problems), but that with preload I've lost the ability to know the name of the module that is being loaded when module-scope code executes (before, I could locate a unit test module using Modules.host, track it's module name, and importNow which would cause the Jest module scope code to execute and register tests mapped to the module name). Now, understandably, modules load outside of my runtime control and therefore all module-scope code executes without any ability to track the module that is executing.

    Peter Hoddie
    @phoddie
    Preload loads JavaScript modules during the build. Therefore, the modules run on your computer, not the target device. Consequently, they cannot call host functions because those may be written for a different instruction set, may be written for a different operating system, and and may depend on hardware resources or APIs that are not present. That's why not all modules may be preloaded. The Modules module is one of those.
    Peter Hoddie
    @phoddie
    So, the Modules module is not an option here. The only native code that runs is XS built-ins, of course, since XS can guarantee that it provides the same functions at build time and execution time. Dynamic import isn't an option because it is asynchronous and modules loaded at preload are synchronous. It might be possible to use importNow on a Compartment instance to synchronously load a module during preload. I don't have time to try it this morning.
    Chris
    @cmidgley
    No worries - don't take any more time on this. Just thought I'd check. I just got a workaround - a bit ugly but seems to work fine.
    Peter Hoddie
    @phoddie
    @cmidgley – here's a bit of trivia for future reference. You asked about how to run code after preload completes. It turns out there's a simple solution: use a Promise:
    Promise.resolve().then(() => {
        // this code runs after all preloads complete but before lockdown of the firmware image
    });
    Chris
    @cmidgley
    Interesting! I'll give it a try (though I have found a workaround that is working ... using the last entry in the preload array as you suggested). This might simplify the manifests though so I'll give it a whirl. thx.
    Lawrence R Kincheloe
    @LokiMetaSmith
    there's a typo, #define ICACHE_XS6RO2_ATTR attribute((section(".rodata.xs6ro2"))) attribute((aligned(4)))
    #define ICACHE_XS6STRING_ATTR __attribute((section(".rodata.str1.4"))) __attribute__((aligned(4)))

    it was listed as

    define ICACHE_XS6STRING_ATTR attribute((section(".rodata.str1.4"))) attribute__((aligned(4)))

    but should be

    define ICACHE_XS6STRING_ATTR attribute((section(".rodata.str1.4"))) attribute((aligned(4)))

    still doesn't fix the issue though

    I gotta figure out formatting in gitter, this is horrible...
    Peter Hoddie
    @phoddie
    Good catch. GCC attributes like that are really difficult to read (and apparently to write reliably too!).
    What's the issue you are looking into?
    Lawrence R Kincheloe
    @LokiMetaSmith
    trying to stage the esp32c3 pull request
    but there's code I commented out to make it work, and figuring out why it isn't working is well, entertaining