Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Peter Hoddie
    @phoddie
    Another possibility is that you have configured the initial slot and chunks in the creation section of the manifest to those sizes. We leave those small by default so that the balance between slots and chunks is found by runtime use.
    Chris
    @cmidgley
    Thanks - I'll dive deeper into it. I'm guessing the "execute preload code after modules have been loaded" question is a no... but can you confirm there is no trick to cause a module to be guaranteed to be last to preload or allow code to execute after all preloads are done (all during preload, so I can do a freeze)?
    Peter Hoddie
    @phoddie

    execute preload code after modules have been loaded

    Preload is based on modules and recursive. If you have code in Module A you want to run only after Module B is loaded, add an import statement to Module A for Module B to make that dependency visible to the language . Then, add Module A to the preload list. When Module A is preloaded, the dependency on Module B will be resolved first causing Module B to preload, then the top level of Module A will be executed. That's standard JavaScript module loading behavior.

    Chris
    @cmidgley

    Yeah - that's what I meant by "a module just for loading tests", but was hoping to avoid having to maintain both the manifest file and that separate include file anytime I change my tests. But it is what it is, and I'll work with it.

    FYI - I have the WebSocket working great now. I have a test suite with 15 tests that hit all sorts of tests (connection loss, early cancel, small messages, large multi-fragment messages, etc) and it's passing all after GB's of testing. Thanks for the hint about having to read all - was a pain to implement, but it's all good now. It's not a trivial change, so not sure you are going to want it ... but in the coming days I'll do a pull and let you decide.

    Peter Hoddie
    @phoddie
    I guess I don't understand what you are trying to do with the tests manifest to offer a helpful suggestion.
    Chris
    @cmidgley
    Not sure it's worth taking up more of your time - I can work with it.
    Peter Hoddie
    @phoddie
    Naive question: if you want a module to run after all other modules have preloaded, why not just put it at the end of the preload array in your top-level manifest?
    Chris
    @cmidgley
    Ummm.... because I didn't think of that? I'll give it a try!
    Chris
    @cmidgley

    Peter - I'm trying to reduce all the warnings I get from the linker on preload. Many I understand, but I'm struggling with a few. One in particular are typescript enums. I've been able to take an enum and freeze it (and will have a CDV update to support this), such as:

    enum MyEnum {
        someValue = 1
    }
    (<unknown>MyEnum) = Object.freeze(MyEnum);

    This appears to successfully freeze the enum, and the warnings for it go away.

    However when I use the enum, such as:

    if (myValue == MyEnum.someValue)
       ...

    I get the linker warning:

    MyEnum: no const

    It's not a big deal from a slot perspective, but I'm getting so many warnings that it's hard to find the important ones.

    Peter Hoddie
    @phoddie

    MyEnum is not const. It isn't the object that the warning is for, but the variable the object is assigned to. Recall that in compileDataView we allow the original variable the enum is assigned to (which is not const) get garbage collected and export a reference to it, so this C enum:

    #pragma language(typescript)
    
    enum Foo {
      one = 1,
      two = 2
    };

    outputs this TypeScript:

    enum Foo {
       one = 1,
       two = 2,
    }
    
    
    export { Foo };

    That was the second solution, as you didn't like the first one because it confused TypeScript in some way. It used an extra variable to get to const since there is no way to do that directly in TypeScript:

    enum Foo_ {...};
    const Foo = Object.freeze(Foo_);
    Chris
    @cmidgley
    Ahhh.... I see. Thanks!
    Chris
    @cmidgley
    Sadly that doesn't work either (*&$# typescript). I can do as you recommend (create enum with altered name, and then create a new const for it), but the new element is a value and not a type, so usage of the enum fails with TypeScript errors (Foo.one reports 'Foo' refers to a value, but is being used as a type here. Did you mean 'typeof MessageType'?).
    Chris
    @cmidgley

    Good news - I've got preload now working on my unit tests so they now get the majority of their work done during preload! Reduced slots by around 20K bytes for a typical test suite. I separated all members that need to be writable at runtime into a writable object within each class, and then got the combination of preload array order and preload code execution to work so I then do a recursive scan on the objects (that define all test suites, test cases, etc) and freeze the non-writable objects.

    The side effect is I get hundreds of linker warnings, as all writable members warn (per test case) and tests themselves have unfrozen elements. Makes the enum issue seem trivial!

    Getting close on preload (wow it's been a ton of work). I've had to eliminate use of Mocking until that proxy WeakMap bug is resolved, but that's let me get closer to getting things working (even though many tests can't be done). Soon I should be able to see if I'm out of the woods on slots / chunks or not...!

    Peter Hoddie
    @phoddie
    Very nice. (Apart from TypeScript's little annoyance.)
    Michael Kellner
    @mkellner
    @LokiMetaSmith in xs/platforms/esp/xsHost.c, please try changing INSTRUMENT_CPULOAD to 0.
    Peter Hoddie
    @phoddie
    FYI – we had a long discussion today about WeakMap with keys in ROM. Obviously it shouldn't crash, and we'll fix that. Actually making it work is complicated because of how the weak map is represented internally. That representation gives amazing performance, even for massively huge weak maps, so we aren't anxious to undo that. But.... we have a couple of ideas on how to allow ROM keys to work. More in a day or so, with any luck.
    Chris
    @cmidgley
    Thanks, Peter. I appreciate the time and effort.
    Michael Kellner
    @mkellner
    @LokiMetaSmith - ignore the above. We found a better solution. With the change (and your branch), I was able to build and run "helloworld" connecting to xsbug with the esp32c3. I was also able to run some wifi examples.
    The change was pushed to Moddable-OpenSource this morning.
    Lawrence R Kincheloe
    @LokiMetaSmith
    o, my! ok, I updated the latest branch, created yet another new branch, merged the changes in debug into it and yes I do seem to have a working HelloWorld!
    alright, it probably needs a lot more testing, but it's looking promising! I'll now port the custom board I built and keep on keeping on.
    Peter Hoddie
    @phoddie
    🎉🎉🎉
    Michael Kellner
    @mkellner
    That's great to hear!
    Chris
    @cmidgley

    The Modules module currently does not work with preload. Is it possible (and not difficult) to enable Modules to work at preload and locate all modules (using host) and use Modules.importNow to import them (which would then pull them into preload)? Basically a runtime way to define preload.

    The problem I'm trying to solve for isn't so much the runtime control over preload (though I thought that was potentially an elegant solution to multiple problems), but that with preload I've lost the ability to know the name of the module that is being loaded when module-scope code executes (before, I could locate a unit test module using Modules.host, track it's module name, and importNow which would cause the Jest module scope code to execute and register tests mapped to the module name). Now, understandably, modules load outside of my runtime control and therefore all module-scope code executes without any ability to track the module that is executing.

    Peter Hoddie
    @phoddie
    Preload loads JavaScript modules during the build. Therefore, the modules run on your computer, not the target device. Consequently, they cannot call host functions because those may be written for a different instruction set, may be written for a different operating system, and and may depend on hardware resources or APIs that are not present. That's why not all modules may be preloaded. The Modules module is one of those.
    Peter Hoddie
    @phoddie
    So, the Modules module is not an option here. The only native code that runs is XS built-ins, of course, since XS can guarantee that it provides the same functions at build time and execution time. Dynamic import isn't an option because it is asynchronous and modules loaded at preload are synchronous. It might be possible to use importNow on a Compartment instance to synchronously load a module during preload. I don't have time to try it this morning.
    Chris
    @cmidgley
    No worries - don't take any more time on this. Just thought I'd check. I just got a workaround - a bit ugly but seems to work fine.
    Peter Hoddie
    @phoddie
    @cmidgley – here's a bit of trivia for future reference. You asked about how to run code after preload completes. It turns out there's a simple solution: use a Promise:
    Promise.resolve().then(() => {
        // this code runs after all preloads complete but before lockdown of the firmware image
    });
    Chris
    @cmidgley
    Interesting! I'll give it a try (though I have found a workaround that is working ... using the last entry in the preload array as you suggested). This might simplify the manifests though so I'll give it a whirl. thx.
    Lawrence R Kincheloe
    @LokiMetaSmith
    there's a typo, #define ICACHE_XS6RO2_ATTR attribute((section(".rodata.xs6ro2"))) attribute((aligned(4)))
    #define ICACHE_XS6STRING_ATTR __attribute((section(".rodata.str1.4"))) __attribute__((aligned(4)))

    it was listed as

    define ICACHE_XS6STRING_ATTR attribute((section(".rodata.str1.4"))) attribute__((aligned(4)))

    but should be

    define ICACHE_XS6STRING_ATTR attribute((section(".rodata.str1.4"))) attribute((aligned(4)))

    still doesn't fix the issue though

    I gotta figure out formatting in gitter, this is horrible...
    Peter Hoddie
    @phoddie
    Good catch. GCC attributes like that are really difficult to read (and apparently to write reliably too!).
    What's the issue you are looking into?
    Lawrence R Kincheloe
    @LokiMetaSmith
    trying to stage the esp32c3 pull request
    but there's code I commented out to make it work, and figuring out why it isn't working is well, entertaining
    Peter Hoddie
    @phoddie
    Could be. The section stuff with GCC has always been a little mysterious. The section attributes are necessary on ESP8266 but I think on ESP32 they aren't really necessary (different versions of GCC, different CPU capabilities). The attribute is still relevant. Maybe you can eliminate the section attribute for your target (maybe we can do that for ESP32 in general?).
    moddable getting love during jsconf budapest from nick!
    Peter Hoddie
    @phoddie
    That looks so great! Thanks for posting. I'm going to have to make time to watch that today.
    Lawrence R Kincheloe
    @LokiMetaSmith
    heads up, esp-idf version 5 has a bunch of migration issues. https://docs.espressif.com/projects/esp-idf/en/latest/esp32/migration-guides/index.html
    Peter Hoddie
    @phoddie
    No doubt! Every major version does. We've survived all the other transitions. We'll get through this one too.
    Peter Hoddie
    @phoddie
    (I just hope audio isn't broken again. Or, at least, that v5 finally correctly fixes the ADC audio woes of ESP-IDF v4.4. @meganetaaan and I have wrestled with that enough.)
    Lawrence R Kincheloe
    @LokiMetaSmith

    I'm still getting error: conflicting types for 'size_t' typedef __SIZE_TYPE__ size_t;
    feeling kind of stuck.

    https://github.com/juniper-garden/moddable/tree/testing_tag3

    Peter Hoddie
    @phoddie
    Will try to take a look today.
    Michael Kellner
    @mkellner
    @lokimetasmith, using your testing_tag3 branch... at line 112 in xs/includes/xs.h change from #if defined(__ets__) to #if defined(__ets__) && !ESP32. And in xs/platforms/esp/xsHost.h, change the vPortEnter/ExitCritical to portENTER_/portEXIT_CRITICAL to get past the next hurdle.
    Michael Kellner
    @mkellner
    Thanks to @LokiMetaSmith, there is now early support for the esp32c3.
    Michael Kellner
    @mkellner
    There are a number of c3 boards out there. I have been using the ESP32-C3-DevKitM-1 from Espressif. For the AI Thinker modules like ESP-C3_32S-Kit or ESP-C3-12F-Kit, I had to set UPLOAD_SPEED to 460800.