xsuse on macOS, ESP8266, and ESP32. All results look reasonable. Here's the start of output from ESP32:Boolean: 0 bytes
Number: 0 bytes
String `String in ROM`: 0 bytes
String `fromCharCode(32)`: 12 chunk bytes = 12 bytes
String `String in ` + 'RAM': 24 chunk bytes = 24 bytes
Object {}: 1 slots = 16 bytes
Object {x: 1}: 2 slots = 32 bytes
Object {x: 1, y: 2}: 3 slots = 48 bytes
Object {x: 1, y: 2, z: 3}: 4 slots = 64 bytes
Date: 2 slots = 32 bytes
BigInt 1n: 12 chunk bytes = 12 bytes
BigInt 100000000001n: 16 chunk bytes = 16 bytes
Function () => {}: 7 slots = 112 bytes
...
Sorry if I was unclear - xsuse works fine, and that's how I figured out the numbers. The documentation, TypeScript file and the other example are all wrong (or at least not portable, as they are wrong on ESP32).
I do see how to find the base of instrumentation to get slots and instrumentation that follows (and I have that working). I found that before that, System Free Memory, was also offset wrong... Just wondering how I should approach accessing any of the instrumentation - for example, is the base of instrumentation (1) always correct until a particular index, and then we have to scan backwards from the end to find the rest? How would I find System Free Memory or Timers, for example?
@phoddie As you know, I've been working on out-of-memory issues, focused on getting preload working. I got much of Instrumentation working (still don't have all the indexes right, see prior comment) and found I needed to use another WeakMap.
I'm now crashing in xsMapSet.js with an exception here. Spent the last two days trying to debug why and finally got a small fragment showing the crash - it's due to preload being used on a class that gets put into a WeakMap. See this gist for the files that recreate the failure. This is a blocking issue for me as I can't run without preload but crash with it. Hopefully there is a simple solution, but it's beyond me to figure out.
yes,
In file included from /home/loki/.espressif/tools/riscv32-esp-elf/esp-2021r2-8.4.0/riscv32-esp-elf/riscv32-esp-elf/sys-include/sys/reent.h:14,
from /home/loki/Projects/esp-idf/components/newlib/platform_include/sys/reent.h:17,
from /home/loki/.espressif/tools/riscv32-esp-elf/esp-2021r2-8.4.0/riscv32-esp-elf/riscv32-esp-elf/sys-include/sys/errno.h:11,
from /home/loki/.espressif/tools/riscv32-esp-elf/esp-2021r2-8.4.0/riscv32-esp-elf/riscv32-esp-elf/sys-include/errno.h:9,
from /home/loki/Projects/esp-idf/components/newlib/platform_include/errno.h:18,
from /home/loki/Projects/moddable/xs/includes/xs.h:130,
from /home/loki/Projects/moddable/xs/includes/xsmc.h:41,
from /home/loki/Projects/moddable/modules/pins/digital/digital.c:21:
/home/loki/.espressif/tools/riscv32-esp-elf/esp-2021r2-8.4.0/riscv32-esp-elf/lib/gcc/riscv32-esp-elf/8.4.0/include/stddef.h:216:23: error: conflicting types for 'size_t'
typedef SIZE_TYPE size_t;
^~
In file included from /home/loki/Projects/moddable/xs/includes/xsmc.h:41,
from /home/loki/Projects/moddable/modules/pins/digital/digital.c:21:
/home/loki/Projects/moddable/xs/includes/xs.h:113:21: note: previous declaration of 'size_t' was here
typedef uint32_t size_t;
^~
make: * [/home/loki/Projects/moddable/build/tmp/esp32/esp32c3/debug/helloworld/makefile:760: /home/loki/Projects/moddable/build/tmp/esp32/esp32c3/debug/helloworld/digital.c.o] Error 1
I just commented out
// #if defined(ets)
// typedef uint32_t size_t;
// #endif
// typedef uint32_t size_t;
// #endif'''
#if defined(__ets__) && !defined(ESP32) won't define it on ESP32 targets. Would that work on the RISC-V target?
Once again a crazy question.... Is there any way to cause code executed during preload to execute after all modules have been preloaded?
Background: I have unit test files (Jest-like) that if I include in the Manifest they are detected and executed. I used to use Modules to find them, but now I am preloading them - which is really cool because all sorts of overhead gets absorbed into flash. As each module is preloaded, when the Jest describe method is execute, I register the test suite. Once all modules are preloaded, I'd like to freeze that object - but I can't do it in main (it runs before they do) and I don't see a way to cause code to execute after they have been loaded. There are ways around this (I could have a module that is just for loading tests), but that has the side effect that you have to keep it and the manifest synchronized or it will fail to work.
Also - can you confirm that slot/chunk allocation used during preload, even if they garbage collect out, will keep the "high water" mark of the slot and chunk usage when the code executes? What I think I see happening is after a big preload (the test stuff above) my slot and chunk numbers are decent (say 8K and 2K), but the allocated size is 80K and 20K. The problem is that when the code runs, it needs less slots and more chunk space available, which garbage collection won't help with.
execute preload code after modules have been loaded
Preload is based on modules and recursive. If you have code in Module A you want to run only after Module B is loaded, add an import statement to Module A for Module B to make that dependency visible to the language . Then, add Module A to the preload list. When Module A is preloaded, the dependency on Module B will be resolved first causing Module B to preload, then the top level of Module A will be executed. That's standard JavaScript module loading behavior.
Yeah - that's what I meant by "a module just for loading tests", but was hoping to avoid having to maintain both the manifest file and that separate include file anytime I change my tests. But it is what it is, and I'll work with it.
FYI - I have the WebSocket working great now. I have a test suite with 15 tests that hit all sorts of tests (connection loss, early cancel, small messages, large multi-fragment messages, etc) and it's passing all after GB's of testing. Thanks for the hint about having to read all - was a pain to implement, but it's all good now. It's not a trivial change, so not sure you are going to want it ... but in the coming days I'll do a pull and let you decide.
Peter - I'm trying to reduce all the warnings I get from the linker on preload. Many I understand, but I'm struggling with a few. One in particular are typescript enums. I've been able to take an enum and freeze it (and will have a CDV update to support this), such as:
enum MyEnum {
someValue = 1
}
(<unknown>MyEnum) = Object.freeze(MyEnum);This appears to successfully freeze the enum, and the warnings for it go away.
However when I use the enum, such as:
if (myValue == MyEnum.someValue)
...I get the linker warning:
MyEnum: no constIt's not a big deal from a slot perspective, but I'm getting so many warnings that it's hard to find the important ones.
MyEnum is not const. It isn't the object that the warning is for, but the variable the object is assigned to. Recall that in compileDataView we allow the original variable the enum is assigned to (which is not const) get garbage collected and export a reference to it, so this C enum:
#pragma language(typescript)
enum Foo {
one = 1,
two = 2
};outputs this TypeScript:
enum Foo {
one = 1,
two = 2,
}
export { Foo };That was the second solution, as you didn't like the first one because it confused TypeScript in some way. It used an extra variable to get to const since there is no way to do that directly in TypeScript:
enum Foo_ {...};
const Foo = Object.freeze(Foo_);enum with altered name, and then create a new const for it), but the new element is a value and not a type, so usage of the enum fails with TypeScript errors (Foo.one reports 'Foo' refers to a value, but is being used as a type here. Did you mean 'typeof MessageType'?).
Good news - I've got preload now working on my unit tests so they now get the majority of their work done during preload! Reduced slots by around 20K bytes for a typical test suite. I separated all members that need to be writable at runtime into a writable object within each class, and then got the combination of preload array order and preload code execution to work so I then do a recursive scan on the objects (that define all test suites, test cases, etc) and freeze the non-writable objects.
The side effect is I get hundreds of linker warnings, as all writable members warn (per test case) and tests themselves have unfrozen elements. Makes the enum issue seem trivial!
Getting close on preload (wow it's been a ton of work). I've had to eliminate use of Mocking until that proxy WeakMap bug is resolved, but that's let me get closer to getting things working (even though many tests can't be done). Soon I should be able to see if I'm out of the woods on slots / chunks or not...!
