just pulled the public branch and did the "make install" for linux, got hit with the following errors.
make install -f simulator.mk
make[1]: Entering directory '/home/loki/Projects/moddable/build/makefiles/lin'
simulator debug : cc main.c
/home/loki/Projects/moddable/build/simulator/lin/main.c: In function ‘onApplicationOpen’:
/home/loki/Projects/moddable/build/simulator/lin/main.c:434:99: warning: ‘.xsa'’ directive output may be truncated writing 5 bytes into a region of size between 3 and 8193 [-Wformat-truncation=]
434 | snprintf(cmd, sizeof(cmd), "/bin/cp -p \'%s\' \'%s.xsa\'", path, gxConfigPath);
| ^~
/home/loki/Projects/moddable/build/simulator/lin/main.c:434:49: note: ‘snprintf’ output between 21 and 8211 bytes into a destination of size 8208
434 | snprintf(cmd, sizeof(cmd), "/bin/cp -p \'%s\' \'%s.xsa\'", path, gxConfigPath);
| ^~~~~~~~~~~~~~~~~~
/home/loki/Projects/moddable/build/simulator/lin/main.c:427:99: warning: ‘.so'’ directive output may be truncated writing 4 bytes into a region of size between 3 and 8193 [-Wformat-truncation=]
427 | snprintf(cmd, sizeof(cmd), "/bin/cp -p \'%s\' \'%s.so\'", path, gxConfigPath);
| ^~~
/home/loki/Projects/moddable/build/simulator/lin/main.c:427:49: note: ‘snprintf’ output between 20 and 8210 bytes into a destination of size 8208
427 | snprintf(cmd, sizeof(cmd), "/bin/cp -p \'%s\' \'%s.so\'", path, gxConfigPath);
| ^~~~~~~~~~~~~~~~~~~
Assembler messages:
Fatal error: can't create /home/loki/Projects/moddable/build/tmp/lin/debug/simulator/main.o: No such file or directory
make[1]: [simulator.mk:59: /home/loki/Projects/moddable/build/tmp/lin/debug/simulator/main.o] Error 1
make[1]: Leaving directory '/home/loki/Projects/moddable/build/makefiles/lin'
make: [makefile:66: install] Error 2
@cmidgley – a fix the issue you ran into, being unable to create a Proxy for a host function primitive, will be pushed in a couple days. Meanwhile, if you want to give it a try before that, in xsProxy.c after line 780 add these lines:
#ifdef mxHostFunctionPrimitive
if ((mxArgc > 0) && (mxArgv(0)->kind == XS_HOST_FUNCTION_KIND))
fxToInstance(the, mxArgv(0));
if ((mxArgc > 1) && (mxArgv(1)->kind == XS_HOST_FUNCTION_KIND))
fxToInstance(the, mxArgv(1));
#endifThere's a corresponding change for Proxy.revocable, but it doesn't sound like you are using that (yet!).
Using Instrumentation to get live info on slots - I'm finding the doc and the first example are wrong, as the numbers are not correct (for example, doc says 11 is SlotHeapSize yet it is actually 14 on ESP32). I see in xsuse it scans to find the correct number - that's how I got to 14. What's the story here?
Also, the type definition for it is very difficult to use. The type InstrumentationOffset is not exported, and really only limits a numeric value and enforces it to be 1-16 (even though different numbers are used). And you can't use the actual names as they are only types and not exported constants. Same issue with WiFi (and perhaps others) on their type definitions. Just an FYI for now, but I think the fix (if instrumentation numbers are static and don't need to be scanned) is to export constants from the .js file and declare them in the .d.ts file (perhaps using a enum, not sure yet). If instrumentation numbers are dynamic, I've got no good ideas for typing ... maybe a dictionary lookup method in .js that does the scan and return the correct number for the instrumentation?
xsuse on macOS, ESP8266, and ESP32. All results look reasonable. Here's the start of output from ESP32:Boolean: 0 bytes
Number: 0 bytes
String `String in ROM`: 0 bytes
String `fromCharCode(32)`: 12 chunk bytes = 12 bytes
String `String in ` + 'RAM': 24 chunk bytes = 24 bytes
Object {}: 1 slots = 16 bytes
Object {x: 1}: 2 slots = 32 bytes
Object {x: 1, y: 2}: 3 slots = 48 bytes
Object {x: 1, y: 2, z: 3}: 4 slots = 64 bytes
Date: 2 slots = 32 bytes
BigInt 1n: 12 chunk bytes = 12 bytes
BigInt 100000000001n: 16 chunk bytes = 16 bytes
Function () => {}: 7 slots = 112 bytes
...Sorry if I was unclear - xsuse works fine, and that's how I figured out the numbers. The documentation, TypeScript file and the other example are all wrong (or at least not portable, as they are wrong on ESP32).
I do see how to find the base of instrumentation to get slots and instrumentation that follows (and I have that working). I found that before that, System Free Memory, was also offset wrong... Just wondering how I should approach accessing any of the instrumentation - for example, is the base of instrumentation (1) always correct until a particular index, and then we have to scan backwards from the end to find the rest? How would I find System Free Memory or Timers, for example?
@phoddie As you know, I've been working on out-of-memory issues, focused on getting preload working. I got much of Instrumentation working (still don't have all the indexes right, see prior comment) and found I needed to use another WeakMap.
I'm now crashing in xsMapSet.js with an exception here. Spent the last two days trying to debug why and finally got a small fragment showing the crash - it's due to preload being used on a class that gets put into a WeakMap. See this gist for the files that recreate the failure. This is a blocking issue for me as I can't run without preload but crash with it. Hopefully there is a simple solution, but it's beyond me to figure out.
yes,
In file included from /home/loki/.espressif/tools/riscv32-esp-elf/esp-2021r2-8.4.0/riscv32-esp-elf/riscv32-esp-elf/sys-include/sys/reent.h:14,
from /home/loki/Projects/esp-idf/components/newlib/platform_include/sys/reent.h:17,
from /home/loki/.espressif/tools/riscv32-esp-elf/esp-2021r2-8.4.0/riscv32-esp-elf/riscv32-esp-elf/sys-include/sys/errno.h:11,
from /home/loki/.espressif/tools/riscv32-esp-elf/esp-2021r2-8.4.0/riscv32-esp-elf/riscv32-esp-elf/sys-include/errno.h:9,
from /home/loki/Projects/esp-idf/components/newlib/platform_include/errno.h:18,
from /home/loki/Projects/moddable/xs/includes/xs.h:130,
from /home/loki/Projects/moddable/xs/includes/xsmc.h:41,
from /home/loki/Projects/moddable/modules/pins/digital/digital.c:21:
/home/loki/.espressif/tools/riscv32-esp-elf/esp-2021r2-8.4.0/riscv32-esp-elf/lib/gcc/riscv32-esp-elf/8.4.0/include/stddef.h:216:23: error: conflicting types for 'size_t'
typedef SIZE_TYPE size_t;
^~
In file included from /home/loki/Projects/moddable/xs/includes/xsmc.h:41,
from /home/loki/Projects/moddable/modules/pins/digital/digital.c:21:
/home/loki/Projects/moddable/xs/includes/xs.h:113:21: note: previous declaration of 'size_t' was here
typedef uint32_t size_t;
^~
make: * [/home/loki/Projects/moddable/build/tmp/esp32/esp32c3/debug/helloworld/makefile:760: /home/loki/Projects/moddable/build/tmp/esp32/esp32c3/debug/helloworld/digital.c.o] Error 1
I just commented out
// #if defined(ets)
// typedef uint32_t size_t;
// #endif
// typedef uint32_t size_t;
// #endif'''
#if defined(__ets__) && !defined(ESP32) won't define it on ESP32 targets. Would that work on the RISC-V target?
Once again a crazy question.... Is there any way to cause code executed during preload to execute after all modules have been preloaded?
Background: I have unit test files (Jest-like) that if I include in the Manifest they are detected and executed. I used to use Modules to find them, but now I am preloading them - which is really cool because all sorts of overhead gets absorbed into flash. As each module is preloaded, when the Jest describe method is execute, I register the test suite. Once all modules are preloaded, I'd like to freeze that object - but I can't do it in main (it runs before they do) and I don't see a way to cause code to execute after they have been loaded. There are ways around this (I could have a module that is just for loading tests), but that has the side effect that you have to keep it and the manifest synchronized or it will fail to work.
Also - can you confirm that slot/chunk allocation used during preload, even if they garbage collect out, will keep the "high water" mark of the slot and chunk usage when the code executes? What I think I see happening is after a big preload (the test stuff above) my slot and chunk numbers are decent (say 8K and 2K), but the allocated size is 80K and 20K. The problem is that when the code runs, it needs less slots and more chunk space available, which garbage collection won't help with.
execute preload code after modules have been loaded
Preload is based on modules and recursive. If you have code in Module A you want to run only after Module B is loaded, add an import statement to Module A for Module B to make that dependency visible to the language . Then, add Module A to the preload list. When Module A is preloaded, the dependency on Module B will be resolved first causing Module B to preload, then the top level of Module A will be executed. That's standard JavaScript module loading behavior.
Yeah - that's what I meant by "a module just for loading tests", but was hoping to avoid having to maintain both the manifest file and that separate include file anytime I change my tests. But it is what it is, and I'll work with it.
FYI - I have the WebSocket working great now. I have a test suite with 15 tests that hit all sorts of tests (connection loss, early cancel, small messages, large multi-fragment messages, etc) and it's passing all after GB's of testing. Thanks for the hint about having to read all - was a pain to implement, but it's all good now. It's not a trivial change, so not sure you are going to want it ... but in the coming days I'll do a pull and let you decide.
Peter - I'm trying to reduce all the warnings I get from the linker on preload. Many I understand, but I'm struggling with a few. One in particular are typescript enums. I've been able to take an enum and freeze it (and will have a CDV update to support this), such as:
enum MyEnum {
someValue = 1
}
(<unknown>MyEnum) = Object.freeze(MyEnum);This appears to successfully freeze the enum, and the warnings for it go away.
However when I use the enum, such as:
if (myValue == MyEnum.someValue)
...I get the linker warning:
MyEnum: no constIt's not a big deal from a slot perspective, but I'm getting so many warnings that it's hard to find the important ones.
MyEnum is not const. It isn't the object that the warning is for, but the variable the object is assigned to. Recall that in compileDataView we allow the original variable the enum is assigned to (which is not const) get garbage collected and export a reference to it, so this C enum:
#pragma language(typescript)
enum Foo {
one = 1,
two = 2
};outputs this TypeScript:
enum Foo {
one = 1,
two = 2,
}
export { Foo };That was the second solution, as you didn't like the first one because it confused TypeScript in some way. It used an extra variable to get to const since there is no way to do that directly in TypeScript:
enum Foo_ {...};
const Foo = Object.freeze(Foo_);