by

Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Modestas Bunokas
    @KT819GM
    it’s for ESP8266 mainly because of ram
    M.K.
    @mk-pmb
    oh ok thanks! then I shall have a look again. still, most of the vendor modules are in C it seems.
    Modestas Bunokas
    @KT819GM
    besides, if you leave filesystem size and boundaries untouched, you even can update firmware and it will leave your spiffs untouched
    they are working on the direction to replace as many C modules as possible to Lua ones
    from what I see
    M.K.
    @mk-pmb
    yeah, under optimal circumstances. it's a bit like knowing how to backup your partition table and a binary image of your OS boot drive makes it a lot easier to experiment with exotic boot loaders. people new to the field rightfully assume a risk of data loss if they try it.
    transfering most modules to LUA sounds like a good idea independently. still, wouldn't it be great to have the capability to change selection of C modules after firmware compile time? would you consider it useless? would it be useful but way to hard to do?
    if I tried myself, could I start by just adding a buffer overflow vuln and then have some exploit framework inject the module code, or is a firmware fundamentally different in that attack vector?
    oh I see that would not survive reboots.
    M.K.
    @mk-pmb
    I should probably leave the task of fetching the bytecode to a LUA script, and try to write a C module that just injects bytecode.
    if it works as planned, people could even download it via wifi without reboot and without requiring flash file memory. (unloading and cleanup might be trickier, dunno.)
    M.K.
    @mk-pmb
    Research notes in case someone else wants to continue later:
    M.K.
    @mk-pmb
    Has anyone tried to load the C modules via LUA require yet? I'm rather noob with both C and LUA but it seems LUA is especially designed to help with this.
    Gregor Hartmann
    @HHHartmann
    It sure would be nice to be able to load c libraries at runtime. But keep in mind that there are only about 40k of ram available on the esp8266. LFS was invented to keep the lua code out of ram to have some more space for application data.
    There is a partition table containing entries for SPIFFS, firmware, LFS and others. As long as your new firmware fits in the existing partition the files on SPIFFS will survive.
    Gregor Hartmann
    @HHHartmann
    There are plans to implement updating the firmware over the air. But this will take some more time.
    M.K.
    @mk-pmb
    Is there way more overhead when loading the lib at runtime, rather than compiling it into the firmware? I'd have thought it takes just a table of function name + pointer for each extern function, can't imagine that would be very much data.
    oh did you mean that any C code loaded has to go to RAM, whereas LUA code can reside on flash, and thus LUA code will always be cheaper than a C module?
    in that case it might be worth trying to invest a few of the precious RAM bytes in a swap mechanism.
    … which probably won't work without hardware support like a Memory Management Unit.
    Modestas Bunokas
    @KT819GM
    LFS play kind a swap (read-only) role for Lua. About C modules, still, it’s easy to build firmware and easy to reload it. You barely need to change C modules at all to be honest. On my daily test unit I have 35 C modules enabled and I still get more than 32K free ram on latest dev. Firmware compile / chip erase / firmware write / LFS load / init.lua is a one-liner bash script.
    M.K.
    @mk-pmb
    ok then I'm probably overthinking it and we should just rewrite the beginner's guide some time to make it sound less complicated. after all, having to decide the module selection upfront, before I had any idea about MCUs at all, was what had turned me away at first. only came back because MicroPython still doesn't have the features I want.
    back then especially because I had no way to export/import my module selection in the cloud build service to incrementally refine my selection, I'd have had to fill the entire form again for each attempt. at least that's going to be easier soon.
    Modestas Bunokas
    @KT819GM
    You sound like a linux guy, isn't native or docker ways looks to you complicated? Mostly 2 files need editing before compile: user_config.h and user_modules.h everything well commented inside files
    M.K.
    @mk-pmb
    Indeed I am a "linux guy", but also a user experience activist. I already made the Github Action that powers https://github.com/mk-pmb/nodemcu-firmware-daily-vanilla/ to make at least that level of ease available to all Githubbers independent of their OS preference. and I'm planning on building a noob-friendly module selection wizard as well.
    Terry Ellison
    @TerryE
    Neither the xtensa toolchain nor the ESP loaders support dynamic libraries. This is an IoT device with very limited RAM and flash memory.
    M.K.
    @mk-pmb
    I know they have few memory of each type, thus the cheap price. I just don't grasp the resulting consequences and limitations in detail.
    when you say the toolchain doesn't support dynamic libs, does it mean there's no known compiler to make shared object files in the first place?
    M.K.
    @mk-pmb
    would it further imply that even if I'd find a compiler that can produce such shared object file, our firmware's LUA's package.loadlib() would be unable to load it?
    Modestas Bunokas
    @KT819GM
    Maybe better approach for this issue we discussing now would be plugin for popular ide like VSCode which would allow you to choose dependencies (parser of config file) and saving it. And for sure ability to erase/write and so on. Would be something like platformio, just for nodemcu?
    M.K.
    @mk-pmb
    I would not be in that approach's target audience, as most of the computers I use for experimenting with my MCU are too weak for VSCode. I don't know platformio, so I can't comment on that.
    ("weak" as in what's left of resources after firefox and VLC eat their share. :-) )
    Modestas Bunokas
    @KT819GM
    Jeeez, if it can run firefox ... vscode should not be a problem at all
    M.K.
    @mk-pmb
    I prefer to have the manual open in Firefox while I'm writing my code. :)
    also this Gitter
    M.K.
    @mk-pmb
    after firmware upload, the greeting message says "LFS: 0x0", is that the size of my LFS? If so, why? because I built it with #define LUA_FLASH_STORE 0x10000
    Modestas Bunokas
    @KT819GM
    You have to define it’s size in the same file I think
    M.K.
    @mk-pmb
    oh, I thought LUA_FLASH_STORE is the size. I'll read again.
    Modestas Bunokas
    @KT819GM
    It is, I’m wrong, can’t check code now on phone
    M.K.
    @mk-pmb
    then at least thanks for verifying the option name. :-)
    Modestas Bunokas
    @KT819GM
    have you done erase before writing new FW?
    M.K.
    @mk-pmb
    nope, will try.
    M.K.
    @mk-pmb
    ran esptool.py erase_flash, re-uploaded firmware, still "LFS: 0x0". I hope it's the amount used then.
    the message is almost for sure from tools/update_buildinfo.sh ln 60, BUILDINFO_TO_STR(BUILDINFO_LFS), I wish someone would have used a more descriptive identifier there. gonna trace it and send a PR.
    Modestas Bunokas
    @KT819GM
    I’m getting LFS: 0x20000 after erase / write
    M.K.
    @mk-pmb
    oh. then at least I know it's the right place for my PR.
    Modestas Bunokas
    @KT819GM
    I’m using spiffs size of 0x40000 and lfs size of 0x20000, and behaviour looks okay for me with latest dev branch
    M.K.
    @mk-pmb
    oh maybe I forgot to activate spiffs
    btw śizes are in bytes, right?
    found "#define BUILD_SPIFFS", it's already active. would you mind sharing your config for comparison?