Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Terry Ellison
    @TerryE
    I've only just got around to reimplementing the Lua apps in Lua5.3 on LFS! The source code is about half as long. :smile:
    Nicol Spies
    @NicolSpies
    I agree fully, what worries me is the lua NodeMCU userbase compared to the ESP-IDF RTOS userbase is shrinking. By moving on, the mover gains and everybody else loose out. Proficiency takes effort and dedication but as have been acknowledged many time on the group, there are things that are beyond the capabilities of the 8266 that is relatively easy on the 32.
    Your 8266 web server springs to mind
    So to jump into the deep end of RTOS and C at the age of 57, I sometimes wonder.
    I think that is the drawing of the 8266 and lua; to do things in a constrained environment that others do not even attempt.
    Terry Ellison
    @TerryE
    The real issue with the ESP8266 and Lua (or microPython) is the lack of RAM. LFS helps a lot. But this makes unrealistic to use RTOS on the 8266. In my case the Wemos D1 ESP32 boards are pretty much module swaps in my existing H/W and have similar pricing. The main user of the ESP8266 chip are the Sonoff and Shelly devices, and I would never consider a NodeMCU firmware for these. Tasmota makes far more sense to me.
    At some point we need to consider moving the ESP8266 branches into frozen support and switch focus to the ESP32 branch.
    Nicol Spies
    @NicolSpies
    Yes, I remember the pre-LFS days when I started with the 8266 starting up with less than 20kb if IRAM. I agree with moving everything over to the ESP32, I believe that is what Johny is working on at $work. It appears as such a different mindset that the risk might be that little of all the 8266 effort will be retained.
    Terry Ellison
    @TerryE
    I am going to support Johny on this.
    Nicol Spies
    @NicolSpies
    Espressif is bringing out new SoCs at such a rate that the 8266 feels like the 4004 processor
    Terry Ellison
    @TerryE
    Yup
    Nicol Spies
    @NicolSpies
    +1 on the support of Johny
    Terry Ellison
    @TerryE
    Or more the 8080 :laughing:
    Nicol Spies
    @NicolSpies
    I suppose the moving across will leave the NON-RTOS behind
    Terry Ellison
    @TerryE
    Yup. Its not so much ESP8266 vs ESP32, but more SDK vs IDE+RTOS is the killer from a support perspective.
    Nicol Spies
    @NicolSpies
    Yes, that is what I am seeing with the new pair of JTAG and RTOS glasses
    So I still believe that for contained environments like your smart-home, the 8266 is perfect, in the runaway stagecoach of cloud IoT it is required to switch just to stay on the board.
    Nicol Spies
    @NicolSpies
    I have to run, nice to chat, cheers
    Terry Ellison
    @TerryE
    Yup enjoy. Busy hosing and killing dead issues. :smiley:
    Nicol Spies
    @NicolSpies
    :thumbsup:
    Patrick Dorion
    @poorandunlucky
    @poorandunlucky Patrick, I can't see the repo. Is it public?

    @TerryE Sorry, I had the repo set to private in case I published my password, which is still a good precaution, so I made a public one... https://github.com/poorandunlucky/lfs-public

    sorry about that 😕

    also very sorry to hear that you were sick, the whole pandemic thing always seems like a real joke until someone you know has complications because of it... some of my mother's friends had problems with it, also, and i didn't know you had it, but i'm glad you pulled through 😊
    Patrick Dorion
    @poorandunlucky
    also copied what you said about globals, and RAM optimization... i didn't know you could get more than around 30K, i load a lot of modules to play with it on my desk like that, but knowing you can get up to 40 comfortably is going to be helpful as a target...
    Terry Ellison
    @TerryE
    Re passwords and other shared secrets, you should never embed them in source that you put in git. A common approach is to have a secrets.lua or secrets.json which you load at startup into local variables in RAM and use .gitexcludeto prevent this file from being added to the repo. However, remember that this will still leave these secrets in a form that can still be read from flash. There are ways to mitigate this but things can get quite complicated to do this properly.
    You need to do a threat assessment, and design your approach according. My preference is isolate my IoT devices on a control LAN interfacing to an RPi which runs a locked- down Linux OS and layer my security over standard Linux stacks.
    I'll take a look at the source.
    Terry Ellison
    @TerryE
    BTW, I've downloaded the ZIP of your repo so you can remove the public attribute
    Terry Ellison
    @TerryE
    @poorandunlucky, your RAM usage is high because the code isn't written with IoT footprint in mind. For example log.luais pretty verbose for what it does uses RAM where there is little to be gained from doing so:
    • module() and package.seeall are Lua 5.0 constructs which are deprecated in 5.1 and run inefficiently on NodeMCU. I am not even sure they work in Lua 5.3.
    • You shouldn't be printing errors -- that's just a good way to miss them in prod systems. Errors are just that and in Lua they should be thrown with an error() statement.
    • In general this code would be fine for running on a real system with lots of RAM, but its not a good fit for a IoT device with ~40Kb RAM. OK, you want a log(message,facility, level ) function and the ability to configure the level from 'DEBUG' to NONE with an option filename, but this IoT functionality doesn't need RAM arrays or this quite verbose logic.
    • for example the strtolvl2()logic could be coded in a dozen lines, run faster and use no RAM arrays:
      level = level:upper()
      if ( level == 'DEBUG' ) then return 7
      -- ...
      elseif level == 'EMERG' or level == 'EMERGENCY'  then return 0
      else error( 'Invalid error level specified: '..level)
      end
    • `lvl2str()2 is just the conjugate, but given that these seem to be private methods anyway so that the API can be sloppy over whether string or numeric log levels would be used, I am not sure if this code is needed.
      I've got other suggestions, but this I am not sure here is the place to do it.
    Terry Ellison
    @TerryE

    BTW, you could also code the str2lvl function something like the following. Perhaps, not the clearest code, but only a 26 instruction function that is fast and has low RAM use in an ESP8266 LFS context. OK, it will miscode error levels like "NOTE", but IMO it does the job. (Note that string.find(s,p,i,true) does a simple fast memory scan.)

    local function str2lvl(s)
      local vals = 'DEB\07INF\6NOT\5WAR\4ERR\3CRI\2ALE\1EME\0'
      local i = vals:find(s:sub(1,3):upper(),1,true) or error('Invalid error level '..s)
      return vals:sub(i+3,i+3):byte()
    end

    Of course there are many ways of implementing this, but some take far less ESP8266 resources than others. All you need is a little lateral thinking in your approach.

    Matthew B White
    @mbwhite

    Hello; just wanted to say thankyou to the hard-work put into this project; I've come back to use the esp8266 that have been lying around not being used for too long.

    Very much enjoying using them. As a professional software engineer by day, I can see the effort that's gone into it :clap:

    rigorka
    @rigorka
    @TerryE Hi, pardon my asking here, but it seems you're the only person who might have a fairly clear idea how/if my problem can be solved. I've had my laptop stolen from me a day after I've written a very large and boring piece of Lua code (SNMP control implementation for some really awkward power management hardware). I did not commit this code to git, so I no longer have the .lua files :( However I still do have the debug esp8266 module where this code has been uploaded after compiling it into LFS image with luac.cross from nodemcu-firmware. I was able to extract the LFS image from esp8266 back to my PC. Now I'd love to give luadec a try to decompile that code, but to do so I need to extract the compiled Lua files (.lc) from that LFS image. Is there any reasonable way to do so?
    Terry Ellison
    @TerryE
    @rigorka, no sorry. Not realistically practical.
    rigorka
    @rigorka
    @TerryE - thanks anyway. At least not gonna waste my time trying, will just rewrite that stuff from scratch.
    Darren Enns
    @darethehair_twitter
    New guy here with a simple/short question: Do either the 'release' or 'dev' versions in the NodeMCU builder produce working LUA firmware for making 'https' calls? Yes, I have added 'tls' options. The docs hint that sites that use CloudFlare to provide HTTPS access may not work with NodeMCUs implemention of TLS. Thanks.
    Marcel Stör
    @marcelstoer
    HTTPS is working indeed - within the constraints documented (you already found those I see).
    Timothy
    @timothyparez
    I cannot seem to find a link in the docs that list the pin numbers? For example the number for the Analog0 ? (https://esp8266-shop.com/esp8266-guide/esp8266-nodemcu-pinout => I have this page but those are the aliases for Arduino, not LUA) Am I missing something?
    Timothy
    @timothyparez
    oh use the gpio and adc modules ok
    Johan Ström
    @stromnet

    Hi, got a weird one.. Suddenly my device fails to boot my LFS image. My init.lua fails node.LFS.get('_init') (nil), so it triggers LFs.reload from the image. which seems to work:
    Erasing LFS from flash addr 0x08a000................................ to 0x0a9fff LFS image loaded
    and then boot continues, aaand.. LFS.get still fails. node.LFS.list reveals no files at all.

    Now, if I remove some (random) code from the built image, it works just fine!

    lfs.img failing is 68056 bytes, working right now is 67815 bytes.. File content is correct on SPIFFS (md5 ok).
    LFS: 0x20000 bytes total capacity
    firmware is dev (+ some stuff of my own but just other modules)

    Am I hitting some kind of size limitation?

    oh, lua 5.3 on esp8266..
    Patrick Dorion
    @poorandunlucky
    did you try to increase the LFS size? you can do that with the partition table editor, really easy to do, just calculate from your current values, and reflash the firmware and your LFS...
    @stromnet (see above) (forgot to @mention you)
    Johan Ström
    @stromnet
    @poorandunlucky I have not, but with 0x20000 I should have plenty of space left.. they are not compressed, right? But, will try that later today!
    Patrick Dorion
    @poorandunlucky
    @stromnet they're not compressed, and it "should", but I've also had a "should" in the same situation as you (not as much though), and just increasing the space worked... Also the default is 0x10000 so that might be your problem right there, as well... it's 0x20000 in the documentation but in the config file it's 0x10000 if i remember correctly... pretty sure increasing your size is going to work, if not, it's going to be interesting, but it's the best starting point i think since you say the only difference is the size of your image, going above 64K, so it's a pretty obvious starting point, i think...
    Johan Ström
    @stromnet
    firmware is built with #define LUA_FLASH_STORE 0x20000, and that LFS: ...total cap was from the boot output. So should be 0x20000. But, let's see now
    Johan Ström
    @stromnet

    Yup, with 0x30000 it boots fine on the image previously "broken". And back to 0x20000, not working.
    Noticed that for the failed ones, it logs
    Loading lfs.img... init.lua:8: system restarting stack traceback: [C]: in field 'reload' � init.lua:8: in function <init.lua:3>

    (i.e. the lua_error from LFS.reload(..) is actually showed. but I guess it could be race between restart/printing that)

    Johan Ström
    @stromnet
    platform_rcr_write works through fine, none of the error paths.. so at least that write is working..
    Johan Ström
    @stromnet
    ...right, that only writes a filename :)
    Patrick Dorion
    @poorandunlucky
    @stromnet maybe Terry will be able to say why it does that when he notices three messages... happy your thing works now, though... may i ask what you use the ESP for and why you chose nodemcu? just curious, wondering what people use theirs for, and why they picked nodemcu...
    Johan Ström
    @stromnet

    Yeah, perhaps.. Easy to reproduce here at least, so I can assist with debugging if needed.
    I'm using it for all sorts of stuff. Started a few years back with just one node for tinkering, and found NodeMCU appealing with the fast development cycles possible with LUA. So started out with some bme280 integrations sending measurements over mqtt.. then my lua "firmware" have kind of grown into modularized code, all modules are in the LFS image, but I enable different ones with different config, depending on the node usage. And OTA for the lfs image.
    So, now I have a bunch of them, in addition of having multiple nodes with bme280's, I also have some 1-wire buses on a few of them (temp sensors, and custom AVR slaves with MoaT fw), some mcp28003 IO muxers, mcp2438 A/Ds for some battery monitoring in one of my battery+sun-powered nodes,..what else.. my house power meter has a Mbus-interface, which I'm reading logging with one node.. and some other powermeters with modbus will be read by another...
    Oh, and for xmas I'm also running some LED lights in the garden, one set via some basic PWM driver.. and two other nodes with 350 RGB-pixels each, receiving streamed "scene data" via E1.31 protocol ("DMX over ethernet"), outputting to the RGB pixels with DMA-backed i2s code . Those last two are done with C modules, not pushed upstream (yet?).

    So yeah.. a bit of this and a bit of that.. :) Dunno what other esp8266 firmwares there are out there these days, but this has worked nice for me. And although the 8266 is a bit overruled by the 32, for alooot of stuff the esp8266 has plenty of power. And with LFS, just have to sprinkle alot of garbagecollect() calls at appropriate places ;)