These are chat archives for esp8266/Arduino

22nd
Mar 2016
Stavros Korokithakis
@skorokithakis
Mar 22 2016 02:26
@ivankravets any plans on switching to flash n chips? https://github.com/cesanta/fnc
tzapu
@tzapu
Mar 22 2016 06:06
or @igrr same question as above :)
or maybe just their modified esptool
andig
@andig
Mar 22 2016 07:44
Fnc doesn't seem valid for Windows without further changes
tzapu
@tzapu
Mar 22 2016 08:26
oh, that answers that then...
how about their modified esptool ?
Ivan Grokhotkov
@igrr
Mar 22 2016 08:55
regarding python esptool — i wasn't able to package it as a binary for every OS. Had issues with 32-bit linux and older OS X releases (<10.7).
That was the reason why i have switched to esptool-ck
regarding faster uploads using a custom stub — i have started going the same route a few months ago with this branch: https://github.com/igrr/esptool-ck/tree/feature/compression
there was not much interest however so i haven't finished that.
tzapu
@tzapu
Mar 22 2016 09:17
well, i guess ota kind of trumps that anyway..
d-anders
@d-anders
Mar 22 2016 11:29
yeah, what i want is quicker compiling/building
guess i better buy a new comp
Ivan Grokhotkov
@igrr
Mar 22 2016 11:35
we can speed up building by pre-building some of the core files into a static library, and shipping that instead of source files
i'm sure some users will frown upon such a solution, and complain that "we ship binaries in their face"
tzapu
@tzapu
Mar 22 2016 11:46
:))))
i m sure they will
@d-anders it doesn t compile that fast for me either, and i guess i ve got what you could say is a fast computer, i7 level, etc
could the above solution of precompiling binaries be optional?
or maybe could arduino ide precompile them once, and then reuse the prebuilt lib?
Ivan Grokhotkov
@igrr
Mar 22 2016 11:49
arduino doesn't have such an option
tzapu
@tzapu
Mar 22 2016 11:49
i guess that s how xcode does it, precompiles all libs, frameworks
Ivan Grokhotkov
@igrr
Mar 22 2016 11:49
by precompiling i mean compiling them before creating a distribution .zip file
that is, when new release is created
tzapu
@tzapu
Mar 22 2016 11:50
but i guess, to keep both camps happy, then you d need to double the mentenance work, for both the boards manager fetch source and compile every time, and for the ones willing to use binaries
Ivan Grokhotkov
@igrr
Mar 22 2016 11:51
yes, it would be possible to satisfy both camps by having two boards manager packages — "full source" one and "optimized" one
tzapu
@tzapu
Mar 22 2016 11:51
could maybe a script compile them in the user s computer, and used that one from now on?
Ivan Grokhotkov
@igrr
Mar 22 2016 11:52
yeah, that might be an option...
tzapu
@tzapu
Mar 22 2016 11:52
i mean, since we re deviating from the norm, i m willing to have a few more steps, do a little bit more setup
also, since i m using the github ver, it wouldn t work very well just with release binaries
that way, i could add a hook in my pull procedure to compile it as well
or whatever, there s a million ways of doing it afterwards
Ivan Grokhotkov
@igrr
Mar 22 2016 11:53
github ver / boards manager ver adds yet another variable to the mix...
tzapu
@tzapu
Mar 22 2016 11:53
yeah, that s why i m thinking, if this could be an optional script/flow someone wanting more speed would be willing to go
Ivan Grokhotkov
@igrr
Mar 22 2016 11:53
yeah, i can probably write a shell script which will do the job on osx and linux
for windows users, someone else will have to write a batch file or something
tzapu
@tzapu
Mar 22 2016 11:54
if most people ara happy, no point changing what s not broken
and i think you said this before @igrr , and you were right, it needs to be simple for everyone to be able to get up and running as fast as possible
even if it s not the absolute best in performance and so on, it gets someone playing, then they read more, then they follow discussions, then they find out they can get something better , etc
at least my two pence :P
that s how i got more and more into this stuff, finding out more and more cool stuff
in time, it will all be refined anyway
i ll shut up now
Ivan Grokhotkov
@igrr
Mar 22 2016 11:58
pretty much yes, that's what Arduino is about — make it easy to start playing with something. move to better tools/frameworks as you learn and develop stuff
tzapu
@tzapu
Mar 22 2016 11:59
i always wanted to move to platform io, it s all nice and everybody is so happy with it etc, but i m concerned that if i do that, it might bend my libs or whatnot and people using plain old arduino might not be able to use them
need to look into that, as the ardu ide really is a pita
but to be honest, if there was no arduino ide, i probably would not have played with this stuff at all
and really, my first firmware for all of this, has been for the esp, i skipped almost completly all the actual arduino hardware :P
and i m quite sure i am not alone
fast compile + ota … how cool would that be :D
if there were such a thing as dynamic linking on the esp, and you could have the prebuilt library linked at runtime on the esp, then the big chunk of code that the SDK is, could be written only once…
but most likely that s not possible, also you d need to include everything you use from the sdk, so it would be bloated
Ivan Grokhotkov
@igrr
Mar 22 2016 12:04
Yes i have experimented with dynamic linking
tzapu
@tzapu
Mar 22 2016 12:04
(bloat is not necessarily an issue if you write it only once)
Ivan Grokhotkov
@igrr
Mar 22 2016 12:04
even wrote a poor man's dynamic loader
it works nicely, but there is noticeable RAM cost due to the relocation table
it does work well on ESP32 though
tzapu
@tzapu
Mar 22 2016 12:05
how in the world did you even get around to learning how to write bootloaders and dynamic loaders, and so on…
Ivan Grokhotkov
@igrr
Mar 22 2016 12:06
re dynamic loaders, i learned a lot from this article: http://eli.thegreenplace.net/2011/08/25/load-time-relocation-of-shared-libraries
and other articles on the same blog filed under "linkers and loaders" tag
tzapu
@tzapu
Mar 22 2016 12:07
for most stuff we all do, there s like a million resources out there that at least point you in a direction or another… but i can t imagine there s more than a handful of people worldwide capable of… and i m sure they are quite busy too :P
yeah…thanks for the article… waay above my head
i guess they need to rellocate the memory in which a shared lib is loaded , because you couldn t possible load all of it, or know where all symbols reside
Ivan Grokhotkov
@igrr
Mar 22 2016 12:12
the idea with a dynamic loader on the ESP is that you place your "loaded" library somewhere in the memory-mapped part of flash memory. you generate a relocation table for this part, and place it into RAM. then you go and fill all the right addresses into the relocation table.
so when the "loaded" code needs some address (say, of an SDK function), it goes into the relocation table and grabs the value from there
this is far from efficient though
tzapu
@tzapu
Mar 22 2016 12:15
so all the sdk would need to be loaded in to ram
Ivan Grokhotkov
@igrr
Mar 22 2016 12:15
no, just all the pointers to SDK functions
tzapu
@tzapu
Mar 22 2016 12:15
or penalty from fetching from flash
Ivan Grokhotkov
@igrr
Mar 22 2016 12:15
the functions themselves are somewhere in flash
(or in IRAM, if that's required)
tzapu
@tzapu
Mar 22 2016 12:16
then except for the relocation store and search, you d have kind of the same performance,
Ivan Grokhotkov
@igrr
Mar 22 2016 12:16
i hope that we will eventually transition to an approach similar to what Nordic is doing with their proprietary bluetooth stack
tzapu
@tzapu
Mar 22 2016 12:16
you ve just got that table to take care of, which you mentioned is too much for the esp8266
Ivan Grokhotkov
@igrr
Mar 22 2016 12:16
they have a "softdevice", which is essentially "SDK libraries" in Espressif speak
and the application doesn't link to the softdevice, it calls its public functions via SVC interrupts
tzapu
@tzapu
Mar 22 2016 12:17
oh
Ivan Grokhotkov
@igrr
Mar 22 2016 12:17
(sorta like linux applications make syscalls)
tzapu
@tzapu
Mar 22 2016 12:17
so you could end up there with the sdk?
but you d need two laywer of abstraction then? one to make it a soft device, and one to structure the calls that call the svc interrupts? or is the calling part simple..
Ivan Grokhotkov
@igrr
Mar 22 2016 12:18
yes, you can. just need to wrap all public functions and make them accessible via svc
calling part is relatively thin
and you don't normally call softdevice functions really often
not that often to make this call overhead significant
all the libc and other "open source" stuff is linked to your application
tzapu
@tzapu
Mar 22 2016 12:20
wouldn t every time you do a digital write or something , call the soft device?
any wifi interaction calls
Ivan Grokhotkov
@igrr
Mar 22 2016 12:20
no, because digitalwrite communicates with peripheral registers directly
tzapu
@tzapu
Mar 22 2016 12:20
ah ha
so it would be only the sdk wrapper functions
everything wifi
Ivan Grokhotkov
@igrr
Mar 22 2016 12:21
and you don't call wifi interactions in a for (i=0; i<100000; ++i) loop
tzapu
@tzapu
Mar 22 2016 12:21
correct
Ivan Grokhotkov
@igrr
Mar 22 2016 12:21
so those should be okay as well
tzapu
@tzapu
Mar 22 2016 12:21
but to keep the current api, you d still need to do a lot of work to make all the calls public, of the sdk, throug the new interrupt svc
Ivan Grokhotkov
@igrr
Mar 22 2016 12:22
normally you write a code generation script in something like perl or python, and it does the job for you
generating jump tables and marshalling arguments...
tzapu
@tzapu
Mar 22 2016 12:22
true, as what you need to expose should be quite explicit
but, what benefit would all this bring in the end?
it would be even more "decoupled" i guess
Ivan Grokhotkov
@igrr
Mar 22 2016 12:24
that would separate application from the softdevice. this solves some licensing issues, and you can have smaller binaries.
for instance, when you are using a GPL-licensed Arduino library with the ESP core, you are in a grey territory...
tzapu
@tzapu
Mar 22 2016 12:25
because of mismatched licenses
as the esp core is not gpl ed and all available
i guess
Ivan Grokhotkov
@igrr
Mar 22 2016 12:25
because the binaries provided to you by Espressif are "Espressif-MIT", and you don't have source for them
tzapu
@tzapu
Mar 22 2016 12:26
is there any plan at some point of making some kind of huge table or list of these wild ideas and/or future plans and getting the pulse of the community about what they see as immediate problems/improvement areas ?
Ivan Grokhotkov
@igrr
Mar 22 2016 12:26
so if you link them with GPL code and start shipping the firmware, you may eventually get a few questions
tzapu
@tzapu
Mar 22 2016 12:27
or is it too much hassle combined with the fact that people don t usually know what s really important?
Ivan Grokhotkov
@igrr
Mar 22 2016 12:27
:) i guess you are right, i should post these things as "issues for discussion" from time to time
tzapu
@tzapu
Mar 22 2016 12:27
i am asking this, because i ve seen some very interesting/wild/SF level ideas around gitter and github issues
and each has a benefit/cost attached
and i m thinking there s not real usage statistics of what people actually use out of the sdk most, etc
or how they use it
Ivan Grokhotkov
@igrr
Mar 22 2016 12:29
i will start working on ESPs full time in a month or so, and things will hopefully start moving at a quicker pace
tzapu
@tzapu
Mar 22 2016 12:29
to determine whatever would help most for most people
oh, that s wonderful news
Ivan Grokhotkov
@igrr
Mar 22 2016 12:29
including getting these "wild ideas" off the ground
tzapu
@tzapu
Mar 22 2016 12:30
well, think of some voting scheme, most people might not care for upload time but might care for long file names on SD s for instance. or rtos so you can write the same firmware for the esp32
of course, some votes might just go to ideas that sound nice, but still...
ota is lovely, but not for every project, so fast uploads might be nicer, etc , etc
i m sure there s a million discussions like that
Ivan Grokhotkov
@igrr
Mar 22 2016 12:33
that's so true. still we can determine the most "hot" topics and try to cover them
tzapu
@tzapu
Mar 22 2016 12:33
yup
anyway, it s a great community as far as i can see
use it :D
tzapu
@tzapu
Mar 22 2016 16:09
well, i tried the fast esptool.py flasher and it is really fast
my esp also didn t work after, so i guess it put gibberish there ...
d-anders
@d-anders
Mar 22 2016 19:22
anyone using platformio?
what do i need to do get clang do autocomplete for digitalRead etc?
Aditya Tannu
@AdySan
Mar 22 2016 20:15
This message was deleted
This message was deleted
oops
xbary
@xbary
Mar 22 2016 20:16
:)
Michael Miller
@Makuna
Mar 22 2016 20:22
I sure hope it was juicy, what ever you deleted.
Aditya Tannu
@AdySan
Mar 22 2016 20:23
nah just saying good things about how awesome people on thic channel are
was recommending it to someone in a PM
Michael Miller
@Makuna
Mar 22 2016 21:00
when trying to use the rtc_mem, I am getting a collect error when it compiles.
undefined reference to `system_rtc_mem_read(unsigned char, void*, unsigned short)'
Ivan Grokhotkov
@igrr
Mar 22 2016 21:01
extern "C" {
#include "user_interface.h"
}
?
Michael Miller
@Makuna
Mar 22 2016 21:02
yup, figured it out and was just about to post, you beat me to it
I really wish the standard of .h was c, and .hpp was c++, this way it could that automatically. But that standard got dropped out of the "standard" early on.
Martin Ayotte
@martinayotte
Mar 22 2016 21:08
Maybe those can be wrapped as ESP.readRTC() and ESP.writeRTC() on our side ?
Ivan Grokhotkov
@igrr
Mar 22 2016 21:18
Personally I would rather "unwrap" them
these functions are wrappers around the simple RTC_MEM[addr] = value
just need to expose RTC_MEM in the esp8266_peri.h
Martin Ayotte
@martinayotte
Mar 22 2016 21:20
That can be a good idea !
Ivan Grokhotkov
@igrr
Mar 22 2016 21:20
these functions do some validity check on input arguments, but no one checks the return code anyway...
so better let it crash on error, in my opinion
Martin Ayotte
@martinayotte
Mar 22 2016 21:20
about the 32bits aligments ?
Ivan Grokhotkov
@igrr
Mar 22 2016 21:21
yep
#1807
Martin Ayotte
@martinayotte
Mar 22 2016 21:22
Ivan Grokhotkov
@igrr
Mar 22 2016 21:23
i know that parts of Espressif code used to write to user area, at least in older SDKs
so i'm not surprised
let's see... here are some defines from their latest LwIP
#define TCP_WND (*(volatile uint32*)0x600011F0)
Martin Ayotte
@martinayotte
Mar 22 2016 21:25
Why they are doing so, in UserSpace region ? and why at those specific/unrelated addresses ? if they need more after the 65, they should have took the next one 66 not the 95
Ivan Grokhotkov
@igrr
Mar 22 2016 21:25
#define TCP_MAXRTX (*(volatile uint32*)0x600011E8)
#define TCP_SYNMAXRTX (*(volatile uint32*)0x600011E4)
#define MEMP_NUM_TCP_PCB (*(volatile uint32*)0x600011FC)
#define DHCP_MAXRTX (*(volatile uint32*)0x600011E0)
are these in user region or system region?
Angus Gratton
@projectgus
Mar 22 2016 21:28
that's interesting. those are system region if the regions haven't moved in the latest SDKs.
Martin Ayotte
@martinayotte
Mar 22 2016 21:28
From what I know, the UserRegion start at 0x60001200, right ? they should stay in their sandbox :-)
Ivan Grokhotkov
@igrr
Mar 22 2016 21:29
right, these ones are fine.
Angus Gratton
@projectgus
Mar 22 2016 21:30
that's really odd they are using rtcmem storage for those, though. in upstream LWIP they are macro constants.
Ivan Grokhotkov
@igrr
Mar 22 2016 21:30
apparently they decided they could make some lwip parameters dynamic this way
Angus Gratton
@projectgus
Mar 22 2016 21:30
yeah - dynamic and persistent across deep sleep? really odd.
Ivan Grokhotkov
@igrr
Mar 22 2016 21:31
and they didn't use normal RAM because they were not sure how to reserve specific addresses
Angus Gratton
@projectgus
Mar 22 2016 21:31
i... guess? not the weirdest choice they've made I suppose
Ivan Grokhotkov
@igrr
Mar 22 2016 21:31
certainly not :)
at least i know that there is an espconn function which modifies the MEMP_NUM_TCP_PCB thing
i'm not entirely sure LwIP expects some constants to change at runtime though...
okay, regarding SDK functions writing to user memory, i saw some posts on the esp8266.ru forum, in the SDK reverse engineering thread. let me find that reference...
Michael Miller
@Makuna
Mar 22 2016 21:36
Does the watchdog functions work? I am trying to trigger them. I see a comment about the timer not being applied, but then what is the default?
Ivan Grokhotkov
@igrr
Mar 22 2016 21:38
there are two watchdogs, hardware one and software one
hardware one is configured with 6 seconds timeout
software one is configured with something like 1-2 seconds, i'm not sure exactly
software one is timer-based (FRC2) relies on timer ISR
so if ISRs are disabled, only the hardware one will work.
Michael Miller
@Makuna
Mar 22 2016 21:41
the software one its definitely not under 3 seconds, but it is under 30 seconds
Ivan Grokhotkov
@igrr
Mar 22 2016 21:44
void setup() {
  while(true){}
}

void loop() {
}
This sketch gets me a reset every ~3 seconds
Michael Miller
@Makuna
Mar 22 2016 21:45
its seems calling any of the delay methods feed the dog, I had a calculation wrong on my getCycleCount(), it is about 3 seconds.
Ivan Grokhotkov
@igrr
Mar 22 2016 21:46
void setup() {
  noInterrupts();
  while(true){}
}

void loop() {
}
this one (triggers hardware watchdog) gives a reset every ~7-8 seconds
Yes, calling delay returns control to the scheduler, which feeds the watchdog
Michael Miller
@Makuna
Mar 22 2016 21:56
its about 3200ms, 3150ms it won't trigger
@skorokithakis we have opened issue for that platformio/platformio#592
Stavros Korokithakis
@skorokithakis
Mar 22 2016 22:34
Ah fantastic
d-anders
@d-anders
Mar 22 2016 22:36
@ivankravets is there an easy way to get autocompletion for code from the "arduino-framework" (digitalRead etc)?