These are chat archives for esp8266/Arduino

27th
Jun 2016
Clemens Kirchgatterer
@everslick
Jun 27 2016 07:02
i'm back to getting tons of linker errors after the jump to 2.3.0 (custom Makefile)
multiple definition of time', multiple definition of_malloc_r', multiple definition of `_free_r', ...
and 1 'undefined reference to `_sbrk_r''
Ivan Grokhotkov
@igrr
Jun 27 2016 07:06
Are you sure this is 2.3.0, and not the latest git?
Clemens Kirchgatterer
@everslick
Jun 27 2016 07:12
latest git
i'm always pulling head
is HEAD not usable at the moment?
Clemens Kirchgatterer
@everslick
Jun 27 2016 07:19
ok, i reverted back to 2.3.0 (checkout 2.3.0) and removed -lc again. this links.
Clemens Kirchgatterer
@everslick
Jun 27 2016 07:27
ah, this is #1752
Ivan Grokhotkov
@igrr
Jun 27 2016 07:31
head is usable in Arduino, but testing other build systems is not part of my test suite
If you want your build system to be tested against future changes, feel free to add a test as a pull request...
you may need to check the changes made to platform.txt and update your include/linker paths and linker flags
Clemens Kirchgatterer
@everslick
Jun 27 2016 07:35
i thought i had recreated all changes in plattform.txt in my Makefile, that's allways the first thing i check, if the build fails
i guess i have to look closer
Ivan Grokhotkov
@igrr
Jun 27 2016 07:45
got your project to work
eh... almost
just duplicate atoi, that's a bug on my side
Ivan Grokhotkov
@igrr
Jun 27 2016 07:50
diff --git a/Makefile b/Makefile
index 0eb2bd9..27ac233 100644
--- a/Makefile
+++ b/Makefile
@@ -123,7 +123,7 @@ DEFINES += -DDEFAULT_LOG_PORT=$(DEFAULT_LOG_PORT)

 C_DEFINES += $(DEFINES)

-LD_STD_LIBS += -lhal -lphy -lpp -lnet80211 -lwpa -lcrypto -lmain -lwps -laxtls -lespnow -lsmartconfig -lmesh -lwpa2 -lstdc++ -lm -lgcc
+LD_STD_LIBS += -lhal -lphy -lpp -lnet80211 -lwpa -lcrypto -lmain -lwps -laxtls -lespnow -lsmartconfig -lmesh -lwpa2 -lstdc++ -lc -lm -lgcc

 START_TIME     := $(shell perl -e "print time();")
 # Main output definitions
@@ -200,7 +200,7 @@ CPP_FLAGS    += -c $(OPTIMIZE) $(WARNINGS) -mlongcalls -mtext-section-literals -

 S_FLAGS      += -c $(OPTIMIZE) -x assembler-with-cpp -MMD -mlongcalls

-LD_FLAGS     += $(OPTIMIZE) $(WARNINGS) -nostdlib -Wl,--no-check-sections -u call_user_start -u _printf_float -u _scanf_float -Wl,-static -L$(SDK_ROOT)/lib -L$(SDK_ROOT)/ld -T$(FLASH_LAYOUT) -Wl,--gc-sections -Wl,-wrap,system_restart_local -Wl,-wrap,register_chipv6_phy -Wl,-wrap,malloc -Wl,-wrap,calloc -Wl,-wrap,realloc -Wl,-wrap,free
+LD_FLAGS     += $(OPTIMIZE) $(WARNINGS) -nostdlib -Wl,--no-check-sections -u call_user_start -u _printf_float -u _scanf_float -Wl,-static -L$(SDK_ROOT)/lib -L$(SDK_ROOT)/libc/xtensa-lx106-elf/lib -L$(SDK_ROOT)/ld -T$(FLASH_LAYOUT) -Wl,--gc-sections -Wl,-wrap,system_restart_local -Wl,-wrap,register_chipv6_phy -Wl,-wrap,malloc -Wl,-wrap,calloc -Wl,-wrap,realloc -Wl,-wrap,free

 # Core source files
 CORE_DIR  = $(ESP_ROOT)/cores/esp8266
and i'll push removal of atoi and other stuff in a few minutes
Clemens Kirchgatterer
@everslick
Jun 27 2016 07:59
a great, thanks so much, it was the -L$(SDK_ROOT)/libc/xtensa-lx106-elf/lib i overlooked
Ivan Grokhotkov
@igrr
Jun 27 2016 07:59
yep
i'm running tests to make sure that removal of atoi and stuff doesn't break anything and will push as soon as these are finished...
Clemens Kirchgatterer
@everslick
Jun 27 2016 08:00
i really should just parse plattform.txt and generate the flags from there
Ivan Grokhotkov
@igrr
Jun 27 2016 08:00
yes, but that's not a simple matter. some stuff gets expanded, you need to take care of that
but, if you decide to do that, the author of makeEspArduino has expressed interest in that as well
perhaps you could talk and come up with some solution
Clemens Kirchgatterer
@everslick
Jun 27 2016 08:12
thx for the hint
Ivan Grokhotkov
@igrr
Jun 27 2016 13:11
@everslick pushed necessary changes to master
Clemens Kirchgatterer
@everslick
Jun 27 2016 14:31
checking it out...
Clemens Kirchgatterer
@everslick
Jun 27 2016 14:42
OTA with http does not work anymore (the same).
looking into it
Ivan Grokhotkov
@igrr
Jun 27 2016 14:46
OTA with http client (get binary from server) or OTA with HTTP server (upload binary to the ESP)?
Clemens Kirchgatterer
@everslick
Jun 27 2016 14:47
upload bin to ESP
ah
the file is too big now
Me No Dev
@me-no-dev
Jun 27 2016 14:48
how big is your flash?
Clemens Kirchgatterer
@everslick
Jun 27 2016 14:48
the new build takes significantly more flash and ram
Ivan Grokhotkov
@igrr
Jun 27 2016 14:48
should take about 3k more RAM and 8-10k more flash
Me No Dev
@me-no-dev
Jun 27 2016 14:48
are you sure you have selected the correct flash size?
Ivan Grokhotkov
@igrr
Jun 27 2016 14:48
depending on the functions used
Clemens Kirchgatterer
@everslick
Jun 27 2016 14:48
eagle.flash.1m192.ld
have to decrease SPIFFS
Me No Dev
@me-no-dev
Jun 27 2016 14:49
thah leaves 800K for both sketches
how big is the sketch?
Clemens Kirchgatterer
@everslick
Jun 27 2016 14:49
Memory usage
Ram: 44836 bytes
Flash: 419307 bytes
Me No Dev
@me-no-dev
Jun 27 2016 14:50
yeah, quite the sketch
two of those will not fit
Ivan Grokhotkov
@igrr
Jun 27 2016 14:50
i think this is the largest i've seen so far.
Me No Dev
@me-no-dev
Jun 27 2016 14:50
me too
Ivan Grokhotkov
@igrr
Jun 27 2016 14:50
you get a prize for that :clap:
still someone has to beat 512k :laughing:
Me No Dev
@me-no-dev
Jun 27 2016 14:51
I guess doable with progmem abuse ;)
Clemens Kirchgatterer
@everslick
Jun 27 2016 14:52
:-D
Ivan Grokhotkov
@igrr
Jun 27 2016 14:52
yeah, i have tried sqiux78's OLED library last weekend, it lets you load a bunch of custom fonts.
i guess with a bunch of fonts one can go above 0.5M
also, images
Me No Dev
@me-no-dev
Jun 27 2016 14:54
defeats the purpose of spiffs :)
Ivan Grokhotkov
@igrr
Jun 27 2016 14:55
well if it's all readonly, then probably not
also, squix78 has a nice web app which generates fonts as C arrays
pretty easy to use, no need for xxd and stuff
it's not convenient for web pages though, since you will probably need to tweak you js and html all the time
but for fonts and images works fine
maybe we need to have some read-only file system as an alternative to spiffs
Me No Dev
@me-no-dev
Jun 27 2016 14:57
like instead of? or together with?
Ivan Grokhotkov
@igrr
Jun 27 2016 14:57
like choose the one you like. the more the merrier.
also if someone has good ideas about flash-friendly key-value storage, i would like to discuss...
and probably pay for the implementation
Me No Dev
@me-no-dev
Jun 27 2016 14:59
rofs should be pretty trivial to implement and upload
Ivan Grokhotkov
@igrr
Jun 27 2016 14:59
exactly, and less overhead than SPIFFS
maybe just use whatever Sprite_tm uses in esphttpd
since Espressif already kinda supports that
Me No Dev
@me-no-dev
Jun 27 2016 15:00
that key-value storage though... to be flash friendly it will need to do something similar to spiffs
Ivan Grokhotkov
@igrr
Jun 27 2016 15:00
and the tool is already written
Me No Dev
@me-no-dev
Jun 27 2016 15:00
well what do you need :D name, size and the data
does it have any sort of catalog?
that is something that could speed up file list and search immensly
currently not so good on spiffs...
Ivan Grokhotkov
@igrr
Jun 27 2016 15:06
IIRC it's based on cpio, so probably no index in Flash. But we could lazy-initialize a dictionary in RAM, which would probably not be too big.
Me No Dev
@me-no-dev
Jun 27 2016 15:09
that will depend on the file count and length of names, but yeah
I think spiffs takes 300ms to run through the flash for 3M partition
so every exists or open of non-existing file will take that much
Hagai Shatz
@hagai-shatz
Jun 27 2016 15:10
~50ms
Me No Dev
@me-no-dev
Jun 27 2016 15:11
@hagai-shatz sure
Hagai Shatz
@hagai-shatz
Jun 27 2016 15:11
I've tested it a few days ago, 47ms to return false for exists.
Me No Dev
@me-no-dev
Jun 27 2016 15:11
with 3M right?
Hagai Shatz
@hagai-shatz
Jun 27 2016 15:11
Yes
Me No Dev
@me-no-dev
Jun 27 2016 15:13
we are lucky we do not wdt in the async server....
Hagai Shatz
@hagai-shatz
Jun 27 2016 15:13
wdt?
Me No Dev
@me-no-dev
Jun 27 2016 15:13
trigger the watchdog
Hagai Shatz
@hagai-shatz
Jun 27 2016 15:14
I don't understand.
Me No Dev
@me-no-dev
Jun 27 2016 15:15
if you take too long to return, you will trigger watchdog reset of the ESP
that is why the sync code uses yield
Hagai Shatz
@hagai-shatz
Jun 27 2016 15:15
What is too long?
Me No Dev
@me-no-dev
Jun 27 2016 15:15
@igrr ?
Ivan Grokhotkov
@igrr
Jun 27 2016 15:17
50ms is not long enough to overflow watchdog timer.
it needs around 1 second
Me No Dev
@me-no-dev
Jun 27 2016 15:17
pfff, that's ages in MCU time :D
Hagai Shatz
@hagai-shatz
Jun 27 2016 15:18
Good to know
By the way, I think SPIFFS cache the last accessed file. It takes more than 4ms to access a file, but if you access it again (exists or open) immediately after, it takes about 0.05ms.
Me No Dev
@me-no-dev
Jun 27 2016 15:21
yes
maybe we can increase the number of files that it caches?
Hagai Shatz
@hagai-shatz
Jun 27 2016 15:23
Exactly! Or allow the sketch to set it based on what is appropriate.
Martin Ayotte
@martinayotte
Jun 27 2016 15:39
talking about SPIFFS enhancements, that would be nice to have a new mtime attribute...
Clemens Kirchgatterer
@everslick
Jun 27 2016 16:02
when i tried to OTA the 'too big' binary, the updater reported '[0] No Error' in Update.printError(out);
Me No Dev
@me-no-dev
Jun 27 2016 16:05
@everslick check if both calcul;ate the available space the same way. also did the update work?
Clemens Kirchgatterer
@everslick
Jun 27 2016 16:08
it 'worked' as in yes it flashed the update. it 'didn't work' as in, the binary did not even boot.
the NO Error was printed for each block, that did not fit anymore
but the flash went on regardless
Me No Dev
@me-no-dev
Jun 27 2016 16:46
anyone interested in adding this to the core:
#include <time.h>
struct tm * getSketchCompileTime(){
  const char * months[] = {"Jan", "Feb", "Mar", "Apr", "May", "Jun", "Jul", "Aug", "Sep", "Oct", "Nov", "Dec"};
  const char * compile_date = __DATE__;
  const char * compile_time = __TIME__;

  int year = atoi(compile_date+7),
  month = 0,
  day = atoi(compile_date+4),
  hour = atoi(compile_time),
  minute = atoi(compile_time+3),
  second = atoi(compile_time+6);

  int i;
  for(i=0; i<12; i++){
    if(memcmp(compile_date, months[i], 3) == 0){
      month = i;
    }
  }

  static struct tm * timeinfo = (struct tm *)malloc(sizeof(struct tm));
  timeinfo->tm_year = year - 1900;
  timeinfo->tm_mon = month;
  timeinfo->tm_mday = day;
  timeinfo->tm_hour = hour;
  timeinfo->tm_min = minute;
  timeinfo->tm_sec = second;
  mktime(timeinfo);
  return timeinfo;
}
@everslick sounds like bug in updater (returning wrong error code)
Clemens Kirchgatterer
@everslick
Jun 27 2016 16:53
can i send html pages in chunks to the browser using the SYNC ESP8266WebServer also?
i know it works in the async server
Me No Dev
@me-no-dev
Jun 27 2016 16:54
afaik no
Clemens Kirchgatterer
@everslick
Jun 27 2016 16:56
hmpf
Me No Dev
@me-no-dev
Jun 27 2016 16:56
should not be hard to implement though ;)
Clemens Kirchgatterer
@everslick
Jun 27 2016 17:05
setting content header and length an calling WiFiClient::write() in a loop and then WiFiClient::stop() ?
Ivan Grokhotkov
@igrr
Jun 27 2016 17:09
no need to stop
Clemens Kirchgatterer
@everslick
Jun 27 2016 17:10
or implementing a sendChunked with a callback to feed it more data
Ivan Grokhotkov
@igrr
Jun 27 2016 17:10
just set the correct content-length first, and then call server.sendContent a number of times
Clemens Kirchgatterer
@everslick
Jun 27 2016 17:10
ah, ok
will try that
Ivan Grokhotkov
@igrr
Jun 27 2016 17:10
nowadays in master i think you can pass the whole thing to sendContent, and it will send is as fast as possible
Me No Dev
@me-no-dev
Jun 27 2016 17:11
there is no content length with chunked output, so it makes sense in cases where you do not know the size of the response beforehand
else there is no point with the sync server
Ivan Grokhotkov
@igrr
Jun 27 2016 17:11
ah, right. there is no support for chunked encoding in esp8266webserver
it's just that you can set content-length if you know it
Clemens Kirchgatterer
@everslick
Jun 27 2016 17:14
i run out of heap to prepare the whole page in rame before sending it
so i need to generate one chank, send it, generate the next aso
Me No Dev
@me-no-dev
Jun 27 2016 17:15
do you know the size of the resulting page though?
Clemens Kirchgatterer
@everslick
Jun 27 2016 17:15
not exactly
but i could just send a little bit more
Ivan Grokhotkov
@igrr
Jun 27 2016 17:15
hmm, sounds like you might really need chunked encoding
Clemens Kirchgatterer
@everslick
Jun 27 2016 17:15
to fill up the missing bytes
Clemens Kirchgatterer
@everslick
Jun 27 2016 17:26
or does anybody have a good idea how to send large dynamic html code (other then the async server, which i cannot switch to in the short term)/
maybe use websocket to set innerHTML on a div
Ivan Grokhotkov
@igrr
Jun 27 2016 17:29
@me-no-dev is going to implement chunked encoding in sync webserver. i think this will take him like 15 minutes.
Me No Dev
@me-no-dev
Jun 27 2016 17:29
what are your issues with switching to async?
I so hate this "Merge remote tracking branch..." crap in my commits
how to get rid of it?
tried rebase and whatnot....
surely it's a one liner in terminal
Clemens Kirchgatterer
@everslick
Jun 27 2016 17:41
some of my code is not 'async safe' yet
and i have no time to sort it out cleanly ATM
i have to say, you guys are awesome! :-D
Me No Dev
@me-no-dev
Jun 27 2016 18:15
ok now let's see if it works :D
Clemens Kirchgatterer
@everslick
Jun 27 2016 18:22
tell, if i can test it
Me No Dev
@me-no-dev
Jun 27 2016 18:41
works, but it has delay at the end that should not be there
Me No Dev
@me-no-dev
Jun 27 2016 18:48
yup working great :) sendContent and sendContent_P
  server.on("/chunked", HTTP_GET, [](){
    server.send(200, "text/html", String());
    server.sendContent("<!DOCTYPE html><html><head><title>Chunked Test</title></head><body>");
    server.sendContent("<p>Chunk 1</p>");
    server.sendContent("<p>Chunk 2</p>");
    server.sendContent("<p>Chunk 3</p>");
    server.sendContent("<p>Chunk 4</p>");
    server.sendContent("<p>Chunk 5</p>");
    server.sendContent("<p>Chunk 6</p>");
    server.sendContent("</html>");
    server.sendContent("");//end of chunked
  });
works on HTTP/1.1 clients
Me No Dev
@me-no-dev
Jun 27 2016 18:53
@everslick if you want to test it: esp8266/Arduino#2199
no need to set contentLength to UNKNOWN anymore if you send empty string in send()
Clemens Kirchgatterer
@everslick
Jun 27 2016 19:15
looks great. i will test it tomorrow. thank you very much for your support!!!
Me No Dev
@me-no-dev
Jun 27 2016 20:00

no need to set contentLength to UNKNOWN anymore if you send empty string in send()

that is no longer true

Clemens Kirchgatterer
@everslick
Jun 27 2016 20:08
so I have to set it to UNKNOWN and send an "" (empty string) as the last chunk?
Me No Dev
@me-no-dev
Jun 27 2016 20:11
yes
the empty string means the end of the page
else it will wait 2 seconds as usual
see the example in the PR
Clemens Kirchgatterer
@everslick
Jun 27 2016 20:28
great. thank you very much
Me No Dev
@me-no-dev
Jun 27 2016 20:37
thanbks for asking for it :D we should have added this long time ago