These are chat archives for esp8266/Arduino

15th
May 2017
Aaron N.
@porkyneal
May 15 2017 07:51
Documentation looks great, learnt a few new things already ;)
Is it possible to have platformio pull the latest git automtaically yet?
Me No Dev
@me-no-dev
May 15 2017 10:10
@porkyneal that depends on @ivankravets to PR the required build file for PIO, as he did for ESP32
Ivan Kravets
@ivankravets
May 15 2017 11:51
@me-no-dev @igrr do you have any plans for a next release of ESP8266 Core for Arduino?
Ivan Grokhotkov
@igrr
May 15 2017 12:00
Plans as in date in the calendar, no, we don't. But we are working towards that :)
Me No Dev
@me-no-dev
May 15 2017 12:37
@ivankravets regardless of the plans/dates we should make the staging branch for pio behave the same as on ESP32
that is the only way to deliver latest code to PIO users that want it
else they have to go through hoops and not necessarily succeed :)
Lars Englund
@larsenglund
May 15 2017 16:43
Hi, I'm using https://github.com/Links2004/arduinoWebSockets to try and send some binary data from my esp to a browser via a websocket. On my esp the webSocket.sendBIN() call returns very fast (dunno if it sends it directly or queues it for sending). On the browser it takes several minutes before I receive the binary data. The data is a 4096 byte buffer.
I also get a "WebSocket connection to 'ws://esp.local:81/' failed: Invalid frame header" on the client after the long wait
So I guess I'm asking if anyone has a working example of sending binary data from a esp to a browser via websockets since I can't seem to get it working..
Steve Nelson
@zenmanenergy
May 15 2017 16:47
are you working from the WebSocketServer.ino file?
Lars Englund
@larsenglund
May 15 2017 16:47
My code for sending data:
void webSocketEvent(uint8_t num, WStype_t type, uint8_t * payload, size_t lenght) {
  switch (type) {
  case WStype_DISCONNECTED:
    Serial.printf("[%u] Disconnected!\n", num);
    break;
  case WStype_CONNECTED: {
    IPAddress ip = webSocket.remoteIP(num);
    Serial.printf("[%u] Connected from %d.%d.%d.%d url: %s\n", num, ip[0], ip[1], ip[2], ip[3], payload);
    webSocket.sendTXT(num, String("D") + " " + sample_idx_1 + " " + num_samples_1);
    Serial.println("Sending binary");
    webSocket.sendBIN(num, (uint8_t *) &samples2s[0], MAX_SAMPLES*2);
    Serial.println("Binary sent");
    }
    break;
@zenmanenergy No, I've copy pasted bits and pieces into another ino
Steve Nelson
@zenmanenergy
May 15 2017 16:49
i was just going to point out the sendbin() in that example is commented out :-)
Lars Englund
@larsenglund
May 15 2017 16:50
Client code:
var connection = new WebSocket('ws://'+location.hostname+':81/', ['arduino']);
connection.binaryType = 'arraybuffer';
var s1 = new Uint16Array(2048);
connection.onopen = function () {  
  connection.send('Connect ' + new Date()); 
}; 
connection.onerror = function (error) {    
  console.log('WebSocket Error ', error);
};
connection.onmessage = function (e) {  

  console.log('onmsg type: ', typeof(e.data));
  console.log('onmsg data: ', e.data);
Steve Nelson
@zenmanenergy
May 15 2017 16:50
what do you have for: samples2s and MAX_SAMPLES
Lars Englund
@larsenglund
May 15 2017 16:51
#define MAX_SAMPLES 2048
uint16_t samples2s[MAX_SAMPLES];
Steve Nelson
@zenmanenergy
May 15 2017 16:54
nothing is jumping out immediately. but why are you expecting it to take a long time?
can you post your whole ino file or is it huge?
Lars Englund
@larsenglund
May 15 2017 16:56
I'm not expecting it to take a long time, quite the contrary
Hmm, It's kindof large but I could pastebin it?
Steve Nelson
@zenmanenergy
May 15 2017 16:57
So, the esp is going fast, but the browser is taking a long time?
Lars Englund
@larsenglund
May 15 2017 16:57
Exactly
Steve Nelson
@zenmanenergy
May 15 2017 16:57
yeah, pastebin it. i'll just give it a shot here and see if i get something different
Lars Englund
@larsenglund
May 15 2017 17:00
But I haven't successfully received and decoded any data yet, just printed out that I get a object type
I started out from the CaptivePortalAdvanced example
Steve Nelson
@zenmanenergy
May 15 2017 17:02
i'll take a look.
Lars Englund
@larsenglund
May 15 2017 17:04
@zenmanenergy Awesome, thanks alot!
Steve Nelson
@zenmanenergy
May 15 2017 17:04
don't thank me yet. i haven't offered anything useful! :-)
on a totally different note, one thing I've started doing is downloading saving my html/pngs into spiffs files. So if you can update your files without recompiling:
void Settings::downloadAndSaveFile(String fileName, String  url){

  HTTPClient http;

  Serial.print("[HTTP] begin...\n");

  http.begin(url);

  Serial.printf("[HTTP] GET...\n", url.c_str());
  // start connection and send HTTP header
  int httpCode = http.GET();
  if(httpCode > 0) {
      // HTTP header has been send and Server response header has been handled
      Serial.printf("[HTTP] GET... code: %d\n", httpCode);
      Serial.printf("[FILE] open file for writing %d\n", fileName.c_str());

      SPIFFS.begin();
      File file = SPIFFS.open(fileName, "w");

      // file found at server
      if(httpCode == HTTP_CODE_OK) {

          // get lenght of document (is -1 when Server sends no Content-Length header)
          int len = http.getSize();

          // create buffer for read
          uint8_t buff[128] = { 0 };

          // get tcp stream
          WiFiClient * stream = http.getStreamPtr();

          // read all data from server
          while(http.connected() && (len > 0 || len == -1)) {
              // get available data size
              size_t size = stream->available();
              if(size) {
                  // read up to 128 byte
                  int c = stream->readBytes(buff, ((size > sizeof(buff)) ? sizeof(buff) : size));
                  // write it to Serial
                  //Serial.write(buff, c);
                  file.write(buff, c);
                  if(len > 0) {
                      len -= c;
                  }
              }
              delay(1);
          }

          Serial.println();
          Serial.print("[HTTP] connection closed or file end.\n");
          Serial.print("[FILE] closing file\n");
          file.close();

      }

      SPIFFS.end();
  }
  http.end();


}
oops, kill the Settings:: unless you it into a class.
Lars Englund
@larsenglund
May 15 2017 17:10
Yeah, I was thinking of doing that but didn't stumble upon a good example, thanks!
Steve Nelson
@zenmanenergy
May 15 2017 17:11
i'm still goofing around with the best way to do it myself.
Lars Englund
@larsenglund
May 15 2017 17:12
So your esp pulls all the files from your development machine at boot/setup?
Steve Nelson
@zenmanenergy
May 15 2017 17:12
what i've quickly learned is that downloading files via wifi is roughly a 100 million-billion times faster than uploading them through usb.
Lars Englund
@larsenglund
May 15 2017 17:13
I was thinking of adding a form for uploading files but automatically pulling them in setup() is way faster if you can bother setting up a local webserver on the development machine to serve them
Steve Nelson
@zenmanenergy
May 15 2017 17:13
yeah, i'm rewriting the class for it, but the basic concept is to download a list of files from the server. I'm sending the list when it connects to the server's websocket.

i.e. 1. connect to the server via websockets

  1. server sends a list of URLS
  2. esp loops over the list one by one and downloads each of them

probably a better way is to send a hash of the file along with the list and compare it to whats currently in the esp. haven't gotten that far yet.

oops, my numbering gotten funky
Steve Nelson
@zenmanenergy
May 15 2017 17:24
hmmm. i was able to load the index.html and it appears to send the data, it's missing hte Chart.js, but that's no big deal.
although I don't have a thermocouple here to test it.
Lars Englund
@larsenglund
May 15 2017 17:28
Did you receive the data too?
Steve Nelson
@zenmanenergy
May 15 2017 17:29
so far all it sent was:
data
:
"D 206 206"
i'm faking the thermocouple.readCelsius() at the moment.
Steve Nelson
@zenmanenergy
May 15 2017 17:32
wait... you're doing webSocket.broadcastTXT(String("") + c + " " + c_millis); not binary
am i looking at the wrong line?
Lars Englund
@larsenglund
May 15 2017 17:34
I do that periodically after the initial connect to keep streaming live data
But when the client first connects I send the whole circular buffer
Line 101 in RakuTemp.ino
Steve Nelson
@zenmanenergy
May 15 2017 17:36
oh, i see. so when they connect, you send the data up to that point, then the broadcastTXT updates it?
Lars Englund
@larsenglund
May 15 2017 17:36
right
(btw, I linked the wrong Chart.js before, correct version is https://cdnjs.com/libraries/Chart.js)
Steve Nelson
@zenmanenergy
May 15 2017 17:41
it does seem to just sit there in the browser even when the esp is displaying a temperature.
if i change the MAX_SAMPLES to a lower number the browser loads quick. although in the JS console when the e.data is an object, the e.data seems to be empty. i'm not sure if that means the lower max_samples broke everything :-)
Lars Englund
@larsenglund
May 15 2017 17:49
Hmm, I tried the same now and it does go fast, will try to add code for processing the data
Steve Nelson
@zenmanenergy
May 15 2017 17:51
have you considered using json in place of binary? it certainly makes the JS side easier.
i have to run and pick up my kids. I'll check back with you later see if i can help more.
Lars Englund
@larsenglund
May 15 2017 17:57
I'm thinking it will be very much slower..
I tried sending each value in the buffer as txt before but it took forever.
And using json will eat up alot of ram to create a json object out of 2048 uint16:s..
Steve Nelson
@zenmanenergy
May 15 2017 18:00
maybe, but you could send it in chunks and it would release the memory. from what i can tell you only have to do the big dataset once upon connect then every 2 seconds afterward
New code that processes the buffer
I have set MAX_SAMPLES to 16 on both sides
But on the client I only get this
onmsg type:  string
(index):26 onmsg data:  D 10 10
(index):25 onmsg type:  object
(index):26 onmsg data:  ArrayBuffer {}
(index):35 idx 0: 7424
(index):35 idx 1: 7424
(index):35 idx 2: 7424
(index):35 idx 3: 7424
(index):35 idx 4: 7424
(index):35 idx 5: 7424
(index):35 idx 6: 7424
(index):35 idx 7: 7424
(index):35 idx 8: 7424
(index):35 idx 9: 7424
(I also fixed so that the chart starts updating with real time data once the buffer is received)
Lars Englund
@larsenglund
May 15 2017 18:16
It was only the wrong endianness, it's reading the data fine now but when I increase the buffer size to 2048 (from 16) it takes several minutes for it to be "received" at the client... why is that? WEBSOCKETS_MAX_DATA_SIZE is 15360 so it should fit easily..
Lars Englund
@larsenglund
May 15 2017 18:27
Using a buffer size of 1024 is seemingly instantly loaded same as a buffer of 16. A buffer size of >1500 leads to the client spitting out a WebSocket connection to 'ws://esp.local:81/' failed: Invalid frame header
Lars Englund
@larsenglund
May 15 2017 18:43
or sometimes WebSocket connection to 'ws://esp.local:81/' failed: One or more reserved bits are on: reserved1 = 0, reserved2 = 1, reserved3 = 1
Lars Englund
@larsenglund
May 15 2017 18:48
A workaround that works fine is to send the buffer in chunks of 1024..
Lars Englund
@larsenglund
May 15 2017 18:58
For reference, here's the working code (with hardcoded chunks): https://gist.github.com/larsenglund/b2714e6d9962ade2ed2a2b440a6e353d
Lars Englund
@larsenglund
May 15 2017 19:57
@zenmanenergy Thanks alot for your help, really nice of you to take the time!
Lars Englund
@larsenglund
May 15 2017 20:25
I will push any further updates to this repo: https://github.com/larsenglund/RakuTemp
Steve Nelson
@zenmanenergy
May 15 2017 21:26
@larsenglund you're welcome! good luck