These are chat archives for SmingHub/Sming

6th
Nov 2015
zhivko
@zhivko
Nov 06 2015 03:02
does anybody have sample for 512kB OTA flash from websocket binary upload?
alon24
@alon24
Nov 06 2015 06:14
@zhivko that sounds interesting
zhivko
@zhivko
Nov 06 2015 07:00
@alon24 you wont need external server to host firmware with that method.
alon24
@alon24
Nov 06 2015 07:10
I too would like to know how to do this
zhivko
@zhivko
Nov 06 2015 07:41
There are methodshave you read this? to receive binary, but
zhivko
@zhivko
Nov 06 2015 08:18
MAybe @robotiko can comment this (is it same like harizanov firmware OTA and if it can be somehow integrated with webdocket binary receive) because currently I havent played with rboot functionality in Sming and therefore I dont have overview of that.
robotiko
@robotiko
Nov 06 2015 08:59
@zhivko @alon24 I looked at the harizanov post ..
sming rboot solution is more mature and robust than the pull... but the push is not available in sming .. yet
however.. all pieces are there to do it
I started playing with the push concept .. but didn't complete
we have many options in sming..
ftp, direct post to httpserver(as in this post) or websocket binary .. the good thing is that all this.. is just how to retrieve the file.. so it loosk quite straightforward from that point to send it to rboot and continue
zhivko
@zhivko
Nov 06 2015 09:22
I see websocket binary as most appropriate for job.
Is there python script to do websocket binary push? Maybe we should go in this direction, and add target to Makefile-project.mk
Any opinion from you guys?
THere is nice lib for python: https://pypi.python.org/pypi/websocket-client/ there is also sample for sending file there...
icodk
@icodk
Nov 06 2015 09:48
Can sming do basic authentication in httpserver ?
hreintke
@hreintke
Nov 06 2015 09:50
@zhivko
I am "Generalizing" rBoot interface to be able to have multiple pull/push implementations.
For push I considered using FTP but websocket should also be an option
@icodk : No, there is currently no authentication in httpserver
hreintke
@hreintke
Nov 06 2015 10:00
@zhivko : No full knowledge of websockets : with http and ftp you have standard based file level retrieves, it there something like that defined for websockets ?
robotiko
@robotiko
Nov 06 2015 10:19
@hreintke websockets is just the channel.. any retry/reconnect logic is to be implemented by the client
Is there python script to do websocket binary push?
you mean an ws client.. I would suggest to add it to the config page itself
sam-fori-yana
@khaaali
Nov 06 2015 14:09
i installed sming by cmd... ifollowed the changes of ESP_HOME = c:\Espressif and SMING_HOME = c:\tools\sming\Sming
now im getting issue with
c:/espressif/xtensa-lx106-elf/bin/../lib/gcc/xtensa-lx106-elf/5.1.0/../../../../xtensa-lx106-elf/bin/ld.exe: cannot find -lsming
collect2.exe: error: ld returned 1 exit status
make: * [out/build/app.out] Error 1
Dmitry Kireev
@kireevco
Nov 06 2015 14:10
@vnvnsairam did you look into wiki - troubleshooting Windows?
sam-fori-yana
@khaaali
Nov 06 2015 14:11
will check
yeah i followed those steps.
Dmitry Kireev
@kireevco
Nov 06 2015 14:14
Did you build SmingFramework?
Jakub Sobieraj
@sobiso
Nov 06 2015 14:16
hi all
Dmitry Kireev
@kireevco
Nov 06 2015 14:17
@sobiso hey, man
Jakub Sobieraj
@sobiso
Nov 06 2015 14:17
is there any way to save (put) large json data to file ?
Dmitry Kireev
@kireevco
Nov 06 2015 14:17
@sobiso co slyhat?
Jakub Sobieraj
@sobiso
Nov 06 2015 14:17
@kireevco lol almost in polish ;)
Dmitry Kireev
@kireevco
Nov 06 2015 14:17
almost, sorry
Jakub Sobieraj
@sobiso
Nov 06 2015 14:18
'co slychac' without polish characters
Dmitry Kireev
@kireevco
Nov 06 2015 14:18
yeah… I just remember that by ear
and I’m a russian speaker, so pardon me
Jakub Sobieraj
@sobiso
Nov 06 2015 14:19
;)
Dmitry Kireev
@kireevco
Nov 06 2015 14:20
so, are you serializing your data? what stops you just write it as a regular file?
Jakub Sobieraj
@sobiso
Nov 06 2015 14:21
on app i do that
```
Dmitry Kireev
@kireevco
Nov 06 2015 14:21
This message was deleted
Jakub Sobieraj
@sobiso
Nov 06 2015 14:21
        String data = request.getPostParameter("data");
        fileSetContent(".event",data);
Dmitry Kireev
@kireevco
Nov 06 2015 14:22
ok
Jakub Sobieraj
@sobiso
Nov 06 2015 14:22
i know, wrong chat mode
Dmitry Kireev
@kireevco
Nov 06 2015 14:22
so, that’s what? sming or the other side?
Jakub Sobieraj
@sobiso
Nov 06 2015 14:22
on js i put
    var string = JSON.stringify(Events);
    var compressed = string;

    $.post( host + '/ajax/event', { 
        data: compressed
    })
and esp restart
Item: data = [{"d":1,"gm":1234,"g":[1,2,3,4,5,6,7,8,9,0],"f":"z","a":2},{"d":1,"gm":1234,"g":[1,2,3,4,5,6,7,8,9,0],"f":"z","a":2},{"d":1,"gm":1234,"g":[1,2,3,4,5,6,7,8,9,0],"f":"z","a":2},{"d":1,"gm":1234,"g":[1,2,3,4,5,6,7,8,9,0],"f":"z","a":2},{"d":1,"gm":1234,"g":[1,2,3,4,5,6,7,8,9,0],"f":"z","a":2},{"d":1,"gm":1234,"g":[1,2,3,4,5,6,7,8,9,0],"f":"z","a":2},{"d":1,"gm":1234,"g":[1,2,3,4,5,6,7,8,9,0],"f":"z","a":2},{"d":1,"gm":1234,"g":[1,2,3,4,5,6,7,8,9,0],"f":"z","a":2},{"d":1,"gm":1234,"g":[1,2,3,4,5,6,7,8,9,0],"f":"z","a":2},{"d":1,"gm":1234,"g":[1,2,3,4,5,6,7,8,9,0],"f":"z","a":2},{"d":1,"gm":1234,"g":[1,2,3,4,5,6,7,8,9,0],"f":"z","a":2},{"d":1,"gm":1234,"g"%3
Fatal exception (0):
epc1=0x34333231, epc2=0x00000000, epc3=0x00000000, excvaddr=0x00000000, depc=0x00000000

 ets Jan  8 2013,rst cause:2, boot mode:(3,0)

load 0x40100000, len 22580, room 16
Dmitry Kireev
@kireevco
Nov 06 2015 14:23
and what happens?
how big is your data?
Jakub Sobieraj
@sobiso
Nov 06 2015 14:23
i thing that data is too large
Dmitry Kireev
@kireevco
Nov 06 2015 14:23
have you thought that esp has limited ram
high chance is that it simply can’t process your data
you need to chunk it
Jakub Sobieraj
@sobiso
Nov 06 2015 14:24
before request
Heap: 
29912
Dmitry Kireev
@kireevco
Nov 06 2015 14:24
or re-architect it
Jakub Sobieraj
@sobiso
Nov 06 2015 14:25
chunk only app site or both app and js ?
side
Dmitry Kireev
@kireevco
Nov 06 2015 14:25
js, and app… so both will know the protocol
I assume
spread it across multiple requests
that’s the quickest hack, I think
do you have a data sample?
Jakub Sobieraj
@sobiso
Nov 06 2015 14:27

yes

{"d":1,"gm":1234,"g":[1,2,3,4,5,6,7,8,9,0],"f":"z","a":2}

this is object and i have 28 objects in array

Dmitry Kireev
@kireevco
Nov 06 2015 14:27
ok, cool
so, do a loop and try to chunk it by 10 in each
slice it
Jakub Sobieraj
@sobiso
Nov 06 2015 14:28
but "g" can be largest example 'g':[1,2,3,4,5,6,7,8,9,1,2,3,4,5,6,7,8]
send first 10, then next 10 then rest (8 or more)...
Jakub Sobieraj
@sobiso
Nov 06 2015 14:28
ok on js side it is no problem but in app ?
Dmitry Kireev
@kireevco
Nov 06 2015 14:29
try it first, see if it breaks
then you will have to teach both sides some «continuation» flag
sam-fori-yana
@khaaali
Nov 06 2015 14:30
@kireevco issue resolved i messed with path configuration thanks!!
Jakub Sobieraj
@sobiso
Nov 06 2015 14:30
ok thx
Dmitry Kireev
@kireevco
Nov 06 2015 14:31
@sobiso like
{"continued":true, [{"d":1,"gm":1234,"g":[1,2,3,4,5,6,7,8,9,0],"f":"z","a":2},{"d":1,"gm":1234,"g":[1,2,3,4,5,6,7,8,9,0],"f":"z","a":2},{"d":1,"gm":1234,"g":[1,2,3,4,5,6,7,8,9,0],"f":"z","a":2},{"d":1,"gm":1234,"g":[1,2,3,4,5,6,7,8,9,0],"f":"z","a":2},{"d":1,"gm":1234,"g":[1,2,3,4,5,6,7,8,9,0],"f":"z","a":2},{"d":1,"gm":1234,"g":[1,2,3,4,5,6,7,8,9,0],"f":"z","a":2},{"d":1,"gm":1234,"g":[1,2,3,4,5,6,7,8,9,0],"f":"z","a":2},{"d":1,"gm":1234,"g":[1,2,3,4,5,6,7,8,9,0],"f":"z","a":2},{"d":1,"gm":1234,"g":[1,2,3,4,5,6,7,8,9,0],"f":"z","a":2},{"d":1,"gm":1234,"g":[1,2,3,4,5,6,7,8,9,0],"f":"z","a":2}]}
(just an idea).
but I would try that it doens’t break
when you send less data
Jakub Sobieraj
@sobiso
Nov 06 2015 14:32
i will try this
Dmitry Kireev
@kireevco
Nov 06 2015 14:33
 String data = request.getPostParameter("data»);
// take «continued flag out, extract array as it is and _append_ to the file (don’t know how, what sming gives us today? any append method?

fileSetContent(".event",data);
kireevco @kireevco is still missing method reference...
hreintke
@hreintke
Nov 06 2015 14:35
@sobiso :
From htttconfig_network appsettings.h file
    void save()
    {
        DynamicJsonBuffer jsonBuffer;
        JsonObject& root = jsonBuffer.createObject();

        JsonObject& network = jsonBuffer.createObject();
        root["network"] = network;
        network["ssid"] = ssid.c_str();
        network["password"] = password.c_str();

        network["dhcp"] = dhcp;

        // Make copy by value for temporary string objects
        network.addCopy("ip", ip.toString());
        network.addCopy("netmask", netmask.toString());
        network.addCopy("gateway", gateway.toString());

        //TODO: add direct file stream writing
        fileSetContent(APP_SETTINGS_FILE, root.toJsonString());
    }
Jakub Sobieraj
@sobiso
Nov 06 2015 14:36
@hreintke i know about this but i send large data
hreintke
@hreintke
Nov 06 2015 14:46
Do you get exception with receiving data or processing json ?
Jakub Sobieraj
@sobiso
Nov 06 2015 15:04
receiving
hreintke
@hreintke
Nov 06 2015 15:31
@sobiso :
I am looking at your message from 15:22.
The line Item : data =[.....]
Does that contain the whole data which has been sent or just a part ?
hreintke
@hreintke
Nov 06 2015 15:41
Can I get a file with the data that you are trying to send ?
I want to do some debugging, if something is to long, it should fail with error and not crash.
Jakub Sobieraj
@sobiso
Nov 06 2015 15:48

yes

{"d":1,"gm":1234,"g":[1,2,3,4,5,6,7,8,9,0],"f":"z","a":2}

this is object and i have 28 objects in array

hreintke
@hreintke
Nov 06 2015 15:52
So the data is just that string, 28 times repeated ?
also with the short names like d, f, a etc ?
According to the string shown 1596 bytes
Jakub Sobieraj
@sobiso
Nov 06 2015 16:02
yes
but "g" array can be largest max 100 elements
hreintke
@hreintke
Nov 06 2015 16:04
according to the message from 15:22 the error occurs "already" with 10 elements ?
Do you have the debug messages from : "before the Item = data..." line ?
Jakub Sobieraj
@sobiso
Nov 06 2015 16:06
yes in this case it is only 10 elements
no but wait a moment i test again
Station configuration was updated to: gospo
AP IP succesfully updated
mode : sta(18:fe:34:a0:a4:b7)
add if0
f 0, scandone
state: 0 -> 2 (b0)
state: 2 -> 3 (0)
state: 3 -> 5 (10)
add 0
aid 2
pm open phy_2,type:2 0 0

connected with gospo, channel 6
ip:192.168.1.11,mask:255.255.255.0,gw:192.168.1.1
cnt 
 -- connected --

- - start server
'/' registered
'/ajax/set-ip' registered
'/ajax/get-networks' registered
'/ajax/connect' registered
'/ajax/send-roller' registered
'/ajax/send-group' registered
'/ajax/set-roller' registered
'/ajax/zero-pin' registered
'/ajax/set-pin' registered
'/ajax/save-pin' registered
'/ajax/load-pin' registered
'/ajax/set-router' registered
'/ajax/save-rollers' registered
'/ajax/save-groups' registered
'/ajax/checkCookies' registered
'/ajax/get-date' registered
'/ajax/set-date' registered
'/ajax/run-sta' registered
'/ajax/event' registered
'/ajax/save-group' registered
'/ajax/nor' registered
addUser: me 123
Heap: 
34056 
0
Connected? :5
0
onAccept state: 0 K=0
Free heap size=33608, K=0
+TCP connection
timeout updating: 70 -> 90
TcpServer onClient  192.168.1.40, activeClients = 1

path=/ajax/event
Host === 192.168.1.11
Content-Length === 3610
Content-Type === application/x-www-form-urlencoded; charset=UTF-8
Item: group_saved = 2
Item: roller_saved = 0
parsed
Request: POST, 3610 bytes
POST WAIT
TCP received: 489 bytes
onReadyToSendData: 1
Item: data = [{"d":0,"gm":1234,"g":[1,2,3,4,5,6,7,8,9,0],"f":"z","a":2},{"d":1,"gm":1234,"g":[1,2,3,4,5,6,7,8,9,0],"f":"z","a":2},{"d":2,"gm":1234,"g":[1,2,3,4,5,6,7,8,9,0],"f":"z","a":2},{"d":3,"gm":1234,"g":[1,2,3,4,5,6,7,8,9,0],"f":"z","a":2},{"d":4,"gm":1234,"g":[1,2,3,4,5,6,7,8,9,0],"f":"z","a":2},{"d":5,"gm":1234,"g":[1,2,3,4,5,6,7,8,9,0],"f":"z","a":2},{"d":6,"gm":1234,"g":[1,2,3,4,5,6,7,8,9,0],"f":"z","a":2},{"d":7,"gm":1234,"g":[1,2,3,4,5,6,7,8,9,0],"f":"z","a":2},{"d":8,"gm":1234,"g":[1,2,3,4,5,6,7,8,9,0],"f":"z","a":2},{"d":9,"gm":1234,"g":[1,2,3,4,5,6,7,8,9,0],"f":"z","a":2},{"d":10,"gm":1234,"g":[1,2,3,4,5,6,7,8,9,0],"f":"z","a":2},{"d":11,"gm":1234,"g"
Fatal exception (0):
epc1=0x34333231, epc2=0x00000000, epc3=0x00000000, excvaddr=0x00000000, depc=0x00000000

 ets Jan  8 2013,rst cause:2, boot mode:(3,7)

load 0x40100000, len 22580, room 16 
tail 4
chksum 0x88
load 0x3ffe8000, len 852, room 4 
tail 0
chksum 0x0d
load 0x3ffe8358, len 10076, room 8 
tail 4
chksum 0x39
csum 0x39
��뢁use rtc mem data
rl+TCP connection
timeout updating: 70 -> 65535
Server timeout updating: 40 -> 90
+TCP connection
timeout updating: 70 -> 65535
Server timeout updating: 40 -> 900
+TCP connection
fs.start: size:2048 Kb, offset:0x0x100000

mount res: 0
hreintke
@hreintke
Nov 06 2015 16:15
OK, good info. Now I know how the data is transmitted.
Just the standard question : You are at latest Sming 1.4.0 and Espressif SDK 1.3.0 ?
Jakub Sobieraj
@sobiso
Nov 06 2015 16:18
yes
zhivko
@zhivko
Nov 06 2015 16:18
@hreintke
@hreintke websocket binary push is thinner than ftp pull i think
Jakub Sobieraj
@sobiso
Nov 06 2015 16:25
@hreintke it is possible to write directly stream to file ?
hreintke
@hreintke
Nov 06 2015 16:25
@sobiso : Thx, will do some investigation/debugging and get back to you.
Probably tomorrow, otherwise sunday
Jakub Sobieraj
@sobiso
Nov 06 2015 16:33
@hreintke ok, big thx
hreintke
@hreintke
Nov 06 2015 16:33
@sobiso : I don't think so. But I have not much experience with spifss FS.
@zhivko :
I experimented with ftp push, not pull.
ESP has ftpserver, not ftpclient. Something like clientside : PUT appfile $ROM0 or PUT spiffsfile $ROM1
You can then upload f.e. using curl
hreintke
@hreintke
Nov 06 2015 16:42
@zhivko :
I have limited knowledge of websockets.
With Http you get do GET /dir/appfile to pull a file, with ftp you "just wait" for the client to do the PUT, and you exactly get the file.
I don't know how a file transfer is handled within websockets
zhivko
@zhivko
Nov 06 2015 16:47
@hreintke OK but I don't want to have ftpserver along with httpserver it seems to me overkill...
@hreintke websockets are already there - I need them for my webapp - and websocket SMING implementation has binary receive event...
I plan to use python to uplad firmware to server - but than details to what adress to download firmware - i still need to study this... I hope there is support in sming for firmware size - like in harizanov code...
hreintke
@hreintke
Nov 06 2015 18:03
@zhivko :
Overkill, can be. But if you don't have a "fully loaded" ESP and have space/option to use an additional service it is a standards based solution which is implemented with relative little effort.
And : What is the websockets standard to transfer files ? In Sming Core we should use as much as possible generic standards instead of sming specific implementations.
Sming/rBoot does not support firmware size.
Beside that I have some other updates i would like to see in the rboot code.
Things like : partition status (valid, loading, invalid...), partition type (application, spiffs FS,..), partition userdata (a 8 byte location for application defined data like version), other rboot options to facilitate a kind of "rboot bios" in order to (almost) never loose contact with an ESP and thus be able to recover failing applications remotely.
All of the above updates should be gathered and discussed with Richard. Then we hopefully can have a "one time update" to rboot instead of numerous little updates.
robotiko
@robotiko
Nov 06 2015 18:36
@hreintke +1 to the logic layer for rboot