These are chat archives for SmingHub/Sming

12th
Apr 2016
commanderz
@commanderz
Apr 12 2016 05:17
hi, guys
can we debug code in pc ?
Patrick Jahns
@patrickjahns
Apr 12 2016 07:23
@commanderz
you can use gdb in combination with your esp device
hreintke
@hreintke
Apr 12 2016 07:38
@patrickjahns :
With pointing to the rtos implementation I was not saying that is not an interesting article.
Just answering the remarks on rtos usability, of which I also didn't say it was not valid. :wink:
Patrick Jahns
@patrickjahns
Apr 12 2016 07:45
@hreintke
no worries
Btw did you have success with FlashWrite ? Thought of writing the HttpRbootUpload these days
hreintke
@hreintke
Apr 12 2016 07:48
Yes close to ready, need some testing to done. Was doing that with ftp to get is useful there too but hit ftp issues there.
Patrick Jahns
@patrickjahns
Apr 12 2016 07:55
is it RTOS only or also NONos? Will provide a class stub for the httpupdate then it can be tested together if you want
hreintke
@hreintke
Apr 12 2016 08:51
I have it in rtos now but it is only located in datasourcestraam.h & cpp.
So will be available at the same time for nonos & rtos.
Do you want me to create a PR to develop or should I make the PR against the HTTP_Post branch for easier usage/testing ?
If you merge your PR into the HTTP_Post branch, I can use that also for easy testing
Or does that make it more difficult for you ?
Patrick Jahns
@patrickjahns
Apr 12 2016 09:28
I am fine with either way - just left the PR open for discussion/remarks. Do you know of a good way to discuss the code after it is merged against a (feature) branch?
hreintke
@hreintke
Apr 12 2016 10:09
You mentioned just the remark I had before on using feature branches.
Best way to handle that is to open an issue for the overall feature discussion.
The positive on merging is that others can PR on the branch, which eases feature development.
There is no way to PR on a PR if you are not the creator of the original PR.
Patrick Jahns
@patrickjahns
Apr 12 2016 10:26
:thumbsup: will merge now and copy over the information from the PR into an issue
Patrick Jahns
@patrickjahns
Apr 12 2016 12:17

@hreintke
preliminary interface:
patrickjahns/Sming@9af3320

There are some things that I need to figure out:

  • how to select the rom slot if there is more than 2 roms ? (in the post data it might be difficult since the slot has to be sent beforehand -> maybe a 2step process? )
  • how to act if authorization is required but the post request is not authorized - it would be good to intercept the request before any file is transmitted (directly after the headers)
hreintke
@hreintke
Apr 12 2016 12:34
@patrickjahns :
I had in mind not to use the rboot httpupdate.
We should make the upload to a flash address which is given in the upload data, there is more that applications to upload. f.e. spiffs FS.
In my first idea of filename conventions i was thinking of !0x10000 but needs to revised, maybe request parameters ?
I am already working on a "rboot class" that should handle switching to other rboot roms and generalize ues of http/ftp/others.
Let's keep the http_update as generic as possible and put other functionality in their own classes.
Remarks ?
hreintke
@hreintke
Apr 12 2016 13:50
This message was deleted
@patrickjahns :
Took the Http_post branch 30 mins ago and flashed the example.
For all upload requests I get a Bad Request reply.
Error log :
path=/upload
host === 10.0.0.210
parsed
Request: GET, nodata
TCP received: 327 bytes
onReadyToSendData: 1
response sendHeader
response sendBody
Stream completed
TCP connection closing
~TCP connection
-TCP connection
onAccept state: 0 K=0
Free heap size=38568, K=0
+TCP connection
timeout updating: 70 -> 90
TcpServer onClient: 10.0.0.219

path=/upload
host === 10.0.0.210
multipart - boundary === ---------------------------191781408624973
content-type === multipart/form-data
content-length === 5615
content-type === application/octet-stream
parsed
Request: POST, 5615 bytes
NETWORK_MAX_HTTP_PARSING_LEN
POST FAILED
SEND ERROR PAGE
TCP received: 1460 bytes
onReadyToSendData: 1
TCP connection closing
~TCP connection
-TCP connection
onAccept state: 0 K=0
Free heap size=38568, K=0
+TCP connection
timeout updating: 70 -> 90
TcpServer onClient: 10.0.0.219

TCP received: (null)
TCP connection closing
~TCP connection
-TCP connection
hreintke
@hreintke
Apr 12 2016 14:42
@patrickjahns :
It is filesize dependent. I presume it is hitting the NETWORK_MAX_HTTP_PARSING_LEN boundary.
With smaller files I get :
No uploaded data
Will move this to #694
Patrick Jahns
@patrickjahns
Apr 12 2016 17:45

@hreintke

We should make the upload to a flash address which is given in the upload data, there is more that applications to upload. f.e. spiffs FS.
In my first idea of filename conventions i was thinking of !0x10000 but needs to revised, maybe request parameters ?

the preliminary interface allows for uploading both rom and spiffs at the same time or individually. Also it's possible to disable spiffs upload - at least that's what I thought.
I see some issues with providing the flash address via http query parameters - mostly if the user enters wrong data the upload will overwrite the wrong sectors and cause trouble. And I bet a lot of people will open issues afterward with "why is Firmware upload not working?".
I like the idea from rboot, to specify "rom slots" from which the program/user can select one

Each rom slot has the rom sector and a spiffs sector start (and length?) - so we can check if something is bad beforehand
Currently there is one issue with selecting the rom slots: it must be done one step before selecting the files.
http forms can not change the query parameters - only the post body. But there is no guarantee that the variable with the romslot (or address) gets send as first part of the request. It might be at any place - but for the upload to work we need to know it beforehand we accept any binary data from rom/filesystem
hreintke
@hreintke
Apr 12 2016 18:49
This message was deleted
hreintke
@hreintke
Apr 12 2016 19:11
@patrickjahns :
Within the rboot class I am using the rboot rom slots as "partition table" (viewable and possibly editable using commands). That provides a "romX" interface for applications that are using it (preventing to update application & spiffs areas which are in use).
For http_post might use this partition table as an option to know the location to store the data.
However, the http_post itself should be full generic, just putting data on a specified location. We cannot assume that applications only want to post applications and spiffs. -> If the user want data to be written to a flash address it should go there.
For rBoot integration I think the rboot class should provide callbacks.
That is what I meant with putting the other functionality in their own classes, using http_post as a "provider"
Patrick Jahns
@patrickjahns
Apr 12 2016 19:18
@hreintke
is there any Serial output function that does not truncate the data?
Generally http_post will always be tied to the webserver - you need a path
hreintke
@hreintke
Apr 12 2016 19:21
Don't understand the question on truncation
Patrick Jahns
@patrickjahns
Apr 12 2016 19:21
If someone is not using a commandline client to push the data (for example curl, or in python requests) - we also need a "normal page" to display an upload form
Currently debugf. and serial.printf does truncate the output
hreintke
@hreintke
Apr 12 2016 19:24
Can the current implementation not be used by curl for pushing files ?
Patrick Jahns
@patrickjahns
Apr 12 2016 19:24
It can - but if it`s not just tied to these clients and you want to use a normal browser - you need a form
But more importantly I need a way to get the full tcp packet shown in Serial to look closer at the data arriving at the esp
with debugf/printf it's truncated and that really doesn`t help
hreintke
@hreintke
Apr 12 2016 19:26
But isn't it up to the application to provide the form.
You mean truncated at a "\0" in the data ?
Patrick Jahns
@patrickjahns
Apr 12 2016 19:27
no it does not show the full data - it shows X chars and then truncates the rest
or removes/doesn`t show
hreintke
@hreintke
Apr 12 2016 19:28
Ah, now I understand. Yes, there is a limit on number of chars printed as a string
Patrick Jahns
@patrickjahns
Apr 12 2016 19:28
is there any function to send more than that - i need the full length to have a closer look
looking with wireshark/fiddler I can see it on my side, but I don`t see what the esp parses in the end - and for catching the issues with firefox I need to inspect all data
hreintke
@hreintke
Apr 12 2016 19:33
The limit is in m_printf.cpp : #define MPRINTF_BUF_SIZE 256
You can increase that in your local environment.
Only noticed just now that m_printf puts the buffer on stack -> should be move to heap.
However I think NONOS will have no problem increasing.
Patrick Jahns
@patrickjahns
Apr 12 2016 19:37
okay will try - any reason for this actually?
hreintke
@hreintke
Apr 12 2016 19:38
Or, maybe better. Create a small debug routine which prints "long buffers" in line of 80 ? chars.
Should be more readable
Patrick Jahns
@patrickjahns
Apr 12 2016 19:38
Alright
About the general http post class - I will try and fix first the parsing and then we can continue talking about the interface
hreintke
@hreintke
Apr 12 2016 19:39
Not a specific reason. It is the limitation of the current m_printf.
With not to much effort we could remove the limitation
Patrick Jahns
@patrickjahns
Apr 12 2016 19:40
I am still not quite sure if I understood you correctly - but if you have a certain design in mind you could create a stub which then adapt to it?
hreintke
@hreintke
Apr 12 2016 19:40
On http_post : agree. let's focus first on correct uplaod.
Yes, I will make sure there is a better description/understanding of the design.
Sorry to to be not clear in the wording.
Patrick Jahns
@patrickjahns
Apr 12 2016 19:45
Okay increasing the limit in the header file doesn`t work -_-
wanted to avoid writing extra unnecessary functions
anyhow
Dont sweat it with the design - its hard to communicate sometimes via chat and github
It is always easier to at least talk or in the end have a meeting. 5 Minute time with a person can save hours of chatting
;-)
Henry Balanon
@balanon
Apr 12 2016 19:48
Hello. Sorry I'm new here. QQ: Does Sming+ESP8266 work with AWS's IoT platform?
robotiko
@robotiko
Apr 12 2016 19:50
@balanon just cheked what is AWS iot
entry point to the platform is mqtt , http and websockets
all that is supported by sming
(websockets client is still not merged but in Pull Requets )
provided sdk code has a rtos client
Henry Balanon
@balanon
Apr 12 2016 19:54
Okay. Wasn't sure because TLS is required and been reading that ESP8266 isn't TLS 1.2 friendly. Wanted to see if someone's successfully connected Sming to AWS IoT
robotiko
@robotiko
Apr 12 2016 19:54
smingrtos might work
Henry Balanon
@balanon
Apr 12 2016 19:54
hm
robotiko
@robotiko
Apr 12 2016 19:54
however tls ..
not sure if sming ssl PR supports tls
Henry Balanon
@balanon
Apr 12 2016 19:59
hm. What's the diff between Sming and SmingRTOS? Looks like they both run on the ESP8266
robotiko
@robotiko
Apr 12 2016 19:59
base sdk
sming uses nonos sdk
smingrtos uses rtos SDK
Patrick Jahns
@patrickjahns
Apr 12 2016 20:00
I really start to hate firefox
If I use fiddler as proxy in-between everything works fine. If i use chrome/ie everything works fine too
@hreintke
can you send me the binary files you had issues with?
Henry Balanon
@balanon
Apr 12 2016 20:04
thanks @robotiko !
hreintke
@hreintke
Apr 12 2016 20:07
@balanon :
The current ssl Sming (PR) is using axTLS and does not support TLS 1.2
@robotiko :
That was one of the disadvantages we discussed before
robotiko
@robotiko
Apr 12 2016 20:08
@hreintke ok dk .. thanks .. as stated.. didn't knwo if it was supported
hreintke
@hreintke
Apr 12 2016 20:17
@patrickjahns :
What is so specific on firefox. Aren't we looking at the tcpstream and shouldn't that be identical between browsers
Patrick Jahns
@patrickjahns
Apr 12 2016 20:25
somehow not - the overall stream yes - but it seems that chrome and internet explorer first hand over the header data to the tcp layer and afterward the body - while firefox hands it over in total
hreintke
@hreintke
Apr 12 2016 20:27
But the content of the tcpstream is the same. All code should be fully independent on tcp stream splits across multiple buffers.
You will never know the behavior of all the clients which are in use in this ever changing world
Patrick Jahns
@patrickjahns
Apr 12 2016 20:31
It should and I think I found the culprit - it was a miscalculation at some point
which never showed when packets where split header/body
hreintke
@hreintke
Apr 12 2016 20:32
Then you should love firefox to give you an early possibility to capture the culpit :smile:
Patrick Jahns
@patrickjahns
Apr 12 2016 20:33
Haha sort of yes - it`s the thing with testing environment and then fully working code
All tests you can think of pass - and then the user takes it and it doesn`t work anymore :shipit:
Henry Balanon
@balanon
Apr 12 2016 20:35
What use cases would someone use the RTOS vs nonos?
Patrick Jahns
@patrickjahns
Apr 12 2016 20:37
@balanon
apparently espressif moves towards rtos with the esp32 platform
it is not known how long the nonos version for esp8266 will receive updates
Henry Balanon
@balanon
Apr 12 2016 20:39
Ah gotcha. Good to know @patrickjahns!
Anyone know pricing on the ESP32 or is that a mystery still?
not sure if it will be the price
but it`s "preorder" with 60days waiting atleast
Henry Balanon
@balanon
Apr 12 2016 20:40
thanks!