These are chat archives for esp8266/Arduino

18th
Jun 2015
Michael Miller
@Makuna
Jun 18 2015 04:55 UTC
@igrr When is it planned to update the normal build manager install? A lot of issues are just due to people running it as it is over a month old now.
Ivan Grokhotkov
@igrr
Jun 18 2015 05:02 UTC
I wanted the update to be based on the new SDK, but there are a lot of issues reported in the last few days related to wdt resets in things which used to work.
But I get your point, better to have at least some update.
Let me test if the basic sketches run at least on my side, if they do I'll push the update today.
Ivan Grokhotkov
@igrr
Jun 18 2015 05:08 UTC
did you have a chance to look at the SSL thing? what's your take on that?
Michael Miller
@Makuna
Jun 18 2015 05:13 UTC
I only have glanced through it. The issues at the first level is that although the esp library exposes Secure methods, our current Wifi isn't based on their libraries (we don't call the related non-secure apis). So that's not easy. I took a quick glance at axTls (which seems to be what the esp library wraps up) but the API surface is daunting compared to the esp library.
There seems to be a lot of issues with hashing support, as some have stated it works on some servers but not others. Although one issue was partially resolved by increasing the buffer size for key to 4k.
Michael Miller
@Makuna
Jun 18 2015 05:18 UTC
The whole issue of putting a cert on a "iot server" seems to be glossed over as "just acceptable" to store the cert on the device unsecured. This is scary, as all it takes is to dump the flash and you can then attack the communications. I am unsure how you would solve this without a secured flash storage of some sort.
Ivan Grokhotkov
@igrr
Jun 18 2015 05:33 UTC
I think the other use case, with an esp acting as a client, is more useful in practice, and doesn't suffer from this insecure flash storage problem. At least on the issue tracker people seem to be more interested in calling HTTPS APIs.
so perhaps it is sufficient to focus on SSL support in WiFiClient for now
Michael Miller
@Makuna
Jun 18 2015 05:50 UTC
axtls seems to be a dead library. Last change was 2008, LibreSsl or BoringSsl are more active (albeit not aimed at constrained environment). Last checkins for these two was days ago.
@igrr was there a reason the esp library wasn't used for connect? Or am I incorrectly reading the code?
Me No Dev
@me-no-dev
Jun 18 2015 06:01 UTC
the I2S example they posted has code for accessing SSL sites
i do not think it's wrapped arround espconn as they use lwip calls there to deal with the network
it's using the RTOS SDK but i doubt that will be an issue
Ivan Grokhotkov
@igrr
Jun 18 2015 08:49 UTC
@Makuna you mean espconn? when I started this project it was buggy as hell... also it adds little value — it's just a wrapper around lwIP which provides a more constrained interface. so there was no point using a wrapper which added little value when i was writing a wrapper myself.
Me No Dev
@me-no-dev
Jun 18 2015 08:50 UTC
there is one advantage for me in espconn
Ivan Grokhotkov
@igrr
Jun 18 2015 08:50 UTC
@me-no-dev is right, the SSL code uses lwIP as it's network interface
which is?
Me No Dev
@me-no-dev
Jun 18 2015 08:51 UTC
and that is the fact that I do not need to poll the server to get the connection callback
Ivan Grokhotkov
@igrr
Jun 18 2015 08:51 UTC
with lwIP you also don't need to poll anything, so that doesn't count as an advangate IMO
you get callbacks much like with espconn
Me No Dev
@me-no-dev
Jun 18 2015 08:51 UTC
i do not talk about lwip :)
i talk about our implementation VS espconn
Ivan Grokhotkov
@igrr
Jun 18 2015 08:52 UTC

well i was replying to

was there a reason the esp library wasn't used for connect

there's no point comparing our library with espconn
they serve different APIs
our library was intended as a drop-in replacement for WiFi Shield library
Me No Dev
@me-no-dev
Jun 18 2015 08:54 UTC
and is not usable in some cases because of that :)
i could not use it for the copter for example
i could use the UDP part
but TCP had to go through espconn (as I'm not familiar with lwip yet)
Ivan Grokhotkov
@igrr
Jun 18 2015 08:56 UTC
you're an advanced user :) i see no reason why you can't use espconn
Me No Dev
@me-no-dev
Jun 18 2015 08:57 UTC
i'm fine with whatever, but user will get wdt reset if the loop get's too busy
it's what happened to me
anyway
i think I went a bit off topic
sorry about that
SSL should be pullable from the I2S example
Ivan Grokhotkov
@igrr
Jun 18 2015 09:05 UTC
not from SSL example i think as it uses FreeRTOS, and as such the interface to lwip is not raw interface but socket interface. but SSL code is present in 0.9.3 SDK, and it uses raw lwip interface which is what we need here
Me No Dev
@me-no-dev
Jun 18 2015 09:06 UTC
yeah, i noticed the socket interface
i loved it... :(
Holger Lembke
@holgerlembke
Jun 18 2015 09:33 UTC
@igrr thanks for looking into it. i have a memory dump of the heap and try to follow this list as outlined in heap_4 and it simply does not work for me.
Ivan Grokhotkov
@igrr
Jun 18 2015 09:36 UTC
the layout of heap node is like this:
struct heap_node{
  heap_node* next;     // 4 bytes
  size_t size;   // 4 bytes
  uint32_t padding1;  // 4 bytes
  uint32_t padding2;  // 4 bytes
  char data[]                   // allocated amount, aligned to 16 bytes size
};
is that what you use in your tool?
Holger Lembke
@holgerlembke
Jun 18 2015 09:36 UTC
yepp
this

''' int i = 0;
while (chain->pxNextFreeBlock) {
i++;

chain=chain->pxNextFreeBlock;

}

Ivan Grokhotkov
@igrr
Jun 18 2015 09:39 UTC
and how are you obtaining the initial value of chain?
Holger Lembke
@holgerlembke
Jun 18 2015 09:40 UTC
unsigned char pucAlignedHeap;
ucHeap = ( unsigned char
)&_heap_start;
pucAlignedHeap = ( unsigned char ) ( ( ( portPOINTER_SIZE_TYPE ) &ucHeap[ portBYTE_ALIGNMENT ] ) & ( ( portPOINTER_SIZE_TYPE ) ~portBYTE_ALIGNMENT_MASK ) );
xStart.pxNextFreeBlock = (xBlockLink
)pucAlignedHeap;
xBlockLink *chain = xStart.pxNextFreeBlock;
and it works for the dump: chain points to the first item in the heap. and this item then points to the heaps end. fail.
Ivan Grokhotkov
@igrr
Jun 18 2015 09:43 UTC
what is the value of portBYTE_ALIGNMENT?
Holger Lembke
@holgerlembke
Jun 18 2015 09:43 UTC

define portBYTE_ALIGNMENT 8

define portPOINTER_SIZE_TYPE unsigned int

define portBYTE_ALIGNMENT_MASK ( 0x0007 )

hm....
Ivan Grokhotkov
@igrr
Jun 18 2015 09:44 UTC
ouch. you should really quote your code blocks :)
Holger Lembke
@holgerlembke
Jun 18 2015 09:44 UTC
#define portBYTE_ALIGNMENT 8
#define portPOINTER_SIZE_TYPE unsigned int
#define portBYTE_ALIGNMENT_MASK ( 0x0007 )
Holger Lembke
@holgerlembke
Jun 18 2015 09:53 UTC
could it be that they sub-allocate?
no. it doesn't make sense
I can send you my code, if it helps...
Ivan Grokhotkov
@igrr
Jun 18 2015 10:04 UTC
i'll check the code again, perhaps i have missed some difference
i'll look into this again tomorrow, have got lots of work planned for today already, sorry
Holger Lembke
@holgerlembke
Jun 18 2015 10:36 UTC
I'm fine with that.
Holger Lembke
@holgerlembke
Jun 18 2015 12:14 UTC
perhaps they use some TLSF?
Ivan Grokhotkov
@igrr
Jun 18 2015 12:39 UTC
no i'm pretty sure they started off with heap4.c, i guess i have missed some minor difference
you can look for yourself
i've annotated part of dissassembly
it's slightly incorrect, success = false is actually result = NULL, but you get the point...
Holger Lembke
@holgerlembke
Jun 18 2015 14:41 UTC
I asked in the espressif forum about details. I shall use os_malloc instead, was the answer. :-)
chad cormier roussel
@chadouming
Jun 18 2015 16:17 UTC
i've learned something today
always check the size of the IC you are ordering if you plan on soldering it
looks nice and easy.
the thing is : this is the actual size of it
IMG_20150618_121427.jpg
Ivan Grokhotkov
@igrr
Jun 18 2015 16:18 UTC
2.5mm? that's huge :)
chad cormier roussel
@chadouming
Jun 18 2015 16:18 UTC
haha, well not that much when you have shaky hand
anyway, i'll still try and have fun
Ivan Grokhotkov
@igrr
Jun 18 2015 16:19 UTC
i was just soldering a balun for my NRF51822 board last night
Holger Lembke
@holgerlembke
Jun 18 2015 16:19 UTC
https://github.com/esp8266/Arduino/blob/esp8266-sdk-1.0/hardware/esp8266com/esp8266/libraries/ESP8266mDNS/README.md is wrong, point 5 (update calls) is obsolete and this way very irritation, because the example doesn't have it.
Ivan Grokhotkov
@igrr
Jun 18 2015 16:19 UTC
yep, forgot to remove it
chad cormier roussel
@chadouming
Jun 18 2015 16:20 UTC
on another note, @me-no-dev , you ever looked into the altImu-v4 for drone gyro,accel,compass and baro ?
Ivan Grokhotkov
@igrr
Jun 18 2015 16:20 UTC
but I made sure it doesn't hurt calling update()
Holger Lembke
@holgerlembke
Jun 18 2015 16:20 UTC
yeah, but /me stupid will ask myself, why it isn't in the example when the readme says it must...
Ivan Grokhotkov
@igrr
Jun 18 2015 16:28 UTC
yeah, that one
chad cormier roussel
@chadouming
Jun 18 2015 16:28 UTC
must be fun
Ivan Grokhotkov
@igrr
Jun 18 2015 16:29 UTC
it was :) the hot air gun kept blowing it away, and if i tried to hold it down with tweezers i disloged it a bit from the right place
ended up building a holder out of three thin steel wires
too bad didn't take a photo
Michael Miller
@Makuna
Jun 18 2015 16:32 UTC
There is this liquid flux that is sticky, it holds parts in place for a few seconds before it bakes off. But timing is critical.
Ivan Grokhotkov
@igrr
Jun 18 2015 16:33 UTC
i'm using some kind of gel flux, but it's not sticky enough
Michael Miller
@Makuna
Jun 18 2015 16:34 UTC
what you really need is a suction part grabbing wand with the head of a needle ;-)
Ivan Grokhotkov
@igrr
Jun 18 2015 16:34 UTC
well it is sticky until you heat it above 280 at which point it gets liquid and starts spreading over the board
yeah)
Michael Miller
@Makuna
Jun 18 2015 16:35 UTC
and bionic eyes to see it with
@igrr you mentioned above that ssl code came with a version of the sdk, was it removed in latter versions?
Ivan Grokhotkov
@igrr
Jun 18 2015 16:37 UTC
right, it was only present in 0.9.3
maybe until 0.9.5... but i have it in 0.9.3
Michael Miller
@Makuna
Jun 18 2015 16:37 UTC
full source or just a library
Ivan Grokhotkov
@igrr
Jun 18 2015 16:38 UTC
source
Michael Miller
@Makuna
Jun 18 2015 16:40 UTC
did you ever glance at boringSSL or any of the others, I ask as I see limitations with axTsl that we believe was used.
Holger Lembke
@holgerlembke
Jun 18 2015 16:41 UTC
lol, watch Louis Rossmann on youtube soldering... :-) /me once managed TTSOP without killing anything. so proud. :-)))
Ivan Grokhotkov
@igrr
Jun 18 2015 16:41 UTC
I didn't check other libraries. If you see it fit the memory/API restrictions then I see no reason not using it...
you're welcome to explore in this area...
Ivan Grokhotkov
@igrr
Jun 18 2015 17:01 UTC
ideally the library should not use socket APIs as it will be harder to port it to lwIP raw API
bonus points for potential to support DTLS
COAP is a great IoT protocol in my opinion and having a secure transport layer for it on the esp would be really cool
also I imagine UDP + DTLS should be less memory hungry than TCP + TLS
Michael Miller
@Makuna
Jun 18 2015 17:09 UTC
I agree about COAP, I already have built two libraries for it, one for Arduino and one for .Net. I need DTLS to finish them.
Me No Dev
@me-no-dev
Jun 18 2015 17:10 UTC
@chadouming you can use any gyro+acc you want. Have not looked at particular modules, rather care what the chips are. MPU6050 is probably the most used one. Latest iterration (MPU9250) has a mag inside as well.
Michael Miller
@Makuna
Jun 18 2015 17:11 UTC
The other issue I am seeing about porting is that they seem to rely heavily on having a filesystem for data. Keys/certs stored in a file.
Oh, and threading ;-)
Ivan Grokhotkov
@igrr
Jun 18 2015 17:12 UTC
yes, threading mostly as the filesystem is easier to have
axTLS is good because it doesn't need either to work
but I imagine there are other options as well
we can actually move the whole thing to FreeRTOS and have tasks with preemptive scheduling, but FreeRTOS doesn't get as much attention as the IoT SDK, and also timer1 is taken so we won't have PWM
Michael Miller
@Makuna
Jun 18 2015 17:17 UTC
OpenSSL, FreeSSL, and BoringSSL are all related works. https://en.wikipedia.org/wiki/Comparison_of_TLS_implementations
chad cormier roussel
@chadouming
Jun 18 2015 17:17 UTC
@me-no-dev I thought this module was quite interesting since it had everything already include, however, just found a mpu9250 and bmp180 module, which sounds even nicer and cheaper
Me No Dev
@me-no-dev
Jun 18 2015 17:17 UTC
yup :)
Michael Miller
@Makuna
Jun 18 2015 17:19 UTC
I glanced at FreeRTOS, and it does support a cooperative tasking (non-preemptive tasking) so that should not require a timer; but I didn't look deep to how it was implemented. Task switching can add a lot of overhead that isn't obvious.
chad cormier roussel
@chadouming
Jun 18 2015 17:20 UTC
i am waiting for a motor and esc so i can make some test, i also ordered some of the mpu6050 but i think i'll get the 9250 module
Michael Miller
@Makuna
Jun 18 2015 17:26 UTC
@igrr Am I correct that axTLS does not support DTLS? It isn't mentioned in the unsupported or supported.
Michael Miller
@Makuna
Jun 18 2015 17:35 UTC
@igrr My belief is that when it comes to crypto, I want it as open as possible and up to date as possible. Not hidden behind espressif's api nor last updated in 2008 ;-)
@igrr And yes, only v9.4 included the source, v9.5 is was hidden.
Holger Lembke
@holgerlembke
Jun 18 2015 18:12 UTC
hmmm. i try to upload files with curl. sometimes the webserver accepts them, sometimes not. when not, onFileUpload is not called.
Holger Lembke
@holgerlembke
Jun 18 2015 18:42 UTC
ok. reason seems to be the timeout in readStringUntil with 1000 ms too short. raised to 5000 ms and it looks much finer.
Me No Dev
@me-no-dev
Jun 18 2015 18:50 UTC
hm... that sounds like something else is the problem
in no case you should need to wait more than a second for data
Holger Lembke
@holgerlembke
Jun 18 2015 18:50 UTC
no. line is slow.
server is slow, everything is cheap and tired.
Me No Dev
@me-no-dev
Jun 18 2015 18:51 UTC
:D
still too much
imaginr that is a whoole second
and it means that time between packages is more than a second
sounds really dial-up like
Holger Lembke
@holgerlembke
Jun 18 2015 18:53 UTC
naaa. its more a systematically problem: what if server really is slow due to other processes etc. and has to think for a moment...
what if data is generated by some slow php scripts?
Me No Dev
@me-no-dev
Jun 18 2015 18:54 UTC
but the HTTP sevrer uses readUntill only for the headers
those at least should come fast
then once you get into the file
Holger Lembke
@holgerlembke
Jun 18 2015 18:54 UTC
not in http_PUT
it collects form data
Me No Dev
@me-no-dev
Jun 18 2015 18:55 UTC
but since you use curl, it should just send the file
so it will parse just 3 lines
Holger Lembke
@holgerlembke
Jun 18 2015 18:55 UTC
as a form...
Me No Dev
@me-no-dev
Jun 18 2015 18:55 UTC
after the headers
the formBoundary
then the file name/content type
then I think one more an then starts the actuall file
Holger Lembke
@holgerlembke
Jun 18 2015 18:56 UTC
ok, I'm not sure what happens, at least 5000 ms fixed the problem...
Me No Dev
@me-no-dev
Jun 18 2015 18:57 UTC
any chace you get the headers + the start of the file coming dumped?
i would love to look at the reuest construct from curl
maybe something there ...
another reason why this timeout should stay short is that if something causes your request to hang, in the next 5 seconds no other request can be handled
which could really quickly turn into wdt reset
Holger Lembke
@holgerlembke
Jun 18 2015 19:01 UTC
POST /upload.html HTTP/1.1
Host: 10.18.121.267
User-Agent: curl/7.42.1
Accept: */*
Content-Length: 7347
Expect: 100-continue
Content-Type: multipart/form-data; boundary=------------------------b5e1af65f3482b26

--------------------------b5e1af65f3482b26
Content-Disposition: form-data; name="file"; filename="working.gif"
Content-Type: image/gif

GIF89a
nothing special.
Me No Dev
@me-no-dev
Jun 18 2015 19:03 UTC
yup, hass everything needed and not much more
Holger Lembke
@holgerlembke
Jun 18 2015 19:03 UTC
as I said, /me cheap
Me No Dev
@me-no-dev
Jun 18 2015 19:04 UTC
:)
i wander at what point you reach the timeout
with the risk to ask too much
any chance you tcpdump it so we can see what each packet contains and when is it sent from the host
maybe curl opens the connections, but actually sends the headers and data later
Holger Lembke
@holgerlembke
Jun 18 2015 19:07 UTC
Parse Form: Boundary: ------------------------736ff3c3164e30c2 Length: 7347
Request: /upload.html
 Arguments:
is what the serial debug shows...
Me No Dev
@me-no-dev
Jun 18 2015 19:09 UTC
when successful or it does not matter?
Holger Lembke
@holgerlembke
Jun 18 2015 19:09 UTC
when not successfull
when successful there are PostArg's
wireshark shows a retransmission with 550 ms time wasted
Me No Dev
@me-no-dev
Jun 18 2015 19:14 UTC
seems like a pause between the headers and the form data
can you confirm?
and who is retransmitting?
retransmission is caused by ACK not received
Holger Lembke
@holgerlembke
Jun 18 2015 19:15 UTC
could be. in total 1.1 seconds for about 8000 bytes. and it does not look bad at all.
Me No Dev
@me-no-dev
Jun 18 2015 19:16 UTC
wireshark can show you the content of the retransmitted packet
what does it contain?
Holger Lembke
@holgerlembke
Jun 18 2015 19:16 UTC
the start of the conversation
Me No Dev
@me-no-dev
Jun 18 2015 19:17 UTC
the request?
POST /blah-blah..?
Holger Lembke
@holgerlembke
Jun 18 2015 19:17 UTC
yes, POST /upload
so it shouldn't harm
Me No Dev
@me-no-dev
Jun 18 2015 19:18 UTC
but why did it not get an ACK on the first one?
can you see the ESP sending it?
should be right after the first transmission
Holger Lembke
@holgerlembke
Jun 18 2015 19:21 UTC
Unbenannt.png
holgerlembke @holgerlembke afk. food and socializing...
Me No Dev
@me-no-dev
Jun 18 2015 19:52 UTC
@holgerlembke no ACK on the first data packet, even though it got it, then large pause from the sender between the ack and the next data packet
both are strange
one on the ESP side and the other on the Sender side
Holger Lembke
@holgerlembke
Jun 18 2015 20:31 UTC
route is esp8266, wifi, router, cable, bigger 3com router, another cable, me.
Me No Dev
@me-no-dev
Jun 18 2015 20:37 UTC
that's nothing compared to internet :)
i do not think it's that at all
the ESP is the slowest of them all
Holger Lembke
@holgerlembke
Jun 18 2015 20:41 UTC
anyway. 5000 fixes it very reliable.
Me No Dev
@me-no-dev
Jun 18 2015 21:13 UTC
fixing it will be the ESP sending the first ACK :)
my ping roundtrip to LA, USA from Sofia, Bulgaria is 800ms
that is on the other side of the globe for me