These are chat archives for ulx3s/Lobby

11th
Oct 2020
Lawrie Griffiths
@lawrie
Oct 11 2020 00:19 UTC
@emard I used to be able to udse wget to get a file (using ftp://) from the ESP32 with pppd, but that doesn't seem to be working now.
Lawrie Griffiths
@lawrie
Oct 11 2020 00:32 UTC
I have wget working now after uploading ufttp.py again.
But socks5 does not seem to work over pppd. There seems to be a problem that the name lookup is dome on the client side with curl, so that fails. But giving it a valid IP address instead makes it hang.
you can start socks server with socks.start(verbose=10) and observe what fails.
BryanJacobs
@BryanJacobs
Oct 11 2020 03:38 UTC
hi all! I've received a ULX3S from the CrowdSupply campaign and have been very impressed with it. It took only a day to get Linux booting on RISC-V with a fully open-source stack. Excellent work.
I've raised one pull request to fix an issue that cost me a bit of time: litex-hub/litex-boards#115
thanks especially for all the revision notes in the official schematics. They made figuring out what was going wrong much easier.
drr
@danrr_au_twitter
Oct 11 2020 04:41 UTC
thanks @emard for the suggestion about flash /WP + /HOLD kept high during SPI transfer. I made the changes here and tagged 1.0.1 version with binaries ( https://github.com/dan-rodrigues/icestation-32/commit/93d9ed627ab0477f58c82c3e2a341f23e3d167fb#diff-3438871748c93ca1cc9d03dbed946b51R182 )
It really should've done that from the start. Hope that resolves it for both flash vendors.
emard
@emard
Oct 11 2020 06:53 UTC
@danrr_au_twitter great I downloaded and will now try. For flash files that don't start from 0, I suggest naming like file_flash@0x200000.bin or similar that contains flash hex address to start writing
emard
@emard
Oct 11 2020 07:10 UTC
@danrr_au_twitter I have uploaded prog-mousic.bin to 0x200000 and the lf
bitstream ics32-ulx3s-85f-pcb-buttons.bit to FPGA but on the screen I see deep blue background and white text"PRESS BUTTON\n\nISSI" I pressd all buttons but nothing happens. This is 85F with ISSI flash
btw what ppp.ifconfig() tells you? Maybe it is setting default gateway, so it cannot reach internet anymore?
Lawrie Griffiths
@lawrie
Oct 11 2020 07:34 UTC
@BryanJacobs It is excellent that you have Litex Vexriscv Linux running on your board. Most of the users on this gitter are using the SaxonSoc version of Vexriscv Linux, and we were having a discussion on the pros and cons between them, but nobody who responded had run the Litex version recently. So I wondered if you would mind answering some questions about it. I am particularly interested in whether it now boots from the SD card, whether it has the root filesystem on the SD card, or in ram, and how long it takes to boot. Which board do you have it running on?
Lawrie Griffiths
@lawrie
Oct 11 2020 07:39 UTC
@kost Here is what I am getting:
>>> socks.start(verbose=10)                                                                                                                           
[1] Server started [0.0.0.0:1080]                                                                                                                     
>>> [2] Got connection from [10.0.5.2:32962]                                                                                                          
[3] client version : 5                                                                                                                                
[3] Client Supported method number : 1                                                                                                                
[3] Client Method : 0                                                                                                                                 
[3] Server select method : 0
From:
root@buildroot:~# curl --socks5 10.0.5.1:1080 http://ifconfig.co
curl: (6) Could not resolve host: ifconfig.co
With an IP address:
[2] Got connection from [10.0.5.2:32964]                                                                                                              
[3] client version : 5                                                                                                                                
[3] Client Supported method number : 1                                                                                                                
[3] Client Method : 0                                                                                                                                 
[3] Server select method : 0                                                                                                                          
[3] client version : 5                                                                                                                                
[3] CONNECT request from client                                                                                                                       
[3] IPv4 : 92.242.132.24                                                                                                                              
[3] Port : 80                                                                                                                                         
[3] Connecting : 92.242.132.24:80
... and hangs
Lawrie Griffiths
@lawrie
Oct 11 2020 07:46 UTC
I am pretty sure the ESP32 can reach the internet, but I don't know what ppp.ifconfig() is?
What is the ppp object?
emard
@emard
Oct 11 2020 07:47 UTC
ppp is the internal pppd instance in the esp32 micropython or idf somewhere internal
mara
@vmedea
Oct 11 2020 07:48 UTC
from what i remember socks5 can do dns lookup for you on the proxy, but many programs (including curl, apparently) don't do this by default
emard
@emard
Oct 11 2020 07:48 UTC
ppp.ifconfig() without parameters will let linux side request IP addresses
mara
@vmedea
Oct 11 2020 07:48 UTC
but with an IP should definitely work… don't know why it fails
Lawrie Griffiths
@lawrie
Oct 11 2020 07:49 UTC
But how do I call ppp.ifconfig(), I hae no ppp object.
mara
@vmedea
Oct 11 2020 07:49 UTC
the socks proxy gets the incoming connection, so the pppd tunnel should be set up correctly
does your initialization script for PPP store the PPP() instance?
emard
@emard
Oct 11 2020 07:50 UTC
ppp of esp32 also gives up after few seconds and I'm thinking to register a watchdog timer to somehow restart it because it's impractical to hurry until it gives up. You have "p" object, then it has p.ppp.ifconfig() and it will print IP settings
Lawrie Griffiths
@lawrie
Oct 11 2020 07:50 UTC
Thanks
emard
@emard
Oct 11 2020 07:51 UTC
p.ppp.isconnected() will return True/False and this can be inidication that ppp needs restaring. Also p.ppp.ifconfig()[0] will return '0.0.0.0' when not connected and '10.0.5.1" when connected
Lawrie Griffiths
@lawrie
Oct 11 2020 07:52 UTC
I connected the LAN7820 Ethernet device, so that I can do DNS lookups on SaxonSoc, then curl --socks5 10.0.5.1:1080 http://ifconfig.co gives:
[3] client version : 5                                                                                                                                
[3] Client Supported method number : 1                                                                                                                
[3] Client Method : 0                                                                                                                                 
[3] Server select method : 0                                                                                                                          
[3] client version : 5                                                                                                                                
[3] CONNECT request from client                                                                                                                       
[3] IPv4 : 172.67.133.228                                                                                                                             
[3] Port : 80                                                                                                                                         
[3] Connecting : 172.67.133.228:80
mara
@vmedea
Oct 11 2020 07:53 UTC
@lawrie the DNS issue can be avoided with curl --socks5-hostname 10.0.5.1:1080 (this makes the DNS lookup happen on the SOCKS server, so that Saxon doesn't need a configured DNS)
Lawrie Griffiths
@lawrie
Oct 11 2020 07:53 UTC
and hangs.
>>> p.ppp.ifconfig()                                                                                                                                  
('10.0.5.1', '10.0.5.2', '255.255.255.255', '194.168.4.100')                                                                                          
>>>
mara
@vmedea
Oct 11 2020 07:55 UTC
but it looks to me that there's something wrong in the socks server, causing it hang with curl's specific request somehow
Lawrie Griffiths
@lawrie
Oct 11 2020 07:56 UTC
Not sure what that 194.168.4.100 ip address is, as I have the ESP32 connected to my router with IP address 192.168.0.142.
emard
@emard
Oct 11 2020 07:56 UTC
let's check can your ESP32 go to internet
Lawrie Griffiths
@lawrie
Oct 11 2020 07:57 UTC
I installed socks, via upip, so it must be able to.
mara
@vmedea
Oct 11 2020 07:57 UTC
OHHHHH i get it
your ESP32 is trying to internet-exit through the ppp
it very likely can't make any internet requests while the pppd connection is open (remember, the orignal reason for PPP on ESP32 is to be able to connect it to the internet through a modem), you'd need something akin to 'nodefaultroute' but on the ESP32 side
194.168.4.100 is a DNS server
Lawrie Griffiths
@lawrie
Oct 11 2020 08:00 UTC
So it is, I recognize it now.
emard
@emard
Oct 11 2020 08:00 UTC
from ntptime import settime
settime()
from time import localtime
localtime()
This will be a good check if it sets time from remote server that esp32 can reach internet. Yes ppp can spoil routing and we need to restore it. Could be down with ifconfig
As soon as I start ppptun() then my routing is spoiled by esp32 ppp, this is the problem
mara
@vmedea
Oct 11 2020 08:06 UTC
how much control does micropython give over the esp32's routing tables ? either want to remove the automatically added (for ppp) default gateway, or change the interface metrics so that wifi is prioritized
emard
@emard
Oct 11 2020 08:07 UTC
I don't know, I guess it's narrow in options. I think I need to sniff traffic at wifi access point to see what esp32 is doing after ppptun is started
emard
@emard
Oct 11 2020 08:15 UTC
Maybe we need again micropython recompiled with IP_FORWARD option
mara
@vmedea
Oct 11 2020 08:19 UTC
it's a long shot but it might be possible to make the socks server bind outgoing connections explicitly on the wifi interface
emard
@emard
Oct 11 2020 08:19 UTC
DNS still works after ppptun, p.ppp.ifconfing() has correct route to outside, but data content doesn't go
Lawrie Griffiths
@lawrie
Oct 11 2020 08:20 UTC
@emard Do you have curl now? It is including in the latest Smp/images/rootfs.tar
emard
@emard
Oct 11 2020 08:21 UTC
I downloaded and have to extract it. I can try that curl itself works bacause I have eth and can reach esp32 over eth, without ppp. Of course final point of interest is to get ppp connection work with routing over wifi
Lawrie Griffiths
@lawrie
Oct 11 2020 08:22 UTC
(My problem with curl on buildroot was that I had to include BR2_PACKAGE_LIBCURL and BR2_PACKAGE_LIBCURL_CURL, but not BR2_PACKAGE_CURL).
emard
@emard
Oct 11 2020 08:35 UTC
curl at saxon itself works. For saxon to reach esp32 I placed manual route as they are not on the same IP subnet:
route add <esp32_ip_at_wifi> gw <ip_of_wifi_router>
If your wifi router assigns same subnet to both wifi and eth you don't need above
root@buildroot:~# curl --socks5 <esp32_ip_at_wifi>:1080 http://ifconfig.co
then works. We just need to prevent esp32 ppp from spoiling wifi routing
Lawrie Griffiths
@lawrie
Oct 11 2020 08:43 UTC
Yes, that worked for me, but the same thing with ---sock5-hostname did not. So names are still get looked up on the SaxonSoc client side.
I did not add the route as they are on the same subnet for me.
p.ppp.ifconfig('') with tuple without default gateway should fix the problem. It needs to be run after the ppp activates the connection.
I'll check if I can route the packets over desired interface with micropython
that is another workaround
drr
@danrr_au_twitter
Oct 11 2020 08:49 UTC
thanks for the ISSI test @emard , that must mean it crashed when trying a quad IO read, but 0xab powerup and 0x9f JEDEC ID read succeeded, so that is good progress! I'll have another look at the quad config + reading incase ISSI has a difference that I missed..
Lawrie Griffiths
@lawrie
Oct 11 2020 08:52 UTC
@kost not sure what call of ifconfig you are suggesting. It needs a tuple of length 4.
I tried p.ppp.ifconfig(('10.0.5.1', '10.0.5.2', '255.255.255.255','')) but that didn't help.
Paul Ruiz
@pnru_gitlab
Oct 11 2020 08:56 UTC
I seem to recall that in 1.11 the ifconfig did not support setting parameters for ppp connections.
3rd is gateway, 4th is dns
p.ppp.ifconfig(('10.0.5.1', '10.0.5.2','',''))
Maybe this would work? Not sure.
Paul Ruiz
@pnru_gitlab
Oct 11 2020 09:02 UTC
Third is netmask i think? Maybe the problem is with the netmask, as lawrie's dns is potentially on a different subnet.
If the wifi side has netmask 255.255.255.0 then 192.168.4.x is a different subnet from 192.168.0.y
I'm looking at the general documentatiom, maybe for ppp is different since it is point to point.
Lawrie Griffiths
@lawrie
Oct 11 2020 09:06 UTC
The call with just two parameters did not work.
Caused an error, but ESP32 seems to have hung now, so starting again.
Paul Ruiz
@pnru_gitlab
Oct 11 2020 09:08 UTC
@kost: yes, it is here:
https://github.com/micropython/micropython/blob/master/ports/esp32/network_ppp.c#L227-#L255
You can still only set the net mask for PPP in 1.12
Yes. Also this worries me. as it sets if (pppapi_set_default(self->pcb) != ESP_OK) {
seems like it is setting default gw.?
without any option or variable.
Paul Ruiz
@pnru_gitlab
Oct 11 2020 09:12 UTC
From 1.11 memory: the GW gets set to the same GW as exists on the Wifi side.
Lawrie Griffiths
@lawrie
Oct 11 2020 09:12 UTC
>>> p.ppp.ifconfig(('10.0.5.1', '10.0.5.2','',''))                                                                                                    
>>> p.ppp.ifconfig()                                                                                                                                  
('10.0.5.1', '10.0.5.2', '255.255.255.255', '0.0.0.0')
Paul Ruiz
@pnru_gitlab
Oct 11 2020 09:14 UTC
@lawrie The wifi side is up when you got this result?
emard
@emard
Oct 11 2020 09:15 UTC
I tried 1.11 also but same problem. esp32 wifi is up and settime() works then I activate ppp (without saxon even connecting) and settime() no longer works
Paul Ruiz
@pnru_gitlab
Oct 11 2020 09:15 UTC

From 1.11 memory: the GW gets set to the same GW as exists on the Wifi side.

Sorry, I meant DNS.

emard
@emard
Oct 11 2020 09:16 UTC
DNS works still, sniffed traffic shows that esp32 with broken routing has requested DNS and the response was send back towards esp32. Further I don't know did esp32 take it or what
Lawrie Griffiths
@lawrie
Oct 11 2020 09:17 UTC
The DNS is currently not being used as I am testing with an explicit IP address.
I could not find any network/socket function in micropython docs to select outgoing interface :-/
anyone have experience with it?
this also means that if routing is not fixed, http/ftp proxy would also not work.
emard
@emard
Oct 11 2020 09:19 UTC
Yes they probably have intended this ppp to work dial-out and automatically switch default interface to ppp while they should'nt
at least, would be good to have it as option
defaultgw=true, false or something
emard
@emard
Oct 11 2020 09:20 UTC
I was thinking, if ppp is started first and then wifi activated, would wifi override ppp and then everything will work :)?
Lawrie Griffiths
@lawrie
Oct 11 2020 09:20 UTC
I have tried starting them both ways round with no luck.
emard
@emard
Oct 11 2020 09:21 UTC
hmmmm I will try this also to make sure
but if that means recompiling micropython to change ppp code, then it is also possible to recompile with NAT, so no real gain here :(

I was thinking, if ppp is started first and then wifi activated, would wifi override ppp and then everything will work :)?

Sounds like good idea to try!

emard
@emard
Oct 11 2020 09:24 UTC
@lawrie I can first start ppptun and then connect to wifi (using wifiman.py from my esp32ecp5). At least now ppp is active and routing works for me but esp32 must be powercycled, not only ctrl-D. I havent' tried to connect saxonsoc yet
maybe running on saxonsoc side with nodefaultroute would help:
pppd /dev/ttyUSB0 115200 10.0.5.2:10.0.5.1 noauth local debug dump nodefaultroute nocrtscts nodetach
BryanJacobs
@BryanJacobs
Oct 11 2020 09:32 UTC
@lawrie I have it (with the above patch) booting from the SD card. It uses SPI from the litex BIOS to load the kernel and initramfs on startup. Booting only takes a few seconds (20 total? Maybe?) at 50Mhz. The root filesystem is entirely an initramfs right now; I have not spent the time to install real SD drivers into the booted system, but I don't foresee any trouble. I'm using the 85F but it's only ~15% full.
Lawrie Griffiths
@lawrie
Oct 11 2020 09:33 UTC
So how free memory do you have in that configuration?
BryanJacobs
@BryanJacobs
Oct 11 2020 09:34 UTC
after finishing boot I have 19MB of userland SDRAM.
(with an absolutely bare minimum initramfs, literally just busybox and dropbear)
Lawrie Griffiths
@lawrie
Oct 11 2020 09:34 UTC
SaxonSoc has:
root@buildroot:~# free
              total        used        free      shared  buff/cache   available
Mem:          25656        5388       16660          36        3608       1865
BryanJacobs
@BryanJacobs
Oct 11 2020 09:35 UTC
that's quite a bit more. I imagine if I were to boot from an actual disk filesystem I would also have that much - the initramfs is 12MB in size.
Lawrie Griffiths
@lawrie
Oct 11 2020 09:35 UTC
But it has things like a framebuffer.
emard
@emard
Oct 11 2020 09:35 UTC
it works for me if I first start ppptun, then wifi and connect. thechnically I start saxonsoc boot, sleep(60), then start ppptun, wifi, socks
root@buildroot:~# curl --socks5 10.0.5.1:1080 http://ifconfig.co
93.128.19.18
Stock binary micropython 1.12 no ip_forward recompilation
Lawrie Griffiths
@lawrie
Oct 11 2020 09:37 UTC
@emard Can you explain that sequence a bit more.
emard
@emard
Oct 11 2020 09:37 UTC
But problem is if ppptun is equipped with watchdog it will restart if when saxonsoc doesn't connect within timeout and then it will again override wifi settings
from time import sleep
import linux85f; sleep(60); import main1.py
main1.py was renamed to not be started by default at boot, it contains ppptun, wifi, and socks.start
# main1.py
import ppptun
p=ppptun.ppptun()
from ntptime import settime
try:
  import wifiman
except:
  print("no WiFi")
import uftpd
try:
  settime()
except:
  print("NTP not available")
import socks
socks.start()
Lawrie Griffiths
@lawrie
Oct 11 2020 09:39 UTC
So do I need your wifiman? It seems to have 3 files. Do I need them all?
emard
@emard
Oct 11 2020 09:39 UTC
Well it has 1 py file and password list = 2 files I guess
Lawrie Griffiths
@lawrie
Oct 11 2020 09:40 UTC
@BryanJacobs Is there support for Ethernet. Or could you add LAN8720 RMII oe enc28j60 SPI support?
BryanJacobs
@BryanJacobs
Oct 11 2020 09:41 UTC
litex has a gpio-ethernet module but I have not yet enabled it. I was planning on using the esp32 for internet access.
my SPI driver right now is bitbanged
emard
@emard
Oct 11 2020 09:41 UTC
as soon as saxon boots you need a prepared script and start it :
#!/bin/sh
stty -F /dev/ttySL0 raw
pppd /dev/ttySL0 115200 10.0.5.2:10.0.5.1 noauth local debug dump nodefaultroute nocrtscts nodetach
BryanJacobs
@BryanJacobs
Oct 11 2020 09:42 UTC
I just got the thing a few days ago - haven't really spent any time optimizing the linux boot...
Lawrie Griffiths
@lawrie
Oct 11 2020 09:43 UTC
@BryanJacobs To use ESP32 you will need to add a uart to the ESP32, and then you are likely to hit the problems we are discussing now.
BryanJacobs
@BryanJacobs
Oct 11 2020 09:43 UTC
I haven't been following the discussion - what's the proble m with using the ESP32's second core to drive a UART over shared pins with the ECP5?
(obviously being careful to let the ESP32 continue to beacon/answer wifi)
emard
@emard
Oct 11 2020 09:45 UTC
In short it's micropython's default options. The defaults are good for "normal" users but are not what we need now.
BryanJacobs
@BryanJacobs
Oct 11 2020 09:45 UTC
okay. I don't really feel a need to run micropython.
the esp32 is well supported by platformio
emard
@emard
Oct 11 2020 09:45 UTC
Of course, you can load binary router to esp32 and let it just work
BryanJacobs
@BryanJacobs
Oct 11 2020 09:46 UTC
cool. I'll keep an eye out if I end up trying to run micropython as a virtual modem.
Lawrie Griffiths
@lawrie
Oct 11 2020 09:47 UTC
@emard Would a binary router still need PPPD?
BryanJacobs
@BryanJacobs
Oct 11 2020 09:48 UTC
I was just going to forward packets. I wasn't going to have the esp32 have a pretend IP and a virtual bridge with the ecp5.
emard
@emard
Oct 11 2020 09:48 UTC
Maybe yes binary exe for esp32 connects to wifi router and provides ppp to serial. Then linux runs pppd and goes to net. On new boards I would provide esp32-rmii wiring so we could skip ppp by expense of occupying practially all remaining pins esp32 has
I have problem in micropython that p.ppp.active(True/False ) doesn't restart ppp. This would allow use to have watchdog resetting it and let linux connect at any time. Now we are fixed to a timeout and if we miss, we could only delete ppp instance to restart, and then it will spoil wifi. If someone has some idea here...
BryanJacobs
@BryanJacobs
Oct 11 2020 09:53 UTC
it would have been nice if one of the ESP32's capacitive-touch wake pins were wired to one of the external pins
emard
@emard
Oct 11 2020 09:53 UTC
Yes they actually are wired to external
BryanJacobs
@BryanJacobs
Oct 11 2020 09:53 UTC
oh?
emard
@emard
Oct 11 2020 09:54 UTC
But I never tried, It may not work as they are shared with fpga. I must check schematics again
Lawrie Griffiths
@lawrie
Oct 11 2020 09:54 UTC
@BryanJacobs It sounds as if that now that Litex Linux supports the SD card it is much faster to load. It needs to support the rootfs on the sd card to enable more RAM. The Ethernet options would be nice. I don't believe it supports access to flash memory. Other things SaxonSoc supports are i2c to the RTC clock for battery-backed-up real time, an HDMI console with a frame buffer alsa audio and access to the oled header for driving LCD displays.
BryanJacobs
@BryanJacobs
Oct 11 2020 09:55 UTC
why would litex not support access to flash memory?
that's just generic SPI
there does exist a linux driver for litex's SD interface; it's just not merged into the mainline kernel yet so I haven't tested it
Lawrie Griffiths
@lawrie
Oct 11 2020 09:55 UTC
It says it doesn't on the Litex Vexriscv readme. Maybe just needs enabling.
BryanJacobs
@BryanJacobs
Oct 11 2020 09:56 UTC
I think it just needs enabling.
emard
@emard
Oct 11 2020 09:56 UTC
No, sorry on v3.0.8 capacitive touch are not connected. :(. Other option would be using esp32 hall sensor and approaching a magnet :). Or in the core I route btn[0]->wifi_gpio0 for some signaling to esp32
BryanJacobs
@BryanJacobs
Oct 11 2020 09:56 UTC
yeah I saw from the schematics that none of the pins that support cap touch wake are wired. It's unfortunate
Lawrie Griffiths
@lawrie
Oct 11 2020 09:57 UTC
Another thing we have with SaxonSoc is a native lcc compiler. That should work on the Litex version too.
emard
@emard
Oct 11 2020 09:57 UTC
New schematics v3.1.3 should route capacitive touch outside shared to gp/gn13 or similar
BryanJacobs
@BryanJacobs
Oct 11 2020 09:58 UTC
I'm not trying to argue for one system or another here. I picked litex to start because I preferred migen over spinalhdl, and neither system seemed to be lacking something crucial.
3.1.3? I've been referencing 3.1.2 schematics
emard
@emard
Oct 11 2020 09:59 UTC
PDF is not released but in kicad it is already 3.1.3, its written on PCB v3.1.3? because that is not yet ready fully
BryanJacobs
@BryanJacobs
Oct 11 2020 09:59 UTC
ah I didn't think to load the kicad files instead of the PDFs. Thanks for the tip.
Lawrie Griffiths
@lawrie
Oct 11 2020 09:59 UTC
I'm not arguing for one over another either. It would be nice to add to the Litex version, the things that SaxonSoc has, so users have a real choice.
BryanJacobs
@BryanJacobs
Oct 11 2020 09:59 UTC
it should be possible to integrate the generated verilog files to use the modules together
all you'd need is a bit of software glue to import spinalhdl modules from migen and/or vice versa
litex uses a wishbone bus though
Lawrie Griffiths
@lawrie
Oct 11 2020 10:01 UTC
I was mainly trying to understand what the Litex version currently supports, and what is doesn't, and what would be easy to add.
BryanJacobs
@BryanJacobs
Oct 11 2020 10:04 UTC
okay. I've tested SDRAM, SPI, MicroSD card access (inc both via SPI and more feature rich). The hardware list is at https://github.com/enjoy-digital/litex
Lawrie Griffiths
@lawrie
Oct 11 2020 10:04 UTC
USB keyboard and mouse support is something that SaxonSoc does not support and @Dolu1990 is looking at adding. I am not sure whether Litex has usb host support which could be added to Linux.
BryanJacobs
@BryanJacobs
Oct 11 2020 10:04 UTC
it lists VGA/HDMI, Ethernet, etc
oh, I actually tested USB ACM support too for serial
https://github.com/enjoy-digital/usb3_pipe looked pretty interesting
Lawrie Griffiths
@lawrie
Oct 11 2020 10:11 UTC
But that looks like it needs Serdes, which I don't believe the Ulx3s has.
BryanJacobs
@BryanJacobs
Oct 11 2020 10:12 UTC
then I guess it'll only be able to hit usb2 speeds
anyway, the feature list is on that page. USB/SDRAM/etc is built into the core so isn't on the list
I'll try connecting USB-HID devices to the second USB port in a few days.
Lawrie Griffiths
@lawrie
Oct 11 2020 10:14 UTC
Of course SpinalHDL generated Verilog is already used by the LiteX version as it uses VexRiscv.
BryanJacobs
@BryanJacobs
Oct 11 2020 10:14 UTC
yeah, I just meant the other soc parts
there's no technical reason I can see preventing bidirectional sharing of soc cores. We don't have any licensing problems here :-D
once someone has written the digital logic it should be possible to use it in different contexts
Lawrie Griffiths
@lawrie
Oct 11 2020 10:25 UTC
@emard I am trying to understand what I need to do to get socks working. It sounds like I need your wifiman. Do I need wifiman.py or main_wifiman.py and what usernames and passwords are in wifiman.conf?
For starting pppd on SaxonSoc, I am currently using screen. Otherwise it hangs up the terminal and if you omit the "nodetach", you don't see errors.
Using screen means it will be slower to attach and ppptun starts.
I would prefer a solution where I didn't have to change main.py.
emard
@emard
Oct 11 2020 10:34 UTC
@lawrie main_wifiman.py is to be named as main.py in order to autostart wifiman, default main.py should not be used as esp32 will immediately connect and we don't want it now as ppp will override it then. I should try to assemble some workaround set that lets you plug power wait and saxon connects to ppp
Lawrie Griffiths
@lawrie
Oct 11 2020 10:57 UTC
I am currently reinstalling micropython on my 85F to fix the full flash problem. If I don't have a main.py then I can't connect via webrepl and it makes things more difficult.
Lawrie Griffiths
@lawrie
Oct 11 2020 11:12 UTC
I don't know how to flash micropython on my Ubuntu 20.04 system. The version of esptool that comes with the system fails to connect.
The version in ulx3s-bin needs pyserial. I assume in needs the python2 version. pip is not available in 20.04, just pip3. Not sure what the best way round this is.
emard
@emard
Oct 11 2020 11:15 UTC
For me python3 works as flasher too.
Lawrie Griffiths
@lawrie
Oct 11 2020 11:18 UTC
I got round it by doing it on my older 16.04 Linux system.
emard
@emard
Oct 11 2020 11:23 UTC
https://github.com/emard/esp32ppp here is some summary with scripts that connect for my board. Saxon bitstream is flashed so it will start at power up automatically
I have also fixed the issue you had @emard : kost/micropython-socks#1
Lawrie Griffiths
@lawrie
Oct 11 2020 11:54 UTC
@emard I have reset micropython with v12 on my 85F and have it starting up using wifiman. I am looking at your solution in esp32ppp, but it all looks a bit error prone, if, for example, you don't start SaxonSoc soon enough, and it is SaxonSoc-specific.
I don't really like having to wait a minute before I can connect with webrepl.
Would a script that disconnects the wifi, runs ppptun and then reconnects, work?
Lawrie Griffiths
@lawrie
Oct 11 2020 12:21 UTC
@emard So I tried you solution and it failed. curl returns an error code. And I am not sure how to diagnose it.
Actually it works if I give the domain name not the ip address, but that relies on having Ethernet connected.
emard
@emard
Oct 11 2020 12:29 UTC
@lawrie I will recheck my scripts again. It is dirty workaround and I'm also not happy to wait 1 min for nothing. A better solution was to ask micropythoneers to let ppp run persistent, not to give up. This will relieve us from time window at least. Disconnecting wifi and again running ppp could work but then wifi connection will bounce until PPP gets connected, still a bad solution
@lawrie does linux-esp32 ppp connect together?
ifconfig

4: ppp0: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 3
   link/ppp 
   inet 10.0.5.2 peer 10.0.5.1/32 scope global ppp0
      valid_lft forever preferred_lft forever
emard
@emard
Oct 11 2020 12:37 UTC
Weellll it doesn't work for me either if I unplug eth cable :). PPP is connected, esp32 can settime() to network and curl doesn't work :)
emard
@emard
Oct 11 2020 12:46 UTC
@kost can I upgrade with just a new upip.install("micropython-socks")
Here are some errors I get:
ping 10.0.5.1
PING 10.0.5.1 (10.0.5.1): 56 data bytes
64 bytes from 10.0.5.1: seq=0 ttl=255 time=27.214 ms
64 bytes from 10.0.5.1: seq=1 ttl=255 time=42.307 ms
root@buildroot:~# telnet 10.0.5.1 1080
Connected to 10.0.5.1

root@buildroot:~# curl --socks5 10.0.5.1:1080 http://140.82.121.4
root@buildroot:~# curl --socks5 10.0.5.1:1080 http://172.67.133.228
error code: 1003root@buildroot:~#
Lawrie Griffiths
@lawrie
Oct 11 2020 12:48 UTC
@emard I changed you main.py to set verbose t see what's going on. It does work for me apart from the hostname issue, and having to be quick starting SaxonSoc.
I think the IP addresses that appear to be for ifconfig.co don't work.
emard
@emard
Oct 11 2020 12:52 UTC
Maybe! Maybe is ipconfig fooled by esp32 networking and doesn't know what to answer. I can try something else to download
Lawrie Griffiths
@lawrie
Oct 11 2020 12:52 UTC
socks5-hostname also gives errors.
emard
@emard
Oct 11 2020 13:05 UTC
Hey this is working, getting yahoo non-ssl homepage
curl --socks5 10.0.5.1:1080 http://87.248.100.216/index.html
Lawrie Griffiths
@lawrie
Oct 11 2020 13:10 UTC
Yes that works for me, but often IP addresses don't work correctly.
We need a solution where we can use host names.
It ought to be ---socks5-hostname
emard
@emard
Oct 11 2020 13:15 UTC
This doesn't work for me if I understood option syntax correctly:
curl --socks5-hostname 10.0.5.1:1080 http://www.yahoo.com/index.html
curl: (7) Failed to receive SOCKS5 connect request ack.
mara
@vmedea
Oct 11 2020 13:16 UTC
it looks like the socks server at least has the code to handle such requests: https://github.com/kost/micropython-socks/blob/master/socks.py#L166
Lawrie Griffiths
@lawrie
Oct 11 2020 13:16 UTC
That should be correct.
lawrie@ubuntu20:~$ curl --help | grep socks5-hostname
     --socks5-hostname <host[:port]> SOCKS5 proxy, pass host name to proxy
emard
@emard
Oct 11 2020 13:21 UTC
@kost we need help :)
Lawrie Griffiths
@lawrie
Oct 11 2020 13:27 UTC
Looks like you need to use --socks5 and --socks5-hostname
But the effect is sometimes "redirect"
No, that relies on a local DNS as well.
Lawrie Griffiths
@lawrie
Oct 11 2020 13:37 UTC
And it was wrong anyway.
There is also this syntax: curl --proxy socks5h://10.0.5.1
But "h" variant again does not work, so perhaps the micropython socks5 does not support it.
Lawrie Griffiths
@lawrie
Oct 11 2020 13:53 UTC
@emard @vmedea The latest Smp/images/rootfs.tar includes netcat.
mara
@vmedea
Oct 11 2020 13:57 UTC
right, the scheme-based syntax like socks5h:// is a different syntax the result should be the same.
it's definitely possible that the code to handle hostname connections in the micropython socks is not working: it looks like it's passing the (domainname, dst_port) to sock.connect as-is instead of doing a DNS lookup explicitly with getaddrinfo. i don't know enough about micropython to know if this would be necessary (https://docs.micropython.org/en/latest/library/usocket.html suggest that is the most portable way though).
great !!!
mara
@vmedea
Oct 11 2020 14:06 UTC

just tested that this works on the ESP32 (only tested without ppp connection running):

import socket
sock = socket.socket()
sock.connect(("www.micropython.org", 80))
sock.write("GET / HTTP/1.0\r\n\r\n")
data=sock.read()

… so as long as the DNS works, that code should work, no need for explicit DNS lookup

mara
@vmedea
Oct 11 2020 14:12 UTC
you could try the above on the esp32 while ppp is running, to see if it somehow interferes
emard
@emard
Oct 11 2020 14:14 UTC
Above works well with ppp connected on my esp32
mara
@vmedea
Oct 11 2020 14:17 UTC
but socks-by-host does not, okay, … must be something in the proxy then, does it still log the [3] Domain name: %s"?
mara
@vmedea
Oct 11 2020 14:23 UTC
i get this while ppp connected:
>>> sock.connect(("www.micropython.org", 80))                                                                                                         
Traceback (most recent call last):                                                                                                                    
  File "<stdin>", line 1, in <module>                                                                                                                 
OSError: 202
Lawrie Griffiths
@lawrie
Oct 11 2020 14:24 UTC
No, that Domain message is not logged.
With the socks5h request, I just get this:
 [2] Got connection from [10.0.5.2:35372]                                                                                                          
[3] client version : 5                                                                                                                                
[3] Client Supported method number : 1                                                                                                                
[3] Client Method : 0                                                                                                                                 
[3] Server select method : 0                                                                                                                          
[3] client version : 5                                                                                                                                
[3] CONNECT request from client                                                                                                                       
[3] client version : 119                                                                                                                              
[5] Closing socket on client                                                                                                                          
[5] Closed socket on client
Whereas with socks5, I get:
[2] Got connection from [10.0.5.2:35374]                                                                                                              
[3] client version : 5                                                                                                                                
[3] Client Supported method number : 1                                                                                                                
[3] Client Method : 0                                                                                                                                 
[3] Server select method : 0                                                                                                                          
[3] client version : 5                                                                                                                                
[3] CONNECT request from client                                                                                                                       
[3] IPv4 : 91.103.219.224                                                                                                                             
[3] Port : 80                                                                                                                                         
[3] Connecting : 91.103.219.224:80                                                                                                                    
[2] Tunnel connected! Tranfering data...                                                                                                              
[2] No data received from local! Closing...                                                                                                           
[5] Closing socket on client                                                                                                                          
[5] Closed socket on client                                                                                                                           
[2] Error sending fromlocal(): [Errno 110] ETIMEDOUT                                                                                                  
[5] Closing socket on client                                                                                                                          
[5] Closed socket on client
Despite the apparent errors, that second one works.
emard
@emard
Oct 11 2020 14:29 UTC
handleconn has try/except loop that will print nothing in case of failure
mara
@vmedea
Oct 11 2020 14:30 UTC
"client version 119" yes that's wrong alright, as if it sees the w in the hostname as the first byte of a new socks request
Lawrie Griffiths
@lawrie
Oct 11 2020 14:31 UTC
Ah, I was just about to comment on that.
emard
@emard
Oct 11 2020 14:34 UTC
I have posted an issue in hope that they will support persistent PPP micropython/micropython#6541
@kost can I upgrade with just a new upip.install("micropython-socks")
Yes, it should work.
emard
@emard
Oct 11 2020 14:45 UTC
Can you check name resolving problem (I just posted an issue shortly decribing situation we have here)
@kost we need help :)
Sorry, was driving back home. I'll check hostname resolving.
emard
@emard
Oct 11 2020 15:19 UTC
In the mantime I loaded micropython with ip forward which @pnru_gitlab recompiled, logged with ssh thru ppp from PC and played a party of tetris. It had random network lag, annoying but playable
with help of ip_forward I can use DNS of LAN and then curl works... a temporary fix until kost clears the bug
mara
@vmedea
Oct 11 2020 15:31 UTC
i think i found the problem: it's adding bytes to a unicode string here https://github.com/kost/micropython-socks/blob/master/socks.py#L169
yep. just commited fixes here: kost/micropython-socks@f4947b1
it was missing decode in one place and encode in other.
updated on pypi as well. so, you can just update with upip.install()
mara
@vmedea
Oct 11 2020 15:38 UTC
i was about to suggest to just make domainname a b'', but that works too :)
sorry about that.
@vmedea your way is actually faster, will fix it that way. thanks.
emard
@emard
Oct 11 2020 15:46 UTC
@kost hey it works thanx for quickly fixing!
emard
@emard
Oct 11 2020 16:04 UTC
this is saxonsoc related: https doesn't work
curl --proxy socks5h://10.0.5.1:1080 https://ifconfig.co/
i guess you need ca-certificates?
and curl with compiled with any ssl library?
(openssl, libressl, etc)
emard
@emard
Oct 11 2020 16:06 UTC
Yes, I think it has some ssl /usr/lib/libssl.so.1.1
curl --version should tell you compiled in libs
emard
@emard
Oct 11 2020 16:13 UTC
curl 7.68.0 (riscv32-buildroot-linux-gnu) libcurl/7.68.0 OpenSSL/1.1.1f zlib/1.2.11
Release-Date: 2020-01-08
Protocols: dict file ftp ftps gopher http https imap imaps pop3 pop3s rtsp smb smbs smtp smtps telnet tftp 
Features: AsynchDNS HTTPS-proxy IPv6 Largefile libz NTLM SSL TLS-SRP UnixSockets
Lawrie Griffiths
@lawrie
Oct 11 2020 16:23 UTC
All working for me now. So we can now fetch files without using Ethernet.
emard
@emard
Oct 11 2020 16:26 UTC
It is a really big improvement, copying files and logging from remote console. Small drawback for this login, micropython with ip_forward compiled is needed but I have on esp32ppp link to binary v1.11 with forward
emard
@emard
Oct 11 2020 16:35 UTC
@lawrie have you got https fetch working maybe?
Lawrie Griffiths
@lawrie
Oct 11 2020 16:36 UTC
No.
The buildroot config files has:
BR2_PACKAGE_LIBOPENSSL=n
BR2_PACKAGE_OPENSSH=n
emard
@emard
Oct 11 2020 16:39 UTC
OK, it's not urgent just asking
Lawrie Griffiths
@lawrie
Oct 11 2020 16:40 UTC
Not sure what the effect of including those is. There is probably more to it than that. There is presumably something to do with certificates. I think @Dolu1990 was concerned about the CPU usage of openssl.
are you including ca-certificates package?
Lawrie Griffiths
@lawrie
Oct 11 2020 16:42 UTC
Don't think so. Not sure where that is specified.
There is a BR2_PACKAGE_CA_CERTIFICATES package.
Could try to add that too.
Might have a go at that tomorrow.
mara
@vmedea
Oct 11 2020 16:53 UTC
ha, tetris > /dev/tty1 just works, output is on the monitor while input still comes from the ttty—besides aspect ratio of 640×480 being horrible on a widescreen monitor it works great :)
@lawrie I think you need something like that in order to have basic set of CA certificates
for SSL/TLS to function
emard
@emard
Oct 11 2020 18:06 UTC
@vmedea :) eheheeehe yes it is light and rather clean, I'm very glad that unix redirection ">" works for terminal output. there is "vi" editor at saxon which can still work this way :). "nano" will not
mara
@vmedea
Oct 11 2020 18:35 UTC
that's good to know thanks !
mara
@vmedea
Oct 11 2020 19:37 UTC
i don't seem to be able to get sound working on saxonsoc on a ULX3S 85F board, trying to play various mp3s using mpg123 -T -f 4096 "$1" as suggested in the README, i hear a tick when it is supposed to start playing but nothing after that, same for speaker-test, are you supposed to just be able to connect any earphones or does it need an external amplifier? there is no mute or volume control or anything?
Lawrie Griffiths
@lawrie
Oct 11 2020 19:58 UTC
It should just work with earphones. You can use alsamixer to set the volume but it should not be necessary.
mara
@vmedea
Oct 11 2020 20:07 UTC
trying some other earphones i do hear sound, really softly
awesome
mara
@vmedea
Oct 11 2020 20:12 UTC
it does skip every few seconds and then logs [...] spinal,lib-audio-out 10094000.audio_out: Start cha=2 rate=44100 buf=8820 per=2940, but i suspect this is because it's having trouble decoding a 320kbit vbr mp3 in real time, cpu usage is very high
Lawrie Griffiths
@lawrie
Oct 11 2020 20:16 UTC
I just tried it and it is very low volume through by earphones. It works better with an amplified speaker. Also I couldn't make alsamixer work. I have not tried this before, I just copied what @Dolu1990 told me for the .asoundrc file.
mara
@vmedea
Oct 11 2020 20:18 UTC
alsamixer (with the .soundrc/.asoundrc file) didn't work for me either
Lawrie Griffiths
@lawrie
Oct 11 2020 20:20 UTC
It should be .asoundrc, I will correct the README
But that doesn't work for me either.
alsamixer worked when I logged out and back in again.
Lawrie Griffiths
@lawrie
Oct 11 2020 20:26 UTC
But it was already at 100%.
Both sound and the HDMI framebuffer are using DMA.
mara
@vmedea
Oct 11 2020 20:29 UTC
i'll try that, but yes i was afraid of that already that it's only used to scale down the volume
speaking of HDMI, streaming sound over that would be interesting
Lawrie Griffiths
@lawrie
Oct 11 2020 20:31 UTC
I think that is possible, but I am not sure anyone has attempted it. SPDIF sound is good too. @danrr_au_twitter's game supports that, but SaxonSoc doesn't.
mara
@vmedea
Oct 11 2020 20:31 UTC
anyhow going to look for amplified speakers
Lawrie Griffiths
@lawrie
Oct 11 2020 20:32 UTC
I have a variety of cheap ones that I used with Raspberry Pis.
emard
@emard
Oct 11 2020 20:35 UTC
using more bits for DAC can make it really loud
mara
@vmedea
Oct 11 2020 20:48 UTC
that game looks really nice, SPDIF sounds like a good option too
drr
@danrr_au_twitter
Oct 11 2020 21:04 UTC
I used the spdif_tx from the ulx projects repo and it works great. Apart from slight wiring changes, just need to use address_out to support stereo output. I think the sample integration was mono only but the tx module supports stereo
drr
@danrr_au_twitter
Oct 11 2020 21:09 UTC
I also spotted a very silly bug in the ISSI flash config in that repo, it doesn’t deassert CS on final flash config command. Ofcourse the winbond one is fine. I’ll correct this later today
emard
@emard
Oct 11 2020 21:38 UTC
@danrr_au_twitter I used mono because stereo didn't synthesize properly with some bigger core for a 128-voice hammond-alike synth which needs a single channel high quality 24-bit. Yes enable stereo 2x16-bit, if it plays sound, everything is fine!