These are chat archives for anacrolix/torrent

12th
Jul 2018
Denis
@elgatito
Jul 12 2018 04:12

The max writeChunk counter?

yes

Denis
@elgatito
Jul 12 2018 04:24
they do have syscall wrap for windows open file - https://golang.org/src/syscall/syscall_windows.go#L247
Matt Joiner
@anacrolix
Jul 12 2018 05:27
i'm working on some private projects, i'll have to get to the buffering after those
mainly OpenFile on Windows is a very expensive operation
i'm not sure what the android situation is, i think they just have rate-limiting to ensure app responsiveness
Denis
@elgatito
Jul 12 2018 05:28
i bet using cache layer for collecting chunks writes should help
i'm not sure about verification of pieces, while pieces data is in the cache, it probably needs own Read() to check in the cache or requested chunks
Matt Joiner
@anacrolix
Jul 12 2018 05:34
oh, did you try with a mmap storage backend? that should demonstrate much better performance on windows
(probably not a big improvement, if any on android)
Denis
@elgatito
Jul 12 2018 05:35
people said mmap was also slow while downloading
Matt Joiner
@anacrolix
Jul 12 2018 05:35
ok thx
Denis
@elgatito
Jul 12 2018 05:36
i will probably also try to code something, to see if there is any difference with the default storage
Matt Joiner
@anacrolix
Jul 12 2018 05:36
(y)
Denis
@elgatito
Jul 12 2018 07:25
there is a user saying we has 100% downloaded file and starts playback and Kodi saying the source is too slow
that means reading is very slow as well
and he's on windows
so that s the issue with openfile in windows, but still strange there is on optimization from syscall implementation in go...
maybe some flag should be set for windows?
Denis
@elgatito
Jul 12 2018 07:59
after reading golang source in file_windows.go I'm not sure it can write fast with concurrent open+writeat calls
Matt Joiner
@anacrolix
Jul 12 2018 08:35
Literally the first thing I'd do is cache file handles
Followed by merging writes
Are you super keen on this?
Denis
@elgatito
Jul 12 2018 08:36
Not even close 😁
Matt Joiner
@anacrolix
Jul 12 2018 08:36
I'm dubious though if mmap isn't faster
That should bypass all the overheads for files
Denis
@elgatito
Jul 12 2018 08:37
How you d mmap a 20gb files?
Matt Joiner
@anacrolix
Jul 12 2018 08:37
64 bit only
Good point :/
Lots of your users are 32 bit on those little devices right?
Denis
@elgatito
Jul 12 2018 08:38
Hm... not sure, about few hundreds probably
Mostly windows
Matt Joiner
@anacrolix
Jul 12 2018 08:39
Noobs :P
I wonder if it's worth supporting it
Have you tried mmapping on 64 bit? Maybe I should make that the default
For 64 bit
Denis
@elgatito
Jul 12 2018 08:40
I can try later mmap option on my windows host
people reported it did not help them
was a bit faster but not much
when I tried to use persistent filehandles on linux - it was much slower than the default which we have now
because you have to mutex all the way
windows, itself, should be fine with async io. but that looks like sync io is used in our case
when I look over io usage - it's almost - 0%, cpu usage - very low, network usage - very low. It just blocks and blocks
Matt Joiner
@anacrolix
Jul 12 2018 09:17
hm
maybe there's something in reader that's causing the slowness
Denis
@elgatito
Jul 12 2018 12:00
they look so dumb straight (reader and writer)
I will try to make a profile on windows
Denis
@elgatito
Jul 12 2018 17:26