These are chat archives for TheHolyWaffle/TeamSpeak-3-Java-API

10th
Aug 2016
Bert De Geyter
@TheHolyWaffle
Aug 10 2016 15:39
@Firedroide about that recent issue with delays
Roger Baumgartner
@rogermb
Aug 10 2016 15:40
Yup?
Bert De Geyter
@TheHolyWaffle
Aug 10 2016 15:40

I'm not so sure about restructuring the commands. I like how it gives a great overview right now of all the commands in a package. Each command deserves to be his own class.

Besides that, how can you send a new command to the server when the answer for the previous command isn't received yet?

Roger Baumgartner
@rogermb
Aug 10 2016 15:43
Shouldn't that work? TCP sockets allow us to send independent of receives, no?
I might be completely wrong on this ^^
Bert De Geyter
@TheHolyWaffle
Aug 10 2016 15:43
I know, I didn't mean on a TCP level
but more on the application level
Because if you send 2 commands simultaneously
there is no way to differentiate the 2 answers we'll get
As in: which answer belongs to which command we've just sent.
Roger Baumgartner
@rogermb
Aug 10 2016 15:44
we don't send them simultaneously, we just don't wait for the response to arrive
the sequential order still exists
Bert De Geyter
@TheHolyWaffle
Aug 10 2016 15:44
oh, are the answers still sequential?
Roger Baumgartner
@rogermb
Aug 10 2016 15:44
yup
Bert De Geyter
@TheHolyWaffle
Aug 10 2016 15:44
Can the server not mix them up?
Since one command might be handled quicker than the other
regardless of whether it came in first
Roger Baumgartner
@rogermb
Aug 10 2016 15:45
TCP guarantees the correct ordering of packets and as far as I know the server handles them synchronously
But you're right, there's no real guarantee for the second one :/
Bert De Geyter
@TheHolyWaffle
Aug 10 2016 15:45
Well, if you're pretty sure it cannot mix up
then feel free to work with that in mind :D
Anyway, you were testing everything on localhost right?
Roger Baumgartner
@rogermb
Aug 10 2016 15:46
Yeah, but with thousands of identical commands and 0 ping. I've not yet tested with lots of ping and different commands that could take different amounts of time to process
Bert De Geyter
@TheHolyWaffle
Aug 10 2016 15:46
You might want to take a look at this little tool to simulate a bad network: https://jagt.github.io/clumsy/
It has options to increase latency, switch up packet orders, ...
Roger Baumgartner
@rogermb
Aug 10 2016 15:47
That's cool, let's hope it works ^^
I've tried a similar tool before and the only thing it did was give me BSODs :smile:
Bert De Geyter
@TheHolyWaffle
Aug 10 2016 15:48
this one is pretty well known
you should be good, but make sure to enter a filter
otherwise EVERYTHING on your desktop will be impacted
Roger Baumgartner
@rogermb
Aug 10 2016 15:48
Thanks for letting me know :smile:
By the way, I need to show you this
[2016-08-10 17:42:03.602] > clientupdate
[2016-08-10 17:42:03.723] [clientupdate] < error id=0 msg=ok
[2016-08-10 17:43:03.602] > clientupdate
[2016-08-10 17:43:03.649] [clientupdate] < error id=0 msg=ok
[2016-08-10 17:44:03.604] > clientupdate
[2016-08-10 17:44:03.642] [clientupdate] < error id=0 msg=ok
[2016-08-10 17:45:03.604] > clientupdate
[2016-08-10 17:45:03.726] [clientupdate] < error id=0 msg=ok
Despite multiple commands being sent at once, the ordering seems to stay consistent
But because of that the timestamps are completely messed up now
Bert De Geyter
@TheHolyWaffle
Aug 10 2016 15:51
I see
Roger Baumgartner
@rogermb
Aug 10 2016 15:52
I'll have to test it a bunch more. The results are extremely interesting though ^^
Bert De Geyter
@TheHolyWaffle
Aug 10 2016 15:52
I wouldn't modify the logging so that answers appear right after the command.
Roger Baumgartner
@rogermb
Aug 10 2016 15:52
Well, I'm not. It just orders it like that for some arcane reason. But there's 0 guarantee for it to show up like that
Bert De Geyter
@TheHolyWaffle
Aug 10 2016 15:53
you mean that the answer is lightning fast?
in your example
Roger Baumgartner
@rogermb
Aug 10 2016 15:53
Nah, in this case it's probably just some random occurrence
Cause I just noticed that I'm a huge derp
I tested that stuff with the synchronous API which waits for the server response before returning FailFish
Bert De Geyter
@TheHolyWaffle
Aug 10 2016 15:54
oh ye, I remember a professor saying once that system.out.println is never printing to console in a guaranteed order
Roger Baumgartner
@rogermb
Aug 10 2016 15:55
There's no way for it to do that without blocking all other calls, yeah
The real output when used with the async API:
[2016-08-10 17:56:00.906] > clientupdate
[2016-08-10 17:56:00.906] > clientupdate
[2016-08-10 17:56:00.906] > clientupdate
[2016-08-10 17:56:00.907] > clientupdate
[2016-08-10 17:56:00.907] > clientupdate
[2016-08-10 17:56:00.908] > clientupdate
[2016-08-10 17:56:00.985] [clientupdate] < error id=0 msg=ok
[2016-08-10 17:56:01.001] [clientupdate] < error id=0 msg=ok
[2016-08-10 17:56:01.002] [clientupdate] < error id=0 msg=ok
[2016-08-10 17:56:01.002] [clientupdate] < error id=0 msg=ok
[2016-08-10 17:56:01.004] [clientupdate] < error id=0 msg=ok
[2016-08-10 17:56:01.004] [clientupdate] < error id=0 msg=ok
But that's pretty cool, tbh. I was able to send all of those commands in 1 round-trip-time
Bert De Geyter
@TheHolyWaffle
Aug 10 2016 15:58
Oh ye, I encountered a weird bug
Roger Baumgartner
@rogermb
Aug 10 2016 15:59
What kind of weird bug? :stuck_out_tongue:
Bert De Geyter
@TheHolyWaffle
Aug 10 2016 15:59
when the command delay is bigger than the "no answer received" timeout then errors start popping up
I don't have a log
but you can recreate it by setting the floodrate to 5 seconds
and using the synchronous api
Roger Baumgartner
@rogermb
Aug 10 2016 16:00
command delay as in round trip time of packets back and forth?
Bert De Geyter
@TheHolyWaffle
Aug 10 2016 16:00
the floodrate delay
Roger Baumgartner
@rogermb
Aug 10 2016 16:01
Isn't that to be expected, though?
Bert De Geyter
@TheHolyWaffle
Aug 10 2016 16:01
well, the commands should be slowed down
but that shouldn't result in "no answer for command X" errors
because of a huge floodrate delay
Roger Baumgartner
@rogermb
Aug 10 2016 16:02
Oooh, now I can follow you
Yeah, that shouldn't happen O.o
I'll do some tests
Bert De Geyter
@TheHolyWaffle
Aug 10 2016 16:03
I encountered this when using the api on a server with strict flood restrictions. So I had to increase the floodrate ms to a huge number in order to make it work.
Roger Baumgartner
@rogermb
Aug 10 2016 16:07
Oh, I see why it happens. Take a look at TS3Query#doCommand, the command timeout time is measured between when the command is enqueued and when the response arrives
Bert De Geyter
@TheHolyWaffle
Aug 10 2016 16:07
but how come the response arrives so late?
Roger Baumgartner
@rogermb
Aug 10 2016 16:07
So if the query is still busy sending other stuff or waiting for the floodrate timeout to expire, that doCommand timer will expire and throw an exception
Bert De Geyter
@TheHolyWaffle
Aug 10 2016 16:08
ah I see
Roger Baumgartner
@rogermb
Aug 10 2016 16:08
Yeah, the timeout currently happens in the wrong place of the API, basically ^^
Basically at the moment it's a guarantee that no matter what happens, x seconds after you call TS3Api.someMethod, you'll either have a response or have an exception thrown
What you're suggesting and what is should probably be is that the call will block forever until the command has been actually sent and then x seconds after that an exception will be thrown
if no response has arrived, of course
Bert De Geyter
@TheHolyWaffle
Aug 10 2016 16:11
yep, pretty much
so that every operation in synchr api will never go past the the timeout
Roger Baumgartner
@rogermb
Aug 10 2016 16:12
That way we'll lose the guarantee that any command invocation will terminate after a fixed amount of time, however
Bert De Geyter
@TheHolyWaffle
Aug 10 2016 16:13
I'm not sure I understand what you're saying :P
from how I view it
on this line
c.isAnswered should become true
no matter if the floodrate is greater than the command timeout
Roger Baumgartner
@rogermb
Aug 10 2016 16:15
one moment, brb
At the moment we're doing this: at the start of doCommand we set ourselves a "due date". If nothing happens until that time, bail out. So even if there are lots of commands in the queue before our command (or in your scenario: the floodrate is greater than the command timeout), there will be a timeout exception despite the query working perfectly
And there's no command timeout for asynchronous commands
Roger Baumgartner
@rogermb
Aug 10 2016 16:20
So yeah, I agree that we should change that behavior
The timeout should be more along the lines of: If no data is received for x time despite a command being in the queue, there must have been some kind of connection problem
Do you agree with that so far?
Bert De Geyter
@TheHolyWaffle
Aug 10 2016 16:32
Yep, so we basically should calculate the due date based when the command is sent to the server and not when it's placed in the queue to be sent to the server.
Roger Baumgartner
@rogermb
Aug 10 2016 16:34
Uhm, no, that's what we're currently doing :smile:
Bert De Geyter
@TheHolyWaffle
Aug 10 2016 16:35
Huh?
Roger Baumgartner
@rogermb
Aug 10 2016 16:35
The problem being that in your case (too high flood rate timeout), that command just doesn't get taken out of that queue fast enough
wait, I misread what you said
Yeah, exactly
Bert De Geyter
@TheHolyWaffle
Aug 10 2016 16:35
I thought so :D
Roger Baumgartner
@rogermb
Aug 10 2016 16:35
Sorry xD
We also maybe shouldn't throw an exception for just that command
We should consider just triggering a normal disconnect in that case
Bert De Geyter
@TheHolyWaffle
Aug 10 2016 16:37
ah I see
what with the users who put try-catch's around their operations to see if it was sent?
Roger Baumgartner
@rogermb
Aug 10 2016 16:38
Yeah, that would be a problem :/
The whole concept of a timeout just doesn't play nicely with the new reconnect system in general
Bert De Geyter
@TheHolyWaffle
Aug 10 2016 16:39
I agree
Roger Baumgartner
@rogermb
Aug 10 2016 16:40
I wonder if I could do some arcane magic so that:
  • if the default ConnectionHandler is set (the one that just drops the connection): Throw some kind of TS3DisconnectException
  • if some other handler is installed: Just wait it out
Bert De Geyter
@TheHolyWaffle
Aug 10 2016 16:40
where would they catch that exception?
Roger Baumgartner
@rogermb
Aug 10 2016 16:41
still at the same place, provided they don't have a reconnect handler installed
It would be even better if the ConnectionHandler could throw that exception on its own
Bert De Geyter
@TheHolyWaffle
Aug 10 2016 16:42
I'm not quite up to date on how the reconnection works.
How does it detect a disconnect right now?
Roger Baumgartner
@rogermb
Aug 10 2016 16:43
Yeah, it's pretty complicated internally right now ^^
In short: If SocketReader stops without being interrupted, we take that as there being a connection problem. In that case, it'll call TS3Query.fireDisconnect, which in turn calls the installed ConnectionHandler
Bert De Geyter
@TheHolyWaffle
Aug 10 2016 16:45
oh ok
because I've tried fiddling myself with disconnect situations
and I tried socket.isConnected and such but they are not reliable :)
Roger Baumgartner
@rogermb
Aug 10 2016 16:45
If that handler doesn't do anything (which is the default implementation), we remain in that disconnected state
Yeah, I've noticed that, too :smile:
Bert De Geyter
@TheHolyWaffle
Aug 10 2016 16:46
stackoverflow says the only way to detect a DC is to write on the socket. If it fails, then you're disconnected.
Roger Baumgartner
@rogermb
Aug 10 2016 16:46
And in JavaLand, it sometimes doesn't even fail but instead it blocks indefinitely
reading from the inputstream throws an exception, on the other hand
or returns null
and that's how we're currently detecting those problems
Bert De Geyter
@TheHolyWaffle
Aug 10 2016 16:47
+
:+1:
Roger Baumgartner
@rogermb
Aug 10 2016 16:48
Anyways, user connection handlers are free to call TS3Query.connect when onDisconnect is called
That's how those control the timeout between the retries / set an upper limit, etc
Bert De Geyter
@TheHolyWaffle
Aug 10 2016 16:49
oh? I was using my own connectionhandler
so it won't reconnect by itself because I do nothing in my onDisconnect?
Roger Baumgartner
@rogermb
Aug 10 2016 16:50
Depends on how you're using it, but most likely no
Bert De Geyter
@TheHolyWaffle
Aug 10 2016 16:50
I thought the connectionhandlers were just a bunch of events we could use to handle some stuff.
Not to do connection stuff ourselves
Roger Baumgartner
@rogermb
Aug 10 2016 16:50
Yeah, I had to change that hole part because the whole inheritance stuff got way to complicated
I've updated the ReconnectExample, you should check that one out
I've introduced these various ReconnectStrategies, which handle all the calls to TS3Query.connect and whatnot for the users
Bert De Geyter
@TheHolyWaffle
Aug 10 2016 16:53
oh ok, now I get it
Roger Baumgartner
@rogermb
Aug 10 2016 16:53
And those will then call the connectionhandler supplied by the user
Bert De Geyter
@TheHolyWaffle
Aug 10 2016 16:54
It'd be cool if the reconnect was handled behind the scenes, and the connectionhandler served only to notify the user .
Roger Baumgartner
@rogermb
Aug 10 2016 16:55
Which part of the whole process do you mean specifically?
I mean if tell TS3Config that you'd like the ability to do reconnects (e.g. with linear backoffs), it'll do that for you
Bert De Geyter
@TheHolyWaffle
Aug 10 2016 16:56
where the user has to call TS3Query.connect in the onDisconnect right now
could this not be done by the api after onDisconnect was fired?
Roger Baumgartner
@rogermb
Aug 10 2016 16:56
No, sorry, they don't
I'm very bad at putting the whole thing into words
So, internally there's a field for one single ConnectionHandler
When the query is created, that field will be assigned depending on the chosen ReconnectStrategy
If that strategy is userControlled, the ConnectionHandler passed along from the TS3Config object is placed in that field
In that case and only that case does the user need to do the calls to TS3Query.connect (or not, whatever they'd like to do)
Bert De Geyter
@TheHolyWaffle
Aug 10 2016 17:01
oh ok, so it does do reconnects behind the scenes :)
Roger Baumgartner
@rogermb
Aug 10 2016 17:01
If the strategy is disconnect, which is the default, the API will create its own ConnectionHandler, which calls ts3Query.exit() in its implementation and also calls the user-supplied handler after that
And if it's any of the reconnecting strategies, it'll create a ConnectionHandler that'll both try to reconnect and call the user handler
Bert De Geyter
@TheHolyWaffle
Aug 10 2016 17:03
Alright, thanks for clearing that up.
Roger Baumgartner
@rogermb
Aug 10 2016 17:03
Yeah, it's hidden if you want it to be hidden, but you also have total control if that's something the user wants to do
I just ran into this problem that I somehow had to enable the user do both with only one interface
And that's why ConnectionHandler is used both internally and by users :stuck_out_tongue:
Bert De Geyter
@TheHolyWaffle
Aug 10 2016 17:04
I'd have seperated them personally
but it seems to work so that's good
Roger Baumgartner
@rogermb
Aug 10 2016 17:05
I wanted to do that at first, too, but then it would've become complicated if the user wanted to control the reconnecting by themselves
Bert De Geyter
@TheHolyWaffle
Aug 10 2016 17:05
ConnectionListener <-> ConnectionHandler
Roger Baumgartner
@rogermb
Aug 10 2016 17:06
In that case we would've had to create some kind of "translation layer" between the API and the "userControlled" connection handler
And that would've also meant that in the end, both would've had to have the same methods
Functionality-wise I mean
¯\_(ツ)_/¯
"Version 1" of the reconnect API (which never made it into any releases) had the user call new ReconnectionHandler() { and they then had to override some abstract methods
And in most cases, it just got extremely complicated and you had to remember all those class names to be even able to reconnect
Roger Baumgartner
@rogermb
Aug 10 2016 17:11
The newer version is much more intuitive, in my opinion at least ^^
Roger Baumgartner
@rogermb
Aug 10 2016 17:19
By the way, the "don't wait for the response before sending" even speeds up asynchronous commands on localhost where there's close to 0 ping :smile:
Without it, I get around 8600 clientupdate commands per second.
With it enabled, it goes up all the way to 20000 commands per second :smile:
Roger Baumgartner
@rogermb
Aug 10 2016 17:38
I've done some tests for the reordering stuff
When you send a ftinitupload command, the server will send one of the parameters back to you
So I've done like a 10k of those commands with a random ID, and every single time they got the same ID back (and not the ID of the command prior to or following that command)
If TS3 were to dispatch these commands via a thread pool, there would've almost certainly been some kind of reordering
So I'm confident that we can introduce this feature with a very low risk