These are chat archives for Ruhrpottpatriot/GW2.NET

26th
Dec 2015
Robert Logiewa
@Ruhrpottpatriot
Dec 26 2015 16:04
Status Update:
I'm currently working on the HttpClient implementation again, but sadly I lost my old branch, so I'm going to ahve to create it from scratch. Going well so far.
One question I'd like some input however:
Currently the ServiceManger wraps the rdeserialized response into an IResponse<T>. Should we keep that, or try to find another way
Steven Liekens
@StevenLiekens
Dec 26 2015 19:56
I wouldn't keep it
Robert Logiewa
@Ruhrpottpatriot
Dec 26 2015 19:56
Then we'd have to find a way to trasport localisation, query time, etc.
Steven Liekens
@StevenLiekens
Dec 26 2015 19:57
yeah I'm not sure what to do with that
Robert Logiewa
@Ruhrpottpatriot
Dec 26 2015 19:58
currently I'm simply switching out the service client, but try to leave as much intact as possible
if that works we can refine further
Steven Liekens
@StevenLiekens
Dec 26 2015 19:58
why?
it's garbage the way I wrote it
Robert Logiewa
@Ruhrpottpatriot
Dec 26 2015 19:59
the service client is out completely
that's for sure
but the converters stay intact for now
Steven Liekens
@StevenLiekens
Dec 26 2015 20:01
can we upload 1.4?
Robert Logiewa
@Ruhrpottpatriot
Dec 26 2015 20:01
I already did
Steven Liekens
@StevenLiekens
Dec 26 2015 20:02
oh really?
Robert Logiewa
@Ruhrpottpatriot
Dec 26 2015 20:02
two days ago
:)
Steven Liekens
@StevenLiekens
Dec 26 2015 20:02
that's great
closed #36 since I can't reproduce it with 1.4.0
Robert Logiewa
@Ruhrpottpatriot
Dec 26 2015 20:06
ok
now to find an alogorithm which calculates the optimal page size
I want to get as less useless data as possible
Steven Liekens
@StevenLiekens
Dec 26 2015 20:08
optimal page size?
Robert Logiewa
@Ruhrpottpatriot
Dec 26 2015 20:09
Ok, a bit lengthier since I have to explain the thought process in the new repository layout.
Steven Liekens
@StevenLiekens
Dec 26 2015 20:12
some example query strings would be fine too
right vs wrong
Robert Logiewa
@Ruhrpottpatriot
Dec 26 2015 20:12
What does the end user want from our library? Probably he wants to query the api for a set of items (not the ingame ones) and get a result. He does not want to fiddle around with page sizes. Most likely he wants to take a set of identifiers and throw them at our library and get the result data. Thus the FindPage methods almost excusively is something for us developers, most end users would use either the Find(TIdentifier), FindAll(), FindAll(IEnumerable<TIdentifier>) methods.
Thus we can reduce the number of methods to those (and Discover() )
However, we simply shouldn't do a FindAll followed by a LINQ filter, since that would send too much data over the wire, therefore pagination is still the optimal way to go, but for us, not the end user.
Now I'm trying to create an algorithm, which makes the least amount of queries for the biggest page size (or else we simply could make n times a single call, which is stupid)
So, an optimal call for a set of 467 calls would be 3, 2 times 200, one time 67
Steven Liekens
@StevenLiekens
Dec 26 2015 20:17
TPL dataflow has some reusable components for batching
Robert Logiewa
@Ruhrpottpatriot
Dec 26 2015 20:17
That is a pretty baseline algorith, but I'm trying to find one which makes calls which are nearly the same size in return
yeah I started looking into that
What I'm tring to do here is to reduce the amount of methods in the repository to a bare minimum (currently: Discover(), Find(TId), Find(IEnumerable<TId>), Find()) of course everything is async
Steven Liekens
@StevenLiekens
Dec 26 2015 20:20
the only problem would be batching the IEnumerable if it has more than 200 items?
can you Take(200) in a loop until there are no more elements?
Robert Logiewa
@Ruhrpottpatriot
Dec 26 2015 20:21
currently the alogorithm goes like this:
Let n be the number of elemts to query and p the page size
Calculate the gdc between n and 200
if there is no gdc (i.e. n is < 200), then make a call with p = n
if there is a gdc make a number of calls equal to gdc(n, 200)
if there are no more elemts left, exit
else make a last call with p = remainder
this is the most basic algorithm
and should work fine for now
Steven Liekens
@StevenLiekens
Dec 26 2015 20:25
careful with IEnumerable because it doesn't guarantee that you can enumerate it a second time after you get its Count()
Robert Logiewa
@Ruhrpottpatriot
Dec 26 2015 20:25
I'm storing |M| inside the method
Steven Liekens
@StevenLiekens
Dec 26 2015 21:02
here's something I don't like that we have
there's no way in json.net to specify the capacity for a list
which we know in advance (X-Result-Count)
so the list capacity grows far beyond what we really need
because we're dealing with json arrays of 200 items
I think the capacity grows to 256
28% overhead
Steven Liekens
@StevenLiekens
Dec 26 2015 21:49
there is a way to get rid of the overhead using JsonConvert.PopulateObject instead of JsonConvert.DeserializeObject
but then it's up to us to create an existing instance of the object to populate
so this is what we have: https://dotnetfiddle.net/C3L3r2
and this is where I want to be: https://dotnetfiddle.net/aINKU9
can we do that?
Robert Logiewa
@Ruhrpottpatriot
Dec 26 2015 21:51
I'll look into it
Steven Liekens
@StevenLiekens
Dec 26 2015 21:52
thanks