These are chat archives for arenanet/api-cdi

21st
Mar 2016
David Reeß
@queicherius
Mar 21 2016 17:25
Hum. Did you do something to mark more API keys as "invalid key"? Some of my users are reporting keys to get removed (which they automatically do if they are invalid) and if I check my stats, over the last few days 54% of the 80k keys got removed as invalid in the backend... D:
O_O what
darthmaim
@darthmaim
Mar 21 2016 17:25
:O
David Reeß
@queicherius
Mar 21 2016 17:26
If it helps, they get removed if the endpoint returns 'endpoint requires authentication', 'invalid key' or 'no game account'
uhh I think there might be a bug
my API keys are returning requires auth but only for the first request
subsequent requests work fine -- feels like some cache issue
wait but yours got invalid key kicked back?
David Reeß
@queicherius
Mar 21 2016 17:27
That was just one of the test keys. But it makes sense, since I remove if that returns too...
that means the underlying OAuth application ceased to exist
do you have a test key handy?
David Reeß
@queicherius
Mar 21 2016 17:28
B75DAFD8-C1DD-934D-936D-B94C64BE19D23DF11235-D835-410F-A056-7621207E7DA8 was invalid key
But that's from a user, so I am not sure if he removed it
I can send you some more that got marked as invalid as a pm if you want
won't be any use
that error message gets kicked back when the underlying OAuth application has been deleted
David Reeß
@queicherius
Mar 21 2016 17:33
Some fun counts:
  • 15th March: 4904 invalid
  • 16th March: 144 invalid
  • 17th March: 135 invalid
  • 18th March: 128 invalid
  • 19th March: 38196 invalid
  • 20th March 1429 invalid
  • 21th March 1289 invalid
(15th I reset the valid status because it happened before that once and I thought it was me)
Edgar Doiron
@Coffee4cr
Mar 21 2016 17:34
MArch 19-21 is the 54% of 80k :P
I'm gonna have to ask around to get more info
David Reeß
@queicherius
Mar 21 2016 17:36
Thanks for looking into it. <3
yeah no probs
Eearslya Sleiarion
@Eearslya
Mar 21 2016 17:56
@queicherius I'm honestly more surprised that you lost over 40k keys and that's only HALF
David Reeß
@queicherius
Mar 21 2016 17:56
Oh they are not lost, just have a flag in the DB. Else I would have killed them myself so many times lol
But yeah, it's quite a lot of keys. That's why it was a fun task to "crawl account statistics without lighting the server on fire" :D
Edgar Doiron
@Coffee4cr
Mar 21 2016 17:57
i'm actually surprise there's not more than 80k keys on gw2efficiency
Nabrok
@Nabrok
Mar 21 2016 17:58
When did you add the chart on the account overview page?
David Reeß
@queicherius
Mar 21 2016 18:05
@Coffee4cr Well, not everyone uses an API key. In fact, it's only ~29% of users. Don't ask me why. Maybe it's too complicated, maybe people feel like they are risking their account, I have no idea.
@queicherius keep those keys around; hopefully we can figure out what the issue is on our end and get them working again
it definitely feels like an issue on our end
Edgar Doiron
@Coffee4cr
Mar 21 2016 18:06
well between me and you @queicherius , I couldn't live without GW2Efficiency, so thank you. Just to be able to search on my account for something, instead of login/logout of every character is the best thing ever
David Reeß
@queicherius
Mar 21 2016 18:07
@Nabrok 23rd of feburary accoding to git
Nabrok
@Nabrok
Mar 21 2016 18:08
I like it. Seems I lost about 270g in dye value on March 15th
David Reeß
@queicherius
Mar 21 2016 18:09
@lye Well, I tried some keys and they were working again. I think it's just that they sometimes fail for the first request and then work again. For now I just removed the part of the code that marks them as invalid in my DB until it's fixed. :)
@Nabrok Thanks! :D I use that feature a lot too. ("Where the hell is my copper fed?!")
Eh. @Coffee4cr Sorry. D:
Edgar Doiron
@Coffee4cr
Mar 21 2016 18:10
:P
Nabrok
@Nabrok
Mar 21 2016 18:12
Kind of surprises me how little my account value has gone down with all the stuff I've poured into the guild hall, scribing, etc
Though most of my value is from skin unlocks
Edgar Doiron
@Coffee4cr
Mar 21 2016 18:12
@Nabrok same
@queicherius if possible, can you dump out all the broken keys and email them to lawton@arena.net ?
I can go through them and triage from there.
David Reeß
@queicherius
Mar 21 2016 18:16
I can send you all that got marked, yeah. Some of them might be legitimately invalid tho, is that a problem?
no, that's fine
oh, also, the last date they worked would be good info too
any format is fine
David Reeß
@queicherius
Mar 21 2016 18:18
The last date I cant get back, I just overwrote that when I reset the state, I can just salvage the keys from the log files (that dont have dates, I know, I know. :cry: )
eh that's not a big deal
David Reeß
@queicherius
Mar 21 2016 18:21
Edgar Doiron
@Coffee4cr
Mar 21 2016 18:22
LOL
David Reeß
@queicherius
Mar 21 2016 18:23
@Nabrok http://i.imgur.com/TZvbtQZ.png I just like this graph. :D
GOD DAMN EMBEDS.
Pat Cavit
@tivac
Mar 21 2016 18:23
:confused: That embed is super-useful?
Nabrok
@Nabrok
Mar 21 2016 18:23
I like the embeds
Eearslya Sleiarion
@Eearslya
Mar 21 2016 18:23
oh oh oh @queicherius speaking of gw2effeciency
Edgar Doiron
@Coffee4cr
Mar 21 2016 18:23
it is, shows that he's rich
David Reeß
@queicherius
Mar 21 2016 18:24
I just dislike that it makes it span the whole width, else it'd be good. Especially for youtube videos
Nabrok
@Nabrok
Mar 21 2016 18:24
My wallet is at 90g at the moment ...
Eearslya Sleiarion
@Eearslya
Mar 21 2016 18:24
Feature request: On the crafting calculator, I'd love it if we could calculate multiple items at once. I was trying to calculate what I'd need for a full set of ascended armor while taking into account my banked mats
Edgar Doiron
@Coffee4cr
Mar 21 2016 18:24
the sad part, he started over 4000g... I don't even have 100g to my name
Also, talking about the crafting calculator, a feature I love about gw2bltc is you see all the items you can craft from the item you're looking at. that would be great :)
Eearslya Sleiarion
@Eearslya
Mar 21 2016 18:25
I once had a PHP script written to calculate crafting multiple items, but..I've been busy with work D:
David Reeß
@queicherius
Mar 21 2016 18:29
@Eearslya @Coffee4cr Yeah, these two things get requested a lot together with having a overview of all recipes. Soonish(tm). More on that, sorry if that's offtopic
Edgar Doiron
@Coffee4cr
Mar 21 2016 18:29
thanks for the link
Eearslya Sleiarion
@Eearslya
Mar 21 2016 18:30
Yeah, I can't even imagine making the backend for a site THAT popular.
Nabrok
@Nabrok
Mar 21 2016 18:30
By the way ... when I was making the silver vale guardian statue, you had the recipe for that but not for the bronze statue.
Eearslya Sleiarion
@Eearslya
Mar 21 2016 18:30
I've been doing most of my backend stuff in C
@Nabrok might have been whitelist issues
Nabrok
@Nabrok
Mar 21 2016 18:32
Looks like the gold trophy doesn't show either.
Also, I just noticed that it will cost me 1,241g to make the 10 silver trophies I need to make the gold one.
But ... scribing changes are coming soon ... right? right?
David Reeß
@queicherius
Mar 21 2016 18:34
@Eearslya well, for me it was quite a learning project as well, so some of the older code is exaclty that... "learning code" >_>
Eearslya Sleiarion
@Eearslya
Mar 21 2016 18:34
That's..pretty much all my code.
@queicherius a lot of these keys appear to be working
not sure if that's because we're kicking back authentication required incorrectly under certain situations
David Reeß
@queicherius
Mar 21 2016 20:01
Hm. Is there any way to get that ironed out or should I just rely on "invalid key" / "no game account" instead?
"no game account" IIRC means the account's been banned
"invalid key" AFAIK means the key's been revoked
sorry, am still going through the list and sorting out which keys fall into which buckets
throwing my "1 request/second" mantra right out the window
Nabrok
@Nabrok
Mar 21 2016 20:04
I think you also get invalid key if you use a key that never existed
you get a slightly different error message if the key isn't in the right format, but yeah, "invalid key" is the backend saying "I don't know what this is"
specifically, that the OAuth2 application specified by the key doesn't exist
still digesting the list, looks like 60% so far are due to a cache-related bug
David Reeß
@queicherius
Mar 21 2016 20:28
Meanwhile I am adding logging that doesn't suck. :D
David Reeß
@queicherius
Mar 21 2016 20:37

And at the cache related bug, how is the saying?

There are two hard things in computer science: cache invalidation, naming things, and off-by-one errors

pretty much. I'll try to have it tracked down and fixed this week.
It manifests by giving "authorization required" the first time a key is used
just pushed out a fix for the erroneous "authorization required" errors
@queicherius that should resolve ~90% of your broken tokens :<
Eearslya Sleiarion
@Eearslya
Mar 21 2016 22:46
@queicherius 1 request/second and 80k API keys? ouch.
ehhh there aren't any hard and fast rules about rate limiting
for unauthenticated data 1 req/second is fairly okay though
authenticated data is even more heavily cached so it's less of an issue
Eearslya Sleiarion
@Eearslya
Mar 21 2016 22:48
and then there's me who yanks all 60k+ items in 19 seconds
paginated, but still
David Reeß
@queicherius
Mar 21 2016 22:48
@lye Thanks! That was fast.
@Eearslya >_>
@Eearslya ehhhh averaged over a day it's probably fine
darthmaim
@darthmaim
Mar 21 2016 22:50
"its probably fine" :fire:
David Reeß
@queicherius
Mar 21 2016 22:53
Well, I am proxying all unauthenticated data, so there is at least that.
That has to be worth enough for spamming 37 req/s authenticated, right, right? (Please dont block me :sob:)
we migrated to better (and dedicated) hardware for the machines serving API data; there's plenty of headroom but if something catches fire I'll try to make changes on my end before breaking y'alls stuff
so no worries there :>
David Reeß
@queicherius
Mar 21 2016 22:55
Phew.
Eearslya Sleiarion
@Eearslya
Mar 21 2016 22:56
@lye Yeah it amounts to about 135req/19s which is...about 7 a second
and it's not like I do it constantly XD
I actually haven't done anything with the API in a month or two