These are chat archives for arenanet/api-cdi

20th
Mar 2017
windwarrior
@windwarrior
Mar 20 2017 09:43
If I just want to put a simple piece of servercode in a cloud
what provider costs me the least amount of money (free would be lovely!)
Javascript only apps are great, but persistance is annoying in JS :')
David Reeß
@queicherius
Mar 20 2017 09:47
Do you need root access? Depending on your needs there is nearly free speech or similar offerings, or you can shell out around 3 bucks a month for a vps (e.g. on OVH)
windwarrior
@windwarrior
Mar 20 2017 09:48
I was looking at Heroku, but I dont know how limited that is
David Reeß
@queicherius
Mar 20 2017 09:48
AWS and Google Cloud also have free plans, if you only need a tiny bit of ressources
windwarrior
@windwarrior
Mar 20 2017 09:48
I hate managing software stacks :')
I just want code in cloud lol
Well its just for the odd tool I make that needs to retain some data
so, free tiers will be fine for a while I guess
David Reeß
@queicherius
Mar 20 2017 09:50
Heroku, Digital Ocean and all the others work fine, but are overpriced for what they offer
windwarrior
@windwarrior
Mar 20 2017 09:52
will look at AWS/Google Cloud!
David Reeß
@queicherius
Mar 20 2017 09:53
As a word of caution, when you blow the free tiers they can get expensive real fast, so checking that once in a while is advised :)
windwarrior
@windwarrior
Mar 20 2017 09:56
hehe they need to make money somehow :P
no offense again!
Eearslya Sleiarion
@Eearslya
Mar 20 2017 15:22
that's all of us
windwarrior
@windwarrior
Mar 20 2017 15:23
what?! are you saying I program bugs
q.q
(to be fair, I have made bugs in a Hello World programs.... it printed Hell World)
@windwarrior lol
windwarrior
@windwarrior
Mar 20 2017 17:24
gotta love being a software developer
you are your own worst enemy
Archomeda
@Archomeda
Mar 20 2017 17:39
i would say some dependencies are enemies too
smiley
@codemasher
Mar 20 2017 17:53
dependencies are the root of all evil. ask a JS dev.
oh wait.
Darqam
@Darqam
Mar 20 2017 17:55
So throughout this API blackout, my guild has determined that the biggest casualty was /v2/quaggans
Quaggan BooOOoot
@QuagganBooOOoot
Mar 20 2017 17:57
poor quaggans
Archomeda
@Archomeda
Mar 20 2017 17:57
oh hey, it lives again
the account that is
darthmaim
@darthmaim
Mar 20 2017 17:58
I emailed the gitter support
they unbaned it
Archomeda
@Archomeda
Mar 20 2017 17:58
why did they ban it in the first place? XD
darthmaim
@darthmaim
Mar 20 2017 17:58
too many too similar messages :(
Archomeda
@Archomeda
Mar 20 2017 17:58
lol
so what if i type too many similar messages myself, i would get banned too? :P
darthmaim
@darthmaim
Mar 20 2017 17:59

Hi

Sending duplicate messages over and over will generally result in our suspicious behaviour algorithms picking up and blocking the account. I've unblocked it now, but be cautious about sending the same exact messages multiple times.

Eearslya Sleiarion
@Eearslya
Mar 20 2017 17:59
but it's a bot
Archomeda
@Archomeda
Mar 20 2017 17:59
that's just stupid
i mean, it didn't even send that many messages
darthmaim
@darthmaim
Mar 20 2017 18:00
¯\_(ツ)_/¯
Archomeda
@Archomeda
Mar 20 2017 18:00
when did you get it back?
darthmaim
@darthmaim
Mar 20 2017 18:00
I sent a ton of messages in a private chat with myself to test it
like 1 or 2 weeks ago?
Archomeda
@Archomeda
Mar 20 2017 18:00
but that's a private chat D:
darthmaim
@darthmaim
Mar 20 2017 18:00
idd
they don't care
Archomeda
@Archomeda
Mar 20 2017 18:00
maybe gitlab changes some stuff now (hopefully)
can't imagine what i would do if discord would ban bots...
darthmaim
@darthmaim
Mar 20 2017 18:01
When the API is back up and I got some free time (...), I will get him running again
Karasu
@Karasuni
Mar 20 2017 18:01
im getting 404 { error: "not found" } on /items
anyone else?
still down
darthmaim
@darthmaim
Mar 20 2017 18:01
The API is offline
Archomeda
@Archomeda
Mar 20 2017 18:02
api has been offline since saturday (or was it friday?)
Karasu
@Karasuni
Mar 20 2017 18:02
WTF why
depends on the timezone, friday evening PST
@Karasuni because it was crashing
Archomeda
@Archomeda
Mar 20 2017 18:03
probable cause: too many requests crashed it
Karasu
@Karasuni
Mar 20 2017 18:03
k
ChieftainAlex
@ChieftainAlex
Mar 20 2017 18:06
has anyone confessed to being the api murderer, apart from lye?
Archomeda
@Archomeda
Mar 20 2017 18:07
judging from the thread, some people seem think that downloading the whole tp history every time seems reasonable, heh
but no
Pat Cavit
@tivac
Mar 20 2017 18:07
there are a lot of really, really bad programmers in the world
:hand:
darthmaim
@darthmaim
Mar 20 2017 18:08
rate limit all non bulk requests on endpoints that support it to 1/min 😈
hey now i like using gw2e
though i guess you only meant on bulk-expanded endpoints
darthmaim
@darthmaim
Mar 20 2017 18:09
idd
Archomeda
@Archomeda
Mar 20 2017 18:09
but i sometimes browse the api manually with only 1 id :(

but tbf, this is the best response from my bot before i changed it:

Archomeda: !quaggan 404
BOTKormir: @Archomeda, the API responded with the following error: not found

Eearslya Sleiarion
@Eearslya
Mar 20 2017 18:18
perfection :ok_hand:

has anyone confessed to being the api murderer, apart from lye?

Eh… Lawton, was the outage caused by someone request thousands of times?

hmmmmmmmm

the plot thickens
Eearslya Sleiarion
@Eearslya
Mar 20 2017 18:21
dammit I wanted those two quotes separate
Archomeda
@Archomeda
Mar 20 2017 18:22

ha

ha

rip

darthmaim
@darthmaim
Mar 20 2017 18:22
2 empty lines in between

markdown

is hard

Eearslya Sleiarion
@Eearslya
Mar 20 2017 18:22

like

this?

nope
darthmaim
@darthmaim
Mar 20 2017 18:22
wow
Archomeda
@Archomeda
Mar 20 2017 18:23
gitter not following specs
smiley
@codemasher
Mar 20 2017 18:37
btw. how is that there are some people with a gazillion TP listings? is the TP API still "public"?
I don't understand the question
Darqam
@Darqam
Mar 20 2017 18:38
people who say they have a "million listings" are probably counting individual items in the same stack
they might actually have a million listings, I'm not sure
Darqam
@Darqam
Mar 20 2017 18:38
in which case, a million listings is not impossible. especially for that guy
smiley
@codemasher
Mar 20 2017 18:41
but how would a sane person push that many listings to the TP?
the onyl way is through the "unofficial" "public" TP API, eh?
Eearslya Sleiarion
@Eearslya
Mar 20 2017 18:45
not really
there are people who do nothing in gw2 but tp
smiley
@codemasher
Mar 20 2017 18:45
omg :D
Eearslya Sleiarion
@Eearslya
Mar 20 2017 18:47
To clarify a possible point of confusion in some of the above posts, 1 item is not equal to 1 transaction. A transaction where somebody buys or sells 1 item is the same as if they bought 250 in a stack. If you’re a bulk trader, your average load (based on naive mathematics) across 1 million individual items will be about 8000 transactions.
@codemasher
smiley
@codemasher
Mar 20 2017 18:48
yea, i know that, but still 8k transactions...
(i know some people who used to play the TP via the API)
Darqam
@Darqam
Mar 20 2017 18:49

an unintended bonus of explicit rate limiting is that I will no longer have to guess what’s a reasonable request rate. Being the mostly nice person that I am, I try to minimize my requests, however once my new backend is up and running, I’d like to make calls more often.

sigh

oh well~
Darqam
@Darqam
Mar 20 2017 18:49
just because you can doesn't mean you should
smiley
@codemasher
Mar 20 2017 18:50
i'm probably the only one who used rate limiting to call /v2/items... :D
I'm not sure I like X-Rate-Limit-Limit
I'm almost leaning towards X-Rate-Limit-Per-Minute
I think it's more clear.
Darqam
@Darqam
Mar 20 2017 18:53
first gw2 api is down, now discord is down... the world is ending team
Eearslya Sleiarion
@Eearslya
Mar 20 2017 18:54
Discord seems fine to me
down for me
they shard though, so maybe it's just a partial outage
smiley
@codemasher
Mar 20 2017 18:55
https://github.com/rtheunissen/guzzle-rate-limiter
$requests = $response->getHeader('ratelimit-remaining');
$seconds  = $response->getHeader('ratelimit-window');
Archomeda
@Archomeda
Mar 20 2017 18:55
rip discord
@codemasher those aren't prefixed with x-?
are those actual like standardized response headers
smiley
@codemasher
Mar 20 2017 18:56
(it's an example - add the x- in mind ;) )
ah fair
smiley
@codemasher
Mar 20 2017 18:56
iunno, but these seem like sane names
X-Rate-Limit-Limit / X-Rate-Limit-Remaining seem like the semi-standard values
smiley
@codemasher
Mar 20 2017 18:57
:+1:
ok, X-Rate-Limit-Limit is somewhat silly :D
I don't like X-Rate-Limit-Reset
Eearslya Sleiarion
@Eearslya
Mar 20 2017 18:57
I think the reset should be an ISO8601 date, not a unix timestamp
smiley
@codemasher
Mar 20 2017 18:58
X-Rate-Limit-No-More-Quaggans
X-Rate-Limit-Quaggan-Feelings: sad
Eearslya Sleiarion
@Eearslya
Mar 20 2017 18:58
X-Rate-Limit-BooOoOooOOOoo
smiley
@codemasher
Mar 20 2017 18:58
lmao
Eearslya Sleiarion
@Eearslya
Mar 20 2017 18:59
On a more serious note, X-Rate-Limit-Per-Minute?
Quaggan BooOOoot
@QuagganBooOOoot
Mar 20 2017 18:59
sad
smiley
@codemasher
Mar 20 2017 18:59
i'd prefer "window"
I don't think I'm ever going to change from per-minute
ideally the window is fixed since it's easier to reason about
Eearslya Sleiarion
@Eearslya
Mar 20 2017 19:00
or a more generic X-Rate-Limit-Maximum
I mean, Limit is a synonym with Maximum
I'm just going to use X-Rate-Limit-Limit for now
Eearslya Sleiarion
@Eearslya
Mar 20 2017 19:00
but it seems less silly than saying Limit-Limit
and then add X-Rate-Limit-Remaining when I can
smiley
@codemasher
Mar 20 2017 19:00
the API i'm currently working with has some really arbitrary limits... :D
@codemasher I did napkin math for mine!
Should be enough to fetch all of /v2/commerce/listings every 30 seconds.
smiley
@codemasher
Mar 20 2017 19:01
heh
fun!
Eearslya Sleiarion
@Eearslya
Mar 20 2017 19:03
I don't foresee GW2Oracle hitting those limits any time soon, that's for sure
yeah it's only really a problem for gw2e
i'm going to add a way to effectively request higher limits
just gotta figure out the administrative bits
Eearslya Sleiarion
@Eearslya
Mar 20 2017 19:05
Yeah and with the way my project's going, that'll be firmly in place before I actually need it
The biggest request I can foresee from GW2Oracle is maybe TP and accounts
I had a really nice feature on an old site of mine that I miss; it would keep track of your account-wide items and compile a list of what you gained/spent that day
Nabrok
@Nabrok
Mar 20 2017 19:09
I have that on the discord bot.
Eearslya Sleiarion
@Eearslya
Mar 20 2017 19:11
I need to figure out a better (smaller) way to store the data though
Nabrok
@Nabrok
Mar 20 2017 19:11
I just store a diff
ChieftainAlex
@ChieftainAlex
Mar 20 2017 19:12
should I be using sessionstorage for api queries?
at the moment, if someone navigates to a page with an API widget, it fetches the information, and if they hit back then forward, it makes a second request when it reloads. Sorry I was wrong, browser is smart enough to not make a second request.
Eearslya Sleiarion
@Eearslya
Mar 20 2017 19:13
I was storing the change in JSON format
[{"item":12452,"change":"-1"}]
Archomeda
@Archomeda
Mar 20 2017 20:25
@lye, how's the api progressing?
literally my entire world is ending
i may not have this done today
Archomeda
@Archomeda
Mar 20 2017 20:25
aw :(
is it that bad?
mostly yes
Archomeda
@Archomeda
Mar 20 2017 20:27
alright, good luck!
Darqam
@Darqam
Mar 20 2017 20:32
<3 Lye
Eearslya Sleiarion
@Eearslya
Mar 20 2017 20:37
WE BELIEVE
darthmaim
@darthmaim
Mar 20 2017 20:37
<3
Eearslya Sleiarion
@Eearslya
Mar 20 2017 20:37
in the meantime
does anyone have any unit tests I can steal example data from
darthmaim
@darthmaim
Mar 20 2017 20:38
which endpoint?
David Reeß
@queicherius
Mar 20 2017 20:38
<3 lye
Eearslya Sleiarion
@Eearslya
Mar 20 2017 20:42
I'm working on items first
darthmaim
@darthmaim
Mar 20 2017 20:43
all items on gw2treasures return the json returned from the api when you append /json to the url
this is the v1 api format though
Eearslya Sleiarion
@Eearslya
Mar 20 2017 20:44
durn
guess I'll rely on the wiki for now
darthmaim
@darthmaim
Mar 20 2017 20:45
it should be only ${itemtype} -> details and icon_file_id+icon_file_signature -> icon
Eearslya Sleiarion
@Eearslya
Mar 20 2017 20:45
Isn't item_id -> id?
darthmaim
@darthmaim
Mar 20 2017 20:46
ugh maybe?
check the wiki for that
Eearslya Sleiarion
@Eearslya
Mar 20 2017 20:46
wiki seems to think so
I'm already gonna have a lot of fun parsing weapon data (which the wiki has an example for, too) so I think I'll be good until it's back
Luckily I think I found a way to separate out the details object into different tables in rails
Darqam
@Darqam
Mar 20 2017 20:49
I think the wiki has an example for most cases, if not the docs on the github I think should show what the wiki missed
smiley
@codemasher
Mar 20 2017 21:14

Isn't item_id -> id?

this is the v1 api format though

i think the request parameter was item_id
darthmaim
@darthmaim
Mar 20 2017 21:15
yep it was, and we talked about the differences between both formats
smiley
@codemasher
Mar 20 2017 21:15
ah
btw. there's no such parameter in v2 (singular id) as it's in the URL.
it's only in the response (where it was in v1 too?)
Eearslya Sleiarion
@Eearslya
Mar 20 2017 21:24
you can still do /v2/items?id=12452
smiley
@codemasher
Mar 20 2017 21:26
does that work? (never tried that tbh.. :D)
darthmaim
@darthmaim
Mar 20 2017 21:26
it works
(well not right now obviously :P)
smiley
@codemasher
Mar 20 2017 21:27
"undocumented functionality"
darthmaim
@darthmaim
Mar 20 2017 21:27
pretty sure its documented somewhere
smiley
@codemasher
Mar 20 2017 21:27
not in the docs i've written :P
Jonathan Andrist
@rwfrk_twitter
Mar 20 2017 22:26
The guy talking about a million listings is legit. I know him. He's probably underestimating. The API gives him a good overview of what money is where, but I dont think he loads more than his 90day via GW2BLTC that often
Darqam
@Darqam
Mar 20 2017 23:28
apparently google cloud is the reason why discord is messing up?
Michael Dougall
@madou
Mar 20 2017 23:28
@lye in an alternate reality would you have just ip blocked the guy being naughty first and then implement the rate limiting?
:angel:
if I had the IP lol
Michael Dougall
@madou
Mar 20 2017 23:29
wouldn't the cdn or something have request logs
what cdn
Michael Dougall
@madou
Mar 20 2017 23:29
:blush: shit
but no the API doesn't keep request logs due to volume
Darqam
@Darqam
Mar 20 2017 23:29
I know I don't know much about that kind of infrastructure, but I'm surprised that that info isn't easilly accessible
I don't know the details but ops had to turn them off at the IIS level
given what I know about IIS
it's probably because IIS is :poop:
darthmaim
@darthmaim
Mar 20 2017 23:30
:D
Michael Dougall
@madou
Mar 20 2017 23:30
welp it answers the question at least
thanks lye :-)
Darrian
@rikkuness
Mar 20 2017 23:31
IIS....oh no
Michael Dougall
@madou
Mar 20 2017 23:34
with the whole updated c# compiler and new "dot net core" stuff, is there a new IIS equivalent?
i should google
:angel:
Pat Cavit
@tivac
Mar 20 2017 23:39
we could log reqs but it's a stupid amount of data and we don't have a ton of infra for "giant fucking stream of nonsense"
it's not all that interesting most of the time anyways
downside to running on real :computer: instead of in a :cloud: I suppose
Darqam
@Darqam
Mar 20 2017 23:40
could it be a log that rolled over? keep like 3 days worth of data and then dump it? Just in case of scenarios like this.
windwarrior
@windwarrior
Mar 20 2017 23:41
no no, you gotta be modern, private-:cloud:
Pat Cavit
@tivac
Mar 20 2017 23:41
IIS used to be real stupid about rolling logs
windwarrior
@windwarrior
Mar 20 2017 23:43
its sad that that means rate limiting, but I think the rate limit is pretty fair
except for @queicherius, but that gets fixed