These are chat archives for arenanet/api-cdi

21st
Mar 2017
yeah i need to figure out what the story is gonna be for getting rate limit exemptions
figure it's going to be (1) have one or more static IPs, (2) an email address, (3) hopes and prayers
i'm really hype about having per-site graphs of api usage
but i'll worry about that when i get the ratelimiting bit deployed (hopefully tomorrow)
Eearslya Sleiarion
@Eearslya
Mar 21 2017 03:18 UTC
3) participation in the api-cdi gitter so we have a quick line of contact
and/or a guage on programming skill
Archomeda
@Archomeda
Mar 21 2017 03:38 UTC
4) your soul
Jonathan Andrist
@rwfrk_twitter
Mar 21 2017 04:02 UTC
Absolutely a fan of 3. Gotta have super quick point of contact somehow for whitelist "Something broke man"
Jonathan Andrist
@rwfrk_twitter
Mar 21 2017 04:44 UTC
Maybe require an in-game primary account name as another point of contact
Edgar Doiron
@Coffee4cr
Mar 21 2017 12:59 UTC
5) Profit!
Werdes
@werdes
Mar 21 2017 16:03 UTC
i doubt those are logged?
Eearslya Sleiarion
@Eearslya
Mar 21 2017 16:13 UTC
No, nothing's logged right now, just the amount of requests. But I think he was saying that whitelisted sites should have to use User-Agents
smiley
@codemasher
Mar 21 2017 16:30 UTC

yea, re this

figure it's going to be (1) have one or more static IPs, (2) an email address, (3) hopes and prayers

@Eearslya "they". thank you <3
Eearslya Sleiarion
@Eearslya
Mar 21 2017 16:31 UTC
wat
smiley
@codemasher
Mar 21 2017 16:32 UTC
user the pronouns they/them when you refer to me ;)
Eearslya Sleiarion
@Eearslya
Mar 21 2017 16:34 UTC
well alrighty then
Jonathan Andrist
@rwfrk_twitter
Mar 21 2017 16:47 UTC
Aww. Lawton are you that Gold Raider? sadface No one deserves siege on their body.
yeah i am
i throw siege on their bodies
it's all fun
lol
i'm laughing more than I should on this
thanks gitter for the HUGE embed
Pat Cavit
@tivac
Mar 21 2017 17:37 UTC
ahahahaha that's so fucking stupid
I love it
Werdes
@werdes
Mar 21 2017 18:27 UTC
rumors say it swims once in the water.
now that's the stuff i'd pay for, if i actually was into minis
Edgar Doiron
@Coffee4cr
Mar 21 2017 18:28 UTC
well you can be into "this" mini
Nabrok
@Nabrok
Mar 21 2017 18:31 UTC
I'd have a mini out much more often if they weren't seen by enemy players in wvw
hint hint
Edgar Doiron
@Coffee4cr
Mar 21 2017 18:37 UTC
who would suspect a fish though
Jonathan Andrist
@rwfrk_twitter
Mar 21 2017 18:49 UTC
Hey, at least minis respect the stealthed state of their owner now.
Nabrok
@Nabrok
Mar 21 2017 18:50 UTC
Wait, they do? When did that happen?
Pat Cavit
@tivac
Mar 21 2017 18:50 UTC
holy shit it totally does start swimming in water
darthmaim
@darthmaim
Mar 21 2017 18:52 UTC

Dear API Devs,

On Behalf of the World Vs World Community we are requesting the WvW part of the API back. The health of the community depends on these services. I understand that it is broken or parts of it are broken. However it is imperative that sites such as mos.millenium.org/na/matchups/ and https://wvwintel.com/ are working even if nothing else works. Please hurry with up with it we need it.

some people...
where's that from? lol
darthmaim
@darthmaim
Mar 21 2017 18:52 UTC
forum
orz
Archomeda
@Archomeda
Mar 21 2017 18:55 UTC
Lol
darthmaim
@darthmaim
Mar 21 2017 18:55 UTC
I would wait to turn on the wvw endpoints for one more day after you get the api running again...
Eearslya Sleiarion
@Eearslya
Mar 21 2017 18:56 UTC
"it is imperative" why
press B
Nabrok
@Nabrok
Mar 21 2017 18:56 UTC
I missed the bit where the wvw community chose him as spokesperson.
Darqam
@Darqam
Mar 21 2017 19:01 UTC
heh " I understand that it is broken or parts of it are broken." Obviously not....
"Please re-enable a broken thing without fixing it"
Archomeda
@Archomeda
Mar 21 2017 19:02 UTC
What was the API license again (or w/e it's called)? Something about no guarantee?
Might want to remind I'm about that :smile:
Him* thanks phone
Darqam
@Darqam
Mar 21 2017 19:03 UTC
I feel like answering...
I really do...
darthmaim
@darthmaim
Mar 21 2017 19:06 UTC
SLA
(Service Level Agreements)
Eearslya Sleiarion
@Eearslya
Mar 21 2017 19:07 UTC
Yeah, the API was specifically made with no SLA
Archomeda
@Archomeda
Mar 21 2017 19:08 UTC
Yeah
Megan Walker
@lambada
Mar 21 2017 19:12 UTC
Something Something cloud doesn't go down something ;)
i'd rather it go down than be charged $100k/hour for "extra capacity" :P
Eearslya Sleiarion
@Eearslya
Mar 21 2017 19:13 UTC
wait what
what charges $100k/h
hiring a fleet of cloud-capturing helicopters
Megan Walker
@lambada
Mar 21 2017 19:14 UTC
:joy:
btw. any chance on GeoJSON once the API is back up and running?
Archomeda
@Archomeda
Mar 21 2017 19:20 UTC
I think improving the ratelimit system wille more priority atm
Will have* thanks again phone
more like going home and crying myself to sleep is more of a priority
darthmaim
@darthmaim
Mar 21 2017 19:21 UTC
<3
Archomeda
@Archomeda
Mar 21 2017 19:21 UTC
Aw :worried:
ChieftainAlex
@ChieftainAlex
Mar 21 2017 19:21 UTC
"in alignment with other ArenaNet official policy on WvW, we've decided to switch off the WvW endpoint permanently"
Archomeda
@Archomeda
Mar 21 2017 19:21 UTC
We still love you lye <3
Darqam
@Darqam
Mar 21 2017 19:21 UTC
But Smiley, he is the voice of WvW community, he is of course serious
David Reeß
@queicherius
Mar 21 2017 19:21 UTC
You got this lye. <3
Archomeda
@Archomeda
Mar 21 2017 19:22 UTC
Also lol @codemasher
smiley
@codemasher
Mar 21 2017 19:24 UTC
"on behalf of the wvw noobs who aren't able to follow the zerg"
just noticed i still have this in my forum signature http://gw2wbot.darthmaim.de/smiley/
is this still the current database, @darthmaim ?
darthmaim
@darthmaim
Mar 21 2017 19:29 UTC
uhhm
yup
contains the latest items of gw2treasures
ChieftainAlex
@ChieftainAlex
Mar 21 2017 19:29 UTC
was the day the api tanked the same as when gw2armory released the embed?
Dominic
@DomGrieco
Mar 21 2017 19:29 UTC
no
smiley
@codemasher
Mar 21 2017 19:30 UTC
ha, nice - didn't expect this script working for that long.. .:D
Archomeda
@Archomeda
Mar 21 2017 19:30 UTC
Doubt it, @madou knows the exact date probably
@ChieftainAlex lol
Dominic
@DomGrieco
Mar 21 2017 19:30 UTC
embeds were released over 2 weeks ago
I was under the impression that the embed used gw2a's internal API
Dominic
@DomGrieco
Mar 21 2017 19:30 UTC
it does
since it's probably faster than using the gw2 api lol
ChieftainAlex
@ChieftainAlex
Mar 21 2017 19:31 UTC
i thought that until none of the data popped up on qtfy skills when the api went down..
but I'm a noob so I got it wrong /giveup
Dominic
@DomGrieco
Mar 21 2017 19:31 UTC
I'm pretty sure thats because he just did updates
darthmaim
@darthmaim
Mar 21 2017 19:34 UTC

well, I'm getting this:

GET https://api.guildwars2.com/v2/specializations?ids=36,11,18&lang=en 404 (Not Found)
GET https://api.guildwars2.com/v2/specializations?ids=4,11,18&lang=en 404 (Not Found)
GET https://api.guildwars2.com/v2/specializations?ids=36,51,18&lang=en 404 (Not Found)
GET https://api.guildwars2.com/v2/skills?ids=30189,14410,14404,14405,14407,30343&lang=en 404 (Not Found)
GET https://api.guildwars2.com/v2/skills?ids=30189,14403,30074,14405,14407,30343&lang=en 404 (Not Found)
GET https://api.guildwars2.com/v2/skills?ids=29941&lang=en 404 (Not Found)
GET https://api.guildwars2.com/v2/skills?ids=30189,14410,30074,14404,30343&lang=en 404 (Not Found)
GET https://api.guildwars2.com/v2/skills?ids=30189,14404,14405,14407,30343&lang=en 404 (Not Found)
GET https://api.guildwars2.com/v2/specializations?ids=36,11,18&lang=en 404 (Not Found)
GET https://api.guildwars2.com/v2/skills?ids=29941&lang=en 404 (Not Found)
GET https://api.guildwars2.com/v2/specializations?ids=4,11,18&lang=en 404 (Not Found)
GET https://api.guildwars2.com/v2/items?ids=46766,46770,24562,24639&lang=en 404 (Not Found)
GET https://api.guildwars2.com/v2/items?ids=46762,24615,24554&lang=en 404 (Not Found)
GET https://api.guildwars2.com/v2/items?ids=48129,48077,48073,48074,48130,48072,24714&lang=en 404 (Not Found)
GET https://api.guildwars2.com/v2/items?ids=48129,48077,48073,48074,48130,48072,24836&lang=en 404 (Not Found)
GET https://api.guildwars2.com/v2/items?ids=67302,67298,37075,37086,39308,37049&lang=en 404 (Not Found)
GET https://api.guildwars2.com/v2/items?ids=39233,39232,37075,37086,39308,37049&lang=en 404 (Not Found)
GET https://api.guildwars2.com/v2/items?ids=67531,68633&lang=en 404 (Not Found)
GET https://api.guildwars2.com/v2/items?ids=70838,76271,24562,24605&lang=en 404 (Not Found)
GET https://api.guildwars2.com/v2/items?ids=70504,24562,24605&lang=en 404 (Not Found)
GET https://api.guildwars2.com/v2/items?ids=74412,67455,76377,76776,77143,72548,24845&lang=en 404 (Not Found)
GET https://api.guildwars2.com/v2/items?ids=67531,12467&lang=en 404 (Not Found)
GET https://api.guildwars2.com/v2/items?ids=67906,77926,77413,77958,77380,80434&lang=en 404 (Not Found)
GET https://api.guildwars2.com/v2/items?ids=48917,12464&lang=en 404 (Not Found)
GET https://api.guildwars2.com/v2/items?ids=74412,67455,76377,76776,77143,72548,71425&lang=en 404 (Not Found)
GET https://api.guildwars2.com/v2/items?ids=70504,24560,24605&lang=en 404 (Not Found)
GET https://api.guildwars2.com/v2/items?ids=70838,76271,24560,24605&lang=en 404 (Not Found)
GET https://api.guildwars2.com/v2/items?ids=46762,36053,24615&lang=en 404 (Not Found)
GET https://api.guildwars2.com/v2/items?ids=46762,24868,24615&lang=en 404 (Not Found)
GET https://api.guildwars2.com/v2/items?ids=46762,24658,24615&lang=en 404 (Not Found)
GET https://api.guildwars2.com/v2/items?ids=79722,71659,75427,76150,70596&lang=en 404 (Not Found)
GET https://api.guildwars2.com/v2/items?ids=48129,48077,48073,48074,48130,48072,24836&lang=en 404 (Not Found)
GET https://api.guildwars2.com/v2/items?ids=67302,67298,37075,37086,39308,37049&lang=en 404 (Not Found)
GET https://api.guildwars2.com/v2/items?ids=67531,68633&lang=en 404 (Not Found)
GET https://api.guildwars2.com/v2/items?ids=67906,77926,77413,77958,77380,80434&lang=en 404 (Not Found)
GET https://api.guildwars2.com/v2/items?ids=48917,12464&lang=en 404 (Not Found)

just loading https://qtfy.eu/build/warrior

smiley
@codemasher
Mar 21 2017 19:35 UTC
caching & databases are for nubs :D
Eearslya Sleiarion
@Eearslya
Mar 21 2017 19:35 UTC
There's a lot of duplicate IDs in those, too
smiley
@codemasher
Mar 21 2017 19:36 UTC
math hard
Dominic
@DomGrieco
Mar 21 2017 19:37 UTC
thats because of the way the embeds are used on the website
multiples that is
he might of changed something recently because I was pretty sure it was from his internal api
Archomeda
@Archomeda
Mar 21 2017 19:41 UTC
Ah, yeah this is a downside of having embeds
Loading things separately
darthmaim
@darthmaim
Mar 21 2017 19:43 UTC
those embeds are just one js file
not iframes or something like that
and you can even queue requests from different iframes if you really wanted to
ChieftainAlex
@ChieftainAlex
Mar 21 2017 19:44 UTC
can't they just make it load the js after loading every individual frame?
darthmaim
@darthmaim
Mar 21 2017 19:44 UTC
there are no frames
ChieftainAlex
@ChieftainAlex
Mar 21 2017 19:44 UTC
element with data attributes*
Dominic
@DomGrieco
Mar 21 2017 19:45 UTC
the js is loaded in the header
Archomeda
@Archomeda
Mar 21 2017 19:48 UTC
Oh
I was under the impression it was iframes
darthmaim
@darthmaim
Mar 21 2017 19:48 UTC
nope, its somewhere in the 342kb of js loaded from gw2armory
Archomeda
@Archomeda
Mar 21 2017 19:50 UTC
Oh wow
That's quite large
Darrian
@rikkuness
Mar 21 2017 19:50 UTC
das what she said
ohhhhhhh
darthmaim
@darthmaim
Mar 21 2017 19:51 UTC
342kb of js just for tooltips
the whole buildeditor we are working on is just 100 or so when build for production
Archomeda
@Archomeda
Mar 21 2017 19:53 UTC
Then it is possible to reduce the amount of requests
darthmaim
@darthmaim
Mar 21 2017 19:53 UTC
indeed
ChieftainAlex
@ChieftainAlex
Mar 21 2017 19:54 UTC
should be easy enough to combine all of the above requests darthmaim quoted in 3 queries.
darthmaim
@darthmaim
Mar 21 2017 19:54 UTC
thats the js that loads the js
blob
ChieftainAlex
@ChieftainAlex
Mar 21 2017 19:55 UTC
lol 4 scripts even better
Archomeda
@Archomeda
Mar 21 2017 19:56 UTC
Ouch
darthmaim
@darthmaim
Mar 21 2017 19:56 UTC
also 150kb webfonts and 300kb tooltip background
gw2armory loads a total of 781kb
Archomeda
@Archomeda
Mar 21 2017 19:57 UTC
Well, does at least 304 work?
darthmaim
@darthmaim
Mar 21 2017 19:58 UTC
yep
Archomeda
@Archomeda
Mar 21 2017 19:58 UTC
Ah good :)
darthmaim
@darthmaim
Mar 21 2017 19:59 UTC
but still
Aralicia
@Aralicia
Mar 21 2017 19:59 UTC
Hi all ! Tbh, what's bothers me the most with gw2armory is the 865Ko png that display a bunch of fake clouds, with a meh resolution.
darthmaim
@darthmaim
Mar 21 2017 19:59 UTC
blob
https://qtfy.eu/build/mesmer/ makes 408 requests with a total of 3.1MB
ChieftainAlex
@ChieftainAlex
Mar 21 2017 20:00 UTC
how popular is qtfy?
darthmaim
@darthmaim
Mar 21 2017 20:00 UTC
no idea
Archomeda
@Archomeda
Mar 21 2017 20:02 UTC
Quite popular under raiders
Do they have cloudflare set up?
Nabrok
@Nabrok
Mar 21 2017 20:02 UTC
Yeah, it's kind of the go-to site for raid builds.
Dominic
@DomGrieco
Mar 21 2017 20:03 UTC

Sessions
130,172

Users
36,597

Pageviews
1,247,971

Yes I have cloudflare set up
That's analytics since release
ChieftainAlex
@ChieftainAlex
Mar 21 2017 20:03 UTC
36597*408?
oh right
Dominic
@DomGrieco
Mar 21 2017 20:03 UTC
release was 15 days ago?
I think
John Carroll
@johncarroll
Mar 21 2017 20:04 UTC
they broke their 25gb per month limit on enjin by a lot
Dominic
@DomGrieco
Mar 21 2017 20:04 UTC
march 6th
Aralicia
@Aralicia
Mar 21 2017 20:04 UTC
blob
Archomeda
@Archomeda
Mar 21 2017 20:04 UTC
Wow
darthmaim
@darthmaim
Mar 21 2017 20:04 UTC
that site is a performance nitghmare
Aralicia
@Aralicia
Mar 21 2017 20:04 UTC
Not API-related, but the site could gain from not loading ALL the youtube embed at load.
Dominic
@DomGrieco
Mar 21 2017 20:05 UTC
it doesnt lol
its loading a tiny screenshot for the lazy loaded video
darthmaim
@darthmaim
Mar 21 2017 20:05 UTC
hqdefault is 1080p (or was it 720p?)
Dominic
@DomGrieco
Mar 21 2017 20:05 UTC
if all those videos weren't lazy loaded
the page would be atleast 30mb load
Archomeda
@Archomeda
Mar 21 2017 20:05 UTC
YouTube embeds are sometimes slow to load as well, at least that was because of AdBlock and Firefox, a couple of years back
Dominic
@DomGrieco
Mar 21 2017 20:06 UTC
do a gtmetrix @Aralicia
Archomeda
@Archomeda
Mar 21 2017 20:06 UTC
At least I don't use that anymore
Dominic
@DomGrieco
Mar 21 2017 20:06 UTC
those images are nothing on the load
damn wiki images are what kill load times and even those are lazy loaded, most of them
Archomeda
@Archomeda
Mar 21 2017 20:07 UTC
Haha, Mediawiki is just slow to begin with
darthmaim
@darthmaim
Mar 21 2017 20:07 UTC
well you have to start somewhere, that page is horribly slow and loads way to much
ChieftainAlex
@ChieftainAlex
Mar 21 2017 20:07 UTC
uh why would you pull images off the wiki?
Dominic
@DomGrieco
Mar 21 2017 20:07 UTC
Ive told the qt people to stop using wiki images
I just made a template for them to fill with builds
ChieftainAlex
@ChieftainAlex
Mar 21 2017 20:10 UTC
everything else is forgivable, but there's a font called "raleway" whose spelling irks me
darthmaim
@darthmaim
Mar 21 2017 20:10 UTC
but 1,247,971 pageviews in 15 days, making 50 api requests (48/sec) each which could be combined to <5... (even when only 50% of the viewed pages are build pages). I'd say we found the reason the API is crashing
Archomeda
@Archomeda
Mar 21 2017 20:10 UTC
Wait, they use wiki images?
Aralicia
@Aralicia
Mar 21 2017 20:10 UTC
@DomGrieco my bad. Since hqdefault.jpg are used by the embed themselves, I assumed the embed where loaded. (funny sidenote, I took the time to gtmetrix, and got a nice "write failed : Broken pipe" error)
ChieftainAlex
@ChieftainAlex
Mar 21 2017 20:10 UTC
I can't see any wiki image requests in the console
Dominic
@DomGrieco
Mar 21 2017 20:12 UTC
wiki images are in cc breakdown, lazy loaded
darthmaim
@darthmaim
Mar 21 2017 20:12 UTC
1247971*50/15/24/60/60 = 48.147 API requests per second
Aralicia
@Aralicia
Mar 21 2017 20:12 UTC
@darthmaim good point. That would be about 175k call/hour on average.
Dominic
@DomGrieco
Mar 21 2017 20:13 UTC
cloudflare flex ssl has been causing a lot of issues on the site that I've been messing with might be why gt just failed
most of the website traffic was the first 3 days of release
I dont recall the api crashing
more then 30% of those analytics are from initial release
windwarrior
@windwarrior
Mar 21 2017 20:15 UTC
it was /v2/items/ that blew up
wasnt it
Jonathan Andrist
@rwfrk_twitter
Mar 21 2017 20:15 UTC
Yes
ChieftainAlex
@ChieftainAlex
Mar 21 2017 20:15 UTC
pity it was v2/items/, otherwise we could blame the official wvw representative
windwarrior
@windwarrior
Mar 21 2017 20:15 UTC
well qtfy has some items on the site
Archomeda
@Archomeda
Mar 21 2017 20:15 UTC
Lmao
windwarrior
@windwarrior
Mar 21 2017 20:15 UTC
but I can hardly imagine that being too much
darthmaim
@darthmaim
Mar 21 2017 20:16 UTC
https://qtfy.eu/build/mesmer/ is making 33 requests to /v2/items
Dominic
@DomGrieco
Mar 21 2017 20:16 UTC
your using the most populated page lol
Archomeda
@Archomeda
Mar 21 2017 20:16 UTC
Nah, the API blew up because of something else
Dominic
@DomGrieco
Mar 21 2017 20:16 UTC
mesmer is like 2x the content of all the other builds
mesmer and druid
Archomeda
@Archomeda
Mar 21 2017 20:16 UTC
Qtfy was online for a while already
When it blew up
Aralicia
@Aralicia
Mar 21 2017 20:17 UTC
Well, at least we're pretty sure it's not Jormag either this time.
darthmaim
@darthmaim
Mar 21 2017 20:17 UTC
warrior is still making 25 requests to items, 24 too many
ChieftainAlex
@ChieftainAlex
Mar 21 2017 20:17 UTC
..probably one guy wondering why his IE5 isn't displaying qtfy just hitting reload when everything breaks.
Archomeda
@Archomeda
Mar 21 2017 20:17 UTC
Well, I can't disagree with the amount of requests per page. It can be made less
ChieftainAlex
@ChieftainAlex
Mar 21 2017 20:17 UTC
anyway regardless of what tipped the API over the edge, rate limiting is a good idea
windwarrior
@windwarrior
Mar 21 2017 20:18 UTC
yeah but those will be cached as well
Archomeda
@Archomeda
Mar 21 2017 20:18 UTC
But I very much doubt it was the main cause
John Carroll
@johncarroll
Mar 21 2017 20:18 UTC
site doesn't even have full minstrels chrono. don't know why anyone bothers
windwarrior
@windwarrior
Mar 21 2017 20:18 UTC
dont think qtfy is putting much strain on the API
Jonathan Andrist
@rwfrk_twitter
Mar 21 2017 20:18 UTC
Qtfy wasnt it
Nothing new there.
Dominic
@DomGrieco
Mar 21 2017 20:18 UTC
metabattle guy was working on implementing embeds
windwarrior
@windwarrior
Mar 21 2017 20:18 UTC
also, minstrel is situational
Dominic
@DomGrieco
Mar 21 2017 20:18 UTC
but I dont think he did anything past testing yet
John Carroll
@johncarroll
Mar 21 2017 20:19 UTC
minstrels is good for everything
windwarrior
@windwarrior
Mar 21 2017 20:19 UTC
not for a speedclearing guild like qT
lets try to remember that
domi illu on golem is 13k dps, minstrel like 4 at most
and if they can get away with not playing any strong healers
they will
John Carroll
@johncarroll
Mar 21 2017 20:20 UTC
lets try to remember this is an api chat :D
Dominic
@DomGrieco
Mar 21 2017 20:21 UTC
that qt site was literally a rushed project that gives anyone the ability to update the site while spending the least amount of money, cms site
nothing fancy
Archomeda
@Archomeda
Mar 21 2017 20:22 UTC
Yeah that's fair
Doesn't mean that there isn't any room for improvement :D
Dominic
@DomGrieco
Mar 21 2017 20:24 UTC
oh yeah things will get fine tuned I just barely have time to spend on it :(
Archomeda
@Archomeda
Mar 21 2017 20:24 UTC
But yeah, moving away from enjin on such a short notice, I understand that b)
Dominic
@DomGrieco
Mar 21 2017 20:25 UTC
I have paid clients that consume most of my time
Archomeda
@Archomeda
Mar 21 2017 20:25 UTC
;) * thanks again 3x phone
Ohh I see
Dominic
@DomGrieco
Mar 21 2017 20:25 UTC
either way back to api, its supposed to be using gw2a internal api
which is cached on a 7 day cycle
something just not working as intended there
@madou will check it out
Archomeda
@Archomeda
Mar 21 2017 20:29 UTC
:+1:
ChieftainAlex
@ChieftainAlex
Mar 21 2017 20:36 UTC

so sending ajax identifier headers is just as simple as adding

$.ajaxSetup({    
    headers: { 'X-please-don-t-ban-me': 'signed chieftain alex' }    
});

before $.getJSON or $.ajax?

pfft markdown.
@ChieftainAlex except that request would no longer qualify as a "simple request" for CORS purposes and would trigger the browser to submit an OPTIONS preflight
Which we don't support, unfortunately.
ChieftainAlex
@ChieftainAlex
Mar 21 2017 20:50 UTC
welp. nevermind that then.
Good luck with fixing the API and let us know what you want us to change (please let it not be the wiki that broke the API...)
pretty sure it's not the wiki
Darqam
@Darqam
Mar 21 2017 20:51 UTC
inb4 some bot mass refreshed a page on the wiki to crash everything
wiki would have fallen over first
Darqam
@Darqam
Mar 21 2017 20:52 UTC
fair
API isn't up so I can't find interwikis for the new fish mini, nor it's mini-id :/
on behalf of all wiki editors out there, Lye please re-enable minis and items endpoints. we NEED them, and should be prioritized
.....
plz no ban
Archomeda
@Archomeda
Mar 21 2017 21:01 UTC
Rofl
windwarrior
@windwarrior
Mar 21 2017 21:06 UTC
well at least it is now clear how useful the API is
Eearslya Sleiarion
@Eearslya
Mar 21 2017 21:07 UTC
Don't know how good we have it until it's gone
Pat Cavit
@tivac
Mar 21 2017 21:11 UTC
holy shit some of these waterfalls you were linking
I would straight-up burn the site down before it shipped like that :fire:
darthmaim
@darthmaim
Mar 21 2017 21:11 UTC
indeed
Eearslya Sleiarion
@Eearslya
Mar 21 2017 21:11 UTC
Waterfalls? The slew of requests?
except, you know, for website network requests
blob
darthmaim
@darthmaim
Mar 21 2017 21:26 UTC
The entire wiki page is smaller than just the js for tooltips
Michael Dougall
@madou
Mar 21 2017 21:42 UTC
Hey boyz - ok lets talk embeds and api requests, api first.
Archomeda
@Archomeda
Mar 21 2017 21:43 UTC
:wave:
Michael Dougall
@madou
Mar 21 2017 21:45 UTC
api requests - yeah they aren't being proxied through my servers. basic idea is i didnt want to pay for it. so - to get around that when an api request is made to gw2api the data is saved to local storage for 7 days. if the apis are working just load the site and then refresh, you won't see another api call unless you clear local storage.
the api requests do get batched, but only on a per embed. i need more dev time to batch it for all embeds (and probably implement lazy loading, qtfy has a lot of embeds that aren't even seen by the user hidden behind multiple tabs)
per embed meaning i can have one that loads like 10 different skills or items, all of those will be batched up and sent as one request (based on resource, obviously).
I really want to steal @queicherius's gw2api.com domain and just have it throw everything into PostgreSQL JSONB fields with extra indexes
Archomeda
@Archomeda
Mar 21 2017 21:49 UTC
hmm, so it's only because of the 404's that we see so many requests currently?
Michael Dougall
@madou
Mar 21 2017 21:49 UTC
yep
David Reeß
@queicherius
Mar 21 2017 21:49 UTC
if ya want the domain, let me know. it's up for grabs
Archomeda
@Archomeda
Mar 21 2017 21:49 UTC
fair
Michael Dougall
@madou
Mar 21 2017 21:49 UTC
funnily enough if you're a regular to qtfy you probably aren't affected by the outage (unless your cache was cleared)
Archomeda
@Archomeda
Mar 21 2017 21:49 UTC
will it get autocleared after 7 days even if the api isn't live?
Michael Dougall
@madou
Mar 21 2017 21:49 UTC
YEP. probably should do something about that...
Archomeda
@Archomeda
Mar 21 2017 21:49 UTC
or spitting out errors?
ouch :D
@lye for what purpose? backup?
Michael Dougall
@madou
Mar 21 2017 21:50 UTC
lets then talk about bundle size and assets being crazy. yeah - i know. theres a lot of shit being downloaded by the embeds that isn't needed. it takes dev time to make it better, and we all know we're just doing this for fun :) these things, they take time!
the embed system was made fast af, and just reuses everything from the armory
@Archomeda searching items by name :|
Michael Dougall
@madou
Mar 21 2017 21:51 UTC
only the thin layer to bootstrap the embeds was fresh
Archomeda
@Archomeda
Mar 21 2017 21:51 UTC
oh, haha
Archomeda
@Archomeda
Mar 21 2017 21:52 UTC
@madou yeah, everyone is doing it in their spare time, good luck :)
@lye, i think i have some code laying around for searching by name
from a long time ago
but it uses mongodb
and something with full text search i think
ah yeah, found it
    /**
     * Gets the item.
     * @param  {Array}    term     The search term.
     * @param  {Array}    params   The additional parameters from the request.
     * @param  {Function} callback The callback that will be called when the action has been executed.
     */
    getItem: function(term, params, callback) {
        var resultLimit = params.limit !== undefined ? parseInt(params.limit) : 10;
        if (resultLimit <= 0 || resultLimit > 50) {
            callback(new restify.BadRequestError('limit out of range. Use limit values 1 - 50'));
        }
        db.items.find({ '$text': { '$search' : term } }, { match_score: { '$meta': 'textScore' }, _id: 0 }).limit(resultLimit).sort({ match_score: { '$meta': 'textScore' }, name: 1 }, callback);
    }
where db is a mongojs object
urgh, this code is so old
i had it running on a free openshift instance
Archomeda
@Archomeda
Mar 21 2017 21:58 UTC
but yeah, mongodb and this text search feature is quite cool
Darqam
@Darqam
Mar 21 2017 22:03 UTC
speaking of databases, I will probably be rebuilding a good chunk of my website, and that will include storage method. Any particular database I should look at to store stuff like /v2/items, achievs, colors (no auth stuff)? I'm aware of mysql and mongodb, just wondering if I would be recommended a specific one over others
Archomeda
@Archomeda
Mar 21 2017 22:03 UTC
postgresql is good as well; i would probably prefer that over mysql
mongodb just stores things differently
if you want to have strict schemas and tables, try sql databases
Eearslya Sleiarion
@Eearslya
Mar 21 2017 22:05 UTC
I'm using postgres right now
Archomeda
@Archomeda
Mar 21 2017 22:05 UTC
mongodb just has documents and stores everything in json
Michael Dougall
@madou
Mar 21 2017 22:05 UTC
@darthmaim see above for some of my comments ;)
darthmaim
@darthmaim
Mar 21 2017 22:05 UTC
saw it
Michael Dougall
@madou
Mar 21 2017 22:05 UTC
should i get the fire pit out
Archomeda
@Archomeda
Mar 21 2017 22:06 UTC
without having strict schemas; so it can be useful for things like /v2/items
darthmaim
@darthmaim
Mar 21 2017 22:06 UTC
:fire: :D
should be all fine when they are cached for now
batching all embeds is nice to have for later
Michael Dougall
@madou
Mar 21 2017 22:06 UTC
things will be a bit better when i do this too: madou/armory-react#225
Archomeda
@Archomeda
Mar 21 2017 22:06 UTC
but otherwise @Darqam i don't know; it's just what you prefer i suppose
Darqam
@Darqam
Mar 21 2017 22:07 UTC
hrm.... I'll have a look at those. I haven't looked much into the differences for now. So was mainly looking for a starting point
Archomeda
@Archomeda
Mar 21 2017 22:07 UTC
currently i like working with mongodb for my discord bot :)
but it doesn't help that i'm rewriting stuff again
Michael Dougall
@madou
Mar 21 2017 22:08 UTC
cool, cya guys im off :shipit:
Archomeda
@Archomeda
Mar 21 2017 22:10 UTC
cya
Darqam
@Darqam
Mar 21 2017 22:12 UTC
yeah, well my current way of storing stuff for my copy of the api is, imo, the worst way I could have done it, so I'm looking for something that won't be hell to work with. I might look at mongo, and postgres later this week to see exactly how they work
Archomeda
@Archomeda
Mar 21 2017 22:13 UTC
with mongodb you just copy the json into it really, and it allows you to search with it :P
Darqam
@Darqam
Mar 21 2017 22:13 UTC
ugh wait, I might have no choice depending on the host capabilities
Archomeda
@Archomeda
Mar 21 2017 22:13 UTC
though, postgresql allows you to search in json as well nowadays
Darqam
@Darqam
Mar 21 2017 22:13 UTC
so does mongodb just make a json file per item then?
so you'd end up with thousands of files?
darthmaim
@darthmaim
Mar 21 2017 22:14 UTC
no
Archomeda
@Archomeda
Mar 21 2017 22:14 UTC
it's bundled
Darqam
@Darqam
Mar 21 2017 22:14 UTC
ok, good
Darqam
@Darqam
Mar 21 2017 22:15 UTC
aaaaand mongo is not supported unless I pay a premium. I've got to stop dealing with Hostgator....
Eearslya Sleiarion
@Eearslya
Mar 21 2017 22:16 UTC
Hostgator? Eesh. I'd recommend a simple VPS that you can install whatever you want on.
Darqam
@Darqam
Mar 21 2017 22:17 UTC
yeah, well I inherited that through me working on another website, and it's been a pain in the ass from day1
David Reeß
@queicherius
Mar 21 2017 22:17 UTC
mongodb is pretty good with documents with multiple structures (e.g. items, which are different based on what they are)
Darqam
@Darqam
Mar 21 2017 22:17 UTC
some day I'll change it... some day
David Reeß
@queicherius
Mar 21 2017 22:19 UTC
I personally dont touch any hoster run by EIG
Darqam
@Darqam
Mar 21 2017 22:19 UTC
I will keep mongodb in mind though, thanks
Eearslya Sleiarion
@Eearslya
Mar 21 2017 22:21 UTC
I've actually got a monster of a dedicated server to myself; 48GB RAM running VMs
Oh, wait, 64GB
Darqam
@Darqam
Mar 21 2017 22:22 UTC
Once I finally move to my own place I'll build my own machine for it, since the scope of the project is so small in usage it shouldn't be a problem. But hell if I'm going to run stuff of my laptop which is my only computer atm >.>
why use mongodb when pgsql has JSONB columns
darthmaim
@darthmaim
Mar 21 2017 22:28 UTC
mysql has json columns too if you need to run it on a webspace that doesn't support and other db
CREATE TABLE items (
  id INTEGER PRIMARY KEY,
  lang CHAR(2),
  blob JSONB
);
CREATE INDEX ON items USING GIN (to_tsvector('english', blob->>'name')) WHERE lang = 'en';
@darthmaim yeah but then you're using mysql
might as well be back on mongo at that point
darthmaim
@darthmaim
Mar 21 2017 22:30 UTC
if you need to run it on a webspace that doesn't support and other db
man mysql's json operators are almost the same as pgsql but look slightly different
that's so annoying
pgsql> SELECT blob->>'name';
mysql> SELECT JSON_UNQUOTE(blob->>'$.name');
darthmaim
@darthmaim
Mar 21 2017 22:33 UTC
:/
Archomeda
@Archomeda
Mar 21 2017 22:33 UTC
they have to be unique
:P
the unique snowflake
Does MySQL support computed indexes?
Eearslya Sleiarion
@Eearslya
Mar 21 2017 22:35 UTC
@lye why must you make me question my database structure
    create_table :items, { id: false } do |t|
      t.primary_key :id, :null => false
      t.string :name, :null => false
      t.string :icon, :null => false
      t.string :description
      t.string :item_type, :null => false
      t.string :rarity, :null => false
      t.integer :level, :default => 0
      t.integer :vendor_value, :default => 0
      t.timestamps
    end
I mean sure you can do that
but that's a lot of work
I have a schema somewhere that fully normalizes /v2/items (or at least, the output from 2 years ago)
darthmaim
@darthmaim
Mar 21 2017 22:36 UTC
no computed indexes in mysql :(
it was 26 tables? or something?
Eearslya Sleiarion
@Eearslya
Mar 21 2017 22:36 UTC
I was gonna do 1 main table and 1 for each type
@darthmaim apparently a new version has "generated columns" which you can index over
darthmaim
@darthmaim
Mar 21 2017 22:36 UTC
but you can create a view with the computed column and index that
5.7.x i think
does the computed column have to be materialized? or can be it virtual
darthmaim
@darthmaim
Mar 21 2017 22:36 UTC
I have no idea
Archomeda
@Archomeda
Mar 21 2017 22:37 UTC
@lye, get gw2api.com, make it your hobby-project to extend the API, and open source it on github :smile:
but i already have a hobby project
Archomeda
@Archomeda
Mar 21 2017 22:37 UTC
and pay the jackpot for hosting when it gets too popular
darthmaim
@darthmaim
Mar 21 2017 22:37 UTC
just one?
Eearslya Sleiarion
@Eearslya
Mar 21 2017 22:37 UTC
I'm gonna make a gw2oracle API too
it's a big one
Archomeda
@Archomeda
Mar 21 2017 22:38 UTC
related?
to gw2? no
i'll probably never get it to a workable state either
ChieftainAlex
@ChieftainAlex
Mar 21 2017 22:39 UTC
@Darqam german translation is best translation: Mini Krytanischer schlapper Fisch
darthmaim
@darthmaim
Mar 21 2017 22:40 UTC
ughh those german translations...
Archomeda
@Archomeda
Mar 21 2017 22:40 UTC
:|
man the forums
Archomeda
@Archomeda
Mar 21 2017 22:41 UTC
@Eearslya every time you mention gw2oracle, i keep thinking about something that can handle NLP and give me a nice result for every request i do
darthmaim
@darthmaim
Mar 21 2017 22:41 UTC
just don't look at them until the api is back up
right
Eearslya Sleiarion
@Eearslya
Mar 21 2017 22:41 UTC
NLP?
natural language processing
Archomeda
@Archomeda
Mar 21 2017 22:41 UTC
^
Eearslya Sleiarion
@Eearslya
Mar 21 2017 22:41 UTC
I'm not good at naming projects ;;
I don't really like the name either tbh
rename it to bookah.com
darthmaim
@darthmaim
Mar 21 2017 22:42 UTC
1380:05:17:32
uhm what?
uhhhh not my fault
Archomeda
@Archomeda
Mar 21 2017 22:42 UTC
like an oracle, "if i want to buy 42 stacks of item x, will i have enough gold leftover to buy 10 stacks of item y" for example :P
@Archomeda you should submit a gw2 integration to wolfram alpha
Archomeda
@Archomeda
Mar 21 2017 22:43 UTC
or "will i have enough gold to buy a legendary at the end of this month"
Eearslya Sleiarion
@Eearslya
Mar 21 2017 22:43 UTC
asuramasterrace.net
Archomeda
@Archomeda
Mar 21 2017 22:43 UTC
although that will be tricky
@lye lol
i don't even want to look at how NLP works
i know the concept, but man...
I think google opensourced a toolkit awhile back
Eearslya Sleiarion
@Eearslya
Mar 21 2017 22:44 UTC
I had the idea for gw2watchdog.net at one time
Archomeda
@Archomeda
Mar 21 2017 22:45 UTC
i mean, even google has an answer for this query: "who was the american president in 1988"
ehh
and it even autocorrected my spelling error lol
"present" lmao
risky click of the day
Darqam
@Darqam
Mar 21 2017 22:50 UTC
"Because as far as I can find on any site (wiki,forums,all wvw sites) they all say [victory points] not exposed." grumble grumble don't say wiki doesn't show it if you haven't looked
I don't update that for nothing
darthmaim
@darthmaim
Mar 21 2017 22:50 UTC
:P
Archomeda
@Archomeda
Mar 21 2017 22:51 UTC
hahaa
one day i wanted to update something in the API namespace
but i couldn't because i didn't have permissions :'(
Darqam
@Darqam
Mar 21 2017 22:52 UTC
ask for perms, vent hands it out like candy
"seeing that I'm not an active contributor here" it was literally my first contribution
creating a new talk page
Eearslya Sleiarion
@Eearslya
Mar 21 2017 22:54 UTC
I had fun updating the API namespace until job
Archomeda
@Archomeda
Mar 21 2017 22:54 UTC
bleh
Eearslya Sleiarion
@Eearslya
Mar 21 2017 22:54 UTC
now @Darqam is doing a good job about it
Darqam
@Darqam
Mar 21 2017 22:54 UTC
<3
that main API page though, needs to be manually maintained for each new endpoint. It is ugly and I hate it
/v2/wvw/matches.documentation
Darqam
@Darqam
Mar 21 2017 22:57 UTC
alomst tried to go there >.>
but then remembered :fire:
:goat:
Archomeda
@Archomeda
Mar 21 2017 22:58 UTC
outputs: This endpoint will be unavailable until further notice, while the rest of the API is online.
Archomeda @Archomeda is not sorry about referring to that forum thread
Eearslya Sleiarion
@Eearslya
Mar 21 2017 22:59 UTC
Blame the forum.
Darqam
@Darqam
Mar 21 2017 22:59 UTC
yes please
just a heads up -- in like half an hour I'm going to try turning the API on with a super low rate limit
Archomeda
@Archomeda
Mar 21 2017 23:00 UTC
woooo
good luck!
darthmaim
@darthmaim
Mar 21 2017 23:00 UTC
:+1:
I want to flood the system with errors basically
Just to make sure everything works
Darqam
@Darqam
Mar 21 2017 23:00 UTC
lol
good luck
Archomeda
@Archomeda
Mar 21 2017 23:01 UTC
my guild discord will probably get spammed with new guild logs... heh
darthmaim
@darthmaim
Mar 21 2017 23:01 UTC
I hope you have a graph setup with all requests that get rate limited
Eearslya Sleiarion
@Eearslya
Mar 21 2017 23:01 UTC
Super low?
like 60/minute
Archomeda
@Archomeda
Mar 21 2017 23:01 UTC
oh, @queicherius which reminds me, i haven't seen support in your library for the ?since param on /v2/guilds/:id/logs
that might be too high
Eearslya Sleiarion
@Eearslya
Mar 21 2017 23:02 UTC
time for me to quickly grab some example items so I can write tests
Archomeda
@Archomeda
Mar 21 2017 23:02 UTC
does it exist? or do i have to make a PR at some point?
Darqam
@Darqam
Mar 21 2017 23:02 UTC
Lye, 1/min, do it :D
Eearslya Sleiarion
@Eearslya
Mar 21 2017 23:02 UTC
1 lucky person on gw2e gets to see something per minute
Archomeda
@Archomeda
Mar 21 2017 23:02 UTC
what error will the API spit out when rate limited?
HTTP 429
Darqam
@Darqam
Mar 21 2017 23:03 UTC
api would be on fire if I let you in
{ "error" : "too many requests" } I think
Eearslya Sleiarion
@Eearslya
Mar 21 2017 23:03 UTC
rate limit exceeded
Archomeda
@Archomeda
Mar 21 2017 23:03 UTC
cool
Eearslya Sleiarion
@Eearslya
Mar 21 2017 23:03 UTC
I thought that's what you said before
i don't remember
Eearslya Sleiarion
@Eearslya
Mar 21 2017 23:03 UTC
we're about to find out
{"text":"too many requests"}
Darqam
@Darqam
Mar 21 2017 23:03 UTC
I like the json error format regardless
the 429 is what you should be looking for TBH
Eearslya Sleiarion
@Eearslya
Mar 21 2017 23:03 UTC
true
David Reeß
@queicherius
Mar 21 2017 23:04 UTC
@Archomeda Feel free to make an issue / PR, probably missed that :)
darthmaim
@darthmaim
Mar 21 2017 23:04 UTC
make a new issue in my php wrapper too :P
Eearslya Sleiarion
@Eearslya
Mar 21 2017 23:05 UTC
I was gonna make a ruby wrapper but I gave up
Archomeda
@Archomeda
Mar 21 2017 23:07 UTC
ehh, i'm not touching PHP :P
i'll create an issue for you then xD
<?php echo "is fun!"
Darqam
@Darqam
Mar 21 2017 23:10 UTC
I like php, it feels like I'm permanently stumbling in the dark and yet getting the results I want >.>
Archomeda
@Archomeda
Mar 21 2017 23:10 UTC
@darthmaim GW2Treasures/gw2api right?
darthmaim
@darthmaim
Mar 21 2017 23:10 UTC
:+1:
Eearslya Sleiarion
@Eearslya
Mar 21 2017 23:10 UTC
Maybe I should just name my site GW2Stuff
Darqam
@Darqam
Mar 21 2017 23:12 UTC
a few things called gw2stuff already :p
Eearslya Sleiarion
@Eearslya
Mar 21 2017 23:12 UTC
dangit
darthmaim
@darthmaim
Mar 21 2017 23:13 UTC
GET YOUR GUILD WARS 2 STUFF EASY
NO MORE LONG FARMING! ALWAYS CHEAPEST PRICE
welp, that name is probably banned in a few chats
Eearslya Sleiarion
@Eearslya
Mar 21 2017 23:14 UTC
I didn't even google it XD
Looks like I'm still stuck with this name for now
ChieftainAlex
@ChieftainAlex
Mar 21 2017 23:15 UTC
wiki won't be hitting you with TP requests when you boot up.
Archomeda
@Archomeda
Mar 21 2017 23:17 UTC
actually @queicherius, does the library support custom parameters at all?
there's also the world (or something) parameter for /v2/wvw/matches (i think?)
David Reeß
@queicherius
Mar 21 2017 23:17 UTC
Yeah, like here.
Archomeda
@Archomeda
Mar 21 2017 23:18 UTC
ah, thanks
i knew i saw it somewhere
David Reeß
@queicherius
Mar 21 2017 23:18 UTC
I probably missed nearly all the custom ones
Archomeda
@Archomeda
Mar 21 2017 23:19 UTC
honestly, the guild log, wvw match and the commerce ones are the only ones that i know of
darthmaim
@darthmaim
Mar 21 2017 23:20 UTC
recipe and guild search endpoints
Archomeda
@Archomeda
Mar 21 2017 23:21 UTC
ah, guild search by name as well yeah; i see it in the source
Darqam
@Darqam
Mar 21 2017 23:22 UTC
https://status.discordapp.com/incidents/dj3l6lw926kl if you guys want to read about that discord crash yesterday
tl;dr distributed systems are hard :crying_cat_face:
wow this sounds familiar
Once the event queues on these session nodes reached a limit, they ran out of available memory and crashed.
Michael Dougall
@madou
Mar 21 2017 23:29 UTC
Good luck @lye !!
Darqam
@Darqam
Mar 21 2017 23:29 UTC
confirmed, gw2 api runs on discord
These patches introduce limitations on connections and message sending
Archomeda
@Archomeda
Mar 21 2017 23:31 UTC
@queicherius running npm test still doesn't work :(
Error: Cannot find module '******\gw2api-client\tests\cache'
on windows
David Reeß
@queicherius
Mar 21 2017 23:32 UTC
That's weird as hell
Also @Darqam thanks, good read
Archomeda
@Archomeda
Mar 21 2017 23:32 UTC
let me try bash in windows
stack trace is quite long btw
npm installs dev dependencies by default right?
i thought you had to npm install --dev
darthmaim
@darthmaim
Mar 21 2017 23:36 UTC
nope, it installs always installs dev dependencies of the root package
Archomeda
@Archomeda
Mar 21 2017 23:37 UTC
well, it does run in bash on windows
but 93 failed tests... wtf
darthmaim
@darthmaim
Mar 21 2017 23:38 UTC
I had 41 or 49 failed tests in @queicherius gw2api-client
no idea why it wasn't always the same number
Archomeda
@Archomeda
Mar 21 2017 23:38 UTC
well, my added test passes, so there's that
David Reeß
@queicherius
Mar 21 2017 23:38 UTC
You need redis running for all tests to pass, since the cache depends on it
darthmaim
@darthmaim
Mar 21 2017 23:38 UTC
run the linter too
Archomeda
@Archomeda
Mar 21 2017 23:38 UTC
oh
darthmaim
@darthmaim
Mar 21 2017 23:38 UTC
because the linter doesn't run when you have failing tests
bumping my "30 minutes" back another hour
perforce died
darthmaim
@darthmaim
Mar 21 2017 23:40 UTC
rip
Archomeda
@Archomeda
Mar 21 2017 23:42 UTC
:(
does anyone know if the sub endpoints of /v2/wvw/matches also support ?world?
Darqam
@Darqam
Mar 21 2017 23:43 UTC
I'd hope Lye knows
@Archomeda they should yeah.
darthmaim
@darthmaim
Mar 21 2017 23:43 UTC
they do
Archomeda
@Archomeda
Mar 21 2017 23:44 UTC
oh wait, i see examples at the bottom of the wiki page
thanks <3