These are chat archives for arenanet/api-cdi

22nd
Mar 2017
Archomeda
@Archomeda
Mar 22 2017 00:05 UTC
let's see, is there any custom parameter missing here that anyone know of?
  • /v2/commerce/exchange (gems, coins)
  • /v2/guild/:id/log (since)
  • /v2/pvp/seasons/:id/leaderboards (ids)
  • /v2/recipes/search (input, output)
  • /v2/wvw/matches (world)
  • /v2/wvw/matches/overview (world)
  • /v2/wvw/matches/scores (world)
  • /v2/wvw/matches/stats (world)
this is what @queicherius's library has in the docs on my branch atm
darthmaim
@darthmaim
Mar 22 2017 00:05 UTC
  • /v2/guild/search (name)
Michael Dougall
@madou
Mar 22 2017 00:06 UTC
You going through a documentation journey? :angel:
Archomeda
@Archomeda
Mar 22 2017 00:06 UTC
well, i updated the docs to reflect my code changes :)
hmm @darthmaim, indeed, i missed that one since the doc structure is different for that one
Michael Dougall
@madou
Mar 22 2017 00:11 UTC
Ah ok cool! Maybe I should move over to use gw2api lib :-)
Archomeda
@Archomeda
Mar 22 2017 00:11 UTC
argh, @queicherius doesn't use ;
if everyone could gather round real quick, I'd like us to gather to share a few moments of silence for perforce
David Reeß
@queicherius
Mar 22 2017 00:12 UTC
:pray:
Pat Cavit
@tivac
Mar 22 2017 00:12 UTC
:fire:
darthmaim
@darthmaim
Mar 22 2017 00:12 UTC
:pray:
Archomeda
@Archomeda
Mar 22 2017 00:12 UTC
stupid me,
:fire: :pray: :fire:
Michael Dougall
@madou
Mar 22 2017 00:15 UTC
:pray:
Darqam
@Darqam
Mar 22 2017 00:16 UTC
:pray:
well i'm going to be here all night
i've got a pending submit which turns the API back on
but the p4 server is hung
darthmaim
@darthmaim
Mar 22 2017 00:23 UTC
:(
and by "pending" i mean "uncancelable in-flight"
lye @lye cackles madly
Archomeda
@Archomeda
Mar 22 2017 00:24 UTC
@queicherius i created the PR at queicherius/gw2api-client#23 and my test failure at queicherius/abc-environment#35
David Reeß
@queicherius
Mar 22 2017 00:25 UTC
@Archomeda Thanks <3 I'll have a look at the environment thing tomorrow and at the client when the API is back up. :+1:
Archomeda
@Archomeda
Mar 22 2017 00:25 UTC
i'm just baffled at npm test... i mean the files are there
David Reeß
@queicherius
Mar 22 2017 00:26 UTC
I have no idea either, will have to test.
Archomeda
@Archomeda
Mar 22 2017 00:26 UTC
but yeah, whenever you have time :smile:
David Reeß
@queicherius
Mar 22 2017 00:26 UTC
Does the windows bash do shell expansion?
@queicherius msysgit's bash shell does
Archomeda
@Archomeda
Mar 22 2017 00:27 UTC
i'm not that familiar with bash at all, what kind of expansion do you mean in this context?
I was guess globbing, e.g. echo *.c ???bar
David Reeß
@queicherius
Mar 22 2017 00:28 UTC
cat {src,tests,.}/**/*.js
yeah that
Archomeda
@Archomeda
Mar 22 2017 00:29 UTC
oh, i should probably note i'm running this in powershell
my best guess, probably not
Is ** a thing?
powershell lols
Archomeda
@Archomeda
Mar 22 2017 00:29 UTC
hey, better than cmd
and webstorm doesn't properly support bash in windows
"stabbing yourself in the foot is better than stabbing yourself in the hand"
you're not wrong
darthmaim
@darthmaim
Mar 22 2017 00:30 UTC
but you're also not right
IT FINISHED
Archomeda
@Archomeda
Mar 22 2017 00:30 UTC
:tada:
darthmaim
@darthmaim
Mar 22 2017 00:30 UTC
Wool
-l
David Reeß
@queicherius
Mar 22 2017 00:30 UTC
grats! :D
now we gotta wait for the configs to propagate
David Reeß
@queicherius
Mar 22 2017 00:31 UTC
I always use ** / globstar for recursive directories, dont know if I picked that up wrong
darthmaim
@darthmaim
Mar 22 2017 00:31 UTC
No that is correct
huh new features for me
darthmaim
@darthmaim
Mar 22 2017 00:32 UTC
  • is only one level, ** is all subdirectories
I normally use regular old borne shell, so y'all and your newfangled features
Archomeda
@Archomeda
Mar 22 2017 00:32 UTC
i've seen this in many libraries that interact with the file system
not only bash
hell, i think even windows supports it
(don't quote me on that)
"hell, i think even windows supports it" -- Archomeda, 2017
Archomeda
@Archomeda
Mar 22 2017 00:33 UTC
good, not me then
David Reeß
@queicherius
Mar 22 2017 00:33 UTC
""hell, i think windows supports it fully" -- Archomedia, 2017" -- lye, 2017
well crap
David Reeß
@queicherius
Mar 22 2017 00:33 UTC
chinese whispers?
Archomeda
@Archomeda
Mar 22 2017 00:33 UTC
lol
that edit is not gonna save you :whistles:
okay I'm seeing traffic
it's routing to the correct rate-limiting servers, yay
Michael Dougall
@madou
Mar 22 2017 00:35 UTC
:fire:
Darqam
@Darqam
Mar 22 2017 00:35 UTC
madou shhhhhh
@queicherius gw2e IPs should be whitelisted so you don't have a rate limit
everyone else -- you're stuck on 60 reqs/min for a bit
Archomeda
@Archomeda
Mar 22 2017 00:36 UTC
:')
Michael Dougall
@madou
Mar 22 2017 00:36 UTC
thanks obama
Darqam
@Darqam
Mar 22 2017 00:36 UTC
damn elitists
Archomeda
@Archomeda
Mar 22 2017 00:36 UTC
pug life... wait
also
actually I'll just slam it myself
Archomeda
@Archomeda
Mar 22 2017 00:36 UTC
yay, my discord bot went crazy and posted the guild log backlog
Darqam
@Darqam
Mar 22 2017 00:37 UTC
I want to test something quick, no :fire: yet plz
David Reeß
@queicherius
Mar 22 2017 00:38 UTC
@lye Thanks <3 All backend things should currently be disabled, so if stuff breaks it's not me!
Archomeda
@Archomeda
Mar 22 2017 00:38 UTC
sadly most of the entries fell off the log :(
dang
Darqam
@Darqam
Mar 22 2017 00:38 UTC
rip, fire
Archomeda
@Archomeda
Mar 22 2017 00:38 UTC
oh well
okay managed to trigger the rate limit
so that's working
darthmaim
@darthmaim
Mar 22 2017 00:42 UTC
:tada:
Eearslya Sleiarion
@Eearslya
Mar 22 2017 00:49 UTC
apparently chrome is not fast enough to hit it with constant ctrl-shift-rs
i totally hit it with chrome
it's once/second for a minute
also why are my graphs broken I am so confused
Eearslya Sleiarion
@Eearslya
Mar 22 2017 00:54 UTC
I tried hitting /v2/gliders a bunch D:
huh, got the same behavior
i think chrome aborts the in-flight requests if you click f5 too fast
umm
hmm
Archomeda
@Archomeda
Mar 22 2017 00:58 UTC
why not run wget in a loop if you're a unix user anyway ? :P
this is interesting
/v2/build gives me the error but /v2/gliders doesn't
Eearslya Sleiarion
@Eearslya
Mar 22 2017 00:58 UTC
I knew it
🤢
gonna be a long night
Archomeda
@Archomeda
Mar 22 2017 00:59 UTC
don't do that to yourself lye :worried:
i mean
i'd rather spend an hour now than get a phone call in two
Archomeda
@Archomeda
Mar 22 2017 01:00 UTC
eh.. hm
and my phone call I mean my housemate gets the phone call and then knocks on my door to tell me he got a phone call
Archomeda
@Archomeda
Mar 22 2017 01:01 UTC
and say a bunch of other things?
like "did you light the server on :fire: again?"
Eearslya Sleiarion
@Eearslya
Mar 22 2017 01:02 UTC
gw2@naga:~$ for run in {1..120}; do curl --write-out %{http_code} --silent --output /dev/null https://api.guildwars2.com/v2/build; done
200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200
Archomeda
@Archomeda
Mar 22 2017 01:03 UTC
just don't forget the time and realize you're still in the office at midnight :(
Eearslya Sleiarion
@Eearslya
Mar 22 2017 01:03 UTC
it looked like it was averaging about 3/s
Darqam
@Darqam
Mar 22 2017 01:03 UTC
Ey, well api was up long enough for me to debug my code snipet <3
did you want more pings to the API?
Eearslya Sleiarion
@Eearslya
Mar 22 2017 01:08 UTC
I'm trying my best to hit the limit
i'm so confused
I added throw 'ratelimit' to the ratelimit code
/v2/gliders is the only endpoint i've found that doesn't hit it
also my local copy is successfully logging per-endpoint requests?
I'm really not sure how this broke
Darqam
@Darqam
Mar 22 2017 01:11 UTC
it's the newest one, so maybe something funny is happening with new stuff?
Eearslya Sleiarion
@Eearslya
Mar 22 2017 01:12 UTC
4 consoles running a curl each couldn't hit limit on gliders
@Eearslya gliders doesn't apply ratelimiting for reasons that I cannot discern
like, it does not execute the ratelimit code
I don't know why
I don't know how
I'm pushing a patch through now
Darqam
@Darqam
Mar 22 2017 01:13 UTC
magic switch not flipping?
gremlins
literally gremlins
Archomeda
@Archomeda
Mar 22 2017 01:14 UTC
@Darqam haha, i noticed @ChieftainAlex requested permissions for me; that response though :D
do you hate gremlins?
Archomeda
@Archomeda
Mar 22 2017 01:26 UTC
well, at least it's working on /v2/items right? that was the biggest culprit before
Eearslya Sleiarion
@Eearslya
Mar 22 2017 01:27 UTC
I'm testing on several endpoints right now
have to sleep between tests though
items does give 429s
it works on /v2/items, tested that locally
it's a lot easier when your local config is 6 reqs/min
Eearslya Sleiarion
@Eearslya
Mar 22 2017 01:29 UTC
tested build, gliders, items, recipes, wvw/matches...yeah, only gliders failed
it makes no sense
I implemented ratelimiting by modifying the router.get bit
so that it invokes the ratelimit check before the controller
Archomeda
@Archomeda
Mar 22 2017 01:30 UTC
is there a hardcoded bit in the ratelimit backend that matches the glider endpoint? :worried:
and skips it?
but in /v2/gliders (which uses the same routing bits as everything else in the same one file) doens't execute it
yeah I put a throw statement in the ratelimit code just to sanity check
and it literally doesn't hit it
Darqam
@Darqam
Mar 22 2017 01:30 UTC
took me forever to dig this up, but Lye this might explain why things are bad (it's fluff, no it wont): Gnomes run your computer
Eearslya Sleiarion
@Eearslya
Mar 22 2017 01:31 UTC
Is IIS hijacking the request somehow?
shouldn't be
since I don't have IIS running locally
and it still repros
Eearslya Sleiarion
@Eearslya
Mar 22 2017 01:32 UTC
welp that's all I had
gremlins
there's probably a totally logical reason behind it, but figuring it out would take 2h
so I just rejiggered how the ratelimiter hooks in
Archomeda
@Archomeda
Mar 22 2017 01:33 UTC
add /v2/gremlins and return {text: "everywhere"}
pretty much
Jonathan Andrist
@rwfrk_twitter
Mar 22 2017 01:42 UTC
Gremlins. Litteral Gremlins.
p4 could have chosen a much better day to throw temper tantrums
dwayna save me
well
I don't think I can actually do anything else tonight
Archomeda
@Archomeda
Mar 22 2017 02:00 UTC
the api isn't gonna fall over? :smile:
i mean
i wouldn't bet on it
Archomeda
@Archomeda
Mar 22 2017 02:00 UTC
alright
still 60/m?
yeah
Archomeda
@Archomeda
Mar 22 2017 02:01 UTC
fair ^^
i'll up it once i get this fix to /v2/gliders
need to be able to verify the fix on live
Archomeda
@Archomeda
Mar 22 2017 02:01 UTC
appreciate what you're doing <3
i'm really upset that this was down for so long, so
Archomeda
@Archomeda
Mar 22 2017 02:03 UTC
have a good sleep though :)
going to pound some tequila and get a couple hours of path of exile in
that should do the trick
Archomeda
@Archomeda
Mar 22 2017 02:04 UTC
:D
:fire: doing a live deploy :fire:
I really hope this magically fixes the perfs
nope
but hard-restarting the processes does lol
WELL THEN
Archomeda
@Archomeda
Mar 22 2017 02:33 UTC
wut
dude i don't even know
there's so much going wrong today I don't even know where to start
Michael Dougall
@madou
Mar 22 2017 02:34 UTC
Perf gainz
okay, verified that /v2/gliders is now magically rate limited
for reasons that I do not understand.
Archomeda
@Archomeda
Mar 22 2017 02:37 UTC
:sparkles:
going to up the rate limit to 600 reqs/minute
@Eearslya nice find on that btw
Eearslya Sleiarion
@Eearslya
Mar 22 2017 02:38 UTC
I started testing /v2/items but the request was too big, loading too slow to hit limit, so I just went with a smaller endpoint. /v2/buildnever occurred to me XD
lol
well thank goodness for that
Michael Dougall
@madou
Mar 22 2017 02:55 UTC
Thanks for your hard work lye
seems like it's working
@madou no problem, thanks for bearing with me
Archomeda
@Archomeda
Mar 22 2017 02:55 UTC
5k requests hit the limit when 60/min? :O
hahahaha yeah apparently
lord only knows
Archomeda
@Archomeda
Mar 22 2017 02:56 UTC
probably still that /v2/items...
Michael Dougall
@madou
Mar 22 2017 02:56 UTC
Re: the armory embeds, I hope after I put a few more smarts in them there won't be any issues at least from my end.
Eearslya Sleiarion
@Eearslya
Mar 22 2017 02:56 UTC
I wonder if any of those spikes were me
Michael Dougall
@madou
Mar 22 2017 02:56 UTC
There is at the very least 7 day caching so...
eh honestly I don't have issue with the embeds using the API
like none at all
the API should really never be down that often
Michael Dougall
@madou
Mar 22 2017 02:56 UTC
Just need to add smarter batching and lazy loaikg
this has just been a magical christmas week
Archomeda
@Archomeda
Mar 22 2017 02:57 UTC
pretty sure you've seen all the lights flickering on your end :P
hoping that they don't start blinking red over night
Archomeda
@Archomeda
Mar 22 2017 02:59 UTC
@queicherius seems someone wants a gw2e mobile app https://www.reddit.com/r/Guildwars2/comments/60s2gv/does_a_gw2_efficiency_app_exist/ :P
^ to give some insight into how many things were making >60reqs/minute
Archomeda
@Archomeda
Mar 22 2017 03:00 UTC
omg
(the scales are different)
(but it's like 20%)
it's probably commerce sites which I expect to be making ~200 reqs/minute
Archomeda
@Archomeda
Mar 22 2017 03:01 UTC
5.2k limit vs 6.5k total
the one on the left doesn't distinguish between datacenter
Archomeda
@Archomeda
Mar 22 2017 03:01 UTC
80% hit the limit then
maybe I should change it
nah you gotta add both lines on the right together
Eearslya Sleiarion
@Eearslya
Mar 22 2017 03:01 UTC
that big spike deeefinitely wasn't me
Archomeda
@Archomeda
Mar 22 2017 03:01 UTC
oh wait
yeah
@Eearslya lmao
Eearslya Sleiarion
@Eearslya
Mar 22 2017 03:02 UTC
This ruby library is fast af though
is it using libcurl
Archomeda
@Archomeda
Mar 22 2017 03:02 UTC
5.2k limit vs (6.5k+3.1k) total
Eearslya Sleiarion
@Eearslya
Mar 22 2017 03:02 UTC
yes
if it is, is it still ruby
Eearslya Sleiarion
@Eearslya
Mar 22 2017 03:02 UTC
I mean
maybe
sorta
if a butterfly lands on the nose of a man with a chainsaw
Archomeda
@Archomeda
Mar 22 2017 03:02 UTC
like 55% hit the limit then
and a tree is cut down
Archomeda
@Archomeda
Mar 22 2017 03:02 UTC
at least in that peak
did the butterfly cut down the tree
Eearslya Sleiarion
@Eearslya
Mar 22 2017 03:02 UTC
yes
lol
WELL I'M GOING HOME
AND I'M TAKING MY BUTTERFLIES WITH ME
Archomeda
@Archomeda
Mar 22 2017 03:03 UTC
:wave:
(I'll be back on with more graphs once I VPN back in)
Archomeda
@Archomeda
Mar 22 2017 03:03 UTC
have a nice evening
Eearslya Sleiarion
@Eearslya
Mar 22 2017 03:03 UTC
Really, though, 300 requests to /v2/items before I could type the next command
Archomeda
@Archomeda
Mar 22 2017 03:03 UTC
you've earned it :)
Eearslya Sleiarion
@Eearslya
Mar 22 2017 03:03 UTC
@lye you should totally publish these graphs somewhere
we'd all love it
Archomeda
@Archomeda
Mar 22 2017 03:04 UTC
api.guildwars2.com/graphs let's go
Darqam
@Darqam
Mar 22 2017 03:17 UTC
should we tell lye the new mini isn't on the API yet? :p
whitelisted the item for the mini
should take about 5 minutes to show up
Darqam
@Darqam
Mar 22 2017 04:37 UTC
<3
Eearslya Sleiarion
@Eearslya
Mar 22 2017 05:59 UTC
Got 55847 items
Performed ItemCrawlerJob from Async(default) in 37316.83ms
:tada:
not doing this a long time ago was stupid and i regret it
Eearslya Sleiarion
@Eearslya
Mar 22 2017 06:23 UTC
2500 overflows, oh lord
i wonder if our rate limiting bit keeps track of which IPs are the most offending
Eearslya Sleiarion
@Eearslya
Mar 22 2017 06:23 UTC
Looks like someone's "update-the-whole-api" routine
you can burst 600 requests
like ... maybe it takes more than that?
does the API have more than 125k or whatever objects/
:salt:
Eearslya Sleiarion
@Eearslya
Mar 22 2017 06:24 UTC
correction: someone's poorly-written update-the-whole-api routine
lol.
ChieftainAlex
@ChieftainAlex
Mar 22 2017 07:29 UTC
api genuinely seems faster to respond.. or my housemates are asleep
but then again, that does 9 requests anyway
Archomeda
@Archomeda
Mar 22 2017 12:45 UTC
oh wow, that graph
Werdes
@werdes
Mar 22 2017 12:50 UTC
it lives \o/ Thanks lye :heart:
windwarrior
@windwarrior
Mar 22 2017 15:32 UTC
To retrieve the entirety of /v2/items you need 280 requests
so you can do that about twice per minute
Archomeda
@Archomeda
Mar 22 2017 15:33 UTC
you'll never need to do that though
windwarrior
@windwarrior
Mar 22 2017 15:33 UTC
nah
Archomeda
@Archomeda
Mar 22 2017 15:33 UTC
i mean, it's cached for 5 minutes server-sided
windwarrior
@windwarrior
Mar 22 2017 15:33 UTC
you literally need to retrieve that once every patch or so
Archomeda
@Archomeda
Mar 22 2017 15:33 UTC
a full update? yeah
you do have to check for new whitelisted items though, but that's just requesting all the ids and diffing it
max 2 requests, if there are new items; otherwise it's just 1 request
Eearslya Sleiarion
@Eearslya
Mar 22 2017 15:34 UTC
Took me 12 minutes just to commit all the items to pgsql once I'd fetched them
Archomeda
@Archomeda
Mar 22 2017 15:35 UTC
then something seems wrong :P
don't think it should take that long
Eearslya Sleiarion
@Eearslya
Mar 22 2017 15:35 UTC
Not sure ActiveRecord likes going that fast
Darqam
@Darqam
Mar 22 2017 15:36 UTC
yeah, the only thing I do with items is take the diff and then check the new stuff. Only other option is to query for a specific id to update. I have yet to crawl the whole /v2/items since I first got the whole database
windwarrior
@windwarrior
Mar 22 2017 15:41 UTC
well if a description gets updated
or an icon gets fixed
you wont have it
Archomeda
@Archomeda
Mar 22 2017 15:41 UTC
yup
darthmaim
@darthmaim
Mar 22 2017 15:42 UTC
German translations of names change every patch it feels like
smiley
@codemasher
Mar 22 2017 15:45 UTC
the same happens in Wildstar but the translations are (mostly) better than in GW2. some of the bad translations however have a slight GW2 feel, makes me wonder if they have the same translators
darthmaim
@darthmaim
Mar 22 2017 15:45 UTC
There were 39269 name changes since I started tracking
smiley
@codemasher
Mar 22 2017 15:46 UTC
wow
Darqam
@Darqam
Mar 22 2017 15:59 UTC
@Archomeda if you wanted, you could always add info on what happens when you hit rate limit for your reddit post
Archomeda
@Archomeda
Mar 22 2017 16:00 UTC
eh... i did mention the 429
Darqam
@Darqam
Mar 22 2017 16:01 UTC
fuck I'm blind
<.<
Archomeda
@Archomeda
Mar 22 2017 16:01 UTC
:smile:
smiley
@codemasher
Mar 22 2017 16:37 UTC
i made a silly thing, but i'm not sure if it works as intended bc i'm dumb. (basically a self-refilling token bucket)
https://gist.github.com/codemasher/10ee438b0dc7a3da9bc8d83040829a60
thanks Obama Gitter
darthmaim
@darthmaim
Mar 22 2017 16:40 UTC
the 39269 item name changes were only german btw, this is the full table:
Language Renamed Items
de 39269
en 256
es 12738
fr 9474
Eearslya Sleiarion
@Eearslya
Mar 22 2017 16:41 UTC
you keep a lot of interesting metrics
darthmaim
@darthmaim
Mar 22 2017 16:42 UTC
:chart_with_upwards_trend:
@Eearslya are you doing it all within one transaction?
Eearslya Sleiarion
@Eearslya
Mar 22 2017 16:42 UTC
@lye pretty sure ActiveRecord is doing a SELECT and INSERT per item
if it's all within one transaction it might cut down on lock contention
(rather than one transaction per select/insert)
Eearslya Sleiarion
@Eearslya
Mar 22 2017 16:46 UTC
how do you put 57k items in 1 transaction
BEGIN;
SELECT * FROM items WHERE id = 93;
INSERT INTO items VALUES(...);
SELECT * FROM items WHERE id = 94;
INSERT INTO items VALUES(...);
COMMIT;
You can use ActiveRecord::Base#transaction which accepts a block to execute for the transaction
so
def scrape()
    Item.transaction do
        actually_scrape()
    end
end
Eearslya Sleiarion
@Eearslya
Mar 22 2017 16:52 UTC
Oooh.
with ActiveRecord::Base#transaction it's automatically COMMIT'd at the end of the block if no exception is thrown; if an exception is thrown the transaction is ROLLBACKd.

Anyway having it all in a single transaction lets you do stuff like

def scrape()
    Item.transaction do
        delete_all_items()
        refetch_all_items()
    end
end

provided that refetch_all_items throws if an unrecoverable error occurs during the fetch, that will atomically update all item records in your db with the latest build

(and since ActiveRecord::Base#save uses a transaction internally -- it means that you'll have one big transaction rather than 70k little ones)
Why is someone fetching /v2/currencies?lang=ko
Archomeda
@Archomeda
Mar 22 2017 16:57 UTC
lol
...
Archomeda
@Archomeda
Mar 22 2017 16:57 UTC
well...
why is ko listed as a valid language
Archomeda
@Archomeda
Mar 22 2017 16:57 UTC
Possible locale 'langId' values include:
  en
  es
  de
  fr
  ko
  zh
ko and zh are practically useless in our use cases :P
ko just returns ErrUnknown
Archomeda
@Archomeda
Mar 22 2017 16:59 UTC
and pretty sure that ko doesn't even return valid stuff
since it's not actually a valid language
zh actually returns data
Archomeda
@Archomeda
Mar 22 2017 16:59 UTC
well, it's not always chinese is it?
I thought zh was using the chinese translations for things
Archomeda
@Archomeda
Mar 22 2017 17:00 UTC
that's what zh is yeah, but pretty sure i came across things in the past that just showed the english translation
some items only exist on one side or the other
Eearslya Sleiarion
@Eearslya
Mar 22 2017 17:00 UTC
@lye I'm curious, do you actually use Rails, or did you just google all of that on the spot
so the NA/EU-only items probably just aren't translated
@Eearslya ruby is gross
Eearslya Sleiarion
@Eearslya
Mar 22 2017 17:01 UTC
I've been liking it
Archomeda
@Archomeda
Mar 22 2017 17:01 UTC
hmm, fair lye
Edgar Doiron
@Coffee4cr
Mar 22 2017 17:01 UTC
oh wow, I just used a discord bot that does "paging" result
Archomeda
@Archomeda
Mar 22 2017 17:01 UTC
paging result?
Edgar Doiron
@Coffee4cr
Mar 22 2017 17:01 UTC
So I play VainGlory on mobile, and this bot can give Match data right, and shows a single match at a time
Archomeda
@Archomeda
Mar 22 2017 17:02 UTC
you lost me
Edgar Doiron
@Coffee4cr
Mar 22 2017 17:02 UTC
blob
they use the reaction arrow
to navigate through the matches
darthmaim
@darthmaim
Mar 22 2017 17:02 UTC
oh wow, thats actually quite clever
Archomeda
@Archomeda
Mar 22 2017 17:02 UTC
oh
lol what a hack
Edgar Doiron
@Coffee4cr
Mar 22 2017 17:02 UTC
so when you click on a reaction, the bot updates the text
David Reeß
@queicherius
Mar 22 2017 17:02 UTC
that's actually amazing
Archomeda
@Archomeda
Mar 22 2017 17:02 UTC
interesting XD
Edgar Doiron
@Coffee4cr
Mar 22 2017 17:02 UTC
I know right!
Eearslya Sleiarion
@Eearslya
Mar 22 2017 17:02 UTC
dam son
Edgar Doiron
@Coffee4cr
Mar 22 2017 17:03 UTC
@Nabrok time to up your game!
lol
Aralicia
@Aralicia
Mar 22 2017 17:03 UTC
That's interesting, but limited, since one person can only add a specific reaction once.
Eearslya Sleiarion
@Eearslya
Mar 22 2017 17:03 UTC
Can the bot remove a reaction?
Darqam
@Darqam
Mar 22 2017 17:03 UTC
unless removing it counts as a same action
Eearslya Sleiarion
@Eearslya
Mar 22 2017 17:03 UTC
that's true too
Aralicia
@Aralicia
Mar 22 2017 17:03 UTC
bot should be able to remove a reaction yes.
Edgar Doiron
@Coffee4cr
Mar 22 2017 17:03 UTC
well the match opens with already 1 vote in right and left arrow
so if you click it goes to 2
Eearslya Sleiarion
@Eearslya
Mar 22 2017 17:03 UTC
removing it should also trigger an event
Edgar Doiron
@Coffee4cr
Mar 22 2017 17:03 UTC
and click back it goes to 1
@Darqam it's the same action if you click it back(remove)
so it flicks between 1 and 2
Eearslya Sleiarion
@Eearslya
Mar 22 2017 17:04 UTC
so it could still work
Edgar Doiron
@Coffee4cr
Mar 22 2017 17:04 UTC
(or more if more people press it)
Aralicia
@Aralicia
Mar 22 2017 17:05 UTC
I see. there's a lot of things that can be done using reactions like that.
Edgar Doiron
@Coffee4cr
Mar 22 2017 17:05 UTC
So yeah, thought i'd share that here, since a lot of you are working on different bots :)
Eearslya Sleiarion
@Eearslya
Mar 22 2017 17:05 UTC
something to keep in mind, yeah
Aralicia
@Aralicia
Mar 22 2017 17:08 UTC
From the top of my head : polling (use reaction 1, 2, 3... to select an answer) and confirming participation (for raids, for example)
Darqam
@Darqam
Mar 22 2017 17:09 UTC
@darthmaim do you have it easily accessible if/when a en api entry gets updated? I'll eventually get my own version of it running (maybe), but until then if you have that, I'd love to be able to regularly check if anything changes as it happens
darthmaim
@darthmaim
Mar 22 2017 17:10 UTC
no sadly not
I'm only tracking item renames
Darqam
@Darqam
Mar 22 2017 17:10 UTC
hm, is even just that something I could occasionally check on?
if not, no worries
darthmaim
@darthmaim
Mar 22 2017 17:11 UTC
I could probably add some sort of API for that
Darqam
@Darqam
Mar 22 2017 17:12 UTC
honestly, if it's even just a webpage I need to load myself I'd be fine with that. It's something I could easily manually check once a week or something. I doubt EN stuff changes that often
darthmaim
@darthmaim
Mar 22 2017 17:12 UTC
gitter why
Darqam
@Darqam
Mar 22 2017 17:13 UTC
got the link for now, I'll take a peak at that
darthmaim
@darthmaim
Mar 22 2017 17:13 UTC
that are all item name changes since sometime in 2013
Darqam
@Darqam
Mar 22 2017 17:14 UTC
alright, thanks. Just want to make sure things are still ok for names at least on the en wiki (they really should be fine I think)
darthmaim
@darthmaim
Mar 22 2017 17:14 UTC
oh wait I think I already have an API
because I was generating those lists for the german wiki
Darqam
@Darqam
Mar 22 2017 17:15 UTC
lol, that csv file has 4 messed up entries
all renamed items since some hardcoded date as wiki table
Darqam
@Darqam
Mar 22 2017 17:15 UTC
if you sort it by name you get 4 items that... got renamed to blank?
oh that's pretty
darthmaim
@darthmaim
Mar 22 2017 17:16 UTC
there are a few items that have a blank name
so who knows...
¯\_(ツ)_/¯
Nabrok
@Nabrok
Mar 22 2017 17:17 UTC
That's clever.
darthmaim
@darthmaim
Mar 22 2017 17:17 UTC
Uhhm, that link obviously only has german items
I can add en for you
Nabrok
@Nabrok
Mar 22 2017 17:17 UTC
Not sure what you'd show with that for GW2 though.
Also, how'd they get the text in columns?
Darqam
@Darqam
Mar 22 2017 17:19 UTC
nah, the en stuff is on the csv file, so I can just auto-format it in notepad
I'll just steal the idea from your link
Edgar Doiron
@Coffee4cr
Mar 22 2017 17:19 UTC
@Nabrok https://api.guildwars2.com/v2/pvp/games maybe for this?
Edgar Doiron
@Coffee4cr
Mar 22 2017 17:20 UTC
@Nabrok not sure how they got the text in column
darthmaim
@darthmaim
Mar 22 2017 17:20 UTC
all en name changes since 2015-10-20 22:30:00
Darqam
@Darqam
Mar 22 2017 17:20 UTC
lol, that works. faster than I could edit
thanks
Nabrok
@Nabrok
Mar 22 2017 17:21 UTC
Yeah, you could have it go through your pvp game history.
Darqam
@Darqam
Mar 22 2017 17:22 UTC
that's.... a lot less than I expected (en name changes that is)
darthmaim
@darthmaim
Mar 22 2017 17:22 UTC
yeah
Darqam
@Darqam
Mar 22 2017 17:23 UTC
you sure that's everything? the csv has 256 en entries, that doesn't look like 256 lines on that link
Nabrok
@Nabrok
Mar 22 2017 17:24 UTC
On my old guild website I used to gather everybody's pvp data and look for matching game ids to display anything that was a guild match (i.e. 2+ in the same match).
darthmaim
@darthmaim
Mar 22 2017 17:24 UTC
all en name changes since 2015-10-20 22:30:00
the others where before that
Darqam
@Darqam
Mar 22 2017 17:25 UTC
oh I see, my bad
darthmaim
@darthmaim
Mar 22 2017 17:25 UTC
changed it to include all
but the german generated table will crash the wiki with that many entries
Archomeda
@Archomeda
Mar 22 2017 17:25 UTC
@Nabrok those bot responses are richembeds
Darqam
@Darqam
Mar 22 2017 17:25 UTC
well, looks like there is at least one that wasn't caught by the wiki (chicken feathers)
darthmaim
@darthmaim
Mar 22 2017 17:25 UTC
thats why I always hardcoded the date of when I last copied it to the wiki
Darqam
@Darqam
Mar 22 2017 17:25 UTC
gotcha
Archomeda
@Archomeda
Mar 22 2017 17:25 UTC
currently i have this
blob
just an example
Nabrok
@Nabrok
Mar 22 2017 17:26 UTC
@Archomeda nice, looks like that's a fairly recent feature (november 2016)
Archomeda
@Archomeda
Mar 22 2017 17:27 UTC
eh, possibly?
it's definitely some time after the webhooks were introduced
Darqam
@Darqam
Mar 22 2017 17:27 UTC
yeah, wasn't around in V8, so it's a V10 or V11 thing
Archomeda
@Archomeda
Mar 22 2017 17:28 UTC
hammerandchisel/discord-api-docs#167
Edgar Doiron
@Coffee4cr
Mar 22 2017 17:28 UTC
hmm so what are the commands/styling to get that?
Archomeda
@Archomeda
Mar 22 2017 17:28 UTC
it's API only
depends on the library you're using for a bot
Edgar Doiron
@Coffee4cr
Mar 22 2017 17:29 UTC
Like webhooks, bots and user accounts will be able to use rich embeds.
isn't that just users?
Archomeda
@Archomeda
Mar 22 2017 17:29 UTC
it's "bots and user accounts"
user accounts can use them as well, but you have to run a bot behind it
Edgar Doiron
@Coffee4cr
Mar 22 2017 17:29 UTC
Actually it's "webhooks, bots and user accounts"
lol
ok
Archomeda
@Archomeda
Mar 22 2017 17:30 UTC
you have "bot accounts" and "user accounts", there's no limitation :P
but you can only do this by using the api directly
i still want a quoting ability...
but yeah, e.g. discord.js has this: https://discord.js.org/#/docs/main/master/class/RichEmbed
Edgar Doiron
@Coffee4cr
Mar 22 2017 17:31 UTC
kk
Darqam
@Darqam
Mar 22 2017 17:31 UTC
archomeda, you can have quoting with selfbotes quite easily ;)
Archomeda
@Archomeda
Mar 22 2017 17:31 UTC
yes true
cba to set it up though :P
and referring to a specific message is still impossible
Nabrok
@Nabrok
Mar 22 2017 17:32 UTC
I guess it's probably time I updated the discord.js version.
Archomeda
@Archomeda
Mar 22 2017 17:32 UTC
even though messages have a snowflake, you can't link them :(
Darqam
@Darqam
Mar 22 2017 17:33 UTC
which is odd given the search links them
Archomeda
@Archomeda
Mar 22 2017 17:33 UTC
yeah...
Nabrok
@Nabrok
Mar 22 2017 17:33 UTC
It just annoys me that it requires node 6, is all.
Archomeda
@Archomeda
Mar 22 2017 17:34 UTC
node 6 is lts now
migrate :P
Nabrok
@Nabrok
Mar 22 2017 17:34 UTC
I know, but it's still not in default ubuntu etc
Archomeda
@Archomeda
Mar 22 2017 17:34 UTC
use nvm
Nabrok
@Nabrok
Mar 22 2017 17:34 UTC
Yeah
Archomeda
@Archomeda
Mar 22 2017 17:34 UTC
i dislike using the default repos actually for this
the release process in both ubuntu and debian just takes too long
and debian release process is just horribly outdated imo
but that's just my 2 cents
@Darqam discord uses special formatting for linking channels, users and roles:
  • channels: <#...>
  • users: <@...>
  • roles: <@&...>
tbf, there's room for linking messages really, like: <!...>
Darqam
@Darqam
Mar 22 2017 17:37 UTC
yeah...
probably a planned addition that hasn't hit yet
Archomeda
@Archomeda
Mar 22 2017 17:38 UTC
yeah, just like the channel categories we've been waiting on for 3 months now :(
except that one has been announced
Darqam
@Darqam
Mar 22 2017 17:39 UTC
purge channel of messages is also awaited by a few I think
Edgar Doiron
@Coffee4cr
Mar 22 2017 17:40 UTC
those two i'm waiting for them
categories more than mass deletes
Darqam
@Darqam
Mar 22 2017 17:40 UTC
https://api.guildwars2.com/v2/items/48762 lol why. Everything else related to wvw season 1 is called WvW Season One ....
Edgar Doiron
@Coffee4cr
Mar 22 2017 17:40 UTC
the bot I have right now let's you archive the pass X messages and outputs you a json. but I'd like the same for delete
Archomeda
@Archomeda
Mar 22 2017 17:40 UTC
i also want the ability to modify the first text channel... you can't delete it or change the "read messages" permission because it's the default text channel >_>
Eearslya Sleiarion
@Eearslya
Mar 22 2017 17:42 UTC
I know a few bots that offer purging on a few criteria
time, user, media
Darqam
@Darqam
Mar 22 2017 17:43 UTC
the issue is, atm a bot can only delete 100 messages at a time, and if you start spamming deletes for a whole channel, it's api abuse and bot might get banhammered
Archomeda
@Archomeda
Mar 22 2017 17:45 UTC
yeah, it's a long process i suppose
also, hammerandchisel/discord-api-docs#208
Darqam
@Darqam
Mar 22 2017 17:48 UTC
oh right, that too
Archomeda
@Archomeda
Mar 22 2017 17:48 UTC
yeah, there was that issue in the past where it caused too much fragmentation when people were mass deleting messages :P
Edgar Doiron
@Coffee4cr
Mar 22 2017 18:20 UTC
I do love their blog post
Anyone happen to know if the ids are the same between /v2/commerce/transaction/history and /v2/commerce/transaction/current?
Checking on my local server now; I can't make heads or tails of T-SQL
no they are not lolol.
Eearslya Sleiarion
@Eearslya
Mar 22 2017 19:01 UTC
rip
man I really want to write gw2api.com
maybe I can take a few days off work or something smh
doing order reconciliation would be trivial with postgresql
maybe I should just secretly install postgresql on the api cluster
and write libpq bindings for our javascript contexts
darthmaim
@darthmaim
Mar 22 2017 19:17 UTC
:P
i'm like
only half joking though
there's a lot of low-hanging fruit; e.g. getting per-build per-resource histories as jsondiffs
Eearslya Sleiarion
@Eearslya
Mar 22 2017 19:18 UTC
if you make it open source, we can all write it
yeah and people could submit pull requests and stuff
it'd basically be a pile of go+postgresql
David Reeß
@queicherius
Mar 22 2017 19:37 UTC
@lye quick question, last changelog you marked /v2/characters/:id/dungeons and /v2/guild/:id/storage as impl. (what does that mean?) and the api says API not active. That supposed to happen?
ChieftainAlex
@ChieftainAlex
Mar 22 2017 19:42 UTC
is http 206 partial content a normal response header from the api or have I messed up?
darthmaim
@darthmaim
Mar 22 2017 19:42 UTC
thats normal
ChieftainAlex
@ChieftainAlex
Mar 22 2017 19:44 UTC
and did the api always tell you which ids got no response from a bulk expanded query?
darthmaim
@darthmaim
Mar 22 2017 19:45 UTC
huh, where?
oh, now I see
ChieftainAlex
@ChieftainAlex
Mar 22 2017 19:46 UTC
I've never really had a look at the headers response before, and I noticed 'Access-Control-Expose-Headers', which returned "X-Result-Total,X-Result-Count,Warning"
and within warning there's a list of ids which failed
darthmaim
@darthmaim
Mar 22 2017 19:47 UTC
never seen that before
darthmaim
@darthmaim
Mar 22 2017 19:48 UTC
I knew about 206, but have never seen the Warning header
ChieftainAlex
@ChieftainAlex
Mar 22 2017 19:50 UTC
essentially I've written some dumb code which attempts to get tp prices for some soulbound stuff
Eearslya Sleiarion
@Eearslya
Mar 22 2017 19:50 UTC
@queicherius think he meant 'implemented'; i.e., he's got a format ready, but maybe not the backend support yet?
David Reeß
@queicherius
Mar 22 2017 19:51 UTC
I thought that was requires backend support (as the other ones in the changelog)
@queicherius ah by "impl" I meant "requires backend support"
the frontend implementation is live but the backend goes through a much longer release process
@ChieftainAlex 206 means that some ids provided (via the ?ids parameter) were invalid; there's a Warning header containing the list of ids that were invalid
I don't think ?page can return 206, but might be wrong
@queicherius you sure you're not getting 429's anywhere? I'm not seeing any perfs for gw2e requests and am a bit concerned
oh derp I'm looking at DFW, all your requests are to FRA. derp derp.
David Reeß
@queicherius
Mar 22 2017 20:53 UTC
Ah okay, thought the impl might be different
Looks pretty normal on my end :) http://i.imgur.com/ngSZ9AW.png
damn that's fancy af
download (1).png
I think my metrics might be broken
i'm not sure how gw2e is above the FRA line
David Reeß
@queicherius
Mar 22 2017 20:54 UTC
o.o
why is FRA handling so much more requests than DFW?
:shrug:
I think there are more apps in the EU
oh I got FRA/DFW mixed up
@queicherius also NA/EU WvW isn't high right now
That's 3k reqs/minute right there
David Reeß
@queicherius
Mar 22 2017 20:55 UTC
:D
okay so what are you using to make that really pretty UI
Is that a lib or is it entirely bespoke
David Reeß
@queicherius
Mar 22 2017 20:59 UTC
its a bunch of npm dependencies (sparkly + cli-table, mainly) and some sketchy job statistic logic with redis hash fields and lists
but it works better than the job statistics that were natively in the package I used, which tended to lock up redis completely with 1 key per completed job /shrug
Eearslya Sleiarion
@Eearslya
Mar 22 2017 21:06 UTC
oh man that's faaancy
darthmaim
@darthmaim
Mar 22 2017 21:08 UTC
I really have to add logging for all my API requests
David Reeß
@queicherius
Mar 22 2017 21:12 UTC
@lye Do you track only the last 2 hours or is there something like a 48 hour chart too? Am curious to see if I can spread that load better, or if it is as even as that graph suggests
gimme a sec
download (1).png
FWIW I'm actually not that concerned about it, since it's not like you're making needless requests
once I get a combined /v2/account I imagine that'll go down by quite a lot
David Reeß
@queicherius
Mar 22 2017 21:14 UTC
That spike when I turned it back on XD
Eearslya Sleiarion
@Eearslya
Mar 22 2017 21:14 UTC
4AM?
those times are PST I think
David Reeß
@queicherius
Mar 22 2017 21:14 UTC
Gonna ask for that graph again in 2 days or so :>
I should make /v2/graphs
David Reeß
@queicherius
Mar 22 2017 21:15 UTC
Yeah, 11 AM GMT
actually, 12 AM
Darqam
@Darqam
Mar 22 2017 21:15 UTC
/v2/graphs please!
Eearslya Sleiarion
@Eearslya
Mar 22 2017 21:16 UTC
Pls do
Darqam
@Darqam
Mar 22 2017 21:16 UTC
also, that steady increase on frankfurt doesn't seem to be gw2e only
Eearslya Sleiarion
@Eearslya
Mar 22 2017 21:16 UTC
No, that's the US waking up and realizing sites work again
@Eearslya except the US would be hitting the DFW DC, not FRA :P
Eearslya Sleiarion
@Eearslya
Mar 22 2017 21:17 UTC
Maybe the US all VPN'd to Europe
Darqam
@Darqam
Mar 22 2017 21:17 UTC
so... coders in europe going to sleep and starting their automatic api crawling then :p
darthmaim
@darthmaim
Mar 22 2017 21:19 UTC
Or US using EU servers
Eearslya Sleiarion
@Eearslya
Mar 22 2017 21:21 UTC
@lye Well, wrapping the whole thing in a Item#transaction brought it from 12 minutes to..8. So that helps.
yay
removing the extra transactions from ActiveRecord::Base#save would probably buy you another 4 minutes
Eearslya Sleiarion
@Eearslya
Mar 22 2017 21:21 UTC
11 items marked as invalid..because of blank names. Dangit.
I don't recall if there's a way to do that though
Eearslya Sleiarion
@Eearslya
Mar 22 2017 21:22 UTC
Yeah, the problem is I have it set to validate_uniqueness_of :id, so every time it inserts, it does a SELECT first
David Reeß
@queicherius
Mar 22 2017 21:23 UTC
I would love to see the client requests of gw2e
queicherius @queicherius may track that with google analytics
Eearslya Sleiarion
@Eearslya
Mar 22 2017 21:23 UTC
Can you adjust that graph to show data points based on user-agent?
you're gonna have to, only sane way for me to get the metrics is with oauth2 :cry:
@Eearslya U-A isn't tracked on my end
Eearslya Sleiarion
@Eearslya
Mar 22 2017 21:24 UTC
Curses
darthmaim
@darthmaim
Mar 22 2017 21:24 UTC
And you can't set ajax user agents
Eearslya Sleiarion
@Eearslya
Mar 22 2017 21:24 UTC
Oh, really? Whoops.
darthmaim
@darthmaim
Mar 22 2017 21:25 UTC
And other headers will trigger a preflight request
Maybe tracking Origin would work
Or a custom query parameter
Eearslya Sleiarion
@Eearslya
Mar 22 2017 21:26 UTC
/v2/items/12542?origin=gw2e
darthmaim
@darthmaim
Mar 22 2017 21:26 UTC
Yeah
eh tracking it that way is kind of flaky
darthmaim
@darthmaim
Mar 22 2017 21:27 UTC
Like the ?wiki=1 the wiki adds
I don't really mind not having per-client request metrics
they'd be interesting to look at but not really actionable :/
David Reeß
@queicherius
Mar 22 2017 21:45 UTC
looking at options for api tracking since GA limit seems a little low... some start with plans of 750€/month - seems reasonable
wait GA costs money?
I don't ever use GA so ... I thought they were paid with user data :/
David Reeß
@queicherius
Mar 22 2017 21:50 UTC
it costs money for > 10m data points / month, if I understand their terms correctly
hahaha what is one to do with such puny numbers
Archomeda
@Archomeda
Mar 22 2017 21:58 UTC
ohhh, those graphs
gw2e using 5k req/min wow
the important part is that there's good reason for it
Michael Dougall
@madou
Mar 22 2017 22:00 UTC
:angel:
Archomeda
@Archomeda
Mar 22 2017 22:00 UTC
is it? ;P
:angel:
Darrian
@rikkuness
Mar 22 2017 22:19 UTC
elasticsearch all the things ;D
ChieftainAlex
@ChieftainAlex
Mar 22 2017 23:21 UTC
sorry to revisit the above, is generating 206's problematic?
darthmaim
@darthmaim
Mar 22 2017 23:21 UTC
no
ChieftainAlex
@ChieftainAlex
Mar 22 2017 23:22 UTC
k good I can forget it
e.g. the last page of a table