These are chat archives for arenanet/api-cdi

31st
Jan 2018
derRAV3N
@derRAV3N
Jan 31 2018 10:51
What are your tactics for storing stuff in the db versus requesting it live from the api? do you have some sort of cache for everything and just refresh it every few minutes from api or are you working with the api and just store extra information in an own db?
I'm currently working on a raid planning tool, not sure if I should just cache characters and their progress, raids and wings and bosses and so on...
For account/characters it shouldn't be to hard to just request stuff live, but looking for their elite spec progress every request isn't that nice, is it?
Archomeda
@Archomeda
Jan 31 2018 13:01
it depends how many times you need that information and for what
there's also the case of delay whenever you request it live
derRAV3N
@derRAV3N
Jan 31 2018 13:56
It will be a website which every group/guild has to host themselves. I think I'll need info about characters nearly every pageload to show core infos and for example elite-specs
Archomeda
@Archomeda
Jan 31 2018 13:59
hmm, if it's going to be on that scale, it's definitely worth it to cache data to speed things up
the api has 5 minute caching generally anyway, so it's useless to request it more than once in 5 minutes
(depending on the endpoint of course)
for example, compare requesting 10 different accounts live to retrieving cached data about 10 different accounts from the same database
the latter one is significantly faster
derRAV3N
@derRAV3N
Jan 31 2018 14:24
yea
I will try to work with the api for now and look how it will evolve, but with caching in mind, so I won't run into trouble if I want to cache afterwards
derRAV3N
@derRAV3N
Jan 31 2018 14:50
I imagine the things that'll be a bit hard to deal with is elite-specs. I somehow just can't get a clear thought about how to integrate them clean into my structure
ChieftainAlex
@ChieftainAlex
Jan 31 2018 18:33
Does anyone happen to have an old copy of the output from the v2/continents endpoint? thinking about hardcoding the Path of Fire map markers until they get fixed.
ChieftainAlex
@ChieftainAlex
Jan 31 2018 19:45
excellent idea, had no idea wayback archive would also crawl son
json*
windwarrior
@windwarrior
Jan 31 2018 21:23
I have contemplated crawling /v2/items every build
since you can basically put it as a single JSON file in the order of tens of MBs
that's the recommended approach tbh
or the approach I recommended in the past
windwarrior
@windwarrior
Jan 31 2018 22:33
one problem @lye, /v2/items/ is changing between builds due to whitelisting, no?
the list of ids might
but the individual item definitions can change between builds
and there's no way to tell which items have changed, so you need to re-scrape 'em
the list of ids changes between builds
typically
but when /v2/build is incremented you have to invalidate your entire cache if you want to pick up minor changes
(e.g. someone inserts a linebreak in the german translation of a given rune's description)
Quaggan BooOOoot
@QuagganBooOOoot
Jan 31 2018 22:36

Comment to Update about API Communications
by (ArenaNet) MikeLewis.7496 in API Development

Hey all,

I wanted to check in as well and give you all a bit of insight into what's going on with the APIs right now. Our first priority is triage; as Ylan mentioned, the two of us will directly be watching GitHub and this forum for reports and feedback on what's most important to you. Once we have established some general ordering for what will be tackled, we're working on finding multiple folks from the GW2 engineering team who can contribute to keeping the APIs up to date and solid. As several people have identified already, having the entire API project tasked to one person is suboptimal in a lot of ways. So we want to address that specifically going forward.

I'm sure you all can appreciate how huge a game like GW2 is and how much effort goes into keeping it living and fresh; so hopefully it isn't a surprise when I say that we have a lot going on and a lot of competing demands on the team's time and attention. While the APIs are something we are very much interested in maintaining and supporting for the future, we do appreciate your understanding that sometimes other things are higher priorities, and so we are trying very hard not to promise things right now. That said, as things do solidify and become clearer, we will definitely be keeping the community up to date.

One loosely related thing I would love some community feedback on - how do you all feel about GitHub and specifically how we've been using it up until now? Is there anything that you would like to see changed in terms of how we collect API requests and track the associated development work?

Werdes
@werdes
Jan 31 2018 22:36
:clap:
now somebody kidnap him and drag him here :smiling_imp:
ChieftainAlex
@ChieftainAlex
Jan 31 2018 22:38
someone already offered to kidnap lawton back to the API earlier by email.. wasn't sure they were joking
windwarrior
@windwarrior
Jan 31 2018 22:38
basically, you have to always query /v2/build, if that is a new number you can delete your cache. You also always have to query /v2/items for a list of IDs due to new items getting whitelisted. You can however cache responses for particular items between builds. Is that right @lye?
please don't kidnap me
Werdes
@werdes
Jan 31 2018 22:39
well you could crawl the full id list from /v2/items once the build changes and after that unknown ids on demand
@windwarrior yeah that's the best approach I think
windwarrior
@windwarrior
Jan 31 2018 22:40
my primary interest is often search, @werdes
so that requires a full scrape ^^
Werdes
@werdes
Jan 31 2018 22:40
ah well, then my approach is probably not that ideal :sweat_smile:
Eearslya Sleiarion
@Eearslya
Jan 31 2018 22:43
I think I'm the only one here who knows what @lye looks like :smiling_imp:
Werdes
@werdes
Jan 31 2018 22:44
good so we have our kidnapper :smiling_imp:
windwarrior
@windwarrior
Jan 31 2018 22:45
you are also in the not-other-side-of-the-world-vicinity right @Eearslya
Archomeda
@Archomeda
Jan 31 2018 22:45
kitnap both people from that forum thread and get them here :P
Eearslya Sleiarion
@Eearslya
Jan 31 2018 22:45
that's true too
windwarrior
@windwarrior
Jan 31 2018 22:46
hmm there are 146 reddit accounts of anet people
how many reddit API calls am I even allowed to make
Werdes
@werdes
Jan 31 2018 22:47
60/m iirc
ChieftainAlex
@ChieftainAlex
Jan 31 2018 22:47
I know what you're thinking, but 146 is too many to work on the API.
windwarrior
@windwarrior
Jan 31 2018 22:47
well my dev tracker still works
so its not hitting that rate limit
oh the rate limit is only for OAuth reqs
Werdes
@werdes
Jan 31 2018 22:49
are the .json files (like /r/guildwars2.json) part of that? seems like they arent
fair
Mike Lewis
@apoch
Jan 31 2018 22:56
I'm already lurking here btw ;-)
windwarrior
@windwarrior
Jan 31 2018 22:57
welcome!
Werdes
@werdes
Jan 31 2018 22:58
somebody stop @Eearslya then D:
Eearslya Sleiarion
@Eearslya
Jan 31 2018 22:58
the wheels are in motion, the plan cannot be stopped
windwarrior
@windwarrior
Jan 31 2018 22:59
GH as a platform is fine by me. I think almost all developers have a GH account for posting issues
Werdes
@werdes
Jan 31 2018 23:00
yup, this gitter chat is quite ideal imho
windwarrior
@windwarrior
Jan 31 2018 23:01
it can get, and thats also my fault, get a bit too chatty
I am just responding to the post now, what is great about this chat is the transparency in communications. Bug reports are no longer thrown over the fence without real acknowledgements and the turnaround of simple tasks like "plz whitelist" was super low.
It also serves as a bit of a hybrid, on one side its a place to discuss the api's development, but also the api's use
Werdes
@werdes
Jan 31 2018 23:04
And we are nicely integrated into the github issue tracking
windwarrior
@windwarrior
Jan 31 2018 23:06
The way GH documents the API is also great
and the process of "pull requesting new endpoints" allows for a bit of involvement of us as well as making it clear when a feature lands
David Reeß
@queicherius
Jan 31 2018 23:11
@apoch :wave:
windwarrior
@windwarrior
Jan 31 2018 23:11
In general, it was great that lye was able to develop the API quite in the open, I think that makes it easier for folks like @queicherius to predict when new endpoints land and implement support beforehand.
David Reeß
@queicherius
Jan 31 2018 23:12
I personally think the Github is a great place for developers as it is used right now, maybe a bit more labeling/cleanup would be nice. Other than that I think you will get a lot more developer input there than on the forums (i might be biased here, I go to the forums once in a blue moon)
And yes, what @windwarrior said is definitely a bonus, sometimes lye announced something was coming, so I could already pre-write the new pages with the proposed format while waiting for release and then when it came out it was less stressful to push out to the public
David Reeß
@queicherius
Jan 31 2018 23:17
This Gitter also has a pretty important function in my opinion, to have a close relationship with the API dev(s). I communicate(d) a lot with @lye when I saw issues popping up, also in private with lists of API keys which can't just be posted in public. And it is just cool to talk with devs! :)
windwarrior
@windwarrior
Jan 31 2018 23:17
Its very cool to talk to gods devs
we've had some constructive discussions here for sure
ChieftainAlex
@ChieftainAlex
Jan 31 2018 23:20
fast responses from both parties when discussing and trying to clarify issues is always good
Mike Lewis
@apoch
Jan 31 2018 23:20
I intend to be here when I can; I may be in lurk mode more often than not but I certainly want to continue supporting this chat as a direct channel
David Reeß
@queicherius
Jan 31 2018 23:20
Yeah, definitely
Werdes
@werdes
Jan 31 2018 23:22
:thumbsup:
David Reeß
@queicherius
Jan 31 2018 23:22
@apoch Very glad to hear that! If it's too much to read because we talk about other stuff, it's sometimes easier to skim for the highlighted mentions, we got used to using that for everything important :)