These are chat archives for arenanet/api-cdi

9th
Apr 2017
Archomeda
@Archomeda
Apr 09 2017 00:04
running 3 full search instances simultaneously, we'll see in like an hour :laughing:
JC123
@JC123
Apr 09 2017 00:28
i'm not even sure if it's actually stored in the dat, maybe it's just sent by the server
it is, but yes very well hidden
Archomeda
@Archomeda
Apr 09 2017 00:43
i can imagine
Archomeda
@Archomeda
Apr 09 2017 13:31
I came across something odd yesterday. Apparently I have 2 of the same files in my .dat. But they are different in format. One uses 64-bit pointers and the other 32-bit. Now because of that I also solved that long mystery of mine of why some stuff had 4 byte padding... They were 64-bit pointers, urgh.
One of the files might be an older file that somehow wasn't deleted :/
Archomeda
@Archomeda
Apr 09 2017 14:01
Also, another post about gw2timer being difficult to interpret https://www.reddit.com/r/guildwars2/comments/64cnww
darthmaim
@darthmaim
Apr 09 2017 14:08
ahrg, why do I have so many other projects already
It would be so easy to make a clean version of that page
JC123
@JC123
Apr 09 2017 14:20
I came across something odd yesterday. Apparently I have 2 of the same files in my .dat. But they are different in format. One uses 64-bit pointers and the other 32-bit. Now because of that I also solved that long mystery of mine of why some stuff had 4 byte padding... They were 64-bit pointers, urgh.
what file?
Archomeda
@Archomeda
Apr 09 2017 14:20
705798 and 1282850
tbh, i only saw similar stuff like same strings and such; haven't done a full check on those files
JC123
@JC123
Apr 09 2017 14:23
ironically those are what you were looking for lol
Archomeda
@Archomeda
Apr 09 2017 14:23
hm?
these files turned out to contain an integer i was searching the entire .dat for
(among some other files)
JC123
@JC123
Apr 09 2017 14:25
yeah, cntc is PackContent aka "client content"
they contain everything the api does and a lot more
32 of them are for the x86 client and 32 are for the x64 client
Archomeda
@Archomeda
Apr 09 2017 14:26
wait, the .dat contains files for both architectures..?
that kind of makes sense, but at the same time doesn't
JC123
@JC123
Apr 09 2017 14:26
only for those files
well and probably some launcher stuff
eg x86 and x64 versions of the coherent stuff
Archomeda
@Archomeda
Apr 09 2017 14:27
i mean, there's the ClientContent86 and ClientContent64 in the manifest; so i suppose they are categorized under that
JC123
@JC123
Apr 09 2017 14:27
yes
Archomeda
@Archomeda
Apr 09 2017 14:28
oh, coherent, yeah fair
if i only had all the stuff i've ever done regarding the .dat and cdn in one place, and not scattered around, i would work more efficiently :P
Archomeda
@Archomeda
Apr 09 2017 15:09
okay cool, i can probably confirm i've found what i was looking for
with two items
yay me
darthmaim
@darthmaim
Apr 09 2017 15:09
:tada:
Archomeda
@Archomeda
Apr 09 2017 15:10
sad part: i haven't figured out yet how to get that data without searching the entire files for a string id
JC123
@JC123
Apr 09 2017 15:14
what are you looking for
Archomeda
@Archomeda
Apr 09 2017 15:21
oh this? just a way to find the relation between string ids and item ids
JC123
@JC123
Apr 09 2017 15:26
youd have parse the cntcs apart from just searching the string id
Archomeda
@Archomeda
Apr 09 2017 15:27
yeah, i have the structure of the cntc files; now i just have to find the references
JC123
@JC123
Apr 09 2017 15:28
sadly you can only easily dump the struct of the pf
Archomeda
@Archomeda
Apr 09 2017 15:28
true
rest is just bytes :P
JC123
@JC123
Apr 09 2017 15:28
then they contain like 60+ different content types that all have their own structs
Archomeda
@Archomeda
Apr 09 2017 15:29
that's a lot haha
JC123
@JC123
Apr 09 2017 15:32
havent looked in a while but used to be something like that
Archomeda
@Archomeda
Apr 09 2017 15:39
wow okay :O
JC123
@JC123
Apr 09 2017 15:41
gw2db just figured out the struct for itemdef at release and dumped it for their data before the api
Archomeda
@Archomeda
Apr 09 2017 16:43
right, there was that too
i'm glad the api exists nowadays :)
windwarrior
@windwarrior
Apr 09 2017 17:48
The passwords: would be wise to encrypt files that are not to be datamined
maybe thats why they are there
Archomeda
@Archomeda
Apr 09 2017 17:49
yes of course :P
JC123
@JC123
Apr 09 2017 18:21
the only thing they really encrypt is some of the strings lol
JC123
@JC123
Apr 09 2017 18:28
and also hashing on the cntc entry names which is why they look like wZY+N.PVPAc etc
windwarrior
@windwarrior
Apr 09 2017 18:34
why is javascript development so annoyingly complex nowadays
yeah to write 'fun-to-write' javascript you need a transpiler, 1700 frameworks, a library to add cats and dogs
I just want code-in-a-file
a framework that includes chalk just so it can put funky colors in your terminal
where did this go wrong JavaScript
Archomeda
@Archomeda
Apr 09 2017 18:43
oh, that looks fun
windwarrior
@windwarrior
Apr 09 2017 18:43
new react+redux project
743 packages in node_modules
really
Archomeda
@Archomeda
Apr 09 2017 18:43
lol
windwarrior
@windwarrior
Apr 09 2017 18:43
who writes all of this
darthmaim
@darthmaim
Apr 09 2017 18:44
sindresorhus :P
well I really like having a module for everything
windwarrior
@windwarrior
Apr 09 2017 18:46
oh yeah sure, but 743 of them
in a basically empty project
Darqam
@Darqam
Apr 09 2017 18:47
what kind of empty project are you making?
darthmaim
@darthmaim
Apr 09 2017 18:47
most of those are probably build tooling and not in the final bundle
windwarrior
@windwarrior
Apr 09 2017 18:48
meh wanted to explore a client side caching infrastructure for the GW2 api
so try to be smart about making calls to the api, storing stuff in session/localstore etc
so that I get niceties as searching in the api
without actually needing server/index
nor overloading the API
Archomeda
@Archomeda
Apr 09 2017 18:51
sounds cool
windwarrior
@windwarrior
Apr 09 2017 18:51
you have about 2.5MB (mobile)/5MB (desktop) localstore
so you can at least on desktop cache all responses from the GW2 api
uhh item endpoint*
darthmaim
@darthmaim
Apr 09 2017 18:52
use indexeddb (50% of hdd space)
and you can index stuff for faster lookups
David Reeß
@queicherius
Apr 09 2017 18:54
in my experiments indexeddb was always incredibly slow
to the point of 1/200 of the speed of localstorage
windwarrior
@windwarrior
Apr 09 2017 18:56
it doesnt seem to have as strict quotas as LS tho
darthmaim
@darthmaim
Apr 09 2017 18:56
well, reading a simple file is also faster than querying a db
windwarrior
@windwarrior
Apr 09 2017 18:56
but a very agressive eviction policy
like, if it needs space, it will evict entire origins
cant find if thats only the temporary stuff, or also the stuff you intend to "persist"
darthmaim
@darthmaim
Apr 09 2017 18:57
persisting will ask the user and will not be evitcted
David Reeß
@queicherius
Apr 09 2017 18:58
that's true, I mostly just need a key/value store, not a full blown DB
darthmaim
@darthmaim
Apr 09 2017 18:59
thats what localStorage is for
David Reeß
@queicherius
Apr 09 2017 18:59
with more storage than local storage :D
(and async)
windwarrior
@windwarrior
Apr 09 2017 18:59
its annoying that you cant ask for more localstorage
Hey User, I want 10Mb pls
darthmaim
@darthmaim
Apr 09 2017 19:00
idb can do that
maybe stuff like navigator.webkitPersistentStorage.requestQuota() even works for localStorage
never tried it
Archomeda
@Archomeda
Apr 09 2017 19:03
doesn't the site mega.nz basically download stuff to localstorage and then showing a popup to the user for downloading it to your hdd?
i do think that chrome asks the user that the site asks for a lot of storage
darthmaim
@darthmaim
Apr 09 2017 19:03
I think they use window.webkitRequestFileSystem()
which gives you a virtual file system where you can create files and stuff
ChieftainAlex
@ChieftainAlex
Apr 09 2017 19:04
@lye - yet another map bug sorry. https://api.guildwars2.com/v2/continents/2/floors?ids=2 returns "all ids are invalid". https://api.guildwars2.com/v2/maps?ids=1163 (capricorn) suggests it can be found on floor 2.
darthmaim
@darthmaim
Apr 09 2017 19:04
this will ask the user for storage
windwarrior
@windwarrior
Apr 09 2017 19:04
MDN is pretty negative about that one tho
darthmaim
@darthmaim
Apr 09 2017 19:05
yep, chrome only afaik so far
windwarrior
@windwarrior
Apr 09 2017 19:05
do not use it, its not a standard and doesnt appear to become one
darthmaim
@darthmaim
Apr 09 2017 19:06
yup
Archomeda
@Archomeda
Apr 09 2017 19:08
:/
darthmaim
@darthmaim
Apr 09 2017 19:09
there is also the some quota api being developed
not sure if you can request more with that
or if it just returns how much you can store
windwarrior
@windwarrior
Apr 09 2017 19:13
yeah now you just get a "full"
the react shopping list example has as examples
Oculus, WhatsApp and Instagram
oh Facebook...
darthmaim
@darthmaim
Apr 09 2017 19:18
well, if you need more storage just use idb
windwarrior
@windwarrior
Apr 09 2017 19:19
yeah first learning a new JS stack
so will take a while :P
darthmaim
@darthmaim
Apr 09 2017 19:19
:P have fun
react+redux is pretty cool
windwarrior
@windwarrior
Apr 09 2017 19:19
it appears to be so
darthmaim
@darthmaim
Apr 09 2017 19:20
redux took me a while, but once you figure it out its really cool
windwarrior
@windwarrior
Apr 09 2017 19:21
its basically a Haskell store type idea
which I really like as an idea
have an immutable store, and mutate it with a list of actions
ChieftainAlex
@ChieftainAlex
Apr 09 2017 19:31
fucking leaflet.
Untitled.jpg
always remember to add the leaflet css.
Archomeda
@Archomeda
Apr 09 2017 19:33
well okay steam.... just installing torchlight 2 crashes steam when it's finished installing...
darthmaim
@darthmaim
Apr 09 2017 19:34
steam wants you to work on our build editor
(not giving up :P)
Archomeda
@Archomeda
Apr 09 2017 19:34
rofl
i just want to play with a friend of mine :sob:
darthmaim
@darthmaim
Apr 09 2017 19:35
^^
Archomeda
@Archomeda
Apr 09 2017 19:38
well, found the problem
steam beta
darthmaim
@darthmaim
Apr 09 2017 19:42
I had the same last weekend, steam was crashing right after starting, had to start it in offline mode and disable the steam beta
ChieftainAlex
@ChieftainAlex
Apr 09 2017 20:44
https://wiki.guildwars2.com/index.php?title=User:Chieftain_Alex/sandbox6 worst map widget I ever wrote, but nice for exploring dungeon floors.
**browser induced siezures may vary
windwarrior
@windwarrior
Apr 09 2017 21:08
you might wanna add some spinner
"I am still working on something.. don't go yet"
because I anticipated it being broken already :')
windwarrior
@windwarrior
Apr 09 2017 21:28
I am liking this react/redux thing so far
pretty slick
darthmaim
@darthmaim
Apr 09 2017 21:29
:)
windwarrior
@windwarrior
Apr 09 2017 21:34
meh trying to find a localstorage backend thing, but everything is unmaintained :(
Archomeda
@Archomeda
Apr 09 2017 21:34
write your own :smile:
darthmaim
@darthmaim
Apr 09 2017 21:35
what exactly do you want to do?
windwarrior
@windwarrior
Apr 09 2017 21:36
not sure yet, I think I actually need something smarter because I want to make decisions about what API responses to retain when storage quotas are exceeded
like its pointless to (long term) retain responses from /v2/commerce because they are invalid when the app gets reloaded
but it is useful to keep /v2/items because that is only changing when a build changes
(still the API-cache thing)
darthmaim
@darthmaim
Apr 09 2017 21:38
I blacklist commerce, wvw/matches, characters, account and everything with access_token
windwarrior
@windwarrior
Apr 09 2017 21:39
well I am trying to make an app framework (and explorer as demo app) that abstracts away from the api in such a way that it caches and lets me use api responses as if they were local data
or planning to make
all clientside, because maintaining servers is effort
darthmaim
@darthmaim
Apr 09 2017 21:42
probably less effort than handling the api data clientside :P
windwarrior
@windwarrior
Apr 09 2017 21:43
oh certainly
but thats not the challenge™
darthmaim
@darthmaim
Apr 09 2017 21:44
:D