;
npm test
... i mean the files are there
echo *.c ???bar
cat {src,tests,.}/**/*.js
**
a thing?
**
/ globstar for recursive directories, dont know if I picked that up wrong
/v2/gliders
a bunch D:
/v2/build
gives me the error but /v2/gliders
doesn't
gw2@naga:~$ for run in {1..120}; do curl --write-out %{http_code} --silent --output /dev/null https://api.guildwars2.com/v2/build; done
200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200200
throw 'ratelimit'
to the ratelimit code
/v2/gliders
is the only endpoint i've found that doesn't hit it
/v2/items
right? that was the biggest culprit before
/v2/items
, tested that locally
router.get
bit
/v2/gliders
(which uses the same routing bits as everything else in the same one file) doens't execute it
throw
statement in the ratelimit code just to sanity check
/v2/gremlins
and return {text: "everywhere"}
/v2/gliders
/v2/gliders
is now magically rate limited
/v2/items
but the request was too big, loading too slow to hit limit, so I just went with a smaller endpoint. /v2/build
never occurred to me XD
/v2/items
...
/v2/items
before I could type the next command
api.guildwars2.com/graphs
let's go
Got 55847 items
Performed ItemCrawlerJob from Async(default) in 37316.83ms
:tada:
/v2/items
you need 280 requests
Language | Renamed Items |
---|---|
de | 39269 |
en | 256 |
es | 12738 |
fr | 9474 |
BEGIN;
SELECT * FROM items WHERE id = 93;
INSERT INTO items VALUES(...);
SELECT * FROM items WHERE id = 94;
INSERT INTO items VALUES(...);
COMMIT;
ActiveRecord::Base#transaction
which accepts a block to execute for the transaction
def scrape()
Item.transaction do
actually_scrape()
end
end
ActiveRecord::Base#transaction
it's automatically COMMIT
'd at the end of the block if no exception is thrown; if an exception is thrown the transaction is ROLLBACK
d.
Anyway having it all in a single transaction lets you do stuff like
def scrape()
Item.transaction do
delete_all_items()
refetch_all_items()
end
end
provided that refetch_all_items
throws if an unrecoverable error occurs during the fetch, that will atomically update all item records in your db with the latest build
ActiveRecord::Base#save
uses a transaction internally -- it means that you'll have one big transaction rather than 70k little ones)
/v2/currencies?lang=ko
ko
listed as a valid language
Possible locale 'langId' values include:
en
es
de
fr
ko
zh
ko
and zh
are practically useless in our use cases :P
ko
just returns ErrUnknown
ko
doesn't even return valid stuff
zh
actually returns data
zh
was using the chinese translations for things
zh
is yeah, but pretty sure i came across things in the past that just showed the english translation
all en name changes since 2015-10-20 22:30:00
WvW Season One ....
/v2/commerce/transaction/history
and /v2/commerce/transaction/current
?
impl.
(what does that mean?) and the api says API not active
. That supposed to happen?
Warning
header
requires backend support
(as the other ones in the changelog)
Warning
header containing the list of ids that were invalid
?page
can return 206, but might be wrong
impl
might be different
/v2/account
I imagine that'll go down by quite a lot
/v2/graphs
/v2/graphs
please!
Item#transaction
brought it from 12 minutes to..8. So that helps.
ActiveRecord::Base#save
would probably buy you another 4 minutes
validate_uniqueness_of :id
, so every time it inserts, it does a SELECT
first
save!
apparently: http://api.rubyonrails.org/classes/ActiveRecord/Transactions/ClassMethods.html
/v2/items/12542?origin=gw2e