These are chat archives for Automattic/mongoose

14th
May 2016
LeonineKing1199
@LeonineKing1199
May 14 2016 01:19

You actually get pretty good performance if you eschew Mongoose in this one instance and try to implement a raw MongoDB call itself. Basically, if I have a schema called Profile and I want to insert a large number of document simultaneously, I'll just use (if I'm recalling correctly):

Profile.collection.insert(docs)

where docs is an array of N documents that I want to insert into the collection.

Mongoose doesn't have much info on this but luckily, the internet does.

Unless you mean you're inserting just one document but it's very large in which case, I don't think there's anything else you can do but get better hardware.

Did you know, it's cheaper to buy 25% faster hardware than it is to write 25% faster software?

Paul "Joey" Clark
@joeytwiddle
May 14 2016 04:55
@vladotesanovic I have been using bulk insert but you should wait for the execute to complete (.execute().then(...)) otherwise your memory may fill up with queued batches! http://stackoverflow.com/questions/25285232/bulk-upsert-in-mongodb-using-mongoose
(I actually run 4 of these operations in parallel, so Node is never waiting for Mongo and Mongo is never waiting for Node.)
Vlado Tesanovic
@vladotesanovic
May 14 2016 07:56
@joeytwiddle This bulk thing can be interesting, i will tested that. Problem with mongoose ( InsertMany for example ) is validation, on each iteration mongoose construct object from json object, and fill up memory
@LeonineKing1199 yea, i am adding more hardware power on server.
@joeytwiddle this Bulk is something similar to transaction, i don't know how i didn't saw this before