These are chat archives for Automattic/mongoose

15th
Sep 2018
ratnesh2581990
@ratnesh2581990
Sep 15 2018 07:41
@lineus thanks for the answer it will solve my problem what I have pasted above but can you help in another problem
Now I am trying to fetch all the order of a particular user By userid which I have made Schema.Types.ObjectId
userid: {
        type: Schema.Types.ObjectId, ref: 'User',
        required: true
    }
but when i tried to run this code it will give me empty orders array here is my code
 Order.find({userid: req.query.userid}, function (err, orders) {
            if (err) {
                res.json({ 'success': false, 'message': err });
            } else {
                res.json({ 'success': true, 'orders': orders });
            }
        }).sort({orderdate: 'desc'});
but when i run same command in my robomongo command line with userid it will return the array of orders by userid so please tell me why this is happening
mirik999
@mirik999
Sep 15 2018 10:10
Hi @lineus , how to create a short variant -> find all where region = 'US' and update something ( only who is from 'US' )
Maurizio Bellemo
@mbellemo_twitter
Sep 15 2018 10:19
best practise suggestion. I have an object, called customer which contains an embedded array of deals. I decided to model it this way because the number of deals might never be more than 100 (worst case scenario). Whenever I get a single customer, I get also all the deals associated which is fine. Then, I send the object to the FE. In this specific case, does it make sense that I handle deal filtering/sorting on the MongoDB/backend or would you handle it on the FE only?
Kev
@lineus
Sep 15 2018 10:49
@mirik999 you can use the updateMany Model method. Something like
Model.updateMany({ region: 'US' }, { capital: 'New Moscow' })
@ratnesh2581990 I don't see anything wrong with find call. When you run that command and it returns no results, can you console.log the value of req.query.userid and see it' a good value?
Maurizio Bellemo
@mbellemo_twitter
Sep 15 2018 11:01
another more technical point. When I add, edit, delete a deal (which is an embedded object), I use the parent call Customer.findOneAndUpdate. I am trying to build a pre hook that tells me when a deal was changed DealSchema.pre('findOneAndUpdate', function(next) { console.log(`findOneAndUpdate`, this); next(); });
but it never gets called... our does the findOneAndUpdate gets propagated from the parent to child?
Kev
@lineus
Sep 15 2018 11:06
@mbellemo_twitter I think that depends on the number of concurrent requests for this route that your server can handle while maintaining a reasonable response time, the number of actual concurrent requests you recieve, the potential cost increase of extra computing depending on your choice of hosting provider vs your budget, and the load time for your worst case customer ( ie someone on slow internet or a junk device or both ). I think the best practice in this case is the one that fits your specific scenario.
@mbellemo_twitter for your second question, sub schema pre query hooks isn't a supported feature in mongoose. You have to add the hook to the parent schema. There are some issues in the mongoose github queue with details.
ratnesh2581990
@ratnesh2581990
Sep 15 2018 11:16
@lineus I have found req.query.userid will return me 5b50375de1b8d4000485ffd6 this value but still no result.
Is It due to the reason that I have make userid as type Schema.Types.ObjectId ??
Kev
@lineus
Sep 15 2018 11:19
mongoose will cast the string to an ObjectId, and if it has any issues casting the string it will return a CastError in the callback.
ratnesh2581990
@ratnesh2581990
Sep 15 2018 11:20
@lineus but there is no error in with any err callback
Kev
@lineus
Sep 15 2018 11:21
is that the exact same id you are using with your robomongo call?
ratnesh2581990
@ratnesh2581990
Sep 15 2018 11:21
yes
image.png
@lineus here is the screenshot of my robomongo it will return the result but when i use that inside file with mongoose it will return blank array
Kev
@lineus
Sep 15 2018 11:27
have you verified that your mongoose app and robomongo are connecting to the same db, and using the same collection names?
ratnesh2581990
@ratnesh2581990
Sep 15 2018 11:28
Yes I have checked that Many times
Kev
@lineus
Sep 15 2018 11:30
can you add a call to `mongoose.set('debug', true) to your mongoose app and share the corresponding find-debug output here?
ratnesh2581990
@ratnesh2581990
Sep 15 2018 11:32
Ok
ratnesh2581990
@ratnesh2581990
Sep 15 2018 11:44
@lineus Here is what i get in console
5b50375de1b8d4000485ffd6
Mongoose: orders.find({ userid: ObjectId("5b50375de1b8d4000485ffd6") }, { sort: { orderdate: -1 }, fields: {} })
Kev
@lineus
Sep 15 2018 11:46
the only difference that is immediately visible between your robomongo call and your mongoose call is that mongoose is adding the sort. Have you tried adding the sort to robomongo or removing it from mongoose to see if it changes the results?
ratnesh2581990
@ratnesh2581990
Sep 15 2018 11:47
ok i will try that also and let you know about the result
still the same result Mongoose: orders.find({ userid: ObjectId("5b50375de1b8d4000485ffd6") }, { fields: {} })
Kev
@lineus
Sep 15 2018 11:50
you also might try moving your callback to exec
 Order.find({userid: req.query.userid}).sort({orderdate: 'desc'}).exec( function (err, orders) {
            if (err) {
                res.json({ 'success': false, 'message': err });
            } else {
                res.json({ 'success': true, 'orders': orders });
            }
        });
sorry, it's been a while since I used callbacks with mongoose, I can't remember if you have to use exec when you add query helpers like sort.
ratnesh2581990
@ratnesh2581990
Sep 15 2018 11:52
Ok No Problem still no success
Kev
@lineus
Sep 15 2018 11:52
yeah, given the debug output, I figured that was a long shot. The debug output looked correct.
Maurizio Bellemo
@mbellemo_twitter
Sep 15 2018 11:53

@lineus thanks. One question though. If it is possible to add the hook on the parent only; how can i detect whether it is an attribute of the parent to be modified, or the subdocument, and which is the subdocument modified? In this specific case I would like to update the updatedAt field of the specific deal modified.

Customer.findOneAndUpdate({
        _id: customerId,
        company_id: req.user.company_id,
        "deals._id": dealId
    },
        { $set: { 'deals.$': body } },
        {
            new: true,
            runValidators: true
        }).then((customer) => {
            if (!customer) {
                return res.status(404).send();
            }
            res.send({ customer });
        }, (e) => {
            res.status(400).send();
        });

That above is the query done using the parent object. How would I detect in the pre hook that the query modifies a subdocument object and how to access that specific one?

Kev
@lineus
Sep 15 2018 11:55
@ratnesh2581990 can you copy and paste the dbName from your mongoose connection (string|options)?
@mbellemo_twitter give me a few minutes, I'll sort out an example :smile:
Maurizio Bellemo
@mbellemo_twitter
Sep 15 2018 11:56
@lineus man, u are great! I would build u a statue :)
ratnesh2581990
@ratnesh2581990
Sep 15 2018 11:57
@lineus dbname is ecommapp
@lineus or should it mean mongodb://<dbuser>:<dbpassword>@ds121861.mlab.com:21861/ecommapp
ratnesh2581990
@ratnesh2581990
Sep 15 2018 12:03
@lineus when i am running the above debug query in robomongo it is not working
image.png
Kev
@lineus
Sep 15 2018 12:04
I was just typing something to that effect. I seem to remember something strange about mlab object ids. let me do some research.
nah, it wasn't what i was thinking. that was just about the way the mlab ui displays the ObjectIds. His queries were still working.
Kev
@lineus
Sep 15 2018 12:15
@ratnesh2581990 what happens when you do a find with an empty object for the condition? do you get results?
@ratnesh2581990 in mongoose
@ratnesh2581990 or Order.count*()?
ratnesh2581990
@ratnesh2581990
Sep 15 2018 12:18
My other queries are working fine except this like this one
router.get('/getorderbyid', function (req, res) {
    if( req.query.orderid ) {
        Order.find({orderid: req.query.orderid}, function (err, orders) {
            if (err) {
                res.json({ 'success': false, 'message': err });
            } else {
                res.json({ 'success': true, 'orders': orders });
            }
        }).sort({_id: 'asc'});     
    } else {
        res.json({ 'success': false, 'message': 'User Id is empty' });
    }
});
my this query is working fine
Kev
@lineus
Sep 15 2018 12:24
@ratnesh2581990 can you change your mongoose schema type from ObjectId to String for the userId path and try the mongoose query again?
or drill into a document in the collection in robomongo and share the path details for the userid path on a specific document?
I have a feeling that the docs in your collection have userid stored as a string
ratnesh2581990
@ratnesh2581990
Sep 15 2018 12:35
But i need its schema type to ObjectId as i have to fetch the user details from different collection with populate query
Kev
@lineus
Sep 15 2018 12:35
are they currently stored as strings?
ratnesh2581990
@ratnesh2581990
Sep 15 2018 12:39
image.png
it is currently stored as in screenshot
Kev
@lineus
Sep 15 2018 12:41
does the userid path look like the top level _id path in that doc? The _id field is cut off in the screenshot
you could also write a quick script that updates or inserts one doc in the collection to change the userid path to an ObjectId and re-run your query to see if it returns that one document.
or use robomongo to do it? i'm not sure, having never used it :smile:
ratnesh2581990
@ratnesh2581990
Sep 15 2018 12:45
screenshot-mlab.com-2018.09.15-18-13-21.png
Kev
@lineus
Sep 15 2018 12:47
yeah, you're going to want to update your docs' userid path to be objectIds. Is this currently in production or just development?
ratnesh2581990
@ratnesh2581990
Sep 15 2018 12:49
development
Thanks its my bad I have to change type of userid in all the document to objectId Thanks for your effort. you are a life saver as always helped me 2 times when i am stuck badly
thanks a lot
Kev
@lineus
Sep 15 2018 12:54
anytime :) I'm glad I could be of service!
@ratnesh2581990 for what it's worth, you can populate from a string to an objectid, mongoose will cast the string for you. You can add the model property to your population object. Here's an example
having said that, I think ObjectId pathType is more explicit and is a better design choice
ratnesh2581990
@ratnesh2581990
Sep 15 2018 13:04
ok i will go with that as of now i am in development phase so i will change the type to object id
and keep in mind your code with strings as well i need anywhere
Thank you for your help.
Kev
@lineus
Sep 15 2018 13:06
@mbellemo_twitter I haven't forgotten about you, I think I uncovered a bug while writing your example :imp: I'm tracking it down, heh.
I learned something too @ratnesh2581990 :)
I really had no idea the ref property could be set on any other path type besides ObjectId.
ratnesh2581990
@ratnesh2581990
Sep 15 2018 13:08
Oh so you also have done some R&D for that and then provide me the answer. You are really a great person
Maurizio Bellemo
@mbellemo_twitter
Sep 15 2018 13:25
@lineus great!
Kev
@lineus
Sep 15 2018 13:53
@mbellemo_twitter I'm curious, are you using the built-in timestamps option on the dealSchema? or is it a manually defined property?
I ask because I'm wondering if setting the timestamps option on the sub schema will give you the updatedAt on the specifc deal without any extra hoops.
Maurizio Bellemo
@mbellemo_twitter
Sep 15 2018 14:01
i have an updatedAt property in the parent, and I want to add another one in the child "deal"
but i want them to be slightly different because i want to update the property in deal ONLY when I modify the deal and NOT the parent
Kev
@lineus
Sep 15 2018 14:02
after testing this explicitly, setting the timestamps option on the child schema does not work #4412 is open for it, we should have it fixed by the next minor release.
Maurizio Bellemo
@mbellemo_twitter
Sep 15 2018 14:02
while if i change the deal i want the customer updatedAt property to be changed as well
is there a timestamps option... because i was using a new property called updatedAt
Kev
@lineus
Sep 15 2018 14:04
yeah, at the moment it only effects child docs when you update the whole document, so it doesn't work in your case.
for now I think a pre hook is the best option. I'll finish up that example, assuming I don't get sidetracked AGAIN, lol.
Maurizio Bellemo
@mbellemo_twitter
Sep 15 2018 14:07
good to know, and thanks for the example :)
Kev
@lineus
Sep 15 2018 14:24
@mbellemo_twitter here's an example that shows using a pre findOneAndUpdate hook to get the query and the update, conditionally update the update doc being sent before findOneAndUpdate runs, and then tests the output to make sure that only the updated nested doc has it's updatedAt property mutated.
for now you'll have to use the "private" property _update to mutate the actual update before it's sent. I'll throw a PR in for a setUpdate method that mirrors the setQuery one.
Kev
@lineus
Sep 15 2018 14:34
wait, there's a mistake :)
hehe
@mbellemo_twitter I updated the gist, returning this from the pre hook was a mistake :)
it would cause updates that didn't meet the nestedId condition to exceed the maxium call stack almost immediately. I added another test to verify the behavior of the non-nestedId condition as well.
Kev
@lineus
Sep 15 2018 14:44
depending on whether you are providing this as an api to other developers or are working on a team ( or like me you don't always explicitly use $set ), you'll probably want to add some more logic to the pre hook to inspect the update doc more thoroughly, as there's no gaurantee the update doc in the pre-hook will have the top level $set property ie Test.findOneAndUpdate({ _id: test._id, 'nested._id': id }, { 'nested.$': o }, { new: true }) is still a valid update.
Maurizio Bellemo
@mbellemo_twitter
Sep 15 2018 15:07

thanks a lot. I am the only one using the API (for internal purpose) so it should be fine. I changed a bit the hook to also update the field of the parent

CustomerSchema.pre('findOneAndUpdate', function () {
    let q = this.getQuery();
    let u = this.getUpdate();
    let dealId = q['deals._id'];

    if (dealId) {
        u.$set['deals.$'].updatedAt = Date.now();
    }
    u.$set['updatedAt'] = Date.now();

    this._update = u;
});

wdyt?

Kev
@lineus
Sep 15 2018 15:13
LGTM :smile: do the tests pass?
Maurizio Bellemo
@mbellemo_twitter
Sep 15 2018 15:26
ehm... i do not have tests :)
but the DB stores the data
Kev
@lineus
Sep 15 2018 15:54
nice! let me know if you have any issues. It's a neat problem to solve :)
Maurizio Bellemo
@mbellemo_twitter
Sep 15 2018 15:56
jfyi... i am just trying it out with my CRUD implementation and of course it is NOT work when i create a deal because i use $push instead $set. Therefore, I guess that i have to change pre condition to support also the $push case
btw, why do we use the dollar syntax. I printed the objects and they are normal object with either set or pull
ah no... sorry
the dollar is included in the object
ahhaha
Maurizio Bellemo
@mbellemo_twitter
Sep 15 2018 16:11
at the end, I solved the problem
CustomerSchema.pre('findOneAndUpdate', function () {
    let q = this.getQuery();
    let u = this.getUpdate();
    let dealId = q['deals._id'];

    if (dealId) {
        u.$set['deals.$'].updatedAt = Date.now();
    }
    if (u.$push) {
        u.$push['deals'].createdAt = Date.now();
        u.$push['deals'].updatedAt = Date.now();
    }
    u['updatedAt'] = Date.now();

    this._update = u;
});
Maurizio Bellemo
@mbellemo_twitter
Sep 15 2018 16:21
the only thing I noticed is that the first time i create a deal, the createdAt is created correctly. When I update the object, createdAt gets deleted from MongoDB... I guess I have to find a way to preserve the createdAt date in the hook. Any suggestion @lineus ?
Kev
@lineus
Sep 15 2018 16:40
@mbellemo_twitter because you're overwriting the whole embedded document ie 'deals.$': body, unless the body object has the createdAt property it will get overwritten in the db.
can you include the createdAt property in body?
Maurizio Bellemo
@mbellemo_twitter
Sep 15 2018 16:42
how would i do it so that it is present in the hook? I would probably would have to send it from the FE to the API... I am not sure it is really the best option
Kev
@lineus
Sep 15 2018 16:48
do you need to overwrite the whole deal doc? can you just update the specific paths within deal?
like { $set: { 'deal.$.something': 'someValue' } }?
you might need to take body and create an update doc that calls out specific paths. I don't think there's another way without providing the createdAt path in body.
I have to step away, be back in the morning probably :smile: always a pleasure @mbellemo_twitter!
Maurizio Bellemo
@mbellemo_twitter
Sep 15 2018 17:12
thanks a million @lineus
Ajay Tanwar
@ajayt365
Sep 15 2018 20:13
hey guys i had a question .why use joi when you have mongoose validations ? creating two schemas for one object doesn't make sense . is it ?