These are chat archives for Automattic/mongoose

10th
Apr 2018
Bharath Kumar P
@pkbharath
Apr 10 2018 04:17
Hi guys, getting Cast to ObjectId failed for value "" at path "_id" for model "Company" in production runs fine in dev
Any insights
Kev
@lineus
Apr 10 2018 08:31
@pkbharath does the error message show you the _id it's trying to cast?
Josh Cole
@saikojosh
Apr 10 2018 09:40
Hey Noders! I'm in need of a contract Full-Stack Node.js Developer to work on a couple of very lucrative projects for me. I run an innovation tech consultancy - we work on lots of discovery projects and get to play around with new technology all the time. Email josh@recombix.com or reply here. Ideally London based.
Arun Gadag
@arun-awnics
Apr 10 2018 10:50
What is wrong in this code:
getLimitedMessages(receiverId, senderId, offset, size, callback) {
        Message.find({ receiverId: receiverId }, (err, messages) => {
                if (err) throw err;
            })
            .limit(parseInt(offset + size))
            .skip(parseInt(offset))
            .sort({ $natural: -1 })
            .exec(callback);
    }
I am setting offset=20 and size=20 and yet, I get all the records from the db instead of 20
Kev
@lineus
Apr 10 2018 10:51
@arun-awnics you're passing in a callback to find and calling exec with a second callback. it should be one or the other.
Arun Gadag
@arun-awnics
Apr 10 2018 10:52
How? I am sorry. I didn't get you
Kev
@lineus
Apr 10 2018 10:53
Message.find({ receiverId: receiverId }, (err, messages) => {and .exec(callback);
you should get rid of the first one
and handle the error in the second one
getLimitedMessages(receiverId, senderId, offset, size, callback) {
        Message.find({ receiverId: receiverId })
            .limit(parseInt(offset + size))
            .skip(parseInt(offset))
            .sort({ $natural: -1 })
            .exec(callback);
    }
make sure the parameter signature on callback handles err
Arun Gadag
@arun-awnics
Apr 10 2018 10:55
@lineus But how do I handle the error in this case?
Oh... Got it
So you are telling me that the error should be handled in the calling function, right?
Kev
@lineus
Apr 10 2018 10:57
callback = function(err, docs) {
  if (err) { return console.error(err) }
  console.log(docs)
}
for example
the callback that gets passed into exec would be the same callback you would pass into the original find call.
you just shouldn't use both at the same time
Arun Gadag
@arun-awnics
Apr 10 2018 10:59
Got it
Thanks :thumbsup: @lineus
Kev
@lineus
Apr 10 2018 10:59
time to put my dad hat on until the kids are off to school :) back in a bit!
any time :)
Arun Gadag
@arun-awnics
Apr 10 2018 11:00
Haha :D
Arun Gadag
@arun-awnics
Apr 10 2018 11:55
@lineus Hey its not working
getLimitedMessages(receiverId, senderId, offset, size, callback) {
        Message.find({ receiverId: receiverId })
            .limit(parseInt(offset + size))
            .skip(parseInt(offset))
            .sort({ $natural: -1 })
            .exec(callback);
    }
It is still returning me all the records present in the db
Arun Gadag
@arun-awnics
Apr 10 2018 12:00
It fetches 20 records the first time I call this method. I got 2000 records the second time I called it
Kev
@lineus
Apr 10 2018 12:02
did you verify that the offset and size values are the same when they are passed into getLimitedMessages?
it might be worth it to explicitly return Message.find, ie return Message.find(..)....
Arun Gadag
@arun-awnics
Apr 10 2018 12:05
offset keeps changing on every iteration
Kev
@lineus
Apr 10 2018 12:06
can you share the loop where you're calling it?
Arun Gadag
@arun-awnics
Apr 10 2018 12:06
1st iteration:
offset=0; size=20
2nd iteration:
offset=20;size=20
and so on...
I am using angular in front end:
getMoreMessages(group: Group) {
    const size = 20;
    this.chatService.getMessages(this.selectedUser.id, group.id, this.offset, size)
      .subscribe((msgs) => {
        msgs.map((message: any) => {
          this.messages.unshift(message);
          this.ref.detectChanges();
        });
      });
  }
offset is changed on every iteration
This method is called on scroll:
onScroll() {
    const scrollPane: any = this.messageBox.nativeElement;
    if (scrollPane.scrollTop === 0) {
      this.offset = this.offset + 20;
      this.getMoreMessages(this.selectedGroup);
    }
  }
Kev
@lineus
Apr 10 2018 12:08
did you try adding return to the Messages.find() in your method?
Arun Gadag
@arun-awnics
Apr 10 2018 12:09
Without any condition you mean?
Kev
@lineus
Apr 10 2018 12:10

instead of this:

getLimitedMessages(receiverId, senderId, offset, size, callback) {
        Message.find({ receiverId: receiverId })
            .limit(parseInt(offset + size))
            .skip(parseInt(offset))
            .sort({ $natural: -1 })
            .exec(callback);
    }

try this:

getLimitedMessages(receiverId, senderId, offset, size, callback) {
        return Message.find({ receiverId: receiverId })
            .limit(parseInt(offset + size))
            .skip(parseInt(offset))
            .sort({ $natural: -1 })
            .exec(callback);
    }
Arun Gadag
@arun-awnics
Apr 10 2018 12:10
Okay... Let me try
Nope...
Does not work
image.png
Still the same
Kev
@lineus
Apr 10 2018 12:14
I'm mocking something up, give me a few minutes and I'll see if I can sort it out.
Arun Gadag
@arun-awnics
Apr 10 2018 12:15
This is how I am calling the service and my callback function:
messageService.getLimitedMessages((req.params.groupId), (req.params.userId), (req.query.offset), (req.query.size), (err, result) => {
        if (err) log.error('err', err);
        res.send(result);
    });
@lineus Just so that you know
Bruno Barros
@bybrunobarros
Apr 10 2018 12:17
Hey everyone! Is there a simple way to update a subdocument which is in an array? Like remove() from the documentation parent.children.id(_id).remove()
Bruno Barros
@bybrunobarros
Apr 10 2018 12:40
Actually I know you can do it like this https://stackoverflow.com/questions/26156687/mongoose-find-update-subdocument#answer-26157458, but can we do it from a document, not from the model?
Kev
@lineus
Apr 10 2018 12:43
@arun-awnics so every time you scroll on the page, you want the app to display all of the previous records + 20 more. Basically a list that grows until the end?
Arun Gadag
@arun-awnics
Apr 10 2018 12:43
@lineus Yes
Exactly
The page displays 20 records...
Kev
@lineus
Apr 10 2018 12:45
here's where I think you're going to run into trouble.. The onscroll() is going to fire A LOT more times than you want to fire off a call to the db.
I don't have a ton of experience, so I could be wrong, but that's the first thing that I can think of.
Arun Gadag
@arun-awnics
Apr 10 2018 12:47
@lineus No no...
In the front end, I am limiting that scroll()
Kev
@lineus
Apr 10 2018 12:48
oh cool
Arun Gadag
@arun-awnics
Apr 10 2018 12:48
So when I said scroll... It waits for a particular point.
So that is taken care
Kev
@lineus
Apr 10 2018 12:48
nice!
Arun Gadag
@arun-awnics
Apr 10 2018 12:49
I did check in the front end to make sure it is calling only once by having a breakpoint. so I can assure you that the API is called only once when I scroll
Kev
@lineus
Apr 10 2018 13:05
@arun-awnics is your code open source? If it is, and you have the current state tagged or in a branch, I'll install it and take a look. In the meantime, I mocked up an example in a gist that proves your query is sound.
I changed the gist just now @arun-awnics, I forgot to use $natural
Arun Gadag
@arun-awnics
Apr 10 2018 13:09
@lineus I changed the query a bit
Check it
getLimitedMessages(receiverId, senderId, offset, size, callback) {
        console.log('receiverId: ' + receiverId + ' senderId: ' + senderId + ' offset: ' + offset + ' size: ' + size);
        if (offset === '0') {
            console.log('first time called');
            Message.find({ receiverId: receiverId })
                .skip(parseInt(offset))
                .limit(parseInt(size))
                .sort({ $natural: -1 })
                .exec(callback);
        } else {
            console.log('else block');
            Message.find({ receiverId: receiverId })
                .skip(parseInt(offset + size))
                .limit(parseInt(size))
                .sort({ $natural: -1 })
                .exec(callback);
        }
    }
This works only the first 2 iterations but returns [] after that
Kev
@lineus
Apr 10 2018 13:11
I just noticed that the output in my gist is returning the right number of records, but not the right records. back to the drawing board.
Arun Gadag
@arun-awnics
Apr 10 2018 13:11
image.png
Did you delete the gist?
Kev
@lineus
Apr 10 2018 13:14
yeah, it was broken :) I'll throw a new one up momentarily. I wrote a cli tool that makes gists via github's api, but it isn't smart enough yet to update them :) so I just kill them dead.
Arun Gadag
@arun-awnics
Apr 10 2018 13:15
Can I dm you? Do you think we are flooding the group?
Kev
@lineus
Apr 10 2018 13:17
you can dm me anytime, but honestly I think this group needs some activity. It would be supremely cool if anyone with the time could chime in more often.
I've only been using node and javascript since late december of last year, so I'm not an expert by any means. I do have time though :)
Arun Gadag
@arun-awnics
Apr 10 2018 13:25
@lineus Alright... I think I got somewhere with this issue
But not entirely correct
getLimitedMessages(receiverId, senderId, offset, size, callback) {
        console.log('receiverId: ' + receiverId + ' senderId: ' + senderId + ' offset: ' + offset + ' size: ' + size);
        if (offset === '0') {
            console.log('first time called');
            Message.find({ receiverId: receiverId }, { skip: parseInt(offset) })
                .limit(parseInt(size))
                .sort({ $natural: -1 })
                .exec(callback);
        } else {
            console.log('else block');
            Message.find({ receiverId: receiverId }, { skip: parseInt(offset + size) })
                .limit(parseInt(size))
                .sort({ $natural: -1 })
                .exec(callback);
        }
    }
Kev
@lineus
Apr 10 2018 13:26
Arun Gadag
@arun-awnics
Apr 10 2018 13:26
I am using mongoose verion 4
Kev
@lineus
Apr 10 2018 13:26
haha, we both removed the skip :)
Arun Gadag
@arun-awnics
Apr 10 2018 13:26
Hence I have to pass skip as an option
But adding skip returns me only _ids of the records
image.png
Now what do you think the issue is?
Kev
@lineus
Apr 10 2018 13:31
I think find is treating that second parameter as a projection, try find({ receiverId: receiverId }, null, { skip: parseInt(offset + size) })
but I suspect that will just take you back to where you were before :) if it doesn't I'll look into why.
you can see in my gist that it worked without skip at all, but as you get to the bottom of the results, your returned cursors are going to potentially get BIG
Arun Gadag
@arun-awnics
Apr 10 2018 13:36
Addind null just rolls me back to the same condition I was facing before
getLimitedMessages(receiverId, senderId, offset, size, callback) {
        console.log('receiverId: ' + receiverId + ' senderId: ' + senderId + ' offset: ' + offset + ' size: ' + size);
        if (offset === '0') {
            console.log('first time called');
            Message.find({ receiverId: receiverId }, null, { skip: parseInt(offset) })
                .limit(parseInt(size))
                .sort({ $natural: -1 })
                .exec(callback);
        } else {
            console.log('else block');
            Message.find({ receiverId: receiverId }, null, { skip: parseInt(offset + size) })
                .limit(parseInt(size))
                .sort({ $natural: -1 })
                .exec(callback);
        }
    }
This doesn't work
you can see in my gist that it worked without skip at all, but as you get to the bottom of the results, your returned cursors are going to potentially get BIG
Do you think it will work without skip then?
Kev
@lineus
Apr 10 2018 13:39
give it a try and see :) assuming res.send is redrawing the page/frame/angularThingy it should.
Arun Gadag
@arun-awnics
Apr 10 2018 13:42
Let me try
:D
Lol :D
It was a cheap shot but it doesn't work
Kev
@lineus
Apr 10 2018 13:43
what is doing?
Arun Gadag
@arun-awnics
Apr 10 2018 13:43
It returns me the same 20 records every time I make a call
getLimitedMessages(receiverId, senderId, offset, size, callback) {
        console.log('receiverId: ' + receiverId + ' senderId: ' + senderId + ' offset: ' + offset + ' size: ' + size);
        if (offset === '0') {
            console.log('first time called');
            Message.find({ receiverId: receiverId }, null, { skip: parseInt(offset) })
                .limit(parseInt(size))
                .sort({ $natural: -1 })
                .exec(callback);
        } else {
            console.log('else block');
            Message.find({ receiverId: receiverId }, null, { skip: parseInt(offset + size) })
                .limit(parseInt(size))
                .sort({ $natural: -1 })
                .exec(callback);
        }
    }
This works but it returns only IDs
Why is that?
Any idea?
Kev
@lineus
Apr 10 2018 13:44
you mean the version without null as the second arg?
Arun Gadag
@arun-awnics
Apr 10 2018 13:45
Yes
getLimitedMessages(receiverId, senderId, offset, size, callback) {
        console.log('receiverId: ' + receiverId + ' senderId: ' + senderId + ' offset: ' + offset + ' size: ' + size);
        if (offset === '0') {
            console.log('first time called');
            Message.find({ receiverId: receiverId }, null, { skip: parseInt(offset) })
                .limit(parseInt(size))
                .sort({ $natural: -1 })
                .exec(callback);
        } else {
            console.log('else block');
            Message.find({ receiverId: receiverId }, null, { skip: parseInt(offset + size) })
                .limit(parseInt(size))
                .sort({ $natural: -1 })
                .exec(callback);
        }
    }
This one
Sorry... Without null I mean
````
getLimitedMessages(receiverId, senderId, offset, size, callback) {
        console.log('receiverId: ' + receiverId + ' senderId: ' + senderId + ' offset: ' + offset + ' size: ' + size);
        if (offset === '0') {
            console.log('first time called');
            Message.find({ receiverId: receiverId }, { skip: parseInt(offset) })
                .limit(parseInt(size))
                .sort({ $natural: -1 })
                .exec(callback);
        } else {
            console.log('else block');
            Message.find({ receiverId: receiverId }, { skip: parseInt(offset + size) })
                .limit(parseInt(size))
                .sort({ $natural: -1 })
                .exec(callback);
        }
    }
This one
Kev
@lineus
Apr 10 2018 13:46
that's because you're using the projection parameter on a path called skip which I'm assuming doesn't exist in your document. in other words, you're removing the skip option from the query, and telling the db to only send the skip and _id paths back.
Arun Gadag
@arun-awnics
Apr 10 2018 13:46
It returns 20 records when offset=0 but returns only ids when offset>0
I did not understand what projection parameter is
But how do I solve this issue?
Kev
@lineus
Apr 10 2018 13:53
the projection field is the second parameter to the Model.find method. it tells mongodb which fields you want to have returned in the cursor docs.
if you have { skip: truthy_value } as the second parameter to Model.find(), it is going to return a cursor with docs, those docs will only have the _id path and the skip path ( which doesn't exist on your docs ) hence you end up with just a cursor full of docs with{ _id:... }only.
Kev
@lineus
Apr 10 2018 14:39

@bybrunobarros

Hey everyone! Is there a simple way to update a subdocument which is in an array? Like remove() from the documentation parent.children.id(_id).remove()

do you know the _id of the subdoc in advance?