These are chat archives for Automattic/mongoose

22nd
May 2017
Julio Borja Barra
@juboba
May 22 2017 13:10 UTC
hey, I'm saving a document and would like to apply an aggregation to the callback value, i.e.:
myDocument.save(function(err, doc) {
// here I get `doc.created_at`and it's not a string. I would like to make it a string.
})
kunokdev
@kunokdev
May 22 2017 19:58 UTC
guys, is there operator in aggegation framework that allows to group by most frequent value in array? https://stackoverflow.com/questions/44117524/while-grouping-in-aggregation-use-the-most-frequent-value-in-array-for-duplicat
Joshua Austill
@jlaustill
May 22 2017 20:48 UTC
@kunokdev your solution is pretty straight forward, add $sum: 1 to get the count of each city_code, then $unwind the array, then sort by the count, then group and use $first. It will end up being 4 stages if it's playing out in my head correctly.
@kunokdev Here is a very nice writeup on the same basic concept http://techbrij.com/mongodb-aggregation-query-records-max
kunokdev
@kunokdev
May 22 2017 21:10 UTC
@jlaustill Hey thanks!
I will have a read.
To be honest I am a bit sloppy with queries
I understand all these stuff and I can program them on Node.js level but when it comes to creating proper query I usually get a bit uncomfortable
Its due to lack of learning things. I just learned as I needed specific thing at specific moment. I'll spend few days to improve my querying skills
Joshua Austill
@jlaustill
May 22 2017 21:12 UTC
@kunokdev Have you ever tried 3t studio? It has a gui for writing these types of queries that is truly amazing
kunokdev
@kunokdev
May 22 2017 21:12 UTC
@jlaustill I am using it but I am not big fan of GUI
@jlaustill I actually write these queries within 3t studio intelshell
but then I put them into code
I know that IDE generates usable queries for me
Joshua Austill
@jlaustill
May 22 2017 21:13 UTC
@kunokdev This is one case where the gui is super helpful. I'm a command line hero/junkie, whichever term you prefer. but for this, 3t studio's aggregation generato is just amazing
kunokdev
@kunokdev
May 22 2017 21:13 UTC
Yea I should really try it, it will help me understand logic behind syntax easier
some of my colleges use it
in fact that's the only way they can query mongo
Joshua Austill
@jlaustill
May 22 2017 21:15 UTC
The best part is that you can export it to a normal mongodb query. and you can import a query from the clipboard. That feature alone is worth the cost of 3t to me
kunokdev
@kunokdev
May 22 2017 21:15 UTC
I actually use free version
can I import query and transform it to GUI ?
in free version
Joshua Austill
@jlaustill
May 22 2017 21:16 UTC
you can in my version, I'm not sure about the free one. Just open the aggregrate screen and see if the open from clipboard button is there. It's the very right icon
I have the enterprise version though, so I'm not sure what is in the other versions honestly
kunokdev
@kunokdev
May 22 2017 21:17 UTC
well the first thing in the morning will be playing with aggregation framework within mongochef
on this PC i only have terminal for ssh
Joshua Austill
@jlaustill
May 22 2017 21:18 UTC
that sounds fun :)
@kunokdev I'm actually working on some node.js code in ssh at the moment that pulls 2.6 million documents from mysql and then does 2.6 million upserts into mongodb. I'm having issues with the bulk statement I'm pretty sure due to the nodejs driver :(
kunokdev
@kunokdev
May 22 2017 21:22 UTC
@jlaustill oh I remember doing same stuff
i streamed
didnt bulk
Joshua Austill
@jlaustill
May 22 2017 21:22 UTC
@kunokdev How did you deal with the 1,000 document limit and the 16mb document size limit?
oh, streaming would be very different :)
kunokdev
@kunokdev
May 22 2017 21:23 UTC
well you are doing mysql to mongo, right?
Joshua Austill
@jlaustill
May 22 2017 21:23 UTC
it's working great on my dev box, which is a VERY fast, I'm only having issues on the server which is VERY slow
kunokdev
@kunokdev
May 22 2017 21:23 UTC
ohhhh i know that
i actually remember spending weeks trying to find memory leak in one of my scripts
Joshua Austill
@jlaustill
May 22 2017 21:24 UTC
It takes about 1.75 minutes on my devbox, and about 23 minutes on my server.
kunokdev
@kunokdev
May 22 2017 21:24 UTC
and the issue was actually server
Joshua Austill
@jlaustill
May 22 2017 21:24 UTC
I'm starting to wonder if that isn't my issue as well!
kunokdev
@kunokdev
May 22 2017 21:25 UTC
btw
few weeks ago i wrote script that fetches data from ftp server
and netiher of node.js libs could do it properly
i tried liiterally all of them and i tried to hack them to work for 5GB file
at the end I used child_process.exec method and ran 2 line of code python script that worked well
Joshua Austill
@jlaustill
May 22 2017 21:27 UTC
I would have used rsync :)
or lftp lol
I'm a bash script junkie
kunokdev
@kunokdev
May 22 2017 21:28 UTC
oh I already see crafts of mixing bash with node
that's so brilliant
Joshua Austill
@jlaustill
May 22 2017 21:28 UTC
I don't know about brilliant, but it's SUPER powerful
kunokdev
@kunokdev
May 22 2017 21:28 UTC
indeed
when I just got started with all these stuff I met that sysadmin who was shell geek, he taught me so much stuff, and I learned a bit of shell over time and just with a small knowledge I could do more than ever before with regular programmer mindset
too bad guy got fired, now I am alone into this with windows-only users who dont even know what terminal is
Joshua Austill
@jlaustill
May 22 2017 21:31 UTC
I worked in a windows only shop for a decade where I couldn't use linux/bash. man that was AWFUL
they would spend SO MUCH MONEY to buy software that I could write 3 lines of bash to do the same thing
kunokdev
@kunokdev
May 22 2017 21:31 UTC
honestly next time I will be looking for a job, first thing I am gonna ask them is which OS they use on their serves
Joshua Austill
@jlaustill
May 22 2017 21:32 UTC
that's how I ended up where I am now! My work laptop runs ubuntu, as do all of our servers except a few
kunokdev
@kunokdev
May 22 2017 21:33 UTC
Have you tried Bash on Windows?
Joshua Austill
@jlaustill
May 22 2017 21:34 UTC
I used cygwin for years, I haven't used microsofts new version at all. I don't even own a windows computer anymore lol