Log file Viewer
#mongodb logs for Thursday the 25th of September, 2014
[00:01:38] <hydrajump> Different Q about MMS. Can the metrics that the MMS agent gathers be accessed via an API without using the hosted or on premise MMS server?
[00:01:49] <donotpet> Hi. If I have a working replica set, my application can write to the primary, and read from the secondaries. After a failure, how can my app know who is the primary?
[00:05:32] <joannac> donotpet: every driver has a replica set connection, which will auto reconnect when the primary changes
[00:06:10] <donotpet> joannac: that is what I hoped you would say! Thank you for the confirmation. It worked that way in my VM setup
[00:13:09] <hydrajump> yeah it would be easier if the MMS agent itself was opensource and/or allowed sending the collected data to something other than MMS server ;)
[00:16:06] <hydrajump> looks like that is planned https://jira.mongodb.org/browse/MMS-1853?jql=component%20%3D%20%22Monitoring%20Agent%22%20AND%20project%20%3D%20MMS
[00:22:41] <hydrajump> joannac: this is helpful http://mms.mongodb.com/help/reference/monitoring/#database-commands-used-by-the-monitoring-agent
[00:23:30] <hydrajump> so I assume that what the MMS monitoring agent does is execute mongo commands on each monogodb instance and than shipping the data to the MMS server
[03:57:04] <d0tn3t> i have 150GB on SQL 2008, i want using mongodb with mssql to improve performance
[04:14:39] <joannac> dump all your data, redesign your schema, modify your data for the new schema, start up some mongod instances, put all your data in mongodb
[04:55:21] <FatBoyXPC> Well, and project, but I assume that's like adding a projection any other time.
[04:56:39] <FatBoyXPC> Boomtime: If I were to gist up what I want my data to look like returned to me, could you point me to an example that's similar to what I want?
[04:57:37] <FatBoyXPC> The documentation might be good but this aggregation stuff isn't easy to understand, to me, at least.
[04:59:02] <Boomtime> so what you are asking for is for me to search for the appropiate documentation link, that is unreasonable
[05:00:17] <FatBoyXPC> I was hoping you'd know where to look. I can match, group, and project just fine. All I'm doing is summing up some numbers with the match/group, then I want a total number of those sums, too. That's all fine and dandy. I just don't know how to then also see all matching results. it's trivial to do it in 2 queries but I feel like it's possible in 1.
[05:00:50] <FatBoyXPC> Well, I haven't projected this specific query yet, but I've done it in the past. It's just not where my hangup is.
[05:01:46] <FatBoyXPC> Yeah, you're too used to noobs coming in and asking for shit to just get done. I understand why you have that attitude, but it really sucks.
[05:02:07] <Boomtime> if you have an intermediate set of results you can just keep stacking your pipeline with more operators.. it's fun
[05:02:49] <FatBoyXPC> But how would that help me in getting all matching results? Ideally the query is find all clients in zipcodes x, y, and z. And they have sent files field, and accepted files field, I want to sum those up per zip code (that part is easy peasy), then total those (also easy), but then return all results from the first bit.
[05:02:52] <Boomtime> so.. $match -> $project -> $group -> $unwind -> $match -> $group is a totally reasonable pipeline
[05:06:19] <joannac> db.coll.find({a:1}) and also db.coll.aggregate([{$match: {a:1}}, ....other aggregation stuff here])
[05:06:51] <FatBoyXPC> it's "find all matching clients where blah blah blah" then aggregate some of the fields in the matched set
[05:07:25] <FatBoyXPC> The idea is the data gets returned to a list of all clients, but there's also a "Totals" table to get a quick glance at the numbers.
[05:11:14] <FatBoyXPC> now, with projection, I can have 2 levels of grouping do get sums of provided zips, and then total sums, right?
[05:12:22] <Boomtime> yes, you can just keep piping into $group until you get the final collated result you want
[05:13:43] <FatBoyXPC> Well yes, but I mean like, I want a group of sums of field A, sums of field B, then another group of total sum field A, total sum field B. But I want both those returned. I imagine I can do that with $project?
[05:15:17] <FatBoyXPC> Boomtime: The idea is sum of field a and sum of field b for zipcode X. sum of A, sum of B for zipcode Y. Then another group of total sum field A, total sum field B.
[05:18:44] <Boomtime> so group on "zip" and then use $sum twice in that stage, A: { $sum: "$A" }, B: { $sum: "$B" }
[05:19:16] <FatBoyXPC> Yeah, I do that for field a sum and field b sum, but what about for total sum A, total sum B?
[05:20:16] <FatBoyXPC> if it's a 'different results' problem then I'll just do the math on the javascript front end when I iterate over the totals, that's trivial enough.
[05:20:26] <Boomtime> you want the intermediate stages results, but you also want to use those results with another operation
[05:21:03] <Boomtime> i can see a way to do it using an array to temporarily hold the intermediate results, but that is kind of tricky
[05:21:28] <FatBoyXPC> If it gets too tricky then it's not worth it, honestly. It's easy enough to do in node or js front end
[05:21:30] <Boomtime> i think you would be better off just summing the completel totals on the client
[05:22:34] <FatBoyXPC> Right. I iterate over them anyway on the client to throw them in a table, doesn't hurt to just add another var and var += var
[05:23:38] <joannac> there's not way of saying "all my documents should look like this, oh except for the last one"
[05:27:15] <FatBoyXPC> Really, since these are all sums, I could do *all* of this math on the js client, and still only have a single db call, but meh, I like having the assurance the db will be right, I suppose.
[05:30:48] <Boomtime> if you try to do all the math on the client you will need to pull half the database across the network - i suspect this will kill you pretty quick
[05:31:12] <Boomtime> you should definitely do what you can on the server, get the results you must, then fill in any missing math on the client at the end
[05:31:13] <FatBoyXPC> Remember - I'm returning the results anyway - and also iterating over them anyway (to put them in a table)
[05:32:38] <Boomtime> no human can possibly make sense of a table containing 349k records displayed all at once, that is silly
[05:32:57] <FatBoyXPC> the total collection size is 349k records. I figure at most they'll see a few hundred.
[05:33:38] <FatBoyXPC> But honestly, I might start limiting to roughly 50 results. They really only care about the highest sellers, really. He has me sort from highest to lowest honestly, but he swears to me it's good to see all the results.
[05:34:12] <Boomtime> excellent, the aggregation can save you having to iterate over the other 348,000+ results just to do some basic math
[12:59:39] <tirengarfio> How to install MongoId?, I mean I have just started with mongodb using this symfony tutorial: http://symfony.com/doc/current/bundles/DoctrineMongoDBBundle/index.html. but when I try to use persist as the tutorial says, Im getting this error: Attempted to load class "MongoId" from the global namespace in /home/tirengarfio/workspace/mongo/vendor/doctrine/mongodb-odm/lib/Doctrine/ODM/MongoDB/Id/AutoGenerator.php line 36. Did y
[13:04:11] <Derick> tirengarfio: did you install the mongodb php extension? I think Symfony/Doctrine can't find it
[13:05:07] <tirengarfio> I installed these three things: apt-get install php-pear > apt-get install php5-dev > pecl install mongo
[13:07:56] <Derick> then I think you've either not added it to the right php.ini, or didn't restart your webserver (or php-fpm if you're using that)
[13:33:16] <FezVrasta> hey guys... if I've this doc: {"foo": [ {"a": true, "b": true}, {"a": true, "b": false} ]} how can I select only the "foos" which have both values true?
[15:43:08] <wc-> hi all, im using pymongo to insert a document into a collection, one of the fields has a value of None in python, when i check it in mongo the value is NaN
[15:43:28] <wc-> i would like that value to be stored as null in mongo, any idea how to force that to happen?
[16:29:58] <rafaelhbarros> tirengarfio: http://docs.mongodb.org/ecosystem/tools/administration-interfaces/
[16:45:15] <obeardly> Hello fellow nerds; is it acceptable to ask questions about MMS in this channel?
[17:02:01] <tirengarfio> I have this schema: https://gist.github.com/Ziiweb/b4ed1f57449c87eec2b2 but when I remove that category, the products associated is not removed.. I expected the deletion in CASCADE
[17:07:32] <obeardly> I've been trying to get some help on MMS to no avail, but I'm not even sure if this is the right place to ask
[17:10:45] <isevi> hello all, I'm trying to set up a mirror of the downloads-distro.mongodb.org site for internal use, and can't find any info on this... any leads?
[17:15:27] <obeardly> isevi: Isn't that just an apache webserver? Can't you just rsync it to your local server?
[18:43:16] <keeger> i am looking to store like an upgrade queue in a document, and i want to return a "view" of that document
[18:44:24] <keeger> so if i have a document: town: townHall { level: 1 }, townHallUpgrade [ { level: 2, atTime: 10:52 am }, { level: 3, atTime: 12:20 pm }]
[18:45:14] <keeger> is there a way i can query it and return: town: townHall { level 2 }, townHallUpgrade [ { level: 3, atTime: 12:20 pm} ] if the query is done at 10:53?
[19:25:48] <dgarstang> i just did a fresh install of mongo. I thought before the admin user had a password assigned, I could connect locally as admin without a password???
[19:30:16] <dgarstang> cheeser: I'm using a library here which, according to tcpdump, is connecting locally as the admin user.... is the username required when using the localhost exception?
[19:33:08] <cheeser> oh, then you don't need the username afaik. because there's no user to have a name.
[19:43:46] <dgarstang> i really don't understand that, given the name of the admin user is arbitrary.
[20:44:51] <James1x0> Anyone know if you can populate a path like path.to.array.$.withref in mongoose?
[20:52:20] <speaker123> hi i've got a mongodump that is not completing. just stalls at 13% of the creation of some bson file. come back >8 hours later, same spot
[20:52:44] <speaker123> how can i figure out what is going on. nothing is else writing to the db (or even reading)
[20:54:09] <javeln> so we just upgraded to mongo-hadoop 1.3.0 and it seems like the "ignore duplicate keys" on insert behavior changed?
[20:54:50] <javeln> is there config option or some way to get the pre 1.3 duplicate key behavior with out downgrading?
[21:18:16] <nicolas_leonidas> but I don't need all the data all I need to know is how many results this query returns
[21:28:11] <frodopwns> i have a collection "Users" …each user has a list called sites….each item in sites is a dict {"id":"id", "roles":[roles]}, how could i query for all users that have a site with an id of X?
[21:46:19] <keeger> in mongo, can i do an update across a collection, using the current value to set another one?
[21:50:17] <daidoji> keeger: not really. Mongo doesn't do well on key manipulations within the document, you're kind of expected to do that all in the app
[21:50:52] <daidoji> although maybe update({field: 'bar'}, { $inc: {field1: "$field2"}}) notation might work
[21:57:53] <daidoji> keeger: yeah thats another thing, there's no explicit casting so if you didn't store them as the BSON "number" then all numeric operations on values within your document won't work
[21:58:28] <daidoji> which basically means you need to mapReduce the values out with teh keys you want to update, and then $set via a javascript function (if you want to do everything in Mongo)
[22:09:44] <decompiled> I use a keyfile and have never seen that error. Any ^M characters or something?