Log file Viewer

[01:41:38] <jparkton> if I have a collection and have email addresses in there how can I create a variable that will access those emails? really new to mongo and meteor so just having fun but not sure how to grab each email as a variable?

[01:44:22] <diegows> tomorrow I'll have a talk about mongodb, does anyone recommend a data source to test mongodb?

[03:17:53] <alnewkirk> geo-location data?

[04:37:44] <testing22> i'm trying to use the aggregation framework to count subdocument items. example: http://pastebin.com/wU4BrEeN the top is the raw document, the bottom is the output that i'm going for with the pipeline. i've found a way to do it with adding each to a set and then unwinding one at a time but it piles up lots of ram for such a simple (i'd think so at least) task. is there any efficient way to do this?

[04:40:49] <testing22> i could add each to a set and then count the set with sum:1, but that requires the unwind which becomes huge with all of the various sets. is there a better way to determine the side of the set within the pipeline?

[06:04:20] <Fandekasp> hi there

[06:06:22] <Fandekasp> I'm a new user to mongo. I'm tring to add a field profile.website to my users, but when I do so, it removes the other profile fields, eg profile.blurb : http://sprunge.us/QHfC

[06:06:51] <Fandekasp> how can I just edit/append new fields without removing the existing ones ?

[06:09:26] <Fandekasp> I tried $addToSet, but it raises error "Cannot apply $addToSet modifier to non-array"

[06:13:39] <joannac> $set

[06:14:46] <Fandekasp> joannac: yes, that's what I do in the pastebin I shared

[06:17:37] <Fandekasp> the problem is that it removes the other profile values ...

[06:18:28] <joannac> mehdb.users.update({_id: '1703b7fa-ba95-4d67-a006-7005ff67c1cb'}, {$set: {"profile.websites": ['http://www.google.com']}})

[06:18:57] <Fandekasp> damn, why didn't I think of this :)

[06:19:02] <Fandekasp> thank you very much joannac

[06:28:23] <Fandekasp> joannac: do you know a way to join elements from an array ?

[06:29:36] <Fandekasp> I would like to format ['http://www.google.com', 'http://www.facebook.com'] in 'http://www.google.com, http://www.facebook.com' for example

[06:33:05] <Fandekasp> hmm I tried to go another way with a for loop, but it doesn't update anything

[06:33:07] <Fandekasp> db.users.find({'websites':{$exists: true}}).forEach(function(user){ for (w in user.profile.websites){ db.users.update({$set: {"profile.blurb": user.profile.blurb + '\n\t - ' + w}})} })

[06:35:26] <Fandekasp> hmm I had forgotten the id in the update, but still not there yet

[06:39:23] <Fandekasp> got it, websites $exists: true return no results

[06:42:33] <Fandekasp> and for loop don't seem to work

[06:52:54] <rohitksh> Hey Guys, can anyone help me with sharding?

[06:52:55] <rohitksh> http://stackoverflow.com/questions/19950668/how-to-store-specific-data-in-particular-shard-and-the-rest-of-the-data-in-other

[10:43:44] <jdar> I just noticed that DB#addUser(…) / DB#removeUser(…) of the mongo-java-driver (2.11.3) doesn't work anymore with Mongo 2.5.3

[10:44:12] <jdar> Can anyone confirm this?

[10:46:55] <jdar> Here is a short gist that demonstrates the problem: https://gist.github.com/thomasdarimont/7464756

[10:47:03] <jdar> db.addUser(USERNAME, PASSWORD); // Doesn't work

[10:48:09] <jdar> I've noticed the addUser API changed a bit on the mongo side since 2.4.6 (it now demands a json structure with the user information)

[10:48:13] <jdar> so if I do

[10:48:15] <jdar> db.eval("db.addUser( { user: 'testadm', pwd: 'testadm', roles: [ 'readWrite', 'dbAdmin' ] } )"); // works

[10:48:26] <jdar> then i can authenticate as expected.

[10:48:49] <jdar> via db.authenticate("testadm","testadm")

[10:48:58] <jdar> should I file an issue for this?

[10:51:01] <kali> jdar: start by looking if it's not filed already :)

[10:52:42] <kali> jdar: have you identified what's the first version where it does not work ?

[10:54:12] <kali> jdar: iirc there were intended and documented changes on addUser between 2.2 and 2.4, so it may be just that

[10:58:24] <jdar> thx kali - I found this one as well: https://jira.mongodb.org/browse/JAVA-1039?jql=project%20%3D%20JAVA%20AND%20text%20~%20%22addUser%22

[10:58:28] <jdar> Deprecate DB.addUser and DB.removeUser

[10:59:18] <jdar> the problem is there a way to add a user that works on 2.5.3+ and older versions?

[10:59:47] <kali> jdar: check out JAVA-909

[11:00:06] <jdar> I just realized that this wasn't working anymore due to some tests failing after setting up the version in a test env.

[11:00:13] <jdar> ok mom

[11:01:38] <jdar> does this mean that a simple driver update to the upcoming 2.12.0 should do the trick under the covers?

[11:02:21] <jdar> will this new driver also work with older mongodb versions?

[12:36:20] <movedx> I have been handed a MongoDB cluster, it's two replication sets consisting of three shards each. I am trying to determine if the databases are being sharded and if so, what is the sharding configuration of each one. I'm logged into the router and I've executed 'sh.status()'. It's telling me "partitioned: false" for a lot (99%) of the databases. Does this flag represent the sharded nature of that particular database?

[12:39:04] <movedx> Right I am guessing it does, as the "test" database, as it's been named, has shard key information and chunk information.

[12:49:59] <kali> movedx: two replica set of three shards ? really ? :)

[13:19:37] <movedx> I don'tknow mate. Don;'t be a smart arse? :)

[13:27:18] <Nodex> I can safely say that you probably won't get help if you abuse people

[14:05:58] <movedx> kali: I apologise. I'm having a bad day. Thanks for your input. Clearing your language means I am lacking some understanding somewhere. I'll work it out.

[14:06:42] <dab> using mongo with PHP - is there any way to get the cursorID and then reconnect to it with a second instance? the reason is if i have a script running that takes a long time, i can have it run in batches and not have to rebuild the cursor which takes a while with sorting

[14:07:08] <kali> movedx: that was my point, really. if you're confused what's where in replica set and shards, your first stop is sharding and replication overview documentation pages

[14:11:13] <Nodex> dab, I dont think so

[14:12:26] <movedx> kali: Would I be more correct to say I have three shards in a replication set, in a cluster of two replication sets?

[14:13:03] <dab> Nodex: i was looking into some of the docs, and it appears at the low level it might be possible, but that the commands are missing from the php driver - i think it may have been implemented in java here : https://jira.mongodb.org/browse/JAVA-178

[14:13:16] <movedx> Our setup here seems to be: {0,1,2} <-> {3,4,5} where the numbers are shards.

[14:13:52] <movedx> ANd {0} and {3} are the primary shards in each set.

[14:25:07] <kali> movedx: nope, you're still confusing :)

[14:25:20] <kali> 1 shard is (usually) a replica set

[14:25:49] <kali> my guess is you have two shards, each is a replica set containing three replicas

[14:32:48] <movedx> Yes, that's it. I think I am confusing a "shard." We have three replica sets in a shard, aye.

[14:33:12] <movedx> So I assume the shards are also in sync somehow?

[14:42:02] <kali> man, read the doc

[14:52:47] <movedx> Thnaks for your patience.

[14:52:57] <movedx> Although, I can see why most people are shifting FROM MongoDB :)

[14:53:19] <Nodex> and why is that?

[14:55:58] <svetlins> Hi guys, in the development version of MongoDB(v2.5.4-pre-) a query of the type db.stuffs.update({"a" : 1}, {"$push" : {"b" : {"1.5" : 1}}}) no longer works. It’s printing "1.5 is not valid for storage." in the mongo shell and silently failing when used from ruby driver(moped). It works on 2.4.7 and all older versions we've used. Does anybody know if that’s the expected behaviour or should I add a bug to the JIRA?

[14:56:46] <Nodex> I would imageine the dot is the causing factor

[14:57:52] <Nodex> -e

[14:57:59] <movedx> Do you lot run the mongos instance directly on your application servers, or as separate, standalone systems?

[15:07:24] <svetlins> Nodex, that's true - no dot, no problem, but our application stores objects in an array which are supposed to have "float" keys

[15:07:55] <svetlins> we could change that but we have a lot of data already that will need to be migrated

[15:09:22] <movedx> Hmm, if it takes ten seconds for a replica set to find a new primary, if not longer (because that doesn't include the time to vote and promote), then what happens to the incoming write operations? Does Mongo simply return a failure notification?

[15:10:45] <Nodex> svetlins : that really was a bad design decision, tbh I am surprised the driver allowed it this long

[15:10:53] <svetlins> yeah, I agree

[15:12:48] <Nodex> at some point you will have to migrate as I am sure that will make it into production code in the near future

[15:13:24] <svetlins> yeah, I guess so. thanks for the advice

[15:13:51] <Nodex> nothing to say you can't run your code on the current version and migrate slowly

[15:28:10] <Honeyman> Hello. Any good docs or hints available on the optimization of aggregation queries? E.g., if I want to sum the values of a field in multiple documents, is it worthy to do [$match, $project, $group($sum)], to decrease the total byteflow in the pipeline, or is it enough to do [$match, $group($sum)] ?

[16:08:48] <movedx> Ah, now that I've made a quick Visio diagram of our infrastructure, it makes more sense to me now.

[16:09:18] <movedx> Still, trying to pull a sub-document from another document is a pain to understand.

[17:29:08] <kas84> hi

[17:29:50] <kas84> I am having performance issues with mongo on my mac and want to upgrade the number of maxfiles that can be open

[17:29:56] <kas84> anybody knows how?

[17:30:02] <kas84> I'm running mavericks

[17:30:16] <retran> what kind of use case did you run into performance issues

[17:30:42] <retran> that's nuts, i havnt had performance issues even running on a 512MB ram server (with a 20GB sized DB)

[17:30:56] <kas84> i am inserting things to the db and querying at the same time

[17:31:09] <kas84> I am doing lots of inserts each second

[17:31:55] <kas84> anyway, I'd like to try to upgrade the maxfiles option

[17:32:07] <kas84> cause I am getting a warning each time I enter mongo console

[17:37:11] <retran> that's really interesting

[17:37:18] <retran> i guess you're keeping the warning a secret from the chan?

[18:47:29] <hkar> Hello, does anyone know, what is size limit of collection (and max number of collections)? Thanks

[18:50:54] <bjori> hkar: roughly a gazillion

[18:51:00] <bjori> hkar: plus/minus few million

[18:51:06] <bjori> hkar: what are you trying to do?

[18:55:01] <hkar> Goopyo: Thanks for answer! I need a large database with users and for each user around 20,000 short entries...

[18:55:43] <bjori> hkar: "short entries".. meaning collections or documents?

[18:57:37] <hkar> bjori: meanig documents

[18:57:47] <Goopyo> I was pingged accidentally right?

[19:01:06] <hkar> Yes, sorry..

[19:02:22] <Goopyo> no problem. hkar mongodb is a document database so it makes most sense typically to put those 20k files in the user document (up to 16mb), might make sense to go to something relational if its they are being stored and related seperately

[19:02:31] <Goopyo> those 20k entries*

[19:06:34] <hkar> Thanks, I thing this is best way for it (document for every user). Thanks a lot!

[19:08:44] <Goopyo> in mongodb it most definitely is. No problem

[19:09:01] <hkar> * the data isn's good for using relational DB.. always have a slightly different structure

[19:42:48] <bjori> hkonkhon: 20.000 document is very low volume for mongodb, you have nothing to worry about :)

[19:42:58] <bjori> whops.. hkar I mean... guess he left

[19:43:26] <bjori> and Goopyo already answered that - thanks! .)

[19:43:27] <bjori> :)

[19:45:20] <gavitron> so let's say I have ~ 40GB of data on a replica set, and one member was offline long enough to go stale. (RS102). is there any reason I can't rsync from a non-stale replica? (as opposed to a full copy or snapshot of same replica)

[19:51:50] <yodude> whom can i ask a few questions about mongodb in private?

[19:51:59] <yodude> who*

[19:59:12] <gavitron> why can't you ask a mongodb question in public?

[20:48:09] <alnewkirk> it puts the lotion on its skin

[20:50:33] <joannac> gavitron: you can only rsync if the files aren't being written to while the rsync runs.

[21:38:39] <hkonkhon> hkonhkon

[21:38:43] <hkonkhon> hkonhkon

[22:03:24] <Rashad> Hello.

[22:03:49] <Rashad> I want to make a website like omegle.com using node.js and socket.io, and perhaps also MongoDB as the database.

[22:04:25] <Rashad> The db is going to store data like names, profile ids, birthdays, and conversations.

[22:04:51] <Rashad> Some of these will have to be stored in Arabic.

[22:05:17] <Rashad> So does MongoDB have known issues in supporting arabic, or whether it supports it at all?

[22:08:11] <Rashad> My second question is: does MongoDB require more or less disk space to store the same data than MySQL?

[22:09:10] <Rashad> If my questions seem dumb or I seem to be asking the wrong questions then please pardon my newbiness.

[22:09:59] <Rashad> However I would still need to know if that is the case.

[22:10:00] <Rashad> Thanks.

[22:11:39] <iBotPeaches> Rashad: MongoDB is utf8 out of the box

[22:12:48] <Rashad> Ok.

[22:13:36] <Rashad> My second question really stems from me not knowing what are the advantages and disadvantages of MongoDB over MySQL.

[22:13:51] <iBotPeaches> well that is a big problem.

[22:14:11] <iBotPeaches> I would start reading the tons of articles online of difference between relational and nosql

[22:15:10] <Rashad> Hmm.

[22:18:58] <Rashad> The thing is that I would really like if I could decide whether MongoDB is going to be at least safe for that kind of app, without having to go into the debate between relational and nosql.

[22:20:08] <Rashad> Do you think that is doable?

[23:51:40] <zymogens> I see that the syntax for $gt is {field: {$gt: value} } - Just wondering can value be another field ... i.e. show docs where 'dueDate' is less than 'now'?

[23:52:01] <zymogens> sorry not 'now' but another saved date.