Log file Viewer
#mongodb logs for Wednesday the 18th of June, 2014
[00:18:18] <cirwin> if I do something like: db.errors.update({status: null}, {status: "open"}, {multi: true}), is it possible for that to clobber concurrent updates that also set the status?
[00:27:03] <joannac> okay, so I guess the question is, if another update comes in db.errors.update({_id: whatever, status: null} {status: "pending"})
[05:51:40] <alirezataleghani> there are a lots of chunks moved over archive shard! write know but no disk space is released!
[07:21:10] <thomastuts> hey guys, anyone around? i have a question regarding schema modelling for mongoose (on a conceptual level)
[07:21:55] <kali> thomastuts: it will probably get more traction if you can ask it in pure mongo terms
[07:22:36] <thomastuts> well basically i have two collections: users and packages - they have a many-to-many relationship
[07:23:08] <thomastuts> i was thinking of storing users and admins in package.users and package.admins respectively so i know who has which role
[07:23:31] <thomastuts> the problem i'm running into is how do i link this relationship to the users collection
[07:23:53] <thomastuts> should i just use one property, e.g. user.packages that has all packages, regardless of role?
[07:28:46] <thomastuts> then i can populate that object when i retrieve a package and merge the 'user' object together with the role field
[07:29:36] <thomastuts> and you'd store a user's packages like user.packages: [_id, _id, _id, _id], correct?
[07:30:11] <thomastuts> well, when i retrieve a package i would like to populate the users collection with actual user objects instead of just their _id, but that's a mongoose-specific question so it's not really relevant here
[07:30:35] <kali> thomastuts: ho, i get the statement. the role is actually a user system-wide characteristic, not part of the actual user-package relation, right ?
[07:31:43] <kali> thomastuts: that will work, but i would actually do a user.packages: [ { _id: ... }, { _id: ... } ] because it is likely that at some point, you'll need some package property in there too
[07:32:34] <thomastuts> would you recommend storing every collection reference like that? [ { _id: ... }, { _id: ... } ] vs [_id, _id, _id]
[07:33:04] <thomastuts> you'd store them inside an object in the array instead of using the objectid itself, so you can add more metadata to something if it's ever needed, correct?
[07:33:19] <thomastuts> sorry for these basic questions, still getting the hang of mongodb and modelling etc
[07:35:36] <kali> ("no problem" would be more appropriate, but i don't think i would pass turing's test this morning)
[07:40:43] <thomastuts> kali: right, i'll let all this sink in a little and then see what i can come up with :) thanks a bunch for the feedback, really appreciate it!
[11:32:18] <bcsantos> i'm trying to get carrierwave uploader to work in padrino with mongoid + gridfs.
[11:32:54] <bcsantos> getting "NoMethodError at /admin/artworks/update/53a0eedcf2c7961066000002 undefined method `bson_dump' for # file: hash.rb location: block in bson_dump"
[11:33:14] <bcsantos> more here >> http://stackoverflow.com/questions/24203977/carrierwave-mongoid-gridfs-padrino-admin-image-upload
[11:57:41] <bcsantos> i'm trying to get carrierwave uploader to work in padrino with mongoid + gridfs.
[11:57:44] <bcsantos> getting "NoMethodError at /admin/artworks/update/53a0eedcf2c7961066000002 undefined method `bson_dump' for # file: hash.rb location: block in bson_dump"
[11:57:48] <bcsantos> more here >> http://stackoverflow.com/questions/24203977/carrierwave-mongoid-gridfs-padrino-admin-image-upload
[13:19:06] <richthegeek> basically we have each client in a separate database, and currently all those databases are in the same replica set. Eventually, those servers are gonna run out of space and more importantly, out of processing room
[13:19:40] <richthegeek> so what I was hoping to do was have each database automatically sharded (ie, all collections in that DB live in the same physical location(s))
[13:20:37] <richthegeek> more particularly, although this might be mad, is that each database resides on 2/3 of the servers in the N-server cluster - ie, "Bob" is an AB, "Alice" is on CE, etc...
[13:43:06] <rspijker> richthegeek: you can do that sort of stuff, although probably not automatically
[13:45:26] <richthegeek> rspijker: yeah i think we'd prefer to do it manually so we can balance processing allocation against workload-size
[13:50:18] <rspijker> richthegeek: if you want a single DB to always reside on a single shard, then you could just set the primary shard for each DB and just not enable sharding on any of the collections
[13:50:25] <rspijker> see: http://docs.mongodb.org/manual/core/sharded-cluster-shards/#primary-shard
[13:51:01] <richthegeek> rspijker: i see, and then i guess i could just use MMS for the backup (which is what the RS is used for at the moment)
[13:51:39] <rspijker> richthegeek: for backup, sure… but the RS is used for more than backup… it’s used for failover…
[13:52:45] <richthegeek> rspijker: at the moment the db lives on the same server as the processing engine (it's a web analytics platform) so if the db goes down then it doesn't make much difference
[13:54:03] <richthegeek> rspijker: presumably in future we'd keep it that way, as the database storage size seems to be less of a restriction than the CPU usage, so failover is handled further up the chain (transactional event submission through the foremost API)
[13:54:08] <rspijker> well… Then all you need to do is decide for yourself what backups are for. Disaster recovery, rollback in time, … ?
[13:54:47] <richthegeek> at the moment, disaster recovery (ie, the main server melts) .. ideally rollback in the future which is something more complex than just replication delay
[13:55:50] <richthegeek> might be getting to the point where we hire an actual devop with mongo experience....
[13:58:17] <rspijker> well, those are the types of consideratinos that should drive the decision on whether you need a RS, whether you need to do backups only through an RS/MMS, or also with FS snapshots, etc.
[14:00:41] <richthegeek> well we've got a few months breathing room before it becomes something we seriously need to consider, and we can always just scale the servers up ... thanks for the help!
[14:06:18] <Skunkwaffle> I know I can modify settings using ini_set(), but there doesn't seem to be an ini_get() function. Is there any way to check the value of a setting from the mongo shell?
[14:39:50] <jayesh> was using this http://satishgandham.com/2012/09/a-complete-guide-to-integrating-mongodb-with-elastic-search/#comment-3626
[14:42:13] <jayesh> but when i do curl -XGET 'http://localhost:9200/mongoindex/_search?q=firstName:John'
[14:42:46] <jayesh> it gives me {"took":9,"timed_out":false,"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":0,"max_score":null,"hits":[]}}
[14:43:35] <jayesh> {"took":9,"timed_out":false,"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":0,"max_score":null,"hits":[]}}
[15:54:56] <vonnieda> Hi folks. I have some confusion on when/how clients can read from secondaries. I have a 3 member replset. Client is connecting using all three in the connection string and slaveOk=false. If I remove the master from the connection string it still reads but if I then remove one of the secondaries it fails with "not master and slaveOk=false". I would have thought it would fail with that after removing just the primary. Can someone tell me why?
[16:05:39] <elfotografo007> when you remove a primary in a 3 member replica set, there is majority (2 of 3)
[16:06:36] <elfotografo007> and the primary, alone, detects that there is no majority, so it will step down as a secondary to prevent rollback
[16:08:59] <elfotografo007> you can find this information here: http://docs.mongodb.org/manual/core/replica-set-elections/
[16:09:36] <vonnieda> elfotografo007: Sorry, I was not clear. I am talking about connecting to a replset from a client. I am confused as to when/why a client will read from a secondary.
[16:11:21] <vonnieda> elfotografo007: Correct, but that is not the behavior I am seeing. I am able to do reads if I specify two secondaries in the connection string.
[16:12:23] <vonnieda> elfotografo007: Simply by connecting from a client? I'm not reconfiguring the replset, just changing the connection string on the client.
[16:13:41] <sjimi> Is it possible to select documents and filter them so you don't have duplicates for a certain key in those documents? If I did not explain meself good enough please say so. I'll try to elaborate.
[16:14:12] <ernetas> Is there anything I need to know before deploying a second mongos to our cluster?
[16:14:21] <vonnieda> elfotografo007: I'm sorry, I don't follow. It sounds like you are saying it reconfigures the replication set, but I am not making any changes on the server - just changing the connection string on the client.
[16:15:49] <elfotografo007> when connecting a client to a replicaSet, you could include all replica set members in the connection string, or you could just include one member of the replicaSet
[16:16:09] <elfotografo007> then, when the client connects to one member of the set, it asks for the entire list of members
[16:18:31] <elfotografo007> with the list of serves, the client determines which server is the primary, and then connect to it
[16:20:46] <vonnieda> Okay, I understand and agree with all that. I am making a pastebin to show you where I am confused. One second.
[16:21:29] <talbott> if there anything else i need to do other than move the files and set the new dbpath
[16:23:13] <vonnieda> elfotografo007: This illustrates the problem I am having: http://pastebin.com/DRQAAwu7
[16:23:55] <vonnieda> elfotografo007: If what you are saying is correct, in the last two cases it should read the config from the secondary and then connect to the primary, right?
[16:29:31] <elfotografo007> vonnieda: what's the output of running db.isMaster() in the mongo console?
[16:43:31] <sjimi> I would like to filter out only one occurrence of these (for example the last one).
[16:45:19] <sjimi> So for example when you have 6 documents in total. With three {'data' : { 'id': 1, ... }, ...} like this. And three {'data' : { 'id': 2, ... }, ...} like this.
[16:47:58] <Jinsa> maybe by filtering: you order your data by DESC to get the last one and then you apply a array_unique to get only one of them ;)
[16:50:28] <sjimi> saml: if you do what you say you only have one document remaining. Where I want to have two documents.
[16:50:55] <sjimi> The solution of my example should be two document: {'data' : { 'id': 1, ... }, ...} and {'data' : { 'id': 2, ... }, ...}
[16:57:51] <gh3go> Hi, I need a little info, mongodb is using any createdAt updateAt system field to keep tracking of create/update of an entry? I noticed that prob it has create date but not update, is this correct?
[16:58:11] <gh3go> Or updating a field will update the createdate with the timestamp of that very moment?
[17:13:01] <saml> var groups = {}; db.docs.forEach(function(doc) { if (!groups[doc.data.id]) { groups[doc.data.id] = doc; })
[17:51:25] <joshua> Is there a good way to test out a field or two to see how ordinal the data is? I want to come up with some ideas for shard keys but would be cool to verify and compare somehow
[17:54:36] <joshua> Maybe I should get a dump of the data and play with it in a sandbox. I could try some aggregation and group things and see the counts... kinda simulate chunks that way
[17:55:41] <joshua> The devs are clueless so we have unsharded data building up on different shards. heh
[19:29:39] <Jinsa> just to be sure: the full text search feature (using command("text", {search:"..."})) will be discarded right?
[20:21:52] <djlee> Hi all,is 1 - 1.5 seconds a reasonable time for the mongo php driver to get a connection from the pool manager. I would have thought that it would be 0.2/0.3 seconds at the most?
[20:39:24] <djlee> tscanausa: i think its MongoHQ, i believe the URI's they give you to use, aren't the most optimum (or at least not for PHP). I've found that if i use the URI's from their status page the connection is instant after first one, so im guessing theres some overhead in resolving the connection every time you connect using their recommended URI's. However i'm emailing them to confirm that im safe to use the other URI's
[20:40:51] <tscanausa> if you are connecting really far away or network issues that could be the case but not nessesrily allways the case
[23:36:26] <justinsd> I am having an issue with mongo where each time I run it with a config script, the config script is ignored and it continued where it left on to use the old replica set data.
[23:51:15] <Kaiju> I'm playing with map reduce for reporting and I've got the hang of the process. I'm wanting to do some complex processing with user records in both the mapping phase and reducing phase. Since the code is shipped into the cluster for processing I'm not sure how including functions into each stage of the mapreduce process works. Does anyone have a good resource or insight on this topic?