Log file Viewer
#mongodb logs for Friday the 14th of September, 2012
[00:08:03] <_m> wzlwzl: Systems guy says he brings up a new node, gets the data in sync, then adds that node as the master
[00:09:22] <_m> Then make it primary and add the empty secondary (which will be synced automagically)
[00:09:50] <_m> He says that's the easiest/most painless way. Obviously, YMMV. Sorry I couldn't provide more information than that.
[00:17:41] <aboudreault> _m, in mongodb, for my example use case, would you create one collection per user?
[00:21:23] <aboudreault> this is a feature of mongodb? haven't see it yet. Currently reading the doc online
[00:22:54] <aboudreault> yeah, that's how I see it. However, in mondb, I do not have pretty foreign key to link with the user.
[00:22:55] <_m> Each user could then have an array with document information stored. selectors on said array are pretty easy to understand/use and fairly performant.
[00:25:14] <_m> Should give you more insight on how arrays within a document work. Which should provide some insight into document design
[00:29:01] <aboudreault> _m, I see ok... any string or value can implicitly be a foreign key if we want
[00:29:15] <_m> aboudreault: Need to head-down for a bit. Will try to answer more questions in a bit.
[00:30:00] <_m> Vile: In cases where I want to store user documents for their retrieval, having no internets is going to mean a lack of my service.
[00:30:37] <_m> Also, I don't have a DC unless you count our Rackspace Cloud as a DC (i wouldn't recommend their product, btw)
[00:32:02] <aboudreault> Vile, yes, very important. I also test my query execution time when developing. This is the same thing for mongodb with explain.
[00:32:08] <_m> As I stated before, *my* use cases and experience tend to lean toward "this is easier to let s3 handle." I can see the usefulness of other techniques and would implement them if my stack leaned to that use-case
[00:32:42] <aboudreault> Vile, that's exactly what I need. user uploaded file. video + images. So GridFS was very nice. still need to read the doc though
[00:33:49] <aboudreault> _m, yeah, S3 is probably a very good solution too. never worked with it. You probably just put a file unique uuid in your mongo documents?
[00:35:00] <aboudreault> and don't have any server, space disk to maintain. so yeah that something to think about.
[01:43:22] <skiz__> I'm trying to set up a single shard on 1 machine with a router/config set up on another (for testing purposes) the config/router seems to be working fine, but when I attempt to add a shard ( which has shardsvr=true ) I get "errmsg" : "couldn't connect to new shard mongos connectionpool: connect failed
[01:43:52] <skiz__> there are no firewall issues, using admin db via the mongos, and I can connect to it remotely on the mongos shard port
[01:46:22] <Dr{Who}> we tried phpmoadmin and it ended up modifying a lot of our collections and we a fun day of fixing stuff. Does anyone know if RockMongo has this same problem?
[02:44:52] <ojon_> http://stackoverflow.com/questions/12417425/efficiency-of-mongodb-mongoskin-access-by-multiple-modules-approach
[03:02:53] <IQrow> Thanks for the welcome, I'm only here for just a second, was making sure this little chat room existed.
[03:14:16] <CannedCorn> hey guys would appreciate any feedback people have to this: http://tylerbrock.github.com/mongo-hacker/
[03:14:37] <CannedCorn> attempting to make the shell more usable, any good ideas will eventually make it into the shell proper
[04:22:04] <Gavilan2> hi! Is there any way to simulate transactions or some kind of atomic things across different documents?
[04:49:56] <tomlikestorock> is there a way to remove a member of a replica set without incurring the remaining members being resynced to?
[10:22:24] <gigo1980> how can i make repareDatabase on an slave node, that it does not block the hole system ?
[11:12:48] <kali> gigo1980: IIRC, a repair on a slave switch it to recovery mode, so the queries are routed to other nodes
[11:16:20] <kali> gigo1980: it might be more practical to just ditch the secondary data and let it do a full sync (this is how we deal with broken secondaries here)
[12:08:18] <typecast> there is some unexpected behavior in one step of my aggregation pipeline that I don't understand. I hope you can give me some pointers on this one
[12:08:47] <typecast> { '$project': { 'bucket': { '$divide' : [ "$first", int(self.bucket_interval) ] }
[12:09:18] <typecast> so what I would expect from the documentation is that bucket is also an integer
[12:09:25] <typecast> at least, that's what I read from http://mongodb.onconfluence.com/pages/viewpage.action?pageId=38207860#AggregationFramework-ExpressionReference-ArithmeticOperators
[12:11:16] <jmar777> typecast: what makes you expect to get an integer? i don't see that specified in the documentation?
[12:11:38] <jmar777> "takes an array containing a pair of numbers and returns the value of the first number divided by the second number."
[12:15:47] <typecast> so, I'm now wondering whether this is normal behaviour, a bug in mongo or a bug in pymongo
[12:20:18] <Gargoyle> is passing safe = true as an option to update() using $set the same as a full document update? eg. makes sure that there are no errors at the expense of application speed?
[12:23:06] <NodeX> I think the default is at least 1 node but you can configure that... if you're in a 1 node system it will sync it to disk
[12:23:33] <NodeX> (It posisbly syncs to disk in a multi node system on at least one node too I would imagine
[12:24:01] <Gargoyle> if two lines of code call update(blah blah, $set => etc) very quickly, is it possible that they would get run in different order on the server?
[13:10:22] <NodeX> Gargoyle : there is a lock but there is no garuntee which one would get saved / updated first - this depends on the latency
[13:12:58] <NodeX> http://www.mongodb.org/display/DOCS/Atomic+Operations#AtomicOperations-%22UpdateifCurrent%22
[13:17:21] <Gargoyle> It's not really a transaction. issue. Just more I need to think a bit more about my app logic.
[13:53:32] <gigo1980> is it posible that one process write data to an sharded cluster with mongo router a, and an other process will read that data from an different mongo router b. is it posible that there can be an data inconsistent
[14:05:05] <USvER> For example i have object like {"_id" : "497ce96f395f2f052a494fd4". users: ["Abby", "Leo"]}
[14:05:37] <termite> USvER: For example i have object like {"_id" : "497ce96f395f2f052a494fd4". users: {$in:["Abby", "Leo"}]}
[14:57:13] <termite> Odd question has anyone know if you can use $in with _id to search for a list of docs
[14:59:52] <termite> if I use {"_id:{"$in":["23wer342wrewfds", "rfw5wfsfsfssfss"]}} returns an empty set every time(ids are just examples)
[15:05:15] <termite> db.targets.find({"_id":{"$in":{["50528097547d7dbc6a000001"]}},"locs.loc":{"$within":{"$center":[[-77.585101,38.283524],100]}}})
[15:06:24] <termite> db.targets.find({"_id" : {$in:["501d9acd0f33b0475c000000"]},"locs.loc":{"$within":{"$center":[[-77.585101,38.283524],100]}}})
[15:08:41] <termite> I am looking for records in a list that also are within a certain radius of position x,y
[15:11:18] <remonvv> Does anyone know if a cursor can time out while iterating over it? Or does it just time out if getmore is not invoked for a long time?
[15:15:52] <termite> kchodorow:tried that but the node.js driver just converts it to a string and it fails
[15:16:09] <remonvv> if(timeItTakesTo(ANSWER_QUESTION) <= timeItTakesTo(ASK_WHY_QUESTION_IS_RELEVANT)) answerQuestion();
[15:18:20] <kchodorow> termite: i'm not familiar with the node.js driver, that's just how it would be in the shell
[15:19:33] <rguillebert> kchodorow, so if the fields of the objects are identical they will be reduced ?
[15:27:04] <kchodorow> rguillebert: it probably has to be an exact match, i.e., {x:1,y:1} won't be the same as {y:1,x:1}
[15:33:17] <remonvv> kchodorow, i still think that's a bit of a bug btw. It's value equality that matters really.
[15:39:47] <andywdc> but i have a collection called users, a collection called items, and i want to make a new collection which stores which items a user has - is that how we are meant to do it in mongodb?
[15:42:44] <Vile> if you have relatively limited amount of items per user - just create an array of the item _id's for user
[15:46:56] <andywdc> well presumably you can do it with mongo...i mean what are the typical use cases for mongo in that case?
[15:47:38] <termite> andywdc: As suggested before just create one collection with documents like {user:frank, items: []}
[15:49:55] <termite> andywdc: Mongo is fast because it is document based and not relational. If your data needs require allot of joins then Mongo isn't a good idea
[15:51:08] <termite> andywdc: mongo would be perfect for receipts. Since the information is not going to change
[15:51:08] <andywdc> sooo if i wanted to store all the items in my factory and never do anything els with the data - then mongodb suffices
[15:52:34] <termite> andywdc: remember that each document has a maximum size so if you have the possibility of infinite orders you are going to slam into that limit
[15:53:50] <termite> andywdc: it's not supposed to be complicated. Using it to store item data for really fast retrieval is a great idea
[15:56:49] <remonvv> kchodorow, no, the thing that in same situations for MongoDB {x:1, y:1} isn't equal to {y:1, x:1}
[15:59:01] <skiz__> I'm working from http://www.mongodb.org/display/DOCS/Sharding+Limits#ShardingLimits-Stepstoshardanexistingcollection (along with 20 other refs) however #7 never seems to happen by itself. is there an easy way to kick it off?
[15:59:46] <noordung> I need to be able to store large amounts of text in a document, and that text may change significantly between saves. What would be the best way to design the schema for that kind of documents? Should I resort to GridFS?
[16:03:42] <noordung> gigo1980, I'm slightly more concerned on relocations that MongoDB would have to make on saves, rather than the limit...
[16:05:57] <noordung> gigo1980, that is an option high on the list, but I was thinking on something more integrated... 16MB should be enough for my 'large' needs... at least initially...
[16:07:28] <noordung> gigo1980, It's just that between writes, Mongo may need to deal with size jumps of megabytes... Say 1MB of text becomes 4MB... Those can be expensive, from what I know...
[16:12:36] <noordung> IAD, Just a question... GridFS is pre-configured to avoid size jumps, correct? It uses the 256k chunks?
[16:16:16] <anthezium> hey i have a collection with lotsa documents (like 5mm) and i'm trying to export it to json, but mongoexport seems content to export 0 records when i run it: https://gist.github.com/3722945
[16:17:52] <wwilkins> anthezium: try mongodump perhaps? or maybe you can use skip and next in the export query option.
[16:18:22] <anthezium> mongodump will only give me bson, right? how would i use skip and next in this situation?
[16:19:09] <wwilkins> anthezium: I think the export has a size limit, so get it to pump out enough records to get under that limit and loop through the whole collection.
[16:21:50] <anthezium> wwilkins: how do i bake those into a query? i can only find examples of skip using the js driver
[16:22:48] <IAD> noordung: so, it looks like not important. "mongod will only use the space it really use. There is no need to set " https://groups.google.com/forum/?fromgroups=#!topic/mongodb-user/hrRlhOwGWWk
[16:23:53] <anthezium> yeah skip and limit are both driver-level ideas, can't be expressed in query language. query language can only do like $orderby, $hint, $explain, etc.
[16:25:12] <anthezium> i guess i could just use mongodump and figure out how to convert the bson to json
[16:27:37] <IAD> noordung: nginx + nginx-gridfs look good http://www.coffeepowered.net/2010/02/17/serving-files-out-of-gridfs/
[16:29:00] <NodeX> [17:23:24] <NodeX> if you have timestamped data just workout the intersects and -q ranges
[16:36:30] <noordung> Since GridFS can be used as a versioned filestore, how does it handle different versions? Are chunks reused?
[16:57:04] <skiz__> can someone give me a hand trying to kick off existing data on a shard to split to another. the chunks look good, but everything is still on the first shard. how can I get it to begin migrating?
[20:06:40] <jiffe98> I've setup a new instance of mongodb on a server, when I connect to it via --host of 127.0.0.1 or its IP it works fine but if I connect to it by hostname it times out
[20:17:53] <jiffe98> I have 2 machines that I am eventually going to replicate between, one works the other does not and I'm not seeing a difference
[20:23:36] <jiffe98> I can also connect to the other machine from the non-working machine, I just can't connect to the local instance of mongodb
[21:09:52] <g-hennux> is there a plausible reason why two of my three mongodb users can't authenticate any more from one day to the next?
[21:10:48] <g-hennux> the log says "auth: couldn't find user admin" – this worked fine just a couple of days ago
[21:13:48] <noordung> GridFS isn't actually something *native* (per-se) to MongoDB, it is just an API over the actual MongoDB documents and collections, correct?
[21:24:46] <noordung> kchodorow, so nothing is stopping me from implementing my own version of a GridFS-like system... :)
[21:31:25] <tomlikestorock> I'm trying to add a new member to my replset, and I can't successfully do it. I keep seeing this in the logs: ERROR: error processing ttl for db: mydbname 10065 invalid parameter: expected an object ()
[21:39:42] <tomlikestorock> I guess I don't fully understand the procedure for adding members to the replica sets with auth turned on. :(
[21:40:54] <tomlikestorock> I set the config to use auth, use the key file, and set the replSet. Then I bring up mongo on the new box. Hop over to the primary, and run rs.add("newhostname:27017")
[21:42:37] <kchodorow> tomlikestorock: can you pastebin the errmsg from running rs.add and the log from the primary?
[21:45:50] <tomlikestorock> kchodorow: just to be clear, I don't need to add any other users to my new replica box when I want to add it to the set, right? I just add my admin user for my own purposes, then go to the primary and say to add?
[21:46:17] <doubletap> i use the form as described in the documentation but i get the errors that the options i am using (-u, -p, or —username, —password) are not valid.
[21:53:27] <doubletap> it is odd because the docs have options that mongo does not show me when i just type "mongo"
[21:56:34] <doubletap> anyone know why my instance of mongo has different options than what is in the docs?
[21:59:02] <jiffe98> anyone know why I could connect to a mongodb server via 127.0.0.1 and the external IP but not hostname rather than a dns problem?
[21:59:25] <jiffe98> I can resolve the hostname fine from the local machine but it times out when trying to connect
[22:26:08] <jiffe98> but then show dbs gives me 'Fri Sep 14 16:25:46 ReferenceError: db is not defined src/mongo/shell/utils.js:1475'
[23:43:55] <statim> is it possible to do a query using the properties of the document itself? for example, a document with cleared_at: 1234, last_message_at: 5678, and id want to run a query returning documents that have cleared_at < last_message_at