Log file Viewer
#mongodb logs for Tuesday the 30th of April, 2013
[00:43:06] <tjmehta> Is there anyway to use mongo's indexes to determine all the values of a particular key by all the documents in a collection.
[02:05:53] <resting> does anyone knows if its possible to expand an array while exporting with mongoexport?
[03:17:00] <spaceinvader> hi, I think I'm having some problems with geoNear: https://gist.github.com/tmenari/5486403. The distance returned is 4km but I think it's more like 2km
[03:40:55] <dukedave> Is there anything I can do at the collection level to define transformations to apply to a path when it's written and retrieved?
[03:41:54] <dukedave> I've trying to represent the same polymorphic doc in two ODMs (Mongoose and Mongoid), but they both insist on using their language's class name
[03:42:51] <dukedave> I'd like to write something ODM agnostic to the doc, but return the class names each ODM expects as two attributes
[03:55:32] <jon_r> dukedave I'm fairly certain mongoid allows you to change the field used to reinstansiate the klass, if it's conflicting with mongoose change it?
[03:59:18] <dukedave> jon_r, ah, well, I've got that far, but it just means that each doc has two klass fields on it, so something like this: { _id: ..., actual: 'data', _mongooseType: 'MyMongooseKlasssName', _mongoidType: 'TheRubyKlassName' }
[03:59:58] <dukedave> I'd like to just store { ... _type: 'KlassName' }, and generate the return types for free
[04:01:23] <dukedave> I looked at monkey patching Mongoid to accept a mapping from string to a Class (and similarly for Mongoose), but they're both pretty hung up on the idea that the String *is* the class name
[04:02:34] <dukedave> But.. What I'm thinking I'll do now, is just use separate collections, and generate the collection name from the klass name
[04:08:23] <dukedave> jon_r, well, my situation is very close to this SO question: http://stackoverflow.com/questions/9172834/choosing-mongodb-collections-structure-for-similar-data-structures
[04:10:15] <dukedave> And we'll be querying only on those two attributes, then iterating through the results
[04:12:23] <dukedave> But now I'm doing some quick testing locally and it seems that the (collection size) cost of storing a _type (or two as appears would be required) for each doc makes it significantly less attractive. I'm using nearly as much space storing the _type as the data :|
[07:00:37] <resting> any idea what could be the problem when a php findOne doesn't return results but the mongoshell does using the same conditions?
[07:19:14] <Aorion> The quick and dirty is that I can connect to the DB, create a collection, and then list collections
[07:20:02] <Aorion> more are such that I can enter data, on what should be a valid collection, and nothing makes it to the server
[07:22:01] <Aorion> The only thing I can think of is that the data is added to the DB before the connection completes
[07:29:40] <Aorion> last time I checked, I wasn't an idiot, so I feel like if there are no errors reported by the library, and the database echoes that my client connected, and there are no errors in the transactions, that the database should have changed state
[07:30:18] <Aorion> there are only so many permuations of the code I can try before this gets ridiculous
[07:35:29] <Aorion> my initial problem was that I use the createCollection method, then collections() and collectionNames() and neither of the latter will return anything
[07:36:03] <Aorion> so if we are opened, and we successfully added a collection, why aren't any of the collection methods working?
[07:37:23] <Aorion> in this code ->http://pastebin.com/uRkd5taR I create the object which causes mongod to echo some initandlisten messages. Then I call the init() method which creates a collection and throws some default entries in there
[07:38:18] <Aorion> the latter was a test after I read on the docs that the names of the db are in the collection name implicitly
[07:39:31] <Aorion> to start, I am under the impression that the datatype of "db.test" in the shell is a collection object for the test collection, is this correct?
[07:41:28] <Nodex> this.db.collection('users_db.users', function(error, user_coll) { ... console.log(user_coll); } ... <--- what does that log
[07:43:24] <Nodex> http://mongodb.github.io/node-mongodb-native/markdown-docs/collections.html <-- perhaps that will work?
[07:44:21] <Aorion> so it happens asynchronously, and I am not sure of the operation for createCollections if the database is not yet opened
[07:46:38] <Aorion> I read the docs, and the docs say, that I should open a connection to the db. The docs also say that I can create a collection with a given name. The docs also say that I can read the collection names using collections() or collectionNames
[07:46:55] <Aorion> the docs also say that errors at any point in that process will be shown by the first parameter of any callback
[07:47:31] <Nodex> my advice is to follow the first part of the last funciton on this page ... http://mongodb.github.io/node-mongodb-native/markdown-docs/insert.html
[07:48:32] <Aorion> very simply, starting at state zero, if I add a collection, I should be able to see that collection
[07:48:33] <Nodex> it works for everyone else but you're somehow the special edge case it doens't work for
[07:49:48] <Aorion> but if I try to do it through node, the connection gets fucked but it doesn't tell me why, so I am left scratching my head
[09:10:30] <ron> Siyfion: he's here. trust me. but... is your question related to mongodb, php, mongodb and php or something completely different?
[09:12:14] <Siyfion> basically I was just wondering what the best / fastest way of copying a load of documents is with a minor change (and change in _id).
[09:13:21] <Siyfion> I need it to create a new product category with the new name specified, then copy all the products in the old category into the new one.
[09:15:40] <Siyfion> ron: Ah I think I've found the answer myself.. basically I just need to query the products to clone, delete their _id's and change their category refs, then insert them all back in again
[09:15:56] <Siyfion> (I'm using mongoose, so that made it slightly harder, but I'll use http://mongoosejs.com/docs/api.html#model_Model.create)
[09:34:47] <ExxKA> Hey Guys. I cant figure out how to group on the day and hour of a timestamp extracted fromt he Id. I can do it from a timestamp field I have created, so I guess I am just missing a small step?
[09:45:40] <BadDesign> ExxKA: You have to contruct the OID manually, the steps are: 1) get a timestamp (uint64_t) for the day and hour you try to build the query for, 2) get a hex representation (uint64_t) of the timestamp 3) construct the OID by converting the hex representation obtained in the previous step to a string and concatenate to it 0000000000000000
[09:46:58] <BadDesign> ExxKA: Then you can query for documents in the collection using something like { \"_id\" : { $gte : { \"$oid\" : \"" + oid1 + "\" }," "$lte : { \"$oid\" : \"" + oid2 + "\" } } }"
[09:48:11] <ExxKA> This query works: db.baskets.aggregate([{ $match: { merchant: ObjectId("517a98d82893817876000002") } }, {$group: { _id: { year: { $year: "$created" }, month: { $month: "$created" }, day: { $dayOfMonth: "$created" }, hour: {$hour: "$created"} }, total: {$sum: "$amount"} } }, {$sort:{_id:1}}])
[09:52:14] <BadDesign> That first 4 bytes from the ObjectID gives you the moment when the document was inserted in the collection
[09:54:45] <diffuse> if i do a batch insert, do the _id's of each document get returned? I am trying to figure out how to retreive this with the C driver.
[09:56:27] <BadDesign> diffuse: no you won't get the id's returned, because the id's should be calculated on the client
[10:03:29] <BadDesign> ExxKA: you should do what I said above in those steps... and { $group: _id : ObjectId("myconstructedoid") }
[10:17:41] <dob_> I am aggregating a collection a using group. I have a field refid and refname. Now I am grouping by refid, but I also want to output refname, but ignore it if it's different and output the first refname found and add it to my group
[10:54:00] <diffuse> how do you get the last insert id of a document with the C driver? Is it possible with mongo_insert or do you have to run mongo_run_command?
[10:55:38] <algernon> (or if the driver does it, and caches it for you, which is not something the C driver does, iirc)
[12:03:51] <tom_2321> hello, the primary node of my mongodb replicaset gets lots of these messages: ... [conn845090] getmore local.oplog.rs query: { ts: .... Is there something wrong with my oploc or the size of my oplog?
[12:42:55] <csengstock> Hi. Is there a low level access to the index structures from within the mongo shell? For example, i want to get the geohash strings for my records (with an attributes indexed by a 2d index). Or even better, i want to query the number of indexed documents, given a particular geohash string. The aim is to use the quadtree structure to efficiently retrieve document distributions over geographic space.
[12:44:06] <csengstock> of course i mean quadtree-like structure. haven't looked for it in depth, but probably the geohash strings are indexed by a btree?
[13:02:06] <aseba> Morning.. I have a question.. I have a mongodb server running. The disk for that server (ubuntu) got full, so I've attached another volume and rsynced all the /var/lib/mongodb to the new volume. I changed the config to load the new directory and ran it again. Everything seemed to be working fine, but now I get "$err" : "Invalid BSONObj size: -286331154 (0xEEEEEEEE) first element: EOO",
[13:05:48] <johnny_fg09384> sometimes mongostats shows me, that one of my mongodb collection is locked over 100%, eg, 105,8%. what does it means?
[13:20:56] <ExxKA> Hey BadDesign. I figured out the problem, using map reduce instead of the aggregation framework. Thanks for your help
[13:30:44] <ExxKA> I used the .getTimestamp() function on the Id to get a Date object I could extract the values from
[14:26:03] <Nodex> we've all heard of cloud computing but has anyone seen this new thing called cloud commuting. It's being pioneered by airlines all over the world
[14:50:16] <Derick> Psi|4ward: 10gen provides consulting: http://www.10gen.com/products/mongodb-consulting
[14:53:42] <Derick> hmm, I was thinking of suggesting to join a MongoDB UG meeting... but not sure if there are any in nürnberg
[14:57:27] <Mortah> hullo. What's the best way to query something that has the same key multiple times? (e.g. the oplog..)
[15:01:11] <Mortah> I'm trying to debug an issue (with our app) and looking at the oplog to see the exact updates made
[15:01:51] <Mortah> unfortunately, mongo shell shows the update as: "o " : { "$set" : { "_version" : 3 }, "$set" : { "_version" : 3 }, "$set" : { "_version" : 3 } } }
[15:02:55] <Mortah> and I can only think its because $set has been specified on the document multiple times... and then somewhere in the client its kindly mangling it
[15:08:11] <Mortah> because depending on which way I query it I get one key or the other (guess it depends on internal ordering of the map)
[15:11:45] <Nodex> anyone know of a neat hipster trick I can apply to a 3 part compound index where I can always query at least 2 of the fields (one is static - cid) and the other 2 vary - (uid, aid), I've been raqkcing my brains but it always results in 2 indexes
[15:15:47] <Nodex> if I mix the aid,uid into an array I can never garuntee order of which is the uid/aid
[15:18:11] <Nodex> my main trouble with sub documents is I put everything in SOLR too and sub docs don't really map back into solor
[15:19:05] <Nodex> Oh well, guess one can't have everything one wants LOL, where would the fun be in that!
[15:20:08] <Mortah> so... if I managed to insert a document into mongo with the same key specified twice {bob: 1, bob:2} I have no way to query for it?
[15:20:09] <Nodex> solr now has awesome json support so it's really close to map mongo-> solr now :D
[15:21:12] <Mortah> as I understood it, bson allows docs with the same key multiple times... so could in theory get one into mongodb
[15:48:16] <Mortah> o2: { _id: ObjectId('517fb43baf6670191daf664c') }, o: { $set: { _version: 3 }, $set: { priority_origin: [ "T" ] }, $set: { updated_date: new Date(1367323738191) } }
[16:05:20] <bartzy> I mean - probably as an int it will be smaller and better for indexing ? or maybe it doesn't matter a lot ?
[16:05:40] <bartzy> Because it will be slightly more comfortable for me to use the hex string in _id , and not its number representation.
[16:57:12] <JoeyJoeJo> Is there a way in the php driver to get the number of results in the cursor without iterating over it first?
[17:00:14] <jeffwhelpley> is it best practice to try and keep documents flat or logically group fields into subdocuments when it makes sense. Or does it not matter at all either way?
[17:55:27] <jok> run 'mongod --rest' and have web interface to DBs and tables with url: localhost:28017/dbName/tableNameand and on localhost:28017/.
[18:13:10] <mfletcher> Im setting up a QA environment. For mongodb, Ive cloned one of our production servers (its a VM), which is part of a replica set. The QA server is going to be a single node, it won't be a part of the production replica set. To ensure this, all I need to do is make sure that the relpSet variable in my mongod.conf is commented out. Is this accurate?
[19:14:38] <dcholth> I'm trying to figure out hte best way to write an API call that mashes multiple collection objects together...
[19:34:25] <dcholth> I think I figured it out... by breaking it down into smaller functions that call a callback, and when all functions finished, it finally sends the response...
[19:52:32] <pwelch> hey everyone, I was wondering if someone could clear something up for me with Replicasets
[19:53:21] <pwelch> I understand a new deployment should use replicasets and not master/slave replication. My question is what happens if a secondary gets a write
[19:59:31] <leifw> pwelch: I know the slave server will protect itself and throw an error, but I don't know how drivers handle that error
[20:00:29] <pwelch> ok, thats what Im confused about. The docs mention writes should go to the primary. but what is the common way everyone is doing that? load-balancer? If it changes then the new primary needs to add itself to the load balancer
[20:01:41] <pwelch> leifw: thanks anyways. I'll hang around for a bit in case some sees the question.
[20:51:00] <wintron_> hey there, any have experience using MongoDB with Vagrant? trying to set up port forwarding between MongoDB on my Vagrant VM and host machine, but can't connect.
[21:07:25] <tphummel> wintron_: a while back i set up a cluster of three vagrant vm's. each with a line like: m.vm.network :hostonly, "192.168.33.10"
[21:10:33] <wintron_> tphummel: hmmm, wonder if should just destroy my vm and startover. can't see why it shouldn't _just work_
[21:15:30] <tphummel> wintron_: is it important that the port be forwarded to the host or just to be able to connect?
[21:15:52] <wintron_> just need it to connect, thought port forwarding would be the easiest way to achieve that
[23:36:28] <Sxan> Are there any well-known strategies for performing counting on collections with moderate write loads?
[23:52:02] <federated_life> you'll want secondaries for many reasons, some of which are scaling reads