Log file Viewer
#mongodb logs for Wednesday the 30th of January, 2013
[00:32:21] <Moussekateer> I'm having an issue using pymongo. I have a field with an integer zero value but pymongo returns zero results when I do find({'field': 0}). How do I get round this
[01:36:12] <carlzulauf> Given a collection containing values like 'changes: {"rent": [125, 150]}' ... how can I find documents where rent[0] is 125, but rent[1] is not ?
[02:03:31] <TimRiker> using the java mongo driver, trying to connect with a mongouri. I keep getting a number parse error. does this work for anyone?
[04:32:39] <quintus> hey, would anyone know how query the keywords field for the value "word2"? Example: {article: name="article", keywords=["word1", "word2", "word3"]}
[07:10:31] <Hirzu> Hi, does someone know how a total newbie could find docs to help do following: I have a document in Mongo and this document contains collection of key value pairs. I have a set of key value pairs that should be sent to the database and then those key value pairs should be matched with the ones stored in the database + averages counted for those values.
[08:03:50] <svm_invictvs> IN other words, would it be bad practice to repeatedly create an instance of DBCollection frequently or for just one or two method calls?
[08:11:13] <Hirzu> Hi, does someone know how a total newbie could find docs to help do following: I have a document in Mongo and this document contains collection of key value pairs. I have a set of key value pairs that should be sent to the database and then those key value pairs should be matched with the ones stored in the database + averages counted for those values.
[08:12:29] <ron> You should be able to do that with either map/reduce or probably better yet, the aggregation framework.
[08:14:06] <ron> http://cookbook.mongodb.org/ and http://docs.mongodb.org/manual/applications/aggregation/
[08:14:17] <Hirzu> I've not been able to find anything that explains to me how to do that. I might be just too dumb or something.
[08:14:52] <ron> No, you just need to understand the concept rather than expecting the solution to be laid out.
[08:18:01] <Hirzu> but Mongo actually seems to have a promise for something bit more. The learning curve is just so deep for me
[08:18:38] <daslicht> How would i do this in MongoDb ? How to deal with future updates of separated MongoDB Documents?
[08:19:34] <daslicht> if i keep it all in one place i would have to update all things manually in future ?
[08:24:21] <daslicht> ron, what do you mean by external indexing ? how do you solved it technically :)
[08:25:32] <NodeX> they dont really have anything to do with trying to solve your user/group problem
[08:25:40] <ron> of course you should use them only if you need them. if you can solve everything you need with Mongo's capabilities, that's great. Mongo has some great features in it.
[08:26:42] <ron> they _could_ be related to the user/group problem, depending on the complexity you want to achieve. for a simple use case, you won't need it.
[08:27:25] <NodeX> daslicht : the one thing to remember with Mongo / Most nosql solutions is that ... one size never fits all
[08:27:59] <NodeX> this is the main reason that SQL solutions don't scale - because they try to fit every model
[08:28:43] <daslicht> do nosql tend to have redundant data stored and thererfore tend to take more space ?
[08:29:27] <ron> one of the reasons nosql databases gained traction is that disk storage nowadays is significantly cheaper than it was 10 and 20 years ago.
[08:30:06] <NodeX> for me the advantages are speed, ease of use, scalability, faster development to name a few
[08:31:17] <NodeX> I never ever used SQL with joins or in a relational way when I was using it because I could always see that as a bottleneck so Mongo was a natural fit for my way of programming
[08:33:21] <svm_invictvs> daslicht: yeah, from what I read (never used it) it's basically a filesystem built on top of documents in the database
[08:34:16] <svm_invictvs> daslicht: Whether or not it's the "right" filesystem to suit your needs that's a different story
[08:34:50] <NodeX> daslicht : I use both and gridfs is certainly more portable and infinitely scalable
[08:35:10] <NodeX> plus you can always add a cache in the way and there is no direct link to a file on the FS
[09:05:40] <Aartsie> i got this error: Cannot overwrite `Users` model once compiled. in my node.js script i use mongoose but i don't know how to fix it :( can somebody tell me how to fix this ?
[10:10:43] <megawolt> but i'm getting "Unable to connect to a member of the replica set matching the read preference Primary" Error 1 of 3 times
[10:13:28] <mandark> Hi there, I have a strange behavior here with '$exists' being inserted, is it a known error / bug / misunderstanding : http://pastebin.com/rZZ3wQaq ?
[10:16:59] <mandark> remonvv: It seems to have the correct behavior with {'_id.url': {'$exists': False}, '_id.year': 2012}
[10:17:57] <remonvv> mandark: No, that's not correct behaviour. The reason it goes "wrong" is because the $exists condition in one case fails and thus it becomes an upsert rather than an update.
[10:18:13] <remonvv> If you try db.foo.insert(doc) with your original case you'll see the error this operation should give.
[10:19:07] <mandark> remonvv: Ok so why does it seems to work with {'_id.url': {'$exists': False}, '_id.year': 2012} ? Cause they tempered the bug removing '$exists' at only the root level ?
[10:19:57] <remonvv> mandark, because that query will always evaluate to finding a document if the collection is non=empty
[10:21:20] <remonvv> That last query is basically saying "find all documents where _id:url is not set. I'm assuming at least one document matches that criteria in your database.
[10:22:00] <kali> remonvv: i would say the root cause is the query is invalid: the "matching" part should be either a valid value, either a "matcher" ($elemMatch in this case)
[10:23:04] <remonvv> There is nothing wrong with the query is there? Any query object that find(c) accepts is valid for update(c, u, false, true)
[10:23:52] <remonvv> What way? checking for the existence of an embedded field or doing it on an _id field?
[10:24:17] <remonvv> The latter is useless ofcourse in the case of an upsert but it should definitely throw an error.
[10:24:20] <kali> remonvv: i think the syntax assume the "value" part of the selector is either a "valid value" or a matcher (something starting with $)
[10:25:13] <mandark> Tricky question, how does mongo can work with an upsert with '{foo: {$gt: 42}}' ? It inserts {foo: 43} to match the condition ? ^-^
[10:25:15] <kali> remonvv: if you write theat with an $elemMatch, it becomes correct { _id: { $elemMatch : { year: 2012, url : { $exists: false} } } }
[10:25:16] <Aartsie> Is it better to build a model for a find and a new model for a save or can i create one model for both ?
[10:27:34] <remonvv> kali: $elemMatch is an array operator. That's possibly even more broken if anything.
[10:27:57] <khushildep> Hi Chaps. Trying to do a mongorestore and I get " what(): locale::facet::_S_create_c_locale name not valid" - my locale is all set for en_GB.UTF-8 - anyone come across this?
[10:28:16] <mandark> kali: That's a good solution but that's false as it does not match the query ...
[10:32:10] <remonvv> Trying to think of valid cases where you'd want an _id based query for an upsert.
[10:32:47] <kali> remonvv: the _id is not the problem... it's the fact you're upserting without providing a fully defined document
[10:33:26] <remonvv> kali: As in, can you think of useful scenarios where you'd do update({_id:.., other:..}, {some update}, false, true)?
[10:33:55] <remonvv> kali: I feel if you find yourself doing that you have a schema issue. But that might be my lack of imagination ;)
[10:35:17] <kali> remonvv: i have cases with upsert looking like { prop1 : value1, prop2: value2, ... propN: valueN }, { $inc: { counter: 1 }} with a unique index on prop1... propN
[10:35:25] <remonvv> kali: Perhaps if your application is in a position to set the _id to an appropriately unique value, but meh : update({_id: UUID}, {$set:{name:"Remon", level:"noob"}}, false, true)
[10:36:27] <kali> remonvv: depends, it can be argued i'm wasting time and space maintaining two indexes instead of one
[10:36:28] <mandark> For interested people about this bug, i opened it here : https://jira.mongodb.org/browse/SERVER-8407
[10:37:00] <remonvv> kali: Can it? Same memory usage give or take and the compound index is less flexible than two smaller indexes.
[10:38:13] <kali> remonvv: this is counting hits on web services endpoints according to a collection of criteria, so write intensive use case
[10:38:23] <remonvv> kali: Yeah if all fields in the compound have high cardinality that's your only route.
[10:38:56] <remonvv> kali: Fair enough. All the more reason to know what happens when inside MongoDB so people can pick the appropriate schema and indexes. More should be written about that sort of thing.
[10:40:31] <kali> ho, i think there is enough said and written about his actually. but most people are using mongodb without reading or watching the material
[10:44:33] <remonvv> Hm, I look for new blogs and articles on a weekly basis and most are fairly basic. If you look at all the @10gen posted tweets about new blogs and presentations for example a lot of people are basically rehashing the same stuff.
[10:45:49] <kali> remonvv: i think most people haven't grasped the basics of btree indexing and what are its consequences in terms of prefixing, ranges queries, ordering and so on
[10:46:31] <kali> remonvv: every official mongodb event i have seen included a presentation on this very topy by a 10gen staffer, so... i think they're trying as much as they can
[10:48:15] <remonvv> kali: Fair enough. All the information I gathered on these topics a year or two ago came from webcasts of 10gen presentations.
[10:49:14] <remonvv> kali: Although that's only a relatively small part of the solution. Documentation and whitepapers are more useful references and that still seems to lack a bit. Online manual for example is rather messy, and that's me being charitable.
[10:54:17] <remonvv> Derick: Undoubtedly but I don't think it's moving in the right direction, nor is the portal layout itself very practical.
[10:55:22] <remonvv> Derick: And regardless, developers shouldn't be exposed to the process of documentation and manuals moving from one iteration of it to the next. Should happen internally and released when it's done.
[10:55:40] <remonvv> Derick: It's hard to get really specific but it's a generally frustrating experience to look for specific information.
[10:57:26] <remonvv> Derick: I think the main issue is the structure. The hierarchy seems to be focussed on dividing section in the most appropriate leaf in the documentation tree. That shouldn't be the "entry point".
[11:00:31] <remonvv> Derick: Basically, the documentation is structured in a way that seems to answer "How do we put all information in an as clear as possible hierarchy" rather than "With what questions would people arrive to this page"
[11:01:29] <remonvv> Derick: Basically, people come to such a site with a question like "How do I do X" and the current site doesn't really facilitate answering that question.
[11:03:41] <remonvv> Derick: Example : "DB admin wants to add a shard to a cluster" and "Developer wants to know what an appropriate shard key might be". Two completely unrelated questions and two complete different roles but they'd both click "Sharding" in the menu now.
[11:24:05] <Derick> I don't see why you have *one* document with the _id of blogs? Don't you just want a collection called blogs?
[11:28:26] <megawolt> i'm getting "Unable to connect to a member of the replica set matching the read preference Primary" Error 1 of 3 times.Could anyone help?
[11:32:27] <Derick> megawolt: post your connection code snippet, and the output of "rs.config()" online somewhere
[11:45:41] <megawolt> mongodb://**.**.***.***:27017,**.**.***.***:27017,**.**.***.***:27017/?replicaset=RSNSN&readPreference=primary&connectTimeoutMS=60000
[11:48:31] <megawolt> Derick: I have changed with machine names but in that time i have more connection problem from stage machine
[11:55:27] <Derick> megawolt: I suggest you write to the google groups list (mongodb-user) - I don't think the c# people hang out here and I'm at a loss now
[12:01:18] <Derick> daslicht: sounds like you're fighting against your ORM - I suggest you ditch it :-)
[12:28:11] <haxplorer> I'm trying to build mongo with v8. Mongo version 2.2.1, v8 from trunk. I used the I_know_I_should_build_with_GYP=yes option in v8 to build it with scons. Mongo detected the v8 libraries while configuring. But while compiling, I get this error - http://pastebin.com/AzHTXy31
[12:38:20] <Zelest> If I plan to store a lot of websites in mongodb (as in, raw html files), is a document field or gridfs the preferred way?
[12:49:16] <remonvv> Zelest: Alright. Well GridFS is specifically designed to store documents that exceed the single document size constraints.
[12:49:35] <remonvv> Zelest: It shouldn't really be used as a generic file storage solution. As such you might as well store the HTML in normal documents.
[12:50:01] <remonvv> Zelest: That said I'd still prefer a solution where the HTML is stored on a file storage solution and you place the queryable metadata for each document in MongoDB
[12:50:33] <remonvv> e.g. Adding a document means uploading it to S3 or something, parse and extract metadata, store that in MongoDB with the S3/CF URL in the document.
[12:52:01] <remonvv> Storing 1MB of data in MongoDB isn't going to be significantly faster than S3. That said S3 as a much larger per upload time overhead ofcourse.
[12:52:16] <remonvv> But the crawling is async so that's managable. If you need it to be faster use more threads to upload.
[12:52:59] <remonvv> Anyway, storing static data that you'll never query against seems unpractical. Might be a good first step though.
[13:08:52] <haxplorer> v8 works well with the development version which is the HEAD of master in github
[13:09:50] <haxplorer> Does anyone know when 2.4 is going to release? Is the roadmap timeline in https://jira.mongodb.org/browse/SERVER the release timeline too?
[13:30:58] <lar1914> learning about shell scripts... Is there a way to NOT echo result of cmd? I only wanna print when I say print('adsf')
[14:08:09] <remonvv> Someone downvotes my SO answer. I want to hunt this person down and shake my fist at him or her in an angry fashion!
[14:18:06] <NodeX> if you know the upper bounds then it's easy but you say you don't so it's not going to be easy
[14:28:44] <NodeX> yer, you need this .... db.foo.find({num_from:{$lte:12345},num_to:{$gte:123456}});
[14:34:54] <NodeX> and you should also loop to insert them as your num_from / num_to are inserted as strings not ints
[14:42:54] <aster1sk> I've terribly slow replication - there's master / slave / arbiter configuration without write concern. On a sigle primary we do ~8000 upserts / second - on replication we're doing ~300
[15:08:54] <aster1sk> So with safe writes on we don't see this issue however we're only updating ~1500 / second.
[15:12:43] <aster1sk> Background : we're migrating ~800 million records from SQL, I've scripted the migration and there are different update queries.
[15:13:09] <aster1sk> So with one of the migration operations we can get as low as 300 QPS which is unbearable on 800 million records.
[15:13:41] <remonvv> Any particular reason you're migrating into a replicaset instead of doing the import first and then sync a secondary?
[15:15:36] <remonvv> Alright, and they're updates rather than inserts because the migration is more complicated than simply bulk inserting the data I assume
[15:16:00] <remonvv> As in, you could generate large JSON files with all the migration steps performed and use mongoimport for a very fast import.
[15:16:44] <remonvv> That doesn't really do that much. unsafe writes simply skip waiting for a getLastError command
[15:17:55] <remonvv> Alright, no option to seperate the increments and other complicated updates from the main import?
[15:18:33] <remonvv> Anyway, your original problem is odd. Same hardware and same setup should result in roughly similar performance for PRI and SEC
[15:23:58] <remonvv> 2 seconds behind, that seems good. I'm assuming this is the safe writes option?
[15:24:42] <remonvv> When you go for unsafe writes, does it start fast but suddenly degrades in performance?
[15:28:29] <joniloky> Is it possible to have MapReduce run on a secondary and somehow push the results to the primary
[15:28:57] <remonvv> aster1sk: It's hard to tell exactly what's going on from this. Might be that the unsafe writes are accepted faster than they can be written to the oplog on PRI. Altneratively the problem is on SEC side and there is some sort of bottleneck that makes it not process the oplog as fast as it's being filled. The former seems more likely. Do you see any hardware resource bottlenecks on SEC? (cpu, hd, etc.)
[15:29:30] <joniloky> I'd like to prepricess data and I don't want the mapreduce to impact my performance
[15:30:24] <remonvv> joniloky: Two: 1) That sort of performance scaling is best done through sharding, and 2) Look into the AF as a M/R replacement since that isn't single threaded and inherently faster and sharding compatible.
[15:31:00] <remonvv> joniloky: There is no way to use m/r without performance degradation. You're doing more work with the same resources. The question is not does it slow down but is it still fast enough for you.
[15:31:17] <remonvv> Full m/r is best implemented on seperate m/r instances through hadoop or similar.
[15:32:23] <joniloky> remonvv: are there any plans to make it possible for m/r to run on a secondary(maybe even hidden) and push the results?
[15:39:14] <remonvv> joniloky: It runs fine on a secondary if you really want to but you shouldn't. Secondary is used for replication and eventually consistent reads ;)
[15:40:29] <remonvv> aster1sk: In the pastie I mean, but that's during a burst of 4-6k writes sec, not the 300 you mention
[15:40:53] <aster1sk> I think we're past the 300 / sec issue now, we've modified the migration script.
[15:41:21] <remonvv> joniloky: no, you'd have to do some hacking to get that data back to primary. It's not something I'd even look into further. Assume it's not possible.
[15:42:09] <remonvv> aster1sk: Still, at this rate 800 million is going to take a bit and performance will degrade slightly as collections grow.
[15:43:24] <remonvv> aster1sk: Other than that I don't have much suggestions for you if migrate first, sync second isn't an option
[15:46:25] <aster1sk> remonvv: yeah I can't seem to figure this one out - we have a strict launch date but this probably means we can't make the deadline.
[15:47:17] <remonvv> aster1sk: Well, unsafe writes to a PRI and then sync will be significantly faster. Or if you want to be super fast you can simply manually copy the data files of PRI to SEC which should be pretty much an insta sync.
[15:47:55] <remonvv> aster1sk: http://docs.mongodb.org/manual/administration/replica-sets/#resync-by-copying-data-from-another-member
[15:57:57] <aster1sk> remonvv: you've been most helpful - I do believe this is the route we will take
[16:01:59] <remonvv> BadCodSmell: Not sure there's an easy solution. You can look for "waiting for connection on port " string in the log
[16:35:54] <remonvv> BadCodSmell: It's relatively simple. If you think of problems that require a persistence solution as different types of dinner then MongoDB is like mayo.
[16:36:12] <remonvv> BadCodSmell: Good with some dinners but not all of them so you have to know which dinner becomes more tasty with mayo
[16:37:09] <remonvv> BadCodSmell: And then there's a bunch of barely informed critics that hate mayo just because other people hate mayo and/or because they don't like the look of it even though they've never tasted it.
[16:46:54] <BadCodSmell> mongo is like a crude sauce that doesn't particular enhance any dish that comes to mind
[17:38:02] <remonvv> BadCodSmell: This is much like stepping into an Apple store and starting to pronounce that iPads suck. A high troll factor.
[17:44:54] <JoeyJoeJo> Does the driver I'm using have any affect on the speed of inserts or queries? For example, if I write a client in python is it any faster or slower than a similar client written in C?
[17:48:25] <MatheusOl> Perhaps the way the drivers serialize objects to BSON format can make some difference on speed
[17:52:55] <matt__> sorry to bother everyone, but I wanted to know if and what is the mongo equiv of mysql's ~/.my.cnf to specify default datbase
[17:53:48] <matt__> sorry to bother everyone, but I wanted to know if and what is the mongo equiv of mysql's ~/.my.cnf to specify default datbase
[17:55:55] <tjgillies> is there a way to select a date range from mongo? like documents created within the last week?
[18:06:23] <marktraceur> OK, I have a bit of a bend-over-backwards query to make. I want to add in a "sort-by" field to a collection, and I need to run some queries to add this field to existing documents. The problem is, this collection's documents are linked together in subgroups, so there will be many documents with sortkey of "0". Is there some way I can use aggregation parameters in an update query?
[18:06:48] <marktraceur> Basically my theory is that I can group by the subgroup ID, and then limit to one per group somehow.
[18:11:00] <marktraceur> Or maybe not? Maybe I'm forced to run a bunch of queries and do it all on a per-document basis? This wouldn't be all too much trouble, but it's obviously not ideal.
[18:13:11] <JoeyJoeJo> Assuming that my shard key is field "a" and I have fields "a","b", and "c", what happens when I do a query for field "c"? How does mongos know on which shard to look for field "c"?
[18:14:39] <JoeyJoeJo> Or I should say, if I search for {c:10}, how does mongos know on what shard to find documents where c == 10?
[18:14:40] <MatheusOl> marktraceur: the same subgroups should have the same sortkey values? Is that what you want?
[18:16:45] <marktraceur> MatheusOl: Each subgroup's sortkeys should be sequential. So I can sort each subgroup.
[18:17:03] <matt__> <MatheusOl> Do you know what I would add to mongorc.js to use a different database?
[18:19:24] <marktraceur> Maybe it would be easier to start a "sorting" collection that kept arrays of ObjectIDs, indexed by subgroup IDs, that I can update with one query instead of three.
[18:22:19] <marktraceur> MatheusOl: That's all right, I'll make do and if someone else has any other ideas I can try those :)
[18:22:22] <matt__> yeah i know, but i like to use config files so i dont have to type it every time
[18:26:22] <marktraceur> MatheusOl: Presumably they're ordered by ObjectId, but I really haven't tested that theory. I'd guess it's the order in which they're created.
[18:28:15] <MatheusOl> marktraceur: If you use the default ObjectId it is already the creation order
[18:28:42] <marktraceur> MatheusOl: That's what it is now, but I want it to be sortable by the users
[18:29:32] <marktraceur> MatheusOl: So I need to either add a new field, which means some complex update queries, or add a new collection, which seems a lot simpler.
[18:29:55] <marktraceur> MatheusOl: It's about 150 nodes, and there are about 7 collections of similar size that I need to do this for also.
[18:30:27] <marktraceur> The subgroups range from having 1 to 30 nodes each. But yeah, it's not tiny.
[18:30:57] <MatheusOl> In this case, why don't you use ObjectId and add the sortable field only when user request it
[18:32:01] <marktraceur> MatheusOl: How would I use the sortable field if only one node had it out of the subgroup? The one node would either be at the beginning or at the end, and not a lot I could do about it.
[18:34:16] <marktraceur> I'm working on this: http://orgcharts.wmflabs.org:8888/#5085aa408fedf26b68000001/
[18:37:07] <marktraceur> And I want to be able to change the visual order of the nodes. But it would mean changing the database somehow and I'm not sure of the best method.
[18:53:31] <Dededede4> Mongodb requires a 64-bit architecture to manage a database of more than 2 gigabytes. It also works with PowerPC64?
[18:55:03] <vr_> I have a db w/ roughly 150K documents in them. There's a field that I'd like to do a sort/limit to get top-100 or so of.
[18:55:33] <vr_> Doing this via mongodb was painfully slow but just grabbing those fields and doing it in a scripting language took no time @ all.
[19:03:28] <ameoba> Dededede4: never heard anything about an x86 requirement for it. They don't have binary downloads so you'd have to compile it yourself.
[19:07:21] <Dededede4> Yes, but mongodb Will he be able to manage a database of more than 2 gigabytes?
[19:36:36] <tomlikestorock> my previously fully operational replica set now has what seems to be a rogue slave, which only reports "loading local.system.replset config (LOADINGCONFIG)" when I try to bring it back up
[19:40:59] <tomlikestorock> I can connect to mongo from each opposite server, but I can't get this replica set back up. I've logged onto the slave, deleted the repl config, brought down the slave, deleted the replicated datafiles, and brought the system back up. After it replicates, the problem reoccurs
[19:44:18] <JoeyJoeJo> Is there a command that will tell me how long the previously run query took? Or is there something I can add to a find() query that will tell me how long it took?
[19:51:02] <owen1> is it possible for a client to access the nearest mongo host in a replicaset (read freference = near) in version 2.0.1?
[21:33:36] <KeepSafe> Hi, what would be a query that would return documents containing a value in at least one attribute be?
[21:36:21] <KeepSafe> ron: would I have to perform a query for each attribute then and create a set in my application?
[21:41:36] <ron> though like mr_smith said, it also depends on how often the query is being executed.
[21:43:26] <mr_smith> everything in nosql subjectively awesome or awful depending on the usage and use case. track the execution and response times and fix it if it's a problem.
[21:44:01] <KeepSafe> ron: the search would search attributes fo user accounts (email/name) as well as all blog posts (title/taglist/author)
[21:46:14] <ron> if you're searching in a title, it sounds like you need some form of full-text search.
[21:47:03] <KeepSafe> ron: a string would be provided, and the fields would be checked for the values
[21:51:41] <cr3> hi folks, I'm trying to use mongoexport --db foo --collection bar | mongoimport --db foo --collection bar --drop, but now my ruby application is reporting errors because it seems that symbols aren't being imported properly. does that ring a bell for anyone?
[21:54:38] <cr3> aha! there's a big disclaimer at the top of the mongo exporting and importing part of the manual that says mongoexport will lose some information when converting from bson to json. so, can someone recommend a way to export the data in a way that I can modify it in a text editor before importing it?
[22:44:45] <sg> So I have a collection of documents where each document has a created_at attribute, and I need to do a select by DATE, i.e. select every document where create_at is a Monday, etc.
[22:45:22] <sg> Anyways, it looks like there's no efficient way to do this, and that I am stuck with having to use $where: function () { this.created_at.getDay() == 1 }
[22:46:11] <sg> I have to do this a number of times, so I'd like to iterate over every document in this collection (over 39mil of them) and just add a new field called day_of_week or something and update said field with the day of the week, that way I can index it for performance later on
[23:02:41] <fommil> hi all – I've been getting "cursor 0 not found on server localhost" using the java driver. Some googling shows that I need to turn off the timeout – is there a way to do this through MongoOptions?
[23:05:13] <sg> db.tweets.aggregate( { $project : { text: 1, day: { $dayOfWeek:"$created_at" } }, $limit : 5 } );