Log file Viewer
#mongodb logs for Wednesday the 5th of August, 2015
[11:14:44] <kenalex> I am designing an inventory application for cell sites at work. each cell sites have similiar attributes and also attributes that unique define them
[11:15:39] <kenalex> i am also interested in logining technicain works against these sites and various system statistics from external systems for these sistes.
[12:51:47] <jamiel> The documentation states "When an index covers a query, the explain result has an IXSCAN stage that is not a descendant of a FETCH stage, and in the executionStats, the totalDocsExamined is 0." ... why would db.foo.find({_id: ObjectId("55be26451900f15070fbeb46") }).explain("executionStats"); say totalDocsExamined: 1 ?
[13:00:48] <defk0n> can someone help me out with creating this query, details >> http://pastebin.com/6cbYtqEf
[13:04:02] <Derick> it is correct in: "_id": "$tags.collection.name" — as you want the value of tags.collection.name to be grouped on
[13:05:26] <defk0n> seems kind of confusing, if you say {"$unwind" : "$tags.collection"} is wrong, and i would do "tags.collection" the value of that is still a list. so it automatically resolves that ?, but group needs a $
[13:08:30] <defk0n> oh oke, and if i may ask a another question, how does mongo index such a structure? taking the current documents in the pastebin as example, will it store in memory all these documents based on their index key? or will it store the index key the value index points to?
[13:11:18] <defk0n> i havent got one yet, just thinking about where to put a index. because if every document has multiple tags.collection.type = tag field how will mongo index that
[13:13:35] <Derick> defk0n: it will create an index entry for each of the elements, both pointing to the full *document*
[13:14:09] <Derick> but in your A/F query, the only index that would be used is one on tags.collection.name, because that's what is used in the first $match
[13:15:55] <defk0n> Derick: but then that would mean there would be alot of duplicate indexes right? for each document a extra index eating up memory, isnt there like a reverse index, the index points to every document who such a field with value
[13:17:00] <defk0n> say every document has 100 tags, and i have 1000 documents, now its 100k indexes
[13:17:45] <Derick> so at max you have 100k index elements, but likely, it's just the number of unique tags
[13:18:03] <Derick> and for each of the unique tags, you then have a list pointing to each document that uses it
[13:18:04] <defk0n> really? thats awesome, thats basically what i was wondering, if it does that optimization
[13:42:36] <jr3> My current mongo database size is 26gigs, I only have 8 gigs of memory on my server.
[13:44:38] <davidhadas> hi, I just upgraded a mongodb from 2.6 to 3.0. It is a single server configrued with a replica set (I was hopping to add to it two more servers once I completed the upgrade)
[13:45:27] <davidhadas> use admin and than show users gets me to a "E QUERY Error: not authorized on admin to execute command"
[13:46:08] <davidhadas> the log is full of "I QUERY [conn1] assertion 13 not authorized for query... " errors
[13:49:00] <defk0n> Derick: you think i should denormalize that structure so i can index the collection itself ? isntead of the embedded document?
[13:56:14] <defk0n> Derick: technically, no, practically yes, the fields are based by model and responsibility, so tags is a Model, having a field meta alongside that would mean Meta is a model also. the document has diffrent models(fields) like Tags, Meta, Thumbnail, Article . example document http://pastebin.com/HP2xhTEX any feedback regarding index would be helpfull or just best strategy to index the tags etc.
[13:57:32] <davidhadas> Seems like any command I give - gets an error of ""not authorized on admin to execute command { authSchemaUpgrade: 1.0 }""
[14:21:49] <davidhadas> Can anyone please help? Did I loose my entire DB? How can I recover from this state?
[14:32:25] <davidhadas> cheeser: thank you - you have helped me start breezing again with this simple "or disable authentication"
[14:54:57] <SomeAdmin> Hi Gents, im collecting info on mongoDB... Question: Is it safe to use btrfs? The posts ive read about mongoDB +btrfs are quite old... (And people wrote that btrfs is still experimental, and should be avoided for prod)
[14:55:49] <deathanchor> love when the dev says, "TTL Index isn't deleting old stuff" I'm like "look works for me. Show me." and they can't show me.
[14:56:11] <jamiel> Hi all, just wondering if anyone can provide clarity on totalDocsExamined in explain() as it doesn't seem to match my understanding where it should report the number of docs that could not be analysed directly in the index.... The documentation states "When an index covers a query, the explain result has an IXSCAN stage that is not a descendant of a FETCH
[14:56:12] <jamiel> stage, and in the executionStats, the totalDocsExamined is 0." ... however db.foo.find({_id: ObjectId("55be26451900f15070fbeb46") }).explain("executionStats"); say totalDocsExamined: 1 ?
[15:29:31] <davidhadas> cheeser: I was able to get thing back to working (after the upgrade) by returning to a standalone configuration. When I try to move back to a replicaSet, seems as if teh configuration is messed up - status: "Our replica set config is invalid or we are not a member of it"
[15:34:25] <davidhadas> Cheeser: Can one do mongodump from a standalone to a cluster configured with a replicaSet? If so - I can create a new DB and simply load it with the data - than start using it instead
[18:00:49] <cheeser> RWOverdijk: http://docs.mongodb.org/manual/reference/operator/query/near/#op._S_near
[18:01:47] <RWOverdijk> One more question if you don't mind... I'm looking at: http://docs.mongodb.org/manual/reference/command/geoNear/
[18:02:15] <RWOverdijk> This also returns the distance, which is very useful. I'm currently running my own haversine after fetching with $near or $nearSphere
[18:02:37] <RWOverdijk> But my results don't match the ones returned by mongo (they're off somewhere around 1 km)
[18:03:46] <RWOverdijk> Now here's my question... I want to sort on a date field. I know it's possible to specify a query argument for geoNear to do filtering, and do a .sort() afterwards. But that sorts the response.
[18:04:12] <RWOverdijk> I want it to filter all documents within X, and filter those based on a date field.
[18:05:47] <RWOverdijk> "If you also include a sort() for the query, sort() re-orders the matching documents"
[18:06:04] <RWOverdijk> I'm not sure if this reorders all results and then applies the limit, or applies the sort on the limited result
[18:06:34] <RWOverdijk> @cheeser, Yes. That's what I have working right now (snippet coming) but it's not returning the distance.
[18:06:38] <tejasmanohar> http://mongoosejs.com/docs/2.7.x/docs/schematypes.html - i used to have a normal index on a field phoneNumber in my users collection and now i have unique index
[18:07:09] <tejasmanohar> unfortunately, mongoose won't change the index already in my db, which i'd think is a normal index- not unique index... since i changed it to unique after.
[18:07:24] <tejasmanohar> can i do this directly in mongo? is there a nice tool for these kinda migrations and schema changes?
[18:09:06] <RWOverdijk> cheeser, The distanceMultiplier should be 6378. I changed it before pasting... So I know that bit is wrong in the gist, sorry.
[18:12:34] <tejasmanohar> i'd rather not re-create the collection for this change. i bet there must be a better way. i guess i can also drop and re-add the index via the shell but sounds less than optimal
[19:41:18] <deathanchor> I need to write some code to handle no timeout cursor and to close the cursor if the program dies or gets killed.
[20:05:07] <bakhtiya> Hey guys - does anyone know what will happen if a remove overlaps with an update (upsert) - so while the remove is running, an upsert occurs - will that new upsert (even though it happened AFTER the remove) still exist?
[20:11:16] <bakhtiya> @deathanchor: so in this case - if I have two async systems trying to modify the same record (identified by the same _id value) - only one of them will happen at a time?
[20:13:15] <bakhtiya> ah, so if I do a remove with some condition - is there a time gap between mongo finding that data, and then removing it? if there is, and say I hit mongo with an insert which happens right in that time gap - will that data be lost?
[20:14:50] <bakhtiya> effectively in this case mongo is removing data that may or may not match the condition in the remove()
[20:19:17] <deathanchor> .remove(search) locks the collection .update(search, upsert :true) will wait until that is done before it does what it does
[20:19:24] <proteneer> i’m implementing a centralized spinlock of sorts using TTLs in Mongo, practically whats a reasonable frequency of polling? 100ms?
[20:26:41] <Doyle> Hey. How does the mongostat lockdb % report over 100%? I googled, but only found a jira that states its' a known sampling issue.
[20:42:18] <proteneer> "The TTL index does not guarantee that expired data will be deleted immediately. There may be a delay between the
[20:42:19] <proteneer> time a document expires and the time that MongoDB removes the document from the database.
[20:42:20] <proteneer> The background task that removes expired documents runs every 60 seconds. As a result, documents may remain in a
[20:42:21] <proteneer> collection after they expire but before the background task runs or completes."
[20:44:37] <deathanchor> in V2.6+ they say: The TTL feature relies on a background thread in mongod that reads the date-typed values in the index and removes expired documents from the collection.
[20:45:50] <Doyle> If you add a member to a replica set that has 2 members, it'll start replicating off the slave, but say the slave has a short oplog, at the end of the replication, will it then attempt to catch up from the primary which has a longer oplog?
[20:47:08] <Doyle> I read that you can control which one it'll attempt to sync from before the initial sync begins. I'll have to set that next time.
[21:06:06] <jpfarias> can I use the $text search for matching documents that has all of the words I wanna search?
[22:06:59] <kexmex> how significant is this warning? ** WARNING: soft rlimits too low. rlimits set to 1024 processes, 64000 files. Number of processes should be at least 32000 : 0.5 times number of files.
[23:05:27] <Doyle> nofile for example, in centos is important to adjust according to the documentation
[23:06:29] <kexmex> looks like startup script set limits high for both, i changed it to be 0.5x, athough warning says" at least 0.5x"
[23:06:48] <Doyle> well, as far as perf goes, even if it doesn't die, it won't be able to complete tasks. Under continuous load it'll just queue up forever.