[01:02:54] <Boomtime> crazydip: these terms mean different things, WiredTiger is a storage engine, it will store whatever it is told to - mongodb uses it to store databases/collections/indexes - gridfs is basically just a very particular schema applied to two chosen collections
[01:03:23] <crazydip> Boomtime: thanks that makes sense... i thought GridFS is like a Schema + Engine together....
[01:05:37] <crazydip> StephenLynx: thanks you as well ;)
[03:32:06] <acidjazz> binge watching too many horror movies .. where most of em people are in dark places using flashlights
[03:32:55] <acidjazz> why arent horror movies sponsored by flashlight copmanys
[03:33:34] <acidjazz> guaranteed to not randomly flickr so you dont see the demon face for 0.25s
[05:53:37] <crazydip> opening connection to MongoDB via pymongo on localhost has a noticeable delay... my python script has nothing but db = MongoClient(uri) and takes 0.615s while connecting via mongo -u blah db -eval 'quit()' -p pw takes 0.101s - what could i be doing wrong?
[05:54:33] <crazydip> if i comment out the db = MongoClient() line the python script takes 0.08s
[08:29:37] <rawl79> I'm running an aggregation over a large collection (that's frequently updated) using Go. I always get "Exec error: PlanExecutor killed"
[08:29:53] <rawl79> any reason why the aggregation statement is being killed?
[09:43:28] <supersym> I find the documentation on enforce-unique-keys-for-sharded-collections extremely confusing
[09:50:54] <supersym> a. MongoDB does not support creating new unique indexes in sharded collections
[09:50:58] <supersym> b. and will not allow you to shard collections with unique indexes on fields
[09:51:36] <supersym> = so it _is_ possible... but anyway, "If you need to ensure that a field is always unique in a sharded collection, there are three options"
[09:51:47] <supersym> - Use guaranteed unique identifiers.
[09:51:48] <supersym> Universally unique identifiers (i.e. UUID) like the ObjectId are guaranteed to be unique.
[09:53:11] <supersym> So it possible but only when using UUID on _id as ObjectID?
[09:53:43] <supersym> Terms are used confusingly (or I'm a n00b, more likely) but anyway I assume ObjectID is a data type and _id is the field name which could be UUID
[09:55:24] <supersym> Oh wait ... UUID __like__ the ObjectID... but ObjectID doesnt __have__ to be UUID does it
[10:09:13] <djlee> Hi all, does nesting a document field have negative performance impacts (i.e. like it has to drag out all documents and parse them in memory). I was to store a bunch of dates relating to different things inside a "metadata" field on my document for cleanliness, but i need to be able to do thing like "where doc.metadata.sometime >= <some time>"
[10:10:17] <djlee> will using a nested document have a noticeable impact on performance vs a top level attribute (i.e. i could store them as metadata_sometime rather than as a nested doc)
[10:42:44] <supersym> in short: No difference due to serialized data read
[10:45:23] <djlee> cheers supersym, managed to find that exact SO post after i asked here :P
[11:31:35] <Lonesoldier728> anyone know best way of limiting docs in mongo http://stackoverflow.com/questions/31854506/mongoose-query-to-keep-only-50-documents
[12:32:40] <kenalex> i am wondering if mongodb would fit my use case
[12:35:12] <kenalex> i am building an application that will manage cell site (mobile network) and their equipment. each cell site has similar and unique attributes. the application is like a catalog of sorts but also will store summary stats from data from other system as to their performance
[12:36:10] <StephenLynx> I think it would fit just alright.
[12:37:20] <StephenLynx> since it looks like you have a very flexible data model and is barely or even not relational at all.
[12:39:00] <kenalex> i attempted modeling the data schema (using visio) and cam up with around 18 tables for the whole application, that's when a friend of mine mentioned document databases
[12:41:21] <kenalex> StephenLynx: i do that as well :) i had visio at work which I normall use for network design, so I gave it a try
[13:21:03] <crised> Can you explain me in simple words what's a document store?
[13:21:13] <crised> I know a bit about relational dbs, key value stores like dynamo db
[13:21:40] <crised> is a document store, a db that indexes every element in the schema?
[13:24:00] <crised> could anyone explain this paragraph: "Document-oriented databases are inherently a subclass of the key-value store, another NoSQL database concept. The difference lies in the way the data is processed; in a key-value store the data is considered to be inherently opaque to the database, whereas a document-oriented system relies on internal structure in the document order to extract metadata that the database engine uses for further optimization"
[13:25:43] <crised> StephenLynx: How is this different for a key value store?
[13:25:52] <crised> key value cannot have other objects inside?
[13:25:58] <StephenLynx> I am not familiar with key value storages.
[13:26:05] <StephenLynx> so I can't tell you that.
[13:26:26] <crised> StephenLynx: how does mongo extract the metadata of each document?
[13:26:37] <Folkol> I think that your quote pretty much pin-point the difference. In a key-value store, the database does not understand, or care about, the contents of the value (i.e. opaque values). In a "document store", you are storing a structured document (for example a JSON document), that the database can reason about.
[13:27:06] <crised> Folkol: Please give me an example of how db can reason abou tit?
[13:28:32] <StephenLynx> or "only return me documents which this field holds an array with this size"
[13:28:36] <crised> see, I don't get the difference between key value, and document
[13:29:28] <Folkol> Yes, but that is not a property of a generic "key-value store". That is a property of DynamoDB. A generic key-value store will store some opaque data for you, under a given key.
[13:29:32] <crised> StephenLynx: mmm, I think Mongo has alot more functions than a key value db engine like dynamodb, I would guess key value and document store are pretty similar
[13:29:53] <crised> Folkol: oh I see, then dynamodb has also propoerties of document store
[13:33:05] <Folkol> I might be mistaken (the only NoSQL databases that I work with Couchbase and Mongo), but I recall that DynamoDB was a simple key/value-store in the beginning, but got support for structured data later on.
[14:14:20] <deathanchor> thorough testing should help you find all your issues
[14:19:30] <troulouliou_div2> hi is it classic in mondodb to duplicates data. i m storing tweets and users in mongodb for test and the tweets have a user_mentoin list of mentionned users
[14:19:53] <troulouliou_div2> should i insert it like this or insert separate users and create somekind of relaiton when retrievng datas ?
[14:36:09] <StephenLynx> "is it classic in mondodb to duplicates data." no
[14:37:48] <troulouliou_div2> StephenLynx, what should i do in this case remove the mentionned_users list and add a new user in the usercolletions ?
[14:38:05] <troulouliou_div2> and just add an id in the mentionned_users ?
[15:31:23] <joshua> Well if you really want to go crazy you can disable the replica set and boot it as a single server but when things come back up you might be in trouble.
[15:33:40] <joshua> Its the kind of thing you have to work out while they are all still up to plan for since you can't change rs.config on a server you can't reach.
[16:07:55] <djlee> Is there any way to do a non datatype specific lookup. Someone has inserted a load of crap into the database and all the integers are casted as strings
[16:08:21] <djlee> is there a flag i can use to make mongo match the value regardless of data type
[16:11:17] <dddh> kenalex: mongodb is not a RDBMs ;(
[16:46:13] <crazydip> i have an issue where opening connection to MongoDB via pymongo on localhost has a noticeable delay... my python script has nothing but db = MongoClient(uri) and takes 0.615s while connecting via mongo -u blah db -eval 'quit()' -p pw takes 0.101s - what could i be doing wrong?
[16:47:22] <crazydip> the database itself is comprised of 1 collection with 1 tiny document
[16:47:40] <deathanchor> how are you measuring this?
[16:50:20] <deathanchor> sounds like a pymongo problem, not a db problem.
[16:50:30] <deathanchor> I use pymongo and have no issues.
[16:51:13] <cheeser> well, you're in luck! pymongo 2.9rc0 was just released. maybe it fixes your issue.
[16:51:14] <crazydip> it takes less time for python to "compile" a whole framework on this computer so there's something wrong somewhere.... it's not "normal"
[16:51:58] <crazydip> deathanchor: oh i would assume it's something with pymongo not mongodb since timing mongo client it's 6x faster
[16:52:14] <dddh> crazydip: do you mean your problem could be related to dns or something else?
[16:52:15] <deathanchor> how about you precompile your code
[17:46:39] <silasx> Well I’ll ask the question anyway: I have a replica set with three members, one of them in startup2 mode and the other two are secondary. One ofthe secondaries (call it 8501) has the highest priority by far … but no election is happening, and none is becoming primary.
[17:46:51] <silasx> Can I force them to have the election and make one primary?
[17:48:56] <cheeser> you should determine why that one is stuck in STARTUP2
[17:49:33] <silasx> well, it’s not stuck, it was recently re-added to the replica set and is legitimately starting up, it has to re-sync.
[17:49:51] <silasx> But shouldn’t one of the secondaries become primary?
[17:50:46] <cheeser> i would expect one to be primary, yes.
[17:50:52] <cheeser> which one was the high priority?
[17:52:34] <cheeser> i would expect so, yes. check the logs and see if there's anything there.
[17:52:50] <cheeser> is there data on the node stuck in start up?
[17:54:10] <silasx> well I recently took it down because it was stuck in “recovering” for a week, then (per the manual) deleted its db directory, and then just started it back up
[18:06:33] <BladeSling> Howdy,I am attempting to optimize a mongo server and one of the indexes that has been created doesn't seem to take priority like I think it should. To me it is fairly obvious which index should be picked but maybe someone here can explain why it picks the other index. Attached is the index executions for each. http://pastebin.com/WnqPBQFq
[18:12:50] <BladeSling> Here is the data from the query planner. I excluded the many other indexes, but if the order of the rejectedPlans has anything to do with it, the index I want is the first one to appear there. http://pastebin.com/ULPBLyqi
[18:17:08] <bakhtiya> Hey, regarding mongo indexes - I'm not seeing a direct performance increase on a distinct() operator on a key I have defined an index for - is there any optimization techniques around this? maybe an index isn't the solution - how can I optimize the distinct() operator
[18:32:17] <silasx> @cheeser thanks for the help, turned out ot be that something was wrong with the autossh tunnel from 8502 to 8501, just had to restard it and 8502 allowed the election and 8501 was restored to its rightful role as primary
[18:32:58] <silasx> *restart the autossh tunnel, I mean
[18:43:59] <jr3> do read replicas act as a load balancer?
[20:09:14] <BladeSling> Can anyone explain why this index gets picked instead of the index that covers the query better? Query planner data: http://pastebin.com/ULPBLyq execution data: http://pastebin.com/WnqPBQFq
[20:12:31] <crazydip> jr3: 1) you should not be launching mongod by hand on a server but have it automated by systemd and 2) you should not be using sudo to launch mongod, nor should you launch it via root, but have a mongodb user / group -- again, this would be automated by systemd (you do have to create the user/group once if your disto package does not do it)
[22:20:21] <leptone> when i add in this .sort() method it changes the order my collection prints in however its not the correct (ascending or descending) order