Log file Viewer
#mongodb logs for Tuesday the 26th of January, 2016
[00:17:21] <Doyle> Hey. For cloud manager, I think a while back you could add a mongos and it would discover the entire cluster. Is that still how it works?
[00:19:25] <Doyle> I also saw a note in the docs about micro instances being unsuitable for deployments with more than 5 mongod processes. Does that scale linearly? t2.micro (1cpu,1GBmem) for each 5mongod's, so 10 would be 2cpu/2gb, 15, 3cpu.. etc?
[00:54:07] <bros> db.orders.find({ 'log.time': { '$lte': ISODate('2016-01-25T23:59:59-05:00'), '$gte': ISODate('2016-01-25T23:00:00-05:00') } })
[01:10:58] <stanier> So I need some input on how to structure a database for one of my projects. I am trying to collect metadata for packages installed across multiple servers.
[01:11:00] <stanier> Assuming package metadata is inconsistent for each installation, I am tempted to store each installation's metadata as a new document then categorize them by name and other consistent values in another table. Is there any better way to accomplish this?
[08:39:33] <mihaj> Hi, I was wondering if MongoDB supports a remove document operation which also returns a copy of removed document to the application?
[10:33:41] <nixstt> what happened to the MongoDate class in php7? i can’t find it in the mongo-php-library
[10:49:38] <nixstt> Derick: mongo-hhvm-driver seems to recommend using mongo-php-library but mongo-php-library is not compatibly with the latest driver, was really curious about performance
[10:50:55] <nixstt> yeah and installing hhvm 1.1.0RC1 actually installs as a different version number
[10:51:50] <sebbel> hey, I've got a document {"Editor": {enabled: true}}, how would I query to get the value of enabled?
[10:52:56] <Derick> nixstt: can you make it ignore deps? or override it? 1.1.0RC1 should work with mongo-php-library 1.0.0
[10:53:16] <sebbel> It is in the Settings collection, so I tried Settings.find({},{'Editor.enabled": 1})
[10:53:19] <Derick> but there is an odd memory corruption bug in the HHVM driver, which is why it isn't 1.1.0 yet
[10:53:51] <sebbel> but that only returns the document as it is .. but i want to extract the value. or am I thinking wrong and should do the processing with javascript afterwards
[10:54:23] <Derick> sebbel: hmm... you might want to change that to: { setting: { 'Editor.enabled' }, value: true }
[10:54:38] <Derick> sebbel: hmm... you might want to change that to: { setting: 'Editor.enabled' , value: true }
[10:55:26] <Derick> nixstt: you sure you got the latest version of the hhvm driver? 1.1.0RC1 should have readConcern in it
[11:21:50] <Derick> nixstt: it seems not... I think that's going to belong in the mongo-php-library. can you file a feature request at https://jira.mongodb.org/browse/PHPLIB ?
[11:29:48] <sebbel> I've tried it with distinct, and actually got true back. but embedded in array :)
[14:07:51] <lipiec> cheeser: thanks, I thought so, do you know some exmaples on github? or is it better to program it mysef?
[14:15:42] <kopki> guys I need help with a query.. ive a doc with customerid(not unique) and accesstime in millis. need unique customer count by time interval. Also customer should be counted only for the first time each day... no matter how many times he has accessed.
[14:16:46] <cheeser> aggregation. filter by time range group by customer id, group by customer id, limit 1, count
[14:23:25] <kopki> can you be a little bit more explicit? not in terms of code.. as in whats actually going on
[14:27:28] <m3t4lukas> kopki: that's the general doc on aggregation: https://docs.mongodb.org/manual/aggregation/
[14:28:51] <m3t4lukas> then you should take a look into the doc of spring data's api to mongos aggregation framework
[15:33:06] <kaimast> hey. my quest for better performance for 2d indexes continues. i pasted a longer question here: http://pastebin.com/PQHCfqk5
[15:33:25] <kaimast> i also wonder if building mongo with "scons mongod" builds it in release or debug mode?
[15:43:02] <magicantler> for automation do you guys like to use your own scripts or mongod configuration files?
[15:54:38] <kaimast> do replica sets actually increase throughput or are they only useful for redundancy? i only see marginal improvements if i add more machines :(
[15:55:44] <Kobbb> Hi guys. I'm looking for a way to make a graphic showing the evolution of the count of documents in a collections by creation date, do you have any leads to give me ?
[15:56:31] <kaimast> mmmh i give clients a list of all replicas do they automatically distrbute reads?
[15:57:28] <kaimast> in this case data is read only and i just want to measure the maximum throughput i can get with mongodb
[16:01:19] <cheeser> there's a "nearest" which is slightly misnamed but it attempts to read from the "closest" machine (secondary or not)
[16:03:42] <Kobbb> I'm looking for a way to make a graphic showing the evolution of the count of documents in a collections by creation date, do you have any leads to give me ?
[16:04:20] <kaimast> This is also weird. Using a collection with 1000 documents and an 2d index, it sometimes has to examine all objects o.O: planSummary: COLLSCAN keysExamined:0 docsExamined:1000 cursorExhausted:1 keyUpdates:0 writeConflicts:0 numYields:8 nreturned:0 reslen:114
[16:04:21] <Kobbb> I thinks aggregations is the key, but I can only get the number of tracks created on a specific month, not the sum of all tracks at this moment
[16:11:17] <kopki> Hey, is there anyway i can run group by on intervals? Lets say I have documents like (s1,1),(s2,2)............ (s100,100) . I require count in range of intervals of five
[16:31:41] <Kobbb> Guys, I need to make various counts on a collection. I need to show how many documents were in the collection at a certain month time, every month in the last 6 ones. The aim is to show the evolution of records. Is that possible with aggregations ?
[16:32:52] <Kobbb> I could do 6 different counts, with a $lt on creation date, but it would be sexier with some aggs :D
[16:36:34] <Tzahush3d> I want to create a forum database type, and i've seen that you can use a relational database in mongodb with the unique ID, and was wondering if it would be good designing it a bit like i would do in MySQL, so for example: this is the deisign
[16:38:07] <cheeser> Kobbb: https://www.mongodb.com/presentations/mongodb-time-series-data-part-1-setting-stage-sensor-management
[16:54:12] <cheeser> i'm not sure the arbiter stores *anything* on disk. i think it just exists to elect a primary.
[17:31:57] <GothAlice> (-Document.name | (Document.name == "Alice")).as_query → {'$or': [{'name': {'$exists': 0}}, {'name': 'Alice'}]}
[17:32:50] <GothAlice> ((Document.name == "Alice") & (Document.age >= 21) & Document.tag.any(("foo", "bar"))).as_query → {'age': {'$gte': 21}, 'name': 'Alice', 'tag': {'$in': ('foo', 'bar')}}
[17:33:33] <GothAlice> So some ordering issues, but other than that, I'm extremely pleased. 102 lines of code.
[17:37:03] <GothAlice> -Document.name & +Document.name → {'name': {'$exists': 1}} — Does the right thing if you redefine a field query operator, though. Phew.
[17:57:34] <GothAlice> Tzahush3d: Indeed, the structure you've constructed is, for the most part, sound.
[17:57:48] <GothAlice> Where it falls short is on storing a concrete list of thread IDs in the Forum itself.
[17:58:37] <GothAlice> Since the threads within a forum are unbounded–no upper limit–that list could grow beyond the maximum size for a single record, and there is no querying improved by doing so.
[18:00:17] <GothAlice> (If you append new threads to the list via $push, then the list is in ID order already so you can just sort threads on _id to get the same result more efficiently. Instead, store the forum ID in the thread document.
[18:39:00] <GothAlice> Ops({'age': {'$lt': 65}}) & (Document.age >= 21) → {'age': {'$lt': 65, '$gte': 21}} muhahahahahaha
[18:39:40] <GothAlice> You can load existing MongoDB queries and updates into it, then add more / modify them with the expanded syntax.
[18:41:24] <GothAlice> When loading up the data for a thread, you pretty much always want to know what forum it's a part of. When loading up a forum, you probably don't actually want a complete listing of every single thread ever posted in that forum; that's overkill.
[18:42:46] <GothAlice> Additionally, if you want the threads back in creation order (i.e. the same as iterating a list of thread IDs stored within the forum) it's as simple as: db.threads.find({}, {_id: 1}).sort({_id: 1})
[18:43:26] <GothAlice> So storing a list of thread IDs within the forum doesn't actually offer any benefits.
[18:47:16] <GothAlice> Since the data is actually wholly stored and there aren't just references to records in another collection, you can paginate ($slice) the results easily, deleting a thread automatically and naturally deletes all the replies, adding a new reply is effectively a single $push (rather than insert and push as is the case with storing thread references in forums), and loading up a specific reply will also fully load up the thread details (if
[18:47:57] <GothAlice> https://github.com/bravecollective/forums/blob/develop/brave/forums/component/thread/model.py#L58 < the thread model from my own MongoDB forums. :)
[19:00:38] <GothAlice> Tzahush3d: https://gist.github.com/amcgregor/6137a34d7c81e53d429f is the model simplified down (and not tied to Python code).
[19:01:44] <Tzahush3d> but i didn't quite understand something, so when i use an ID, DB wise it inserts the whole 'line'?
[19:02:17] <GothAlice> If you use an ObjectId as a field's value, you get an ObjectId in that field.
[19:02:44] <GothAlice> If you use a document (object) as a field's value (as in the stat and flag examples) then you get the whole document/object as a "sub-document".
[19:03:02] <GothAlice> Similarly, the comments (replies to the thread) are just a list of embedded (sub) documents.
[19:03:36] <GothAlice> Not references (ObjectId or DBRef) to records in another collection of replies stored separately.
[19:03:58] <GothAlice> And yeah, MongoDB has no joins so all ObjectIds are manipulated and looked up manually.
[19:06:02] <Tzahush3d> another question, in the model, for the forum, you are not using an ObjectID, can i ask why? ( _id: String, # Slugs, not ObjectIds.)
[19:22:53] <GothAlice> Tzahush3d: Which is better, /forum/56a7c603927cc64d2c598d17 or /forum/general? ;)
[19:23:19] <GothAlice> Since the "slug" (path element in the URL) is naturally uniquely identifying, we can skip ObjectId generation and just use the slug as the ID.
[19:25:44] <GothAlice> So yeah, actually, the "forum:" entry against a thread needs to be a string, not ObjectId.
[19:28:16] <cheeser> each document in a collection has a primary key called _id. if you don't give it one, mongodb will generate an ObjectId value and use that.
[19:44:40] <GothAlice> cheeser: Now that MongoDB has schema validation support, any hope I could issue some pull requests to pymongo to add schema declaration support and my 120-line query builder? :3
[19:59:14] <GothAlice> cheeser: https://gist.github.com/amcgregor/62e8b69fe1bbdcd5f944 < prototype so far
[20:31:45] <hernanhht> hey guys, I'm trying to hint updates in 2.6. Since I can't use the $hint keyword I'm thinking about using Index Filters for this. Are any of you use it for this kind of thing ?
[21:23:16] <cheeser> you should just ask so that if someone comes along and sees your question, (s)he can answer right away without having to wait for you to explain your problem.
[21:29:33] <TruPilot> I'm looking for a way to kill an op in Mongo from within PHP. Any suggestions?
[21:37:40] <m3t4lukas_> eweiser: just ask your question and don't ask whether anyone can answer a question you are going to ask
[21:41:22] <negatratoron> hey, i've got a collection for which 'db.collection.count()' returns 3571, but when I mongoexport it I get 3871 documents
[22:15:31] <eweiser> so I'm using the latest version of the java mongo client. i'm using an ExecutorService, and calling shutdownNow() at shutdown. my worker threads are continuously checking the interrupt flag, and doing some shutdown cleanup behavior if it's interrupted, before actually shutting the thread down. however, i saw my threads weren't doing the proper cleanup behavior. problem is that the java client catches InterruptException (which clears the interrupt flag) and d
[22:16:06] <eweiser> from what i've read, not re-throwing InterruptException or re-setting the flag with Thread.currentThread().interrupt() is bad practice; is there any reason they aren't doing this or is it worth submitting a ticket
[22:19:01] <GothAlice> eweiser: Because the question relates directly to the underlying mechanisms of a driver, a JIRA ticket filed against the appropriate language driver would likely be more productive. On https://jira.mongodb.org you'd file this against the JAVA project.
[22:50:26] <Spynxic> Is there a way to get the current key from a cursor when iterating through the values of a collection? Or, alternatively, get a list of keys (similar to db.getCollectionNames except for a particular collection opposed to the entire db)