Log file Viewer
#mongodb logs for Saturday the 27th of December, 2014
[02:35:59] <_roland> Is there a way to project the array index in the results of the $unwind stage in an aggregation pipeline?
[02:39:32] <_roland> mm...seems there is an issue report asking for this feature. https://jira.mongodb.org/browse/SERVER-4588. Perhaps I should conclude this isn't possible (yet)?
[02:42:26] <bmillham> Hi _roland. I'm having similar issues with aggregation. It seems to loose unwound and embedded documents (in a list)
[02:43:42] <bmillham> It's real annoying to me that after aggregation, you get a dict instead of an object, so you can't use dotted notation, but have to use dict notation.
[03:25:45] <_roland> bmillham: I would be happy to use map reduce instead, but that has its own share of problems.
[03:26:20] <bmillham> I've thought of that, but it seems real annoying to have to write javascript...
[03:26:41] <_roland> bmillham: not sure I understand your distinction of dict vs object in this regard. can you share an example?
[03:27:05] <bmillham> I like the bug report the you linked, Circular references, bug points to bugs points back to original bug
[03:27:14] <_roland> bmillham: I don't mind js as long as it gets the job done. what annoys me with m/r is limitations such as the map result getting too bit to process with reduce.
[03:28:18] <_roland> bmillham: ah gotcha. this sounds like a particular quirk of the python driver, or the way it represents mongo query results back in python?
[03:28:30] <Aric> if I have an entry with an array of values like {orders: [1,2,3]} will it still find with findOne({orders:1}) ?
[03:33:17] <Aric> I just had a chance to test it and it worked also but I'm new to mongo and will read the documentation
[03:34:17] <_roland> Aric: personally I'm not really into the groove of this query language. I have to constantly look up how this or that operator behaves. It doesn't come naturally to me. Perhaps it has to grow on me.
[03:36:44] <bmillham> Same here _roland. But I'm getting the hang of it. Aggregation is still a bit of a mystery to me.
[03:53:26] <bmillham> I haven't really noticed thing being slower, in fact, there is one query in the MySQL database that I'm converting to MongoDB runs in 300ms on Mongo, compaired to minutes on MySQL
[04:46:16] <Aric> Looking for help in building a family tree: http://stackoverflow.com/questions/27664423/mongo-building-a-family-tree
[04:47:03] <ali_> I'm trying to replicate my mongo database on another VM so I can do some testing on it, what's the quickest way to do that?
[06:40:12] <phillips1012> what would be the best way to store short binary data (2-6kb thumbnails) in a document? should I base64 them, or is there some better way to store non-ascii data?
[08:57:06] <hulkk> Please help me i have database like this : http://pastebin.com/gZ8RpsQ2 i want to add to "One" a new array not "structure" but "data" when i using update it replaces the "structure" with "data" but i don't want replace i need add to it
[09:09:23] <theotherguy> is it possible to do something like this in mongo, SELECT message FROM inbox GROUP BY contact_id
[09:09:49] <theotherguy> basicaly want to get the last message from an series of messages, by contact (like in email)
[09:18:33] <hulkk> joannac i using $set but it acutally replaces the "structure" with "data" but i wanna add it to it.
[09:36:02] <joannac> $set: {"records.two": ["data"]} gets you a document looking like records: {two: ["data"]}
[09:37:30] <hulkk> joannac i tryed to imagine something like records.two and in this subdocuments array data which holds some values and array structure which holds also some values
[10:09:52] <hulkk> joannac i don't even know if my layout is correct i don't know when use subdocuments and when arrays
[10:12:25] <hulkk> http://docs.mongodb.org/manual/tutorial/model-embedded-one-to-many-relationships-between-documents/ there is adresses :[ {} , {} ] mean's adresses is a array and has subdocuments ?
[14:25:54] <_roland> Hi all. I'm trying to use the aggregation pipeline to get grab a top-level property along with a property of a subdocument that's buried inside an array property of the top level document.
[14:26:27] <_roland> I can get the values I want (using $unwind and $project) but now the value from the subdocument in the array property is projected as a subdocument itself. How can I flatten that?
[14:45:51] <_roland> ok. it seems I can achieve that with $let. Is that the only way? It seems so complicated?
[14:51:50] <merpnderp> I want to write a scheduling/appointment app and would love to use Mongo, but I'm not sure if a document store is correct for this domain.
[16:30:59] <hulkk> look this is how my db looks http://pastebin.com/Bbg7E11p why i can't create it like this ?
[16:46:24] <dimon222> here are no actual column names in mongo, its kinda document-like db, where you just store a string which is json, nothing is check'd in there and its stored as it is
[16:47:44] <dimon222> in theory you can make something like you want, but it wont be using any of the mechanisms of mongo like search and etc, and will be limited to the size of document (everything will be stored in single document)
[18:22:54] <_roland> dimon222: I don't understand. Why can't hulkk create his db like that? You just said basically anything goes. Why not hulkk structure? What would prevent him?
[18:35:04] <dimon222> as far as i understood, he wants one of fields to be stored in one array, and another field in another array
[18:35:59] <dimon222> in mongo you dont have separation on columns and values, you just have document
[19:06:36] <dimon222> full partition snapshot (depends on how your platform works with it) and copydb/clone command
[19:08:35] <dimon222> i'm kinda lover of copydb command. http://docs.mongodb.org/manual/reference/command/copydb/#dbcmd.copydb
[19:11:11] <dimon222> but if your platform is not really good with snapshoting, then it may take more time
[19:11:24] <hulkk> dimon222 when i doing $set on subdocument aka {} it allways rewrite my subdocument field i never get a list.. i need to wrap my subdocument also in a array to can have more subdocuments below ?
[19:11:49] <dimon222> for sure copydb took for me like 1.5 hrs to copy 70GB of data between two mongo servers (both are Amazon)
[19:12:43] <ali_> dimon222: Did it lock your main db? my issue is that i have a script inserting data into my main db every 15 mins. if it locks the db, there could be trouble.
[19:13:41] <dimon222> ali_, strange, it was only locking my db when it reached index stage, in the rest its not supposed to put locks at all. Even docs are saying "copydb does not lock the destination server during its operation, so the copy will occasionally yield to allow other operations to complete."
[19:14:26] <dimon222> oh nvm, thought you meant it locks yours, its not supposed to lock unless you do anything with adding new indexes
[19:14:26] <ali_> dimon222: I haven't yet tried it, so I don't know, I just don't want it to interfer with my 15 minute script that inserts data
[19:18:10] <dimon222> maybe better check this page - http://docs.mongodb.org/manual/reference/operator/update-array/ , so I don't say something i'm not completely sure
[19:21:50] <hulkk> dimon222 i mean some user info is easy and working i have already a login system in php .. but i need now something like like records (nameOfRecord(recordinfo=info, recordinfo2=info)) ...
[19:23:59] <dimon222> imo for that stuff you better use relational database, mongo is better just as store for fast accessible data
[19:25:23] <hulkk> dimon222 no i simply want my users to can create a "record" and also specific record names that holds some data / my page allows users to create custom records with custom fields and add data to it for example u create a record "income" with fields like "event, price, date, etc" and u add your incomes there.
[19:26:19] <hulkk> dimon222 and i migrated from sql to mongodb because in sql it's impossible to do something like this since u need to specify first a "scheme" and than use it .
[19:28:05] <dimon222> anyway, aware of row limitations then, 16MB per document (seems like user document for you?)
[19:30:27] <buhman> if I want to reference a document by something like the sha1 of one of its fields, can mongo maintain said hash for me, or must I calculate and update it manually in its own field in that document?
[19:30:37] <hulkk> dimon222 i basically have one document user , password, member_since, records wich i want to embbeed to can do what i mentioned
[19:31:11] <dimon222> hulkk, i think you can use this thingy http://docs.mongodb.org/manual/reference/operator/update/push/#up._S_push
[19:34:40] <dimon222> buhman, hint - mongo doesn't actualy generate hashes, it always done on yours side
[19:36:04] <dimon222> check this page http://docs.mongodb.org/manual/tutorial/create-a-hashed-index/
[19:36:18] <hulkk> dimon222 i using this function to create a record in mongodb which actually looks like this when i findOne from shell http://pastebin.com/MYtDRAB6 here is the php class http://pastebin.com/cyjbeCZe
[19:37:10] <hulkk> dimon222 so ok by now it works good it created for me a record with name "chata" which i entered from my website and has 2 fields inside the structure.
[19:42:51] <hulkk> dimon222 well i neeeded something to loop my "$fields" and put it inside the mongodb array query $key is 0,1,2,3 in mongodb it's not showing it converted it to array
[19:43:52] <dimon222> oh ok, probably better would be use $push instead of $set, at least according to docs http://docs.mongodb.org/manual/reference/operator/update/push/#up._S_push
[19:45:56] <hulkk> dimon222 ok i will try, how you would use my foreach loop to loop $fields and put it in the array i can't use loops inside array()'s
[19:50:07] <hulkk> dimon222 however maybe i will go for more collections instead of arrays ... seem's like mongo db is really not supposted to
[19:53:13] <dimon222> buhman, probably this stuff is supposed to be used for sharding mainly, and not really for tihs purpose
[19:53:50] <hulkk> dimon222 but in relational databases u can't really go for custom fields by user defined i will stay on mongo just using more collections together
[19:54:09] <dimon222> i'm not qualified enough to say 100%, but you can try wait someone more experienced in here and ask on stackoverflow for additional details. Some mongo devs are actually monitoring stackoverflow mongodb tags.
[19:55:18] <dimon222> hulkk why not? table with schema like "nameOfField","valueOfField", "foreign key userid"
[19:57:20] <dimon222> well you can keep using collections of course, but it may take more time, and probably more calls to DB
[19:59:18] <dimon222> hulkk, depends, but for anything related to working with users better use relational database. MongoDB is cool, decent and fast, but unfortunately, not flexible enough to do such important things as inner joins, unions, complex queries and etc
[20:02:14] <dimon222> which can be necessary if you think about expanding experience to something like "groups", "permissions", or even "user posts". Like you can fit it in one document of course, but 16MB limit, long downloading and more data that can be redundant in most cases. Or you can use different collections, but that may lead to secuity problems and synchronization problems. Like if you decide to delete user, you will have to do many calls to delete user data from
[20:02:14] <dimon222> different collections, while its just something like cascade delete in relational database.
[20:44:43] <hulkk> dimon222 i read somewhere google or facebook is using mongodb .. i just can't belive it when a single document can hold only 16MB ..
[20:46:22] <hulkk> Derick idk u have to keep in mind that one document will be full in one day and everything crash ppl will cry etc,
[20:50:53] <Derick> and there are many many articles, tutorials etc out there showing you how to create a database schema where you won't run into this limit
[20:52:32] <cheeser> but putting everything about a user in to one document is just foolish in so many ways.
[20:54:19] <hulkk> Derick can you send me a tutorial , best the easiest how i will not run into this limit ? it would be friendly and would help me i already searching for some limit bypass but can't find a decent one.
[20:56:08] <cheeser> http://blog.mongodb.org/post/87200945828/6-rules-of-thumb-for-mongodb-schema-design-part-1
[21:08:11] <hulkk> cheeser cool ok i have a user document and my user_comments document is full now create a user_comments2 ... awful the data model design second link you posted is only refferencing by id's from one document to other.
[21:24:56] <cheeser> don't put comments in the user collection. have a comments collection and a reference back to the user.
[21:37:53] <R00t_> I'm making an application for a school assignment, and I put together a drawing of how I wanna make the database with mongodb
[22:06:10] <cheeser> in a sense, there are single document transactions since document updates are atomic but that's about it.
[22:08:38] <_roland> Hi all. trying to calculate a sum on that nba games dataset using the aggregation framework: year, team name, and sum of wins. But I get "invalid operator: $sum" error. Any ideas? http://pastebin.com/0eMXYf8m
[22:11:45] <Hestil> should I 'implement' the case of failing and rollback by remove / update to the previous state?
[22:23:54] <_roland> mm, seems I have to pull $sum outside the definition of the _id field for the $group stage.
[22:24:18] <_roland> Yet the mongo db manual does give a $sum example where the $sum is inside the _id group... http://docs.mongodb.org/v2.2/reference/operator/aggregation/group/
[22:27:22] <iliakan> I don't want to enable replication yet and the opLog is needed to stream into ElasticSearch
[22:31:24] <iliakan> @loannac: so the right way would be to setup this mongo as repl master, without slaves?
[22:32:38] <joannac> _roland: there's only one example on that page, and the $group is not inside the _id part...
[22:38:39] <joannac> iliakan: http://docs.mongodb.org/manual/tutorial/convert-standalone-to-replica-set/
[22:39:09] <joannac> Once you see the words "expand the replica set", you can stop - you don't want to expand the replica set (just keep it at one member)
[22:39:48] <joannac> _roland: there's only one example on that page, and the $group is not inside the _id part...
[22:48:43] <dimon222> <iliakan> I don't want to enable replication yet and the opLog is needed to stream into ElasticSearch
[22:50:11] <dimon222> however, to enable replication you dont need second server that will be a copy, you can just specify set name, and enable replication, it will just enable opLog
[22:51:00] <dimon222> if you want a replica set, you can add all settings and ip:port:credentials on your slave, but your current oplog server will be primary, thats pretty much what u need
[23:38:20] <hulkk> db.iom.update({name:'max'}, {$set : {arros:{'data':['jedna','dva']}}}) it creates me 2 subdocuments like arros{ data{} } but when i change 'data' to 'structure' is basically just override my 'data' i can't use push since it is not a array
[23:46:09] <hulkk> also why when i use dot notification i have to wrap it in " brackets before i could use it without
[23:50:10] <hulkk> WHY does $set only update the same "field" : "value" over and over again ?????????????????????????????????????
[23:54:39] <hulkk> cheeser well when i run this "db.iom.update({name:'max'}, {$set : {"arros.structure":{'eewe':'eedsae'}}}, {upsert:true})" twice with other field : value it just replacing that same field : value and not adding
[23:55:04] <hulkk> cheeser so i really have to use arrays allways when i want to "append" some field: value as mutch as i run the update query