Log file Viewer
#mongodb logs for Friday the 18th of July, 2014
[05:17:27] <joannac> you'll want to redesign your schema and then manipulate the data rather than doing a straight import
[05:25:24] <bluesm> joannac: Well. Could you recommend me any article how to change scheme from traditional "Database->Table->Joins model" ?
[07:31:23] <bluesm> What should I do if my database scheme is clearly "joinable".... That is "I have a Text_decriptors" table which is joined with the "word_descriptors" table...
[07:32:13] <bluesm> (Like text contain words, and words are assigned to specific text) (I need split words since it is language learning app, where every word/phrase needs translation)
[09:05:51] <rspijker> bluesm: I’d probably just go with a texts collection and a words collection. Your app will probably always be in the context of a specific text. So then you can just handle the ‘join’ in the application layer.
[09:06:10] <rspijker> That is, if you are working with a specific text, you can just make the textId pat of your queries on the word collection
[09:14:55] <bluesm> rspijker: And do you think I should go with joining myself with the "repetitions_words_descriptors" that in normal SQL database have fields : "user_id" (linking to the user) , "word_descriptor" (linking to the word with translation), and "repetition_count", and "easiness factor" .... and other properties of the repeated word (The
[09:16:12] <rspijker> I think you should embed most of these properties, where it makes sense at least
[09:17:40] <bluesm> rspijker: Embed in the user? Like array of the users's words in the User document ?
[09:18:09] <rspijker> That doesn’t make much sense to me, but again, I don;t know how your application works
[09:22:12] <bluesm> rspijker: Ok. I'll experiment. Now performance isn't that much of the deal. Thanks for time and effort!
[09:23:04] <bluesm> rspijker: I think that I need create separate document for repetitions_words_descriptors since I'll need to sort them (like by "repetition_count" field)
[09:24:36] <rspijker> The main thing is that the way in which the data is stored makes sense to you and makes sense in the way your application uses the data
[09:42:47] <slap_stick> hey, i was reading about replica sets and from what i can tell, you are not able to define a replSet via the configuration file that gets passed in to mongod, but you have to provide it to the mongod as a parameter i.e. --replSet "name" is that really the case?
[09:50:54] <ksinkar> hello I am running on ubuntu 12.04 and have installed mongodb using apt-get install. Is it a good idea to use the standard mongodb package provided by Ubuntu?
[10:36:27] <emps> hii im getting --- warning Cannot detect if NUMA interleaving is enabled. Failed to probe "/sys/devices/system/node/node1":
[10:38:55] <braz> can you try "dmesg | grep -i numa" to see if NUMA is enabled but best is to turn it off via numactl and disable NUMA via the BIOS
[10:43:13] <ruairi> re numa, just say no: http://docs.mongodb.org/manual/administration/production-notes/#mongodb-and-numa-hardware
[11:27:03] <devastor> Any ideas why there's a limit of 1024MB for the chunk size even when manually trying to move a chunk to another shard? How can I move a jumbo chunk bigger than that?
[11:30:12] <cheeser> this might help: https://groups.google.com/forum/#!topic/mongodb-user/YErQ7gu2Ry8
[11:34:09] <nfroidure_> does some of you use mongodb with relationnal datas ? How do you handle constraints, joins as you would with a traditionnal SQL database ?
[11:37:51] <devastor> cheeser, I read it already, but it doesn't help , because the chunk can't be split anymore and the mongo does not allow setting the chunksize to bigger than 1024, at 1024 it still says that it's too big
[11:39:27] <braz> emps http://stackoverflow.com/questions/18416671/starting-service-mongodb-fail-on-permission-denied-sys-devices-system-node-no
[11:40:24] <devastor> cheeser, it worked for him when he set it to 120 instead of 10000 (because anything bigger than 1024 is just ignored), but 120 or 1024 is not enough in our case
[11:42:50] <bmcgee> Hey guys, I just updated my java driver from 2.11.x to 2.12.x and have found a breaking change in the behaviour of tailable cursors. If I add the option QUERYOPTION_TAILABLE the driver automatically adds QUERYOPTION_AWAITDATA forcing the cursor to block for new data when calling next(). This did not happen in 2.11.x and from what I can see in the API there is no way of unsetting the await data option.
[11:46:30] <bmcgee> “There are no circumstances where a user would not want to also set Bytes.QUERYOPTION_AWAITDATA, so if the former is set, the latter will now also be set.”
[11:48:30] <bmcgee> I have an actor which tails a collection and on each read exhausts the cursor creating batches. When hasNext() returns false it schedules another attempt at reading a bit later
[11:49:33] <bmcgee> except now if i don’t receive any data for a while the thread for that actor will be tied up
[11:54:47] <bmcgee> cool, in the meantime I need to make a choice: do I like cursors from aggregation results more than i like a tailable cursor which doesn’t await data….
[12:04:10] <keabard> Hi everyone! I have performance troubles doing bulk upsert with PyMongo, anyone already experienced that ?
[12:12:17] <Guest10682> I'm afraid, I'm not sure to be at the right place to ask this: I have a polygon that I want to carve up with a predefined number of subpolygon with the same area each. Does anyone have an idea ?
[12:15:49] <bmcgee> I would suggest a room concerned with C++ development then or programming in general
[12:17:08] <Guest10682> It is not a development problem, I'm looking for a mathematical algorithm (the underlying idea for such algorithm)
[12:25:18] <remonvv> bmcgee : I object to your implication that C++ programmers are the most obvious heroes to solve that (wo)man's mathematical problem!
[12:26:44] <bmcgee> remonvv: as a Java/Scala developer I intended no such implication. What little part of my brain I devoted to processing the question simply flagged the keyword C++, thus the resultant semi-sarcastic remark was a little half baked
[12:33:33] <remonvv> bmcgee : I'll admit, albeit reluctantly, that that is a punishment that fits the crime.
[12:38:37] <bmcgee> i like Scala quite a bit, gets overly fancy in places. Been curious about Kotlin and Go for a while as a result
[12:39:51] <remonvv> Yeah I've dabbled in Scala and that was exactly my impression. A bit too much of everything.
[12:47:55] <bmcgee> cheeser: I just thought I’d be sneaky and try to go around the API with some oh so very hacky reflection manipulation of the _options flag in the DBCursor…… but yeah, sneaky didn’t work :(
[13:22:46] <uehtesham90> how can i output the results of a find() to a new collection in the mongo shell??
[13:23:25] <uehtesham90> do i do something like: db.source_collection.find(query).copyTo(destination_collection)
[13:26:18] <bmcgee> uehtesham90: only method I know for outputting the results to another collection is with the aggregation framework
[13:26:58] <uehtesham90> but the problem with that is that i can group by the specific fields, but there it will only output those fields
[13:27:25] <uehtesham90> i also want to output additional fields which are not part of group parameter....is there a way to do that in aggregation?
[13:28:44] <bmcgee> uehtesham90: I was wrestling with a smiliar problem where I would have multiple versions per key I wanted to group by, and what I wanted was the first document for each group id. But I wanted the whole document. I messed around with specifying all the document fields with a $first parameter before I just re-structured the problem to avoid it.
[13:32:09] <bmcgee> I did something like { $group: { _id: { foo: 1, bar: 1 }, strField: { $first: ‘strField’ }, intField: { $first: ‘intField’ } …… }
[13:56:58] <rspijker> why would you need to group if you want to get the results of a find uehtesham90 ?
[13:57:12] <rspijker> ‘additional fields of a document’ don’t make much sense in the context of grouping...
[13:58:13] <uehtesham90> so what i want is to get all records that satisfy a certain query and then sort them by some fields and output the result directly to either a new collection or even better a csv file
[13:59:04] <uehtesham90> i dont know how to output the results of my find and sort to csv file. i cannot output to new collection as ill have to use aggregation
[14:02:04] <rspijker> uehtesham90, so how about: db.coll.aggregate({$match:{query}}, {$sort:{“field”:direction}},{$out:”collection”})
[14:02:58] <rspijker> if I want csv output, what I usually do is use robomongo as a client and just use javascript to output csv then copy if into my text editor manually...
[14:16:48] <rspijker> hmmm, I’m guessing $binary is not an operator, it’s just something to denote that there is binary data there...
[14:18:43] <disorder20> yeah the problem is I'm using a fixtures lib for mongo which doesn't support gridFS directly
[14:21:16] <remonvv> I don't think manually modifying the chunks collection will end well. That collection is managed by mongos and you might cause consistency issues.
[14:22:34] <rspijker> disorder20: don’t know your problem, but if the option exists to use an external tool, there is mongofiles ….
[14:23:25] <rspijker> remonvv: not sure.. The chunks themsleves aren’t managed by the cfg server though. Which is the big difference
[14:24:51] <disorder20> this small library I found just loads an external js file and load automatically all objects into mongo
[14:25:29] <remonvv> "Let's call these chunks too, it's roughly similar!" "But..." "Thank you for your attention, get to work"
[14:33:14] <remonvv> You can store chunks in chunks, when a chunk is sharded they are split into seperate chunks and distributed across chunks
[15:57:35] <ggoodman> I'm going to be doing queries that hit a couple fields via $text, potentially an array field and sorted descending by another field.
[15:58:05] <ggoodman> I've read that there is some nuance in the order I use to define this type of index to maximize performance.. can anyone chime in?
[18:16:55] <uehtesham90> can i run two aggregation commands on two different mongo shells without affecting the other.....both commands aggregate from the same collection on the same key but different values
[19:41:15] <hrrld> There's some example code on this page: http://docs.mongodb.org/manual/tutorial/create-an-auto-incrementing-field/ that defines a function called "getNextSequence" ... Is this javascript running inside mongo?
[19:50:39] <thebope> Currently we do things like this in the queries in my java application withField("someObject.someField in", someList)
[20:15:06] <smik> I have question about what would be a good way of storing "Following" kind of information. User A is following user B. What I did is that have a 'following' array in for a user and add ObjectID of all users he is following. But how can I determine (efficiently) how many people follow a particular user.
[21:26:49] <bob_db> hello, have a question: for text search, do I have to do anything for find to match non-alphanumeric characters?
[21:27:05] <bob_db> this query, for example, does not return results even though I know it should: db.collection.find( { $text: { $search: "+term" } } );
[21:32:41] <Pulpie> Is there a way to batch update in a way I can just update all the documents I need to with one command?
[21:40:23] <bob_db> is it possible to do the text search I mentioned previously without having to do a regex?
[22:04:25] <future_phd> I noticed that I don't have to index my collection by date if I want to query it and have results sorted by date. Should I index anyway, or is this reliable in general?
[22:06:26] <future_phd> basically, im saying that I noticed that documents are stored in the order that they are inserted, and I have a field for the date of insertion, and, so far, I don't have to index by that field
[22:06:44] <future_phd> but I don't know if this could go wrong after sharding or something later on in the db lifecycle
[22:09:03] <kali> future_phd: but if you're using objectIds as _id, you can use the "free" _id index to get roughly a creation date order
[23:00:08] <cirwin> mongorestore on my unloaded mac book pro is proceeding at about 3MB/s — is there any way to speed it up?