Log file Viewer
#mongodb logs for Wednesday the 15th of June, 2016
[00:23:57] <jgornick> StephenLynx: I finally found a solution to what I was trying to do: https://gist.github.com/jgornick/eace131a44a2b594d51f1b3d9f94e6ca
[00:25:51] <StephenLynx> it still doesn`t remove multiple elements from the same sub-array, does it?
[00:25:51] <jgornick> The gist of the gist, ha funny, is to show the schema and the update statement :)
[00:45:29] <klaxa> hi, i must be stupid: https://gist.github.com/klaxa/04cb745b9955ab45a85464f19cc8da3d
[00:46:26] <klaxa> the first file is demonstrating how i get the duplicated keys error, the second file is the index i created and the third file is the pythoncode (pymongo) i used to create said index (not that it should matter really)
[00:52:02] <klaxa> if i change the value of "path" to something that doesn't include "music" it works
[00:57:50] <Boomtime> @klaxa: i'm not sure a unique 'text' index could possibly be what you want.. i didn't even know that was possible
[00:59:02] <Boomtime> your answer is probably here -> https://docs.mongodb.com/manual/core/index-text/#index-entries
[00:59:25] <Boomtime> that explains why you get dup key violations - the index contains a lot more entries than you think
[01:00:37] <klaxa> so i'm guessing it's building a reverse index with the strings tokenized to find stuff faster
[01:01:16] <Boomtime> right, you're asking for a unique constraint on the combination of every stem in the text field combined with the second field
[01:03:19] <Boomtime> mongodb doesn't care what the value is, it just indexes whatever is there - when you say 'text' you are asking mongodb to explicitly inspect the type and deconstruct it as a tokenized string field
[01:05:02] <klaxa> yeah, i get it now, text is a special case of string (which is just a special case of data)
[01:48:30] <sector_0> I have a database that stores user information, and I have need for a user id (or something to that effect), should I use the _id field for this, rather than make a separate userID field?
[02:00:55] <Boomtime> @sector_0: be aware that _id is also just a field like any other, you can specify your own value for it
[02:04:41] <StephenLynx> while if you use this field as an index in the first place, you don`t have to do that.
[08:30:05] <shayla> Hi guys. I've got the following query : http://pastebin.com/BpR1GNPu What I need is to make the count ($sum : 1) only for field that respect some $match condition but I don't want to add it on the general $match to the query but just for the count. Is there a way to do it on the same query or I need to do two distinct query?
[09:44:58] <jelle> I have an issue with showing some stats data to customers, the initial query is slow, but doing it the second time it's super fast. Any tips how I can make the initial query faster? (I'm using a hdd backend mongodb)
[09:45:23] <jelle> I havne't checked if the first query has a lot of page faults but I guess it does
[11:46:37] <kknight> I have made an app . Now I want to make a new collection for a model .. how to do that?
[13:23:21] <shayla> Hi guys, i'm having a problem working with mongodb. This is what I do in mongodb family: {$sum: { $cond: [ { $eq: ["$content.products.product", 99] }, 1, 0 ] } } . I need to do that query in php so i do something like 'family' => ['$sum' => ['$cond' => ['$eq' => ['$content.products.product', 99]], 1, 0]]
[13:37:27] <jayjo_> I have a field in my document that is a UTC timestamp in seconds (since the epoch). Is there a way to interact with dates in a query document, or do I have find what time in seconds is certain date marks and go from there? For a concrete example, to get all records from the last day or something similar
[13:44:06] <jayjo_> This data is from a provider that exports to json. Is there a way to have it automatically convert? Or do I run something periodically to convert it?
[13:49:33] <StephenLynx> with the node.js driver you just give it a date object that the driver handles the conversion.
[14:08:12] <m1dnight_> when I do db.collection.find().sort(..);, does that do the sort in the db engine, or does it first fetch the entire table and then sort in memory?(outside of the db)
[14:32:02] <kurushiyama> m1dnight_ That depends on your setup and your indices. (and order in which you give the sort params)
[14:54:26] <jayjo_> So, the data is already inserted using mongoimport, and the _t field is an integer. I can use .forEach() in the mongo console. Is there a way to automate this within mongo or do I just set a cronjob to execute periodically? Like in sql I could use triggers or view functions - is there an analog?
[15:10:00] <jayjo_> Can I use anonymous functions in the javascript shell? liek this: { _t : { $gt: function() { var d = new Date(); d.setDate(d.getDate() - 2); return d.getTime(); } } }
[16:05:29] <scmp> end of what stream? the docs could be a little bit more specific. And stream is not mentioned anywhere else.
[16:06:05] <scmp> it's a large collection, a simple query if field a exists and field does not exists, and a combined index on both fileds, not sparse.
[16:11:04] <hardwire> The docs DO cover it. But I think you may want the functionality to meet some expectations that aren't described.. and I'm not sure there's more to say about that flag.
[16:13:27] <scmp> what i expect is that winningPlan mentions the index and i don't understand why it's not able to select the index
[16:21:26] <idioglossia> StephenLynx, he means if the collection is super big, does mongo limit the query to return, say, the first N items (where N is extremely large but not infinite) instead of returning all of them
[18:46:40] <deostroll> Hi. Anyone out here can answer this question: http://softwarerecs.stackexchange.com/q/33891/8530
[18:48:26] <StephenLynx> get document with field X, on the next query, ask for the minimum value of X to be the highest X you got on the first query
[18:55:23] <StephenLynx> sequential access is pretty much a standard feature of anything reading data.
[18:58:03] <deostroll> in the shell I've tried setting batchSize, but instead it returned all the documents
[19:04:17] <StephenLynx> the other day had one guy here that was neck deep in trouble because of users crud'ing directly
[19:04:38] <StephenLynx> killing the database with overly complex queries and asking him to just fix it.
[19:04:48] <deostroll> my c# client, for e.g. needs those drivers/libraries installed; if I were to talk to direct mongodb
[19:10:38] <deostroll> so if at all I need to get the next set of records from the cursor after I've exhausted it...
[19:12:54] <StephenLynx> you said that you have exhausted it. in that case you don't do anything else with the cursor.
[19:13:36] <jgornick> Hey folks, is it fair to say that appending a value to an array is going to be much more performant than prepending?
[19:14:42] <StephenLynx> besides, when you edit documents, I heard that the bottleneck lies on updating indexes if the document size changes enough.
[19:15:48] <jgornick> StephenLynx: Based on that information, it doesn't matter if you prepend or append.
[19:20:51] <t3chguy> Hello, I'm just getting acquainted to Mongo, and am wondering whether I can use db.collection.save with $currentDate to be able to track a modifiedDate value with the document;- Cheers
[19:21:28] <jgornick> StephenLynx: Yeah, that's what I'll have to do because there are situations where I want it sorted.
[19:26:45] <t3chguy> StephenLynx: so would I just shove the whole document as the $set and its _id as the query?
[19:47:12] <t3chguy> StephenLynx: all my documents are pre-existing, so wouldn't even need to upsert, I have a task that needs to go through each and update some, thus .save would have been ideal for its syntax of just being the document itself but I also need to update the modifiedDate
[19:49:14] <StephenLynx> an update that isn't marked to update many or if it uses an indexed field to find the documents to be updates are optimized.
[19:52:23] <sequence> Hey channel, is there an existing solution for resilience in the face of a replica set primary being stepped down? Something like a smart driver that will reconnect and reissue queries, or a MITM proxy (like dvara)?
[19:56:37] <sequence> GothAlice: looking at something like https://emptysqua.re/blog/save-the-monkey-reliably-writing-to-mongodb/, it seems like reconnecting and reissuing is something the (Python in this case) client has to take care of.
[19:57:10] <sequence> Even though that is quite old, I'm still seeing Exceptions with our use of the Java driver (3.2.2)
[19:57:51] <sequence> `rs.stepDown()` triggers a `com.mongodb.MongoSocketReadException: Prematurely reached end of stream`
[19:58:26] <GothAlice> sequence: http://api.mongodb.com/python/current/api/pymongo/mongo_client.html < note the paragraph under the MongoClient() constructor section at the top starting with "The client object is thread-safe and has connection-pooling built in."
[19:58:56] <GothAlice> You *do not* want the driver automatically re-trying every query. The results could be disastrous.
[19:59:39] <Derick> GothAlice: no driver should attempt to re-issue a write when a connection breaks down, or a new node gets elected as primary
[20:01:33] <GothAlice> Derick: Ah, found it. Indeed, I was referring primarily to the "reconnect" part of the query, not the "reissue" part.
[20:03:50] <GothAlice> sequence: At work we divide our queries into a matrix of risk. One dimension revolves around write concern, another around pattern of retry. From "no need to retry, failure is OK" through "retry is trivial, just repeat the operation" (a la $set), to "yowza, we need to run some custom code to determine what to do next".
[20:04:33] <GothAlice> (Such as checking a document version number and calculating a new atomic operation to "catch up".)
[20:16:40] <tesjo> Is there a db query called offset.Could not find any on doc..But I dont get error when using offset.I am using mongoid
[20:37:57] <pokEarl> do you lose the ordering of documents or something when you do a compressed mongodump?
[20:40:38] <pokEarl> have some script i'm running to generate test data and supposedly it generates the test data into a database, then creates a dump of that test data and uploads it somewhere. When I run tests against the generated database they all pass but when i download the uploaded dump restore it and run against them they fail, but guess its something else weird thats going on then :(
[20:48:42] <pokEarl> does something else change then? just ran tests against a database everything passes, then did manual mongodump, dropped the database and did a mongorestore and now they fail