Log file Viewer

[02:47:00] <anohigisavay> hi. i have some records that look like this: {'dataSource': {'app':1, 'spv':2}, 'tags': ['a', 'b',...]}

[02:47:22] <anohigisavay> i want to get the distinct values of tuple (app, spv, tag)

[02:47:33] <anohigisavay> for all records. what should i do?

[05:03:24] <JaVaSan> Hi there, I have a document with some nested documents (three levels). I have a list of IDs and I need to check if all these IDs exist in any nested document (the ones in the third level). Any advice on how to accomplish this?

[05:05:01] <JaVaSan> Sorry, guys. Never mind. My scenario is a way simpler one.

[08:45:12] <ams_> We're hitting errors with "BSONElement: bad type " -2"" which look like corruption. My best solution is to --repair?

[08:45:44] <ams_> When I google about data corruption errors most people talk about syncing from replicas, in our dev env we only have 1 mongodb instance. Do most people run multiple for this failure case?

[08:51:20] <ams_> assertion 10334 BSONObj size: 872415853 (0x3400026D) is invalid.

[08:51:23] <ams_> is what I'm getting

[08:53:35] <ams_> And when I call db.repairDatabase(): exception: BSONElement: bad type -2

[08:55:45] <ams_> But mongodump works fine

[09:22:01] <ams_> Ah, but mongorestore does not

[09:22:10] <ams_> Any pointers for unpicking a messed up bons file?

[10:19:50] <pokEarl> Is there any reason why I should be wary of logging all queries to a database for a while just to see whats going on? like is this an intense exercise for mongodb or something? :(

[12:13:59] <JaVaSan> Hi, is there any way to store a field with a name, but retrieve it with another (it can be something configure during query, if possible)?

[12:15:04] <JaVaSan> Note: I'm not talking about virtual here, once in virtual you still keep the original records returned. Unless I use a virtual and somehow does not return only the other fields I don't want...

[12:22:14] <JaVaSan> It seems the virtual field is not retrieved in query.

[16:16:15] <Owner> Is there a way to autoscale mongodb on amazon aws?

[17:10:31] <kba> say I havea collection X with 10 properties a,b,c,...,j or so. Each of these then have a list with 1000 pretty rich documents.

[17:10:43] <kba> If I just want the name of my a,b,c,...,j, is that a heavy task for the engine?

[17:11:10] <kba> I'm not sure how it's structured on disk or in memory, like if it has to go through all the subdocumnts in a to find the title of b

[17:11:46] <kba> or if all the properties a,b,c,... are right "next to each other" in memory

[17:12:16] <kba> I just want pulling out a,b,c,...j to take as long as if I was pulling out the entire collection.

[17:12:42] <kba> so I'm wondering if I should just create another collection that caches just the name of a,b,c,....,j, or so

[17:45:55] <jr3> is it common to capitalize mongoose model variables?

[17:48:14] <StephenLynx> I strongly suggest you don't use mongo.

[17:48:28] <StephenLynx> it is an awful piece of software infamous for doing an awful job.

[17:48:46] <Derick> mongo or mongoose? :)

[17:48:46] <StephenLynx> and it seems to be the only source of issues with people using node.js

[17:48:49] <StephenLynx> mongoose

[17:48:52] <StephenLynx> :v

[17:49:19] <StephenLynx> I have tendency of not paying too much attention lol

[18:00:13] <BadApe> silly question, so i have a document, i want to make the name property unqiue, is that something i should be doing in code?

[18:00:46] <StephenLynx> you need an unique index. it can be done in your code or outside it using the terminal.

[18:01:00] <StephenLynx> I like to always ensure indexes on initialization of my systems

[18:01:24] <BadApe> i was told not to use ObjectId references yesterday

[18:02:10] <StephenLynx> theres no particular reason to not do it.

[18:02:27] <BadApe> hmm

[18:02:28] <cheeser> BadApe: who told you that?

[18:02:30] <StephenLynx> relations in general must be done carefully, but _id isn't any different than any others.

[18:03:10] <BadApe> cheeser, well i am using the vert.x api, and i asked how i could do it using their async api, and they told me they won't support it because i shouldn't use it

[18:03:23] <cheeser> they're misinformed

[18:03:45] <BadApe> sigh

[18:06:39] <BadApe> right now they only let me use embedded documents

[18:09:16] <Owner> anyone find a way to autoscale replica set on aws?

[18:12:33] <cheeser> as in, to autoprovision a new node?

[18:12:54] <Owner> yeah like add more nodes as demand increasees

[18:13:08] <cheeser> replica sets aren't really for scling like that.

[18:13:12] <cheeser> scaling

[18:13:24] <Owner> oh?

[18:13:28] <Owner> how do you scale capacity

[18:13:33] <cheeser> sharding

[18:15:58] <BadApe> newbie question, is the a ui to design schema?

[18:16:02] <BadApe> err

[18:16:11] <BadApe> i mean admin indexes

[18:16:25] <Owner> so how can i add shards dynamically?

[18:16:43] <cheeser> i don't know of a system that does that for you

[18:17:07] <Owner> ok

[18:21:23] <Owner> so is it possible to chance the number of shards as you grow?

[18:21:47] <Owner> oh

[18:21:48] <Owner> nevermind

[18:25:36] <cheeser> you can add shards on the fly yes.

[18:35:54] <stickperson> anyone know how to clone a collection or db in pymongo

[18:48:21] <deathanchor> mongo has a UI?

[18:48:34] <daidoji> stickperson: db.collection.copyTo()

[18:48:47] <daidoji> might be one for databases too, I'm not sure about that one though

[18:48:59] <deathanchor> stickperson: http://docs.mongodb.org/master/reference/method/db.collection.copyTo/

[18:49:35] <deathanchor> http://docs.mongodb.org/manual/reference/command/copydb/

[18:50:24] <stickperson> deathanchor: ah, not implemented in pymongo

[18:51:05] <deathanchor> copyTo?

[18:51:08] <deathanchor> or copydb?

[18:51:17] <deathanchor> all things can be run as runCommand()

[18:54:34] <stickperson> deathanchor: just saw copydb is available. i want to copy from an external server to my local one

[18:54:38] <stickperson> http://api.mongodb.org/python/current/examples/copydb.html

[18:54:48] <daidoji> there you go

[18:55:34] <stickperson> daidoji: not sure what to put as “todb” though.

[18:55:35] <deathanchor> I run all my replicaSets with w:0 and single member :)

[18:55:48] <stickperson> my client is the external client

[18:56:18] <stickperson> i think i have to do it in the mongo shell :/

[18:56:42] <daidoji> stickperson: whatever you want to call the new db?

[18:57:17] <stickperson> daidoji: i think client has to be wherever you want to copy to, so in my cast my local client.

[18:57:28] <stickperson> so yea, todb would be the db name i want to clone to

[18:57:53] <stickperson> but the source server is pw protected, so i’d have to call it from the mongo shell

[19:02:30] <daidoji> stickperson: well there you go then

[19:06:26] <deathanchor> stickperson: read that wiki page, there is something about that

[19:06:59] <deathanchor> documentation, for the walrus!

[19:08:52] <stickperson> got it working. thanks :D

[23:39:08] <yopp> I don't think there's enough information on this ticket to point to a bug in the server, and since we keep the SERVER project for reporting bugs in (or feature suggestions for) the MongoDB server I'm going to close this ticket. If you can provide evidence of a bug in mongod please post it here so we can reopen this ticket.

[23:39:12] <yopp> really?

[23:39:52] <yopp> what the fuck

[23:51:33] <cheeser> which ticket?

[23:51:45] <yopp> https://jira.mongodb.org/browse/SERVER-20149

[23:52:48] <cheeser> to me, that looks like either a post to mongodb-users or a support ticket.

[23:53:11] <cheeser> triaging it might lead to a server bug and thus a ticket but that looks more like a question than a bug report

[23:53:32] <StephenLynx> I can agree that the info only points that something is wrong somewhere.

[23:53:53] <StephenLynx> and not to a bug itself

[23:54:33] <Boomtime> yopp: the reason you are getting these comments is (at least in part) because of stats like this -> nscannedObjects:44547 ... nreturned:1

[23:54:51] <yopp> Okay, to simplify: we had 2.x for couple years, and it was fine, except the fact, that we had issues with IO brandwidth

[23:54:54] <Boomtime> that isn't a problem with the server, it's the server doing the best it can in light of badly targeted queries

[23:55:13] <yopp> We're upgraded our small cluster to the 3.0 with WT, to reduce the IO using the compression

[23:55:39] <Boomtime> compression trades IO for CPU

[23:56:10] <Boomtime> yes, it reduces IO, by spending CPU, if your IO was because of badly targeted queries, then now your CPU will be loaded for the same reason

[23:56:21] <yopp> And it was fine. Then we're upgraded our main cluster to 3.0 and now we'e facing the issue, when WT is randomly goes nuts

[23:56:40] <StephenLynx> I would wait until 3.1 to use WT

[23:56:44] <StephenLynx> when it will become default

[23:59:23] <yopp> The thing is: we're restarting the mongo, and everything is fine. I can see from MMS that it goes on the full speed. It can run for day or so. Then something is happening and we're seeing slowly increasing CPU usage, with slowly rising query response time

[23:59:26] <yopp> With the _same_ workload