PMXBOT Log file Viewer

Help | Karma | Search:

#mongodb logs for Wednesday the 8th of January, 2014

(Back to #mongodb overview) (Back to channel listing) (Animate logs)
[02:51:10] <topwobble> can anyone recommend a HTML tool to inspecting mongo data?
[02:52:24] <mrpoundsign> topwobble: you mean like pma for mongodb?
[02:52:53] <mrpoundsign> http://docs.mongodb.org/ecosystem/tools/http-interfaces/
[02:53:25] <topwobble> i don't know pma
[02:53:41] <topwobble> which is your favorite in that list?
[02:55:06] <mrpoundsign> oh idk, haven't used one in a while. :) there's more here: http://docs.mongodb.org/ecosystem/tools/administration-interfaces/
[02:55:46] <mrpoundsign> I used RockMongo for a little while. It was decent.
[02:59:11] <lqez> topwobble: how about tadpole / https://github.com/hangum/TadpoleForDBTools
[02:59:52] <topwobble> thx all. will check em out
[03:00:04] <topwobble> the simplest will do for my use
[08:00:31] <Varun_Krishna> Hello everyone. I have a question
[08:11:34] <Varun_Krishna> I would like to move the data of the database in myserver to mongodb. Is it possible to do this in mongodb?
[08:21:56] <Varun_Krishna> \part
[09:33:03] <master_op> hello, i'm fowllwing this tuto, http://dba.stackexchange.com/questions/22813/mongodb-returning-the-points-nearest-to-a-location-with-distance
[09:33:41] <master_op> after the first db.runCommand i got this error
[09:33:49] <master_op> { "errmsg" : "no geo index :(", "ok" : 0 }
[09:37:13] <tiller> Hi there
[09:38:27] <tiller> I've some mongos/c/d servers running on 4 machines, and I try to set proper iptables
[09:38:40] <tiller> But when I set, on 10.0.0.250 for example, those ip tables: http://pastebin.com/7rAZD9xR
[09:39:00] <tiller> If I remove the "ACCEPT ANY", my replication stop working
[09:39:19] <tiller> do you have any idea of what I'm missing? (and yes, port number are correct)
[12:43:20] <djlee> Does a compound unique index apply in both directions. So if i have a unique index on "FieldA" and "FieldB", and i enter 1 into FieldA and 2 into FieldB, can i make it so the unique constraint will also stop 2 being inserted to FieldA and 1 being inserted in FieldB
[13:19:52] <Nodex> djlee : unique constraints are on the index as a whole
[13:25:56] <djlee> Nodex: right ok, i presume it was a one way uniqueness so "1,2" != "2,1". Clearly not then, which saves me writing a load of management code :D. Thanks for your reply
[13:44:53] <singh_abhinav> is it good practice to always set write concern to Journaled
[13:45:57] <KamZou> Hi, is this possible to use 2 or more shards in parallel for writing ?
[13:53:52] <Nodex> singh_abhinav : that;s down to the compliance your app needs
[13:54:35] <Derick> KamZou: can you elaborate what you're trying to do?
[13:54:46] <Derick> Nodex: a little bird is telling me that j=1 is going away...
[13:57:21] <KamZou> Derick, i'm not using balancer (cause i've got some contraints) and i'd like to know if there's a way to configure mongodb to write not only on one shard but on many ? (at least 2)
[13:58:53] <cheeser> it'll write to the shard for that shard key
[13:59:10] <cheeser> each shard can have it's own replSet, though, iirc
[14:01:02] <Derick> cheeser: correct
[14:01:31] <Derick> KamZou: do you want all the data twice, or just parts to each shard?
[14:01:44] <ron> cheeser: YOU GET A COOKIE!
[14:03:05] <KamZou> Derick, parts to each shards
[14:04:53] <Derick> KamZou: when why not use the mongos balancer?
[14:05:30] <Derick> mongos will make sure half your data gets written to each shard (with a properly chosen shardkey)
[14:05:52] <KamZou> Derick, because my dedicated servers haven't the same volumetry at all
[14:06:20] <KamZou> Derick, some of my shards are currently full of space but i want to keep this retention
[14:06:21] <Derick> KamZou: then you need to do the sharding by hand, and your app needs to know to which of the shards to write
[14:08:09] <KamZou> Derick, my apps doesnt "have to know" which shard to write... The backend is the mongoS who dispatch to the correct shard (i set manually)
[14:08:39] <Derick> the writes will be in parallel.... why do you think they are not?
[14:08:41] <KamZou> Derick, but i'd like to know if i could have 2 shards at the same time in the way i do with one today
[14:08:50] <Derick> huh?
[14:08:54] <Derick> you already have more than one shard
[14:09:01] <Derick> what do you mean by "2 shards"
[14:09:05] <Derick> you mean 2 clusters?
[14:09:25] <KamZou> mmm 2 shards in write at the same time
[14:09:51] <Derick> you mean you want one document in two shards?
[14:11:04] <KamZou> no, i'd like a document A in the shard1, and a document B on shard 2 (if it's possible)
[14:11:13] <KamZou> (without the use of balancer)
[14:12:58] <Derick> KamZou: but you said you are already doing that, by having configured mongos yourself
[14:14:07] <KamZou> Derick, i guess i'm unclear
[14:14:20] <Derick> I think so :-)
[14:14:23] <cheeser> ron: i did it all for the cookie.
[14:14:41] <singh_abhinav> Nodex: that will make app little bit slow right
[14:15:15] <KamZou> i disabled balancer on my mongoS. Right know i've many shards. Each time a shard is "full" i manually edit the shard where "all" data goes. So data go to one shard, i'd like them to go to 2 or more
[14:15:38] <cheeser> why would you do that? do you hate your weekends that much?
[14:16:07] <Derick> how do you edit the shard?
[14:16:20] <Derick> a document always only lives on one shard, never more than one
[14:16:29] <Derick> otherwise mongos doesn't know where to get the document from
[14:21:57] <KamZou> cheeser, ahah.. You're right
[14:23:11] <KamZou> I'm just discovering this parameter : http://docs.mongodb.org/manual/tutorial/configure-sharded-cluster-balancer/#change-the-maximum-storage-size-for-a-given-shard
[14:24:23] <KamZou> i've got to know if the dedicated-server's volumetry is the only reason why we didn't use the balancer right now
[14:24:54] <KamZou> If it's i'll activate it with this kind of configuration (max storage size / shard)
[14:26:01] <Derick> oh, max size
[14:26:03] <Derick> that is handy
[14:28:45] <KamZou> why ?
[14:29:25] <KamZou> Mm i thought "touchy'"... Nevermind
[15:32:18] <tiller> Hi there
[15:32:44] <tiller> Question: which instance [mongod|mongoc|mongos] communicate with which instance?
[15:32:54] <tiller> (to apply my firewall rules)
[15:33:14] <tiller> mongod communicate together when replicated
[15:33:16] <Derick> tiller: everything needs to talk to everything
[15:33:23] <Joeskyyy> Depends a bit, if you have a mongos I'd probably start there with firewall rules
[15:33:32] <tiller> mongos communicate together?
[15:33:36] <Joeskyyy> Since a mongos is your "router" in the mongodb world so to speak.
[15:33:55] <Derick> tiller: that's the only exception. But all mongos's talk to all mongoc's and mongod's
[15:34:03] <Derick> mongod's also talk to all mongoc's
[15:34:12] <tiller> m'okay
[15:34:16] <tiller> Thanks :)
[15:35:31] <Derick> actually, mongod's will only talk to mongod's in the same shard (and to mongos and the 3 mongoc's)
[15:35:37] <Derick> so you can isolate per shard
[15:35:49] <tiller> per shard?
[15:35:53] <tiller> not per replicate?
[15:36:02] <Derick> each shard is a replicaset
[15:36:10] <Derick> in a replicaset, all mongods need to talk to eachother
[15:36:24] <Derick> but the don't talk to mongod's in a different shard's replicaset
[15:36:29] <tiller> yes ok. I misunderstood what you said :)
[16:30:55] <nollidj> i'm interested in mapping homogeneous documents in a collection to a stream of homogeneous documents, which will be inserted into a different collection. is mongodb's map-reduce function the right mechanism for doing this?
[16:45:00] <TehShrike|Work1> What happens if ensureIndex is called for a key that already has an index, but with different options than were used on the first index?
[16:51:31] <Derick> TehShrike|Work1: afaik, it creates another index - but test it?
[16:52:05] <TehShrike|Work1> The docs say about ensureIndex "Creates an index on the specified field if the index does not already exist."
[16:52:29] <Derick> it takes less time to test than it takes to find it in the docs though ;-)
[16:52:37] <kali> Derick: i think it is silently ignored :)
[16:52:49] <kali> TehShrike|Work: i think it is silently ignored
[16:53:10] <TehShrike|Work> Derick: not really, I don't know how to tell the difference between an index being rebuilt versus being silently ignored
[16:54:33] <cheeser> i'm with kali
[16:54:56] <Nodex> db.foo.getIndexes()....
[16:58:40] <TehShrike|Work> Nodex: thanks!
[16:59:39] <Nodex> you can also tell if it's building one with currentOp() iirc
[17:02:08] <TehShrike|Work> Looks like it silently ignores it, even if the options are different. Good to know
[17:12:47] <cheeser> because the name is already there.
[18:16:02] <Veejay> Hi, if I have a MongoDB replica set running locally, how can I manually kill the primary or make it so that it becomes unavailable (to see if the re-election procedure happens as expected)?
[18:16:32] <ehershey> get the pid and kill it?
[18:16:45] <ehershey> with -9 if you want to simulate real sudden instant failure
[18:17:04] <Veejay> Isn't that brutal?
[18:17:11] <Veejay> Or maybe brutal is what I actually want
[18:17:34] <kali> i think you do want brutal.
[18:17:34] <Veejay> I was looking for a more mongo-centric procedure but yeah I guess that'd work fine
[18:17:38] <Veejay> OK
[18:17:40] <Veejay> I'll try that
[18:18:31] <Veejay> Though it does beg a question
[18:18:47] <Veejay> Is db.printSlaveReplicationInfo() the way to get the info about which one is the primary?
[18:19:03] <Veejay> And will I have the info about the process PID that way (to know which one to kill)
[18:21:58] <ehershey> well it depends what exactly you want to simulate
[18:22:03] <ehershey> I'd get teh pid via the os with ps
[18:22:05] <ehershey> the
[18:22:46] <ehershey> I would get state info from rs.status()
[18:58:25] <ShortWave> hi all
[19:02:40] <nollidj> is there a good way when running the mapReduce function to have the resulting records merged with an existing collection rather than overwriting it?
[19:03:52] <nollidj> ah, maybe the action key will do it
[19:05:20] <gregmash> does anybody here know about mongoose?
[19:06:44] <Nodex> gregmash : best to just aks the question ;)
[19:06:47] <Nodex> ask*
[19:06:52] <gregmash> ok
[19:07:05] <gregmash> i need help cloning data in mongodb using mongoose
[19:07:17] <gregmash> and if it can be done with node streams
[19:07:30] <gregmash> i have not seen an example of streaming data into mongoose
[19:08:27] <nineteen1ightd> I have a document with a field that contains an array of subdocuments. Those subdocuments contain a field that itself has an array of subdocuments, e.g. {foo: [{bar: [{baz: True}]}]}. I can find documents in the collection easily enough based on a value in the inner most subdocument, however, I really want to apply updates to a particular set of those inner most subdocuments...
[19:10:44] <nineteen1ightd> I tried something like db.coll.update({'foo.bar.baz': True}, {$set, {'foo.bar.$.qux': true}})
[19:11:17] <loopj> does mongo have any way to sort a collection by the length of an array in each document yet?
[19:11:42] <loopj> we dont have a seperate count field, and I dont want to make one for this task (building an admin page)
[19:11:42] <cheeser> you could probably do that using the aggregation framework
[20:04:51] <wc-> hi all, im having some issues trying to do a disctinct query with 2 fields using the aggregation framework
[20:05:02] <wc-> i feel like im over complicating things
[20:05:56] <wc-> in sql it would be like a count with 2 columns
[20:09:38] <GM001> Hello everybody, does anybody know how to check failover history
[20:10:19] <GM001> to check when there was flips between primary and secondart servers
[21:02:00] <tripflex> aloha peoples!
[21:27:41] <tripflex> hey guys so wondering what the "best" and most efficient way to do this would be
[21:28:32] <tripflex> I have a collection that has a value "status", status is either equal to complete or incomplete, in building my web app to output into tables i'm currently using two find calls but my assumption is this isn't the most efficient way to do it
[21:28:36] <tripflex> so right now i have this
[21:30:23] <jmagder> Good afternoon (EST). Been playing with mongodb for a few weeks now and it is amazing. Been learning about improving search queries with hints, but I can't seem to get a hint to apply
[21:30:32] <tripflex> app.db.models.TPV.find({ OwnedBy: theowner, TpvStatus: 'incomplete' }, function (error, incompletedata){ app.db.models.TPV.find({ OwnedBy: theowner, TpvStatus: 'complete'}, function (error, completedata){ callback (incompletedata, completedata)
[21:30:46] <tripflex> now that's using node, express, and mongoose
[21:30:52] <tripflex> but it's basically a find inside of a find
[21:31:15] <tripflex> so I can pass the returned data in two objects, completedata and incomplete data, and then use swig template to output the tables
[21:31:15] <jmagder> I have a compound index consisting of a 2D key, followed by a multikey index.
[21:32:34] <jmagder> I'm putting in a $hint: { random_point: '2d', tags: 1}
[21:32:49] <jmagder> But it always uses a GeoSearchCursor, with isMultiKey set to false.
[21:36:58] <tripflex> here's a pastebin that should help identify my question:
[21:36:59] <tripflex> http://hastebin.com/luxusepime.lua
[21:39:51] <tripflex> whoops its in same collection, here's update one
[21:39:52] <tripflex> http://hastebin.com/jabiqoruwa.lua
[22:55:44] <djlee> If i have two mongo _id's. Is there any such thing as a "lower" id? Like with a integer id in mysql, 10 is smaller than 15. Is there any such thing in mongo?
[23:03:47] <Joeskyyy> djilee: Not really… the objectID is compromised of a few factors, such as unix epoch, procID, and a random value counter
[23:03:52] <Joeskyyy> http://docs.mongodb.org/manual/reference/object-id/
[23:04:04] <Joeskyyy> So short answer: not really
[23:08:12] <djlee> thanks Joeskyyy, suppose i'm looking for something that mimics mysql's autoincrement field, and theres a good reason mongo doesn't have such a thing. I'll have to start thinking about solving my problem in a different manner
[23:09:11] <tripflex> djlee: http://www.querymongo.com
[23:09:12] <tripflex> :P
[23:10:21] <djlee> tripflex: not sure that helps me, but i'll bookmark it, might come in handy :P
[23:10:34] <tripflex> yeah does for me lots
[23:10:40] <tripflex> at least to point you in the right direction
[23:12:00] <nineteen1ightd> Is there an update I can do to $set a new field on all embedded documents in an array?
[23:12:18] <nineteen1ightd> My research, thus far, says no...
[23:12:33] <djlee> i don't think there is, but is there a way of implemnting a bidirectional unique constraint in mongo? So if FieldA is 1 and FieldB is 2, i cannot then enter FieldA:2, FieldB:1. So basically the uniqueness would be on both 1,2 and 2,1 ?
[23:14:03] <nineteen1ightd> I'm not terribly familar with Mongo, however, I'm guessing a unique index on (FieldA, FieldB) is unique on that ordering of their values?
[23:15:27] <djlee> nineteen1ightd: yep, uniqueness is only on the order you specify from what i can gather from my tests. Which makes sense since first and foremost it's an index
[23:16:12] <djlee> just going through all the possible good solutions i can think to my problem, before i resort to coding something a little less nice and performant to handle things.
[23:17:10] <djlee> nineteen1ightd: not sure if this helps you but: http://stackoverflow.com/questions/6758339/updating-embedded-document-property-in-mongodb
[23:22:38] <nollidj> mongo automatically indexes the _id field of documents, right?
[23:22:56] <Joeskyyy> Yup
[23:23:18] <nollidj> so if i have _id fields set to values like { foo : bar, baz : { quux : blech } }, i should see that the index is being used when i make a query like { _id.foo : bar } }
[23:23:21] <nollidj> ?
[23:23:36] <nineteen1ightd> Yeah, unfortunately the $ operator in the update part only operates on the first matche element in the array...
[23:24:16] <Joeskyyy> Hmmm, that's actualy a good question nollidj. Pretty sure it SHOULD
[23:24:20] <Joeskyyy> is it not in your testing?
[23:25:08] <nollidj> my query is more complicated than that (uses $or more than once), so i'm checking to see if a simpler query does
[23:26:41] <nollidj> when "nscannedObjects" is the cardinality of your collection, that's bad, right?
[23:27:29] <nollidj> i can't confirm that an index is being used --- indexOnly is false, but that makes sense since the query i'm running targets parts of the _id field
[23:27:45] <Joeskyyy> you can tell from the "cursor" field
[23:27:50] <nollidj> ah, thanks
[23:27:51] <Joeskyyy> If it's "BasicCursor" there's no index being used
[23:28:03] <nollidj> it's BasicCursor, so i'm vindicated
[23:28:05] <nollidj> sort of
[23:29:53] <nollidj> bizarre. given documents like { _id : { key : value, key : { nested : stuff }, ... }, value : thingy }, queries like { "_id.key" : value } don't appear to be using an index
[23:29:57] <nollidj> still, that's something i can fix. good to know
[23:30:20] <nollidj> Joeskyyy: thanks for the pointers
[23:30:35] <Joeskyyy> No problem.
[23:30:43] <nollidj> i'll probably be back for another round of tangling with this stuff later
[23:30:46] <Joeskyyy> Now you have me curious haha so I'm trying to poke
[23:31:15] <nollidj> np. i have to catch a bus, but i'll be on after 10 EST
[23:32:39] <Joeskyyy> nineteen1ightd: Did you get yours solved? Because it sounds like you are looking for the positional solution
[23:32:46] <Joeskyyy> http://docs.mongodb.org/manual/reference/operator/update/positional/
[23:33:02] <Joeskyyy> Forgive me if I'm not understanding you exactly though :P it is about time to go home so my brain is drained
[23:35:24] <Joeskyyy> nollidj: FWIW, when I created a new index on "_id.foo" it suddenly works
[23:35:36] <Joeskyyy> Not sure why _id didn't solve that in the first place though :\\
[23:37:07] <Joeskyyy> Especially since you'd have to create an index for all possible fields to accomplish that, which is not pretty at all
[23:37:36] <Joeskyyy> I.e.
[23:37:36] <Joeskyyy> > db.test.find({"_id.baz":{"quux":"blech"}}).explain().cursor
[23:37:36] <Joeskyyy> BasicCursor
[23:37:36] <Joeskyyy> > db.test.find({"_id.foo":"bar"}).explain().cursor
[23:37:36] <Joeskyyy> BtreeCursor _id.foo_1
[23:52:33] <Joeskyyy> Ahhhh, I found it nollidj: http://docs.mongodb.org/manual/core/index-single/#indexes-on-subdocuments
[23:52:56] <Joeskyyy> Querying the subdocuments must match the ordering, which makes sense when you think about how the indexing is working on something like that
[23:53:08] <Joeskyyy> I.e.
[23:53:09] <Joeskyyy> db.test.find({"_id":{"foo":"bar", "baz" : { "quux" : "blech" }}}).explain().cursor
[23:53:09] <Joeskyyy> BtreeCursor _id_
[23:53:21] <Joeskyyy> So you may want to create some indexes yourself if you don't have a bunch already.
[23:53:38] <Joeskyyy> quittin time for me, have a good one everybody :D