pmxbot IRC Log Viewer

[00:20:53] <blizzow> I'm getting out of memory errors when starting mongodb. eeep! It's happening while it's trying to preallocate the journal file...

[00:20:53] <blizzow> tcmalloc: large alloc 8589934592 bytes == (nil) @

[00:20:53] <blizzow> Thu Sep 11 00:14:32.487 out of memory, printing stack and exiting:

[00:44:25] <joannac> blizzow: hrm, that seems large

[00:47:35] <blizzow> joannac: yes, I know. :(

[00:54:00] <joannac> blizzow: could you paste the log files from mongod start to crash?

[00:54:05] <joannac> in pastebin

[00:55:13] <blizzow> http://pastie.org/9543584

[01:02:22] <joannac> thanks blizzow

[01:19:35] <blizzow> any idea why mongo would poop itself like this?

[01:28:47] <ceejh> So... that mongo ruby driver update today

[01:29:20] <ceejh> kinda broke my ubuntu 12.04 envs... seems like it suddenly requires sasl for authentication

[01:37:53] <Boomtime> @blizzow: "Thu Sep 11 00:37:26.941 out of memory, printing stack and exiting:"

[06:52:41] <jjcc> Q: is it possible to create a collection on mongodb startup? use-case: database files are stored in ramdisk

[08:17:04] <RaffoPazzo> I've managed to cross-compile and run mongo for a MIPS board and I did some benchmarking vs SQLite3. Is anybody intereted in hearing some of that ?

[08:17:57] <RaffoPazzo> There are also a couple of open question that the experts may be willing to try give an answer to

[09:04:42] <emfl_> Hello

[09:06:00] <Derick> hi

[09:07:21] <emfl_> I'm trying to install the mongo gem on windows 2012 R2

[09:07:31] <emfl_> And I get error about missing sasl.h file

[09:08:27] <emfl_> I extracted that file from the cyrus_sasl archive and then added it to the mingw include folder. I then had to remove a definition of struc iovec ...

[09:08:55] <emfl_> Now I get undefined reference to `_imp__sasl_decode64'

[09:08:59] <emfl_> I'm running out of idea

[09:28:36] <godelmang> yo dudes im still having problems accessing a particular element of an array that is a subdocument of a collection

[09:28:53] <godelmang> ive tried $slice and dot notation, i cant figure either method out

[09:41:34] <godelmang> my question refined/elaborated: im trying to access an element of an array thats a subdocument in a mongodb collection, i just cant figure it out even after reading the docs. tried dot notation and $slice both, cant figure either out! :( db.mycoll.find({},{"foo.bar":1, "_id":0}) returns the whole array, shouldnt db.mycoll.find({},{"foo.bar.0":1, "_id":0}) return the first element of that array?

[11:14:26] <voidspace> after applying a replicaset config change (via "replSetReconfig") subsequent operations can fail with "need most members up to reconfigure, not ok"

[11:14:30] <voidspace> we're using Go and mgo

[11:14:57] <voidspace> I'd like to know how to *tell* when the new config has been applied and the replicaset is healthy and ready

[11:15:35] <voidspace> my approach was waiting for "replSetGetStatus" to report that the majority of members are in state 1,2 or 7 *and* all have an uptime > 0

[11:15:39] <voidspace> however this isn't enough

[11:15:47] <voidspace> is there a right/better way to check?

[11:55:00] <skot> voidspace: the mgo driver should be tracking the state of the replica set members by calling isMaster on each to detect when one of them reports "master:true" meaning it can take writes.

[11:56:13] <voidspace> skot: ah, ok

[11:56:15] <skot> but the error message you are reporting isn't about a primary being elected but that you don't have a majority of members online to reconfigure the replica set. Why are you doing a reconfig?

[11:56:38] <voidspace> skot: we're testing our low level function - Add, Remove, Set - that change the replicaset members

[11:57:15] <voidspace> skot: in general we'll do Add (to add new members) and Remove (generally only on down members)

[11:57:27] <skot> You might want to read the rules of replica sets in the docs since you can't *always* make changed to replica set config.

[11:57:49] <voidspace> skot: ok

[11:57:58] <voidspace> skot: we do always see the changes made *eventually*

[11:58:10] <voidspace> skot: but we don't want random sleeps, we'd like to be able to tell

[11:58:16] <skot> If only 1 of a 3 member replica set is up then you can't reconfig because that could lead to more than one config logically

[11:58:44] <voidspace> right

[11:59:01] <voidspace> skot: wait, so if you have two members down you can't add more to replace them...

[11:59:08] <voidspace> in that situation

[11:59:20] <skot> If you have a majority down you can't without a force.

[11:59:44] <voidspace> right

[11:59:46] <voidspace> fair enough

[11:59:47] <skot> And if you do that, you are taking responsibility for the possible bad outcome, like if they aren't down but just unreachable

[12:00:05] <voidspace> that isn't the particular problem we have

[12:00:13] <voidspace> but it's definitely worth bearing in mind as we don't check for it

[12:00:45] <skot> Yeah, it is a distributed consensus system with a shared config, which requires consensus to change consistently

[12:02:30] <voidspace> skot: so why, when replSetGetStatus reports that a majority of members are up (state is in 1, 2 or 7 and uptime > 0) do we see this error?

[12:02:55] <skot> I think the only reason you see things eventually work out is that you wait for members to be online in large enough number to form a majority

[12:02:55] <voidspace> skot: we make the config change then wait for replSetGetStatus to report what we think is a healthy replicaset

[12:03:49] <skot> I expect you may be seeing things from only one perspective when the replica set member require consensus. -- ie, your monitoring node sees things different and at different times than the other nodes.

[12:04:09] <voidspace> skot: right, that's one possibility we thought might be the case

[12:04:25] <voidspace> skot: so we need to wait for a majority to report the new state we're waiting for

[12:04:34] <voidspace> although it's disturbing that replSetGetStatus can lie

[12:04:35] <skot> Without more data i can't judge, but the logs will include information about what each member sees

[12:05:07] <skot> replSetGetStatus reports each members view of the world, not a consolidated one.

[12:05:37] <skot> I don't think it is lying so much as you might be misinterpreting what you get back :)

[12:06:12] <voidspace> it's a subjective status not an objective one

[12:06:16] <voidspace> we want an objective one

[12:06:24] <voidspace> it seems we have to ask all of them and work it out for ourselves

[12:07:04] <skot> That is what the distributed system does to ensure consensus and for elections in replica sets

[12:07:20] <voidspace> skot: right, and we weren't taking that into account

[12:07:24] <voidspace> skot: thanks for your help

[12:07:33] <voidspace> skot: I'll have to consider how to change our code appropriately

[12:07:34] <skot> It sounds like you really just want to wait until one of the member is primary

[12:07:47] <skot> (and try to avoid known cases where there can't be one)

[12:07:50] <voidspace> skot: we're seeing replSetGetStatus report one in a primary state

[12:07:56] <voidspace> but it seems that's not the right way to check

[12:08:03] <voidspace> and we should be polling all of them for isMaster

[12:08:22] <skot> The driver does this under the covers, so you can probably just ask it.

[12:08:36] <voidspace> I don't know if mgo does this yet

[12:08:49] <voidspace> we've written replcaset support with the goal of getting it into mgo

[12:08:57] <voidspace> although there is some lower level support

[12:09:17] <voidspace> there is an isMaster implementation now in mgo - so we can get rid of ours for example

[12:09:18] <skot> hard to say, it should be otherwise failover wouldn't work correctly.

[12:09:24] <voidspace> ok

[12:09:30] <voidspace> so I need to talk to Gustavo anyway

[12:09:34] <voidspace> as we have an ipv6 bug

[12:09:52] <voidspace> mongo < 2.7 doesn't work with ipv6 addresses in the form [::1]:port

[12:10:00] <voidspace> and mgo *only* works with them in that format

[12:10:11] <voidspace> so mongo + mgo + ipv6 replicasets just doesn't work yet

[12:10:21] <skot> Yeah, I'm sure he can comment. I was pretty sure mgo kept ping latencies for read preferences selection and tracked replica set members

[12:10:41] <voidspace> probably

[12:10:48] <voidspace> there's a lot of code in cluster.go :-)

[12:11:27] <skot> yeah, I remember discussions with him about the decay/avg algo for latency tracking since it was different from other languages

[12:11:47] <skot> anyway, good luck.

[12:13:29] <skot> so I looked at my old emails and mgo def. support a replica set connection with member health monitoring

[12:13:51] <voidspace> skot: I have some digging to do, I appreciate your help

[12:22:18] <uberg0su> Hi, im need to prepare big bang migration for mongodb using golang. Do you know maybe any usefull library for that? Anyadvice welcome.

[13:25:41] <emfl_> Anyone played with mongodb cookbook ?I'm trying to configure a replicaset

[15:46:50] <saml> rs.status() gives me "errmsg" : "loading local.system.replset config (LOADINGCONFIG)"

[15:46:52] <saml> what's wrong?

[15:48:55] <saml> https://gist.github.com/saml/5d1553866ed398bf341c what's wrong with my replicaset?

[15:52:13] <saml> oh dns problem

[16:26:27] <pgentoo-> I need to store several million documents/day, but can tail them off after say 1 month. If i need to backup the data before destroying it, is there any benefit to using a TTL collection vs just indexing on a date field, manually backing up a given day before issuing a "remove data from that day" command?

[16:28:34] <asker> hey guys, im using php to connect to mongodb. i have a mongocursor reference on a query result. does php hold the connection until the is no reference on the mongocursor?

[16:30:21] <kas84> hi

[16:31:01] <kas84> I would like to migrate default configurated mongodb instance to a directory per db configuration

[16:31:07] <kas84> what do I need to be aware of?

[16:49:55] <pgentoo-> Here's another one for you... I'm getting "E11000 duplicate key error index" for the _id index, while doing an update/upsert (An $addToSet) with {"_id":"myidhere"} as the only selection criteria. This just happened all of a sudden, don't remember seeing it before. In the case i'm seeing this, the array that the $addToSet is operating against doesn't exist yet (but in other cases it works

[16:49:55] <pgentoo-> fine). Any ideas??

[16:52:17] <pgentoo-> WEird, i just saw the same thing for a document where hte item i'm calling $addToSet on does in fact exist, and already has another value in it. hum...

[16:54:47] <freeone3000> I'm reading http://docs.mongodb.org/manual/core/authentication/ and I've used db.addUser() to add a username and password to a database, but users that don't supply a username or password are still able to log in. How can I disable this? Each mongo is in a replication set.

[17:04:37] <freeone3000> I added keyFile= to my mongod.cfg, but now the server fails to start, with no entries in mongod.log. What causes this?

[17:41:41] <edrocks> can anyone help me with $elemMatch projection? I have it getting just the matched item in my array but I also need some info from the outer object the array is on

[17:42:18] <edrocks> ie {name: "ed", favoritecolors:["blue","green

[17:42:41] <edrocks> ie {name: "ed", favoritecolors:["blue","green","red"]} I can get the one item from favoritecolors but I also need the name part

[17:44:56] <paulkon> you need to wrap it in an $and

[17:45:27] <paulkon> although by default multiple key-values are $and-ed

[17:47:08] <edrocks> I just found out I need to use $ instead of $elemMatch then I can add my fields to the sort document

[17:52:49] <saml> how can I take unhealthy member of replicaset out of replica config?

[17:53:04] <saml> by reading documents, right?

[17:58:49] <saml> so if there's an unhealthy node in replicaset, connecting to the replicaset is really slow using MongClient

[18:35:48] <Soop> Hey guys. I was wondering if anyone could give me some guidance with some schema design

[18:56:42] <partycoder> Soop, i would but i have to get going now

[19:33:08] <yopp> hey

[19:33:18] <yopp> is it possible to make conditional $inc?

[19:33:29] <cheeser> conditional how?

[19:34:05] <yopp> do not increment over max value

[19:34:21] <cheeser> filter out those documents in your query

[19:34:22] <yopp> or for example when it's negative increment, do not go below zero

[19:34:38] <cheeser> inc'ing will never go below 0...

[19:34:45] <yopp> really?

[19:34:46] <cheeser> oh, i see.

[19:35:10] <cheeser> you mean the value you're incrementing by is negative. not that you're incrementing a negative value.

[19:35:30] <cheeser> but yeah, either way filter with a query

[19:35:33] <yopp> yeah. For example I want to atomically update balance

[19:36:58] <yopp> cheeser, for example. Current document: {_id: 1, balance: 15}. I'm doing something like $inc => { balance: -20}. I will end up with balance: -5

[19:37:38] <yopp> But I'd like to have something like $inc => {balance: -20, $min: 0}, so balance will be set to zero, not to -5

[19:54:00] <skot> no, there is no way to do that. You can of course only apply the update if the balance will not drop below some value. update({_id:1, balance:{$gt:20}}, {$inc:{balance:-20}}) and then check that the update applied.

[19:54:18] <cheeser> right. that's what i was getting at.

[19:54:31] <skot> so the query will check to make sure that the balance is greater than the decrement

[19:55:07] <skot> You can also use $max(balance, 0) but that is a different modifier and cannot be mixed with $inc in the same call

[19:55:32] <skot> (since it is unclear which should run first, or exclusive)

[19:56:02] <skot> see: v

[19:56:03] <skot> http://docs.mongodb.org/manual/reference/operator/update/max/

[20:01:26] <yopp> :(

[20:01:28] <yopp> skot, thanks

[20:22:52] <freeone3000> When using a replication set, how can I disable no-user no-pass login?

[20:29:18] <SpNg> I installed mongodb on centos using yum. I’m trying to change the data location via the /etc/mongod.conf file, but now I can’t get the service to start using “service mongod start” it fails saying “debpath (/data/db) does not exist” /data/db is not the dir I’m trying to point the data to

[20:44:21] <freeone3000> I set auth=true in all databases on the replset, but they don't seem to be replicating any more.

[21:02:07] <ehershey> SpNg: which package exactly did you install?

[21:12:25] <freeone3000> Found my problem. THe keyfile cannot be in a home directory.

[21:16:40] <SpNg> ehershey: mongodb-org

[21:17:00] <ehershey> did you change anything other than the dbpath in /etc/mongod.conf?

[21:21:08] <SpNg> ehershey: nope

[21:23:01] <SpNg> ehershey: here’s the error when I try to run mongod directly

[21:23:01] <SpNg> http://pastebin.com/HHbBwrnj

[21:27:31] <ehershey> when you run it manually it won't use the config file unless you pass --config/-f

[21:27:36] <ehershey> so the error makes sense in that case

[21:27:50] <ehershey> try: mongod --config /etc/mongod.conf

[21:27:57] <ehershey> and see if there's a syntax error or something

[21:28:02] <ehershey> that 'service mongod start' might be hiding

[21:30:31] <SpNg> ehershey: I get ERROR: child process failed, exited with error number 1

[21:33:55] <SpNg> ehershey: pretty detailed error, huh?

[21:49:23] <ehershey> that's the error if you run mongod --config /etc/mongod.conf ??

[21:49:58] <ehershey> maybe temporarily comment out fork=true in the config file

[21:50:20] <ehershey> or check /var/log/mongodb/mongod.log

[22:05:51] <SpNg> ehershey: figured it out. the new data folder was not created, and it was not creating the folder for me.

[22:05:56] <SpNg> thanks for the help!

[22:12:08] <ehershey> no problem!

Log file Viewer

Help | Karma | Search:

#mongodb logs for Thursday the 11th of September, 2014