Log file Viewer
#mongodb logs for Tuesday the 19th of August, 2014
[02:26:31] <asher^> I'm running 2.4.9 and I cant seem to create new users for some reason. I'm using a user that has userAdminAnyDatabase on admin but I'm getting the error couldn't add user: not authorized for insert on admin.system.users at src/mongo/shell/db.js:128
[02:27:31] <asher^> { "_id" : "admin.userAdmin", "user" : "userAdmin", "db" : "admin", "credentials" : { "MONGODB-CR" : "3744eb0bff2414bce0f4a9e8c732e9bc" }, "roles" : [ { "role" : "userAdminAnyDatabase", "db" : "admin" } ] }
[04:57:13] <user123321> I would just like to make the backup DB come active immediately when the main goes down.
[05:00:25] <user123321> joannac, I was going through the mongodb docs, and I read that I need a minimum of 3 DB servers, OR, 2 Db servers + arbiter :/, any advice?
[05:09:21] <user123321> joannac, So I should not go with 2 DB servers only? (without even an arbiter)
[05:12:58] <Boomtime> it is not recommended, having 2 servers will mean that (by default) you are dependant on both of them being up or no primary will exist
[05:13:03] <joannac> No, having an even number of nodes can cause problems if you ever need to elect a new primary
[08:45:38] <prologic> Anyone had a chance to look over https://gist.github.com/therealprologic/c85697889f0f834180ce ? I asked before and I'd appreciate if anyone has either run into this or can give me some direction to fixing this particular edge-case I've run in to?
[09:04:55] <Siyfion> I have an array of objects that I am allowing the user to modify, I (ideally) want to be able to $push and $pull objects into the same array... However I get the "have conflicting mods in update" issue...
[09:05:20] <Siyfion> Is there a better way of doing this, other than splitting the update operation into two separate queries?
[09:06:00] <Siyfion> (The only other way I can think of is to overwrite the array entirely with the new set... but it's a bit inefficient?)
[09:51:00] <rspijker> I think I’m going insane… Why does db.coll.find({“field”:{$not:/value/}}) say no results, yet db.coll.count({“field”:{$not:/value/}}) gives me 11000?
[10:00:37] <rspijker> an explain on the find also tells me it uses a BasicCursor and the nScanned corresponds to the number of docs in the collection, n is also the same as the count. Yet the cursor is empty and there are no actual results to be had
[10:03:18] <rspijker> Derick: yeah, I was beginning to suspect I was, in fact, not insane. Good to have some confirmation :)
[10:15:38] <rspijker> hmmm, it does work (the mecahnism) on different collections on the same DB system
[11:11:37] <Siyfion> rspijker: Okay, that's kinda my first thought, just a shame, as it means that it's not atomic :/
[11:11:37] <rspijker> bin: they’re called bulk inserts: http://docs.mongodb.org/manual/core/bulk-inserts/
[11:27:28] <rspijker> Siyfion: vote for this, I suppose: https://jira.mongodb.org/browse/SERVER-1050
[11:27:29] <durre> hey! if I want to query a field case insensitively with regex, will I be using the index then?
[11:30:39] <rspijker> it will (I think) use it to match against, as in, match against the index instead of the actual documents
[11:32:33] <durre> what I'm trying to decide is. If I have a collection with 4 million documents, will a regex query be troublesome? the alternative is storing a lowercase representation
[12:05:53] <Viesti> doing "rs.reconfig" on the current primary gives me this: exception: initiation and reconfiguration of a replica set must be sent to a node that can become primary
[12:06:14] <Viesti> doing the same on a secondary gives me this: replSetReconfig command must be sent to the current replica set primary.
[12:22:50] <kali> Viesti: i did not know about that. reconfig accepts a "force" argument that may help
[12:23:35] <kali> Viesti: or you try it in two step. first remove the priority on the current primary, then add it on the new wanted primary
[12:43:13] <tscanausa> chris31jct: I recommend just telling everyone one what string are using and what error you are getting.
[12:44:56] <chris31jct> i don't know my url for the connector requirement. mongodb is all new to me, but i have a client requesting to use informatica cloud for interacting between salesforce and mongodb
[12:45:59] <chris31jct> i just installed mongodb, created a collection of Accounts and now I can't see it at Informatica Cloud
[12:49:33] <chris31jct> i test the connection and infa cloud says its fine. but when i do a task, says there is no object in mongodb. but i have a collection in my db with data
[12:51:42] <tscanausa> localhost doesnt seem right but I know nothing about informatica cloud maybe you can ask them for help?
[12:52:53] <chris31jct> yes i can do that. but just wanted to know if there is a way to get the url where i installed it.
[12:53:35] <chris31jct> in my laptop, just clicked install and that's it. i mean the url where it is located.
[12:57:05] <chris31jct> yeah. downloaded the setup, clicked install. followed some steps to begin mongo services, created a collection of accounts in mydb database and the went to infa cloud
[12:59:31] <tscanausa> is your computer publicly accessible on the internet ( which does not mean does it have internet access but can it be accessed like a server)?
[13:05:03] <tscanausa> well here is my conclusion, infa cloud is a web service that needs to be able to access mongo, http://locahost:8080/query is a local address to the service accessing it. so infa cloud would try to access mongo at its install location ( aka on the same server as the web service ).
[13:06:30] <tscanausa> I am 99% sure your computer is not access able to the public internet, if it were you would be able to change your connection string in infa cloud to be http://PUBLIC_IP:8080/query . this proble goes away when using another cloud service like in the user docs like AWS.
[13:06:59] <tscanausa> because it has a public ip address accessable by both you and infa could and so each can access it the same way
[13:15:04] <chris31jct> i think infa can connect to mongo db because when i test connection it says is successful
[13:15:29] <chris31jct> im getting a The selected DSS target connection has no associated objects. Select a different target connection. error
[13:18:46] <chris31jct> after i input the required information for the connector as per the pdf you sent me, it check if information is correct and accessable
[13:19:50] <chris31jct> infa cloud uses a secure agent in my computer and i think it is the one actually making the connection between the infa cloud services and my computer which is where i connect for uploading flat files and now mongodb
[13:22:21] <chris31jct> I have sqlserver installed also in my computer and have made a successful connection to infa cloud and can see my tables.
[14:01:17] <nomiz> In node-mongodb-native I get an error when I drop an not existant collection, where I would expect a result = false, like in the mongo shell. Why?
[14:11:13] <tscanausa> it does that because that is what the client says it will do. http://mongodb.github.io/node-mongodb-native/api-generated/db.html#dropcollection
[14:34:37] <Shakyj> mongoimport docs say --drop drop all collections, is there a version that just drops the collection you are importing to
[14:36:16] <kali> dob_: http://docs.mongodb.org/manual/reference/configuration-options/#operationProfiling.slowOpThresholdMs
[14:42:04] <tscanausa> the ubuntu mongodb package by default uses standard logging and so it might be of MongoDB's opinion or the packagers opinion that a log rotate file is not needed since it is not used by default. there is also a ticket somewhere that would have mongoDB rotate its own logs
[16:14:56] <gustavnikolaj> Hi. I have a question that I cannot find the answer for. What is considered best practice when writing an API based on express; Making one connection per worker, or making a new connection per request?
[16:16:21] <gustavnikolaj> tscanausa: do you have any resources, off the top of your head, of how to do that? All the examples I find make a new connection.
[17:39:11] <Diplomat> Hey guys I have a weird problem.. For some reason mongodb doesnt start on ubuntu
[17:45:20] <mango_> try running through: http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/
[17:52:41] <Diplomat> it appears it like starts up for a moment and then shuts down and then its impossible to start it
[17:54:37] <Diplomat> dafuq it didnt install it correcly or something because when i tried to install it again then it wants to install itself again
[18:59:45] <brammator> pymongo question: is there any difference between "find({..}, sort={..}, limit=..., skip=...)" and "find({...}).sort(...).skip(...).limit(...)"?
[19:29:48] <jasondockers> I can't seem to append a BSONObjectBuilder with the return value of a map? I can't pinpoint the problem with it. The map is defined as std::map<std::string, std::string>, I'm appending like: cDoc << theMap[q] /*(where q is a string)/* << q;
[20:23:07] <baegle> Is there a way to get a document from one database into another database via the mongo shell?
[20:35:38] <amcrouse> I am trying to setup a geographically distributed replica set of four members. I am following the docs at http://docs.mongodb.org/manual/tutorial/deploy-geographically-distributed-replica-set/ however I have a question. Under the topic “Deploy a Geographically Redundant Four-Member Replica Set” the first point states that “One host (e.g. mongodb4.example.net) must be an arbiter. This host can run on a system that is also used for an application
[20:35:38] <amcrouse> server or on the same machine as another MongoDB process.” On the arbiter page at http://docs.mongodb.org/manual/core/replica-set-arbiter/#replica-set-arbiter-configuration it states “Important: Do not run an arbiter on systems that also host the primary or the secondary members of the replica set.” These two statements seem to be contradictions. Can I please have some clarification. I think having an arbiter running on the secondary in site A w
[20:40:09] <amcrouse> The sysadmin wanted to have 2 in each site. If site A went down for a long time then we would manually switch site B over to the the primary.
[20:41:03] <joshua> If one site goes down you only have 2 members and can't get a consensus on a vote
[20:42:23] <amcrouse> Yes, that is why it would need to be a manual process to switch. Our business requirements state this need.
[20:42:36] <cheeser> amcrouse: it's probably safe to run a mongos and an arbiter on the same box, e.g.
[20:44:18] <joshua> Ideally you would want an arbiter in each location so you have an odd number of members, or get a 3rd location. But the ideal isn't always the reality. :)
[20:47:21] <joshua> Hrm actually if you have an arbiter in each location that might have other complications. They went over all this in the DBA course but its been a while
[20:48:14] <cheeser> 2 arbiters would bring the cluster to 6 making it theoreitcally possible for both sides to elect a primary in the even of a network partition
[20:48:35] <jeffj> it's almost like mongodb doesn't really have a real solution for not fucking itself in the event of a network partition.
[20:48:49] <joshua> Cause if they both are voting and both have a primary, when the network comes back you might have data collision
[20:49:02] <cheeser> it's almost like mongo does but you have to know what you're doing in the first place.
[20:49:54] <jeffj> i love distributed databases where it's possible to have both sides of a partition elect a primary. what happens when the partition heals. how is conflict resolved?
[20:50:40] <kali> amcrouse: arbiters are meant to act as tie-breaker in a two "data" node replica set
[20:50:50] <joshua> They had you bring back things in certain order after adding data but I don't remember half of how it plays out. hah
[20:51:49] <kali> amcrouse: i don't think there is really a point in having more than one in a replica set. anyway, you must break the symmetry between your two datacenter: if they are split, only one of them can elect a primary
[20:52:32] <amcrouse> mango_: I don't see where in the docs it says its for testing. I was under the impression that the guide was valid for production. Sorry, I've been reading all day and my eyes are old
[20:54:04] <amcrouse> kali: So do you think running an arbiter on the same machine as the secondary in site A would be okay? Or would it be best to run it on a separate machine?
[20:58:07] <kali> amcrouse: well, the recommend approach is to put the arbiter on a host that you want on the same side of a potential split than the primary. a good place is often you application server (not the mongodb server, but whatever uses mongo)
[20:59:48] <kali> amcrouse: you will not be able to have automatic failover for a failure of dc A and automcatic failover for a failure of dc B. you have to pick one. it's the same problem as replica set, but at another scale
[21:04:09] <amcrouse> I don't think mongo is capable of doing automatic failover from sites (Nor do I think most programs are able to). The fact that you have to place a majority in site A makes it somewhat clear that site A will never give up priority. I'm wondering if placing an arbiter in site A's secondary would have any adverse effects during an election.
[21:05:05] <kali> amcrouse: not much, but in that case, i would rather tamper with the votes weight (even if i know it's frown upon)
[21:08:11] <kali> joshua: it's not the same. priority have no impact of the existence of a consensus
[21:09:15] <joshua> True true, but if you want to make primaries go back to where you want when the network is optimal again it helps
[21:09:59] <kali> joshua: sure, but in the end it is a completely orthogonal problem to the consensus building
[21:10:37] <joshua> I might have to start thinking about this stuff again once I switch jobs next month.
[21:11:42] <kali> joshua: if you have 3 nodes in DC A, 3 in DC B, no arbiter and no vote tampering, in case of a split, you get two DC with 50% of the votes. no matter what you do with priorities, you have six nodes doing nothing :)
[21:14:06] <kali> amcrouse: if one DC is splitted out (or down), the two remaining still have 66% of the votes and can choose a primary
[21:15:25] <kali> amcrouse: it might be usefull for you to setup a test replica set (you can do it on a laptop) and simulate the various failure scenario to get a feel on how this things react
[21:15:41] <mango_> joshua: I started M102 as well, but gave up too, Im going to start next month, it seems very in depth.
[21:16:30] <joshua> When it got to the ones where it was like your business requirements are XYZ and ABC happens and its asking how long before blah blah blah. I got fed up and stopped there
[21:17:05] <joshua> I guess doing all those calculations to figure out how long you can leave a DB down without doing anything made me think I wouldn't want to work there. heh
[21:17:13] <amcrouse> kali: However this does not solve for if the DC's cant see each other. Nor does this allow for automatic failover for DC's. Only one DC will be eligible for priority since http://docs.mongodb.org/manual/core/replica-set-architectures/ "Keep a Majority of Members in One Location" and "Distribute Members Geographically"
[21:18:08] <kali> amcrouse: well, consider this: just three nodes, each one in a different DC. let's forget about the performance implications
[21:18:31] <joshua> One example I saw had 2 nodes in the primary, then a second location with a single node and arbiter, and a third location with just a single node
[21:19:49] <joshua> If A and B are up, and you have 3 members plus arbiter the arbiter won't vote for itself
[21:20:42] <kali> joshua: yeah, it's nice, as long as A and (B or C) is up and can see each other, A stays primary
[21:23:33] <kali> but there are dozen of options. two nodes in A, two nodes in B, one arbiter in C would work too (and you have two locations that can handle write traffic as well as read-on-secondary)
[21:25:34] <amcrouse> kali: According to the docs you should not do automatic failover of datacentres. All examples that I have seen have placed a 0 priority on mongod instances outside of the primary site
[21:27:08] <amcrouse> I would tend to agree with that as split brain can happen if their were certain conditions. Then reconciliation would result in data loss or an incorrect state.
[21:27:50] <kali> amcrouse: ok. let me state that i never managed cross DC operations, only cross "zones" (as in ec2), so there might be practical issues. i was merely trying to give example of working and/or broken voting pools
[21:29:17] <kali> cheeser: do you know if people manage to get cross-DC working replica set ? with actual working failover strategies ?
[21:31:14] <Matadoer> question guys... would mongodb be good to use for storing financial stuff? or should I just use a MySQL cluster
[21:38:14] <Matadoer> what if there wasn't any transactions and we were just storing bitcoin wallets (no private keys being stored)
[21:42:14] <Matadoer> we just store the encrypted wallet payload and then the user decrypts it with the private key on the client side
[21:59:16] <jeffj> > don't think mongo is capable of doing automatic failover from sites (Nor do I think most programs are able to) <- this isn't true, there are a number that are. and in the days of cloud, even a single location at a single provider can encounter very significant partitions.
[23:03:18] <melvinrram> When doing an aggregate and using grouping, how do you get the first instance of a combination?
[23:06:14] <cirwin> hi guys, i have a sharded cluster with 5 replica sets; and for some reason one of the replica sets has 2x the volume of data than the others
[23:07:20] <Boomtime> @melvinrram: $first: http://docs.mongodb.org/manual/reference/operator/aggregation/first/#grp._S_first
[23:09:43] <cirwin> we did a resync of this replica set recently, so I assume it was part of that process that broke things
[23:10:38] <cirwin> I know, that's why I'm very confused that I'm seeing one replica set behave differently from the other 4
[23:11:25] <Boomtime> what do you mean when you say "resync this replica-set" <- this statement makes no sense, you can resync a member of a replica-set, or you can drain and re-add a shard
[23:12:28] <Boomtime> then you just re-sync'ed a member of a replica-set, that does not affect the cluster overly much
[23:13:03] <cirwin> right, it's the only difference about the "broken" replica set which is why I brought it up
[23:14:32] <cirwin> about a year ago we followed the instructions on the mongo site to set up the index and then shard
[23:15:14] <Boomtime> fyi, you would be better off saying "shards" there, the fact they are replica-sets is incidental
[23:20:25] <Boomtime> i would have to check what it measures, but that might include unsharded collections in the total
[23:21:49] <alejandrocgf> hello, has anyone manage to auth with x509 in node.js? whatever i do, the driver crashes :(
[23:24:25] <Boomtime> then my bet is the chunk ranges are not consistent, yuo would need to go through the chunk collection
[23:26:26] <melvinrram> I have it working with one field but I want it for the first combo of multiple fields
[23:27:24] <Boomtime> it only makes sense to select $first on the group, so if you grouped by multiple fields then i believe it selects the first item that is in that group
[23:30:58] <melvinrram> I don't think I fully understand yet but I believe I have my answer inside http://stackoverflow.com/questions/11418985/mongodb-aggregation-framework-group-over-multiple-values so I'll go do some experiments.
[23:39:21] <edrocks> is it better to use an ssh tunnel between all your api/web server nodes and your db or to make a cross zone vpn that includes all api/web/db nodes?
[23:39:31] <SubCreative> Can someone give me the best way to scan/loop over a collection and execute a function if a value = true?
[23:40:33] <edrocks> a loop would be slow and prone to an out of memory crash because you need to keep the whole dataset in memory to loop over it. With an iter it just grabs one at a time from the db
[23:44:24] <joshua> melvinrram: If you haven't read it yet, this has some good info about choosing a shard key http://docs.mongodb.org/manual/tutorial/choose-a-shard-key/
[23:46:55] <melvinrram> joshua - hmm not sure what that has to do with what I need but I'll definitely take a look.
[23:48:12] <edrocks> why are you suppost to use exactly 3 config servers in production? Wouldn't more be better?
[23:52:01] <edrocks> idk what jsc is but in golang its called iter http://godoc.org/gopkg.in/mgo.v2#Iter
[23:52:48] <edrocks> then you would do you would use iter.next if your data could be large to step through 1 by 1
[23:52:57] <joshua> I'm guessing you don't want more config servers than you need cause it has to make more connections and keep more of them in sync.
[23:53:49] <edrocks> just do whatever the equivalent would be of fetching one document do work on it then get the next then instead of grabbibg everything at once