Log file Viewer
#mongodb logs for Thursday the 12th of March, 2015
[00:03:47] <daidoji> GothAlice: I think mongo-connector looks a bit too automagic for what I have in mind
[00:04:00] <daidoji> I'm looking for something more like https://github.com/sheharyarn/mongo-sync but with a bit more battle testing
[00:04:28] <GothAlice> My typical approach is to temporarily add a hidden replica, let it sync, and stop it when it's done. Bam, instant local copy.
[00:06:21] <grazfather> GothAlice: Yeah I thought about it a lot and I think we need a new collection :(
[00:33:34] <GrubLord> I’m doing some mongodb admin and I’d like help with a little issue - anyone around?
[00:34:12] <GrubLord> Cool. =) So, I’ve been following the LDAP instructions here: http://docs.mongodb.org/manual/tutorial/configure-ldap-sasl-openldap/
[00:34:34] <GrubLord> Everything works nicely, up to the point where I do the auth() function check at the bottom
[00:34:45] <GrubLord> Where I get “Error: 2 PLAIN mechanism support not compiled into client library.”
[00:35:21] <GrubLord> What I’d like to know is - where is this mechanism not compiled into? MongoDB itself? saslauthd? Do I have to recompile mongo from source?
[00:35:59] <GrubLord> On here: http://search.cpan.org/~mongodb/MongoDB-v0.707.2.0/lib/MongoDB/MongoClient.pm I see an option for building Mongo with SASL via Perl, but the previous page seems to imply that’s not necessary.
[00:41:19] <GrubLord> setParameter=authenticationMechanisms=PLAIN” to the mongod.conf causes it to crash on startup
[00:44:47] <GothAlice> I'd expect to see client crashes if the CXX driver is compiled without --use-sasl-client, and issues if the installed SASL lib wasn't compiled with PLAIN support. Similarly, server-side.
[00:46:24] <GrubLord> What confuses me is that the saslauthd conf file is supposed to be set to MECH=ldap… saslauthd doesn’t even list a “PLAIN” mechanism in its list of mechanisms.
[01:04:33] <GrubLord> running this: mongod --dbpath /var/lib/mongo/ --auth --setParameter saslauthdPath="/var/run/saslauthd/mux" --setParameter authenticationMechanisms=PLAIN
[01:04:43] <GrubLord> gives me: BadValue Illegal --setParameter parameter: "authenticationMechanisms”
[01:04:57] <GrubLord> Even though that’s exactly how they do it in: http://docs.mongodb.org/manual/tutorial/configure-ldap-sasl-openldap/
[01:48:09] <drlewis> Is there a way to slow down mongoimport to avoid disrupting my production servers?
[04:21:01] <Freman> so... mongo is pretty magical... I have a small collection... { "name" : "role1", "description" : "a test role", "permit" : [ "scheduled", "persistent" ], "matches" : [ "shan" ] }
[04:21:58] <Freman> I'd like to do a find where doc.matches.filter(function(match){ var re = new RegExp(matches, 'i'); return match.test($GIVENARG) });
[04:27:37] <Freman> or am I going to have to grab the entire collection and do it in app? (not really a problem, it's a small collection and it'll never be very big, but I thought it'd be cool to do it in mongo :D)
[04:33:02] <jclif> We're trying to get a 1.6 TB dataset from MMS but it's projected to take around 50 hours; is this something that people just grin and bare?
[04:37:39] <Freman> dn.cluster.find({$where: "var found = this.matches.filter(function(match){ var re = new RegExp(match, 'i'); return re.test('someshanserver'); }); return found.length > 0;"}) works :D
[05:51:55] <arussel> is there a way to have a selector that returns an object but not its fields (without know all the fields in advance) ? something like {"myobj.*" : 0}
[05:56:04] <cbuckley> hi there, I'm having trouble updating my mongodb insall on CentOS 6 to 2.4.13. yum keeps telling me its obsoleted by mongodb-org 2.6.8. Does anyone have any information on how to accomplish this, or have a URL I can look at?
[06:03:44] <joannac> cbuckley: http://docs.mongodb.org/v2.4/tutorial/install-mongodb-on-red-hat-centos-or-fedora-linux/#manage-installed-versions
[07:47:40] <kali> somebody knows what this mean: "08:41 -!- kali #mongodb Cannot change nickname while banned on channel" ?
[08:22:52] <Cygn> Is there any function in the mongodb query syntax that accepts a string and returns a regex expression? f.e. "/[a-Z]/" would return /[a-Z]/ ?
[11:53:32] <tiwest> I've just upgraded php from 5.4.* to 5.6.6 on a CentOS 5.6 box and now it looks like the MongoDB driver is out of date. Is anyone able to point me in the right direction for upgrading that?
[12:16:32] <Freman> '$where': "return this.matches.filter(function(m){new RegExp(m, 'i').test('someshanserver')}).length > 0" :D
[13:04:56] <klevison> Is there a "tool" way to show easily a query (that has loc fields) into a map.
[13:13:53] <steerio> is this behaviour (expected order of keys in compound ID) normal or a bug? http://pastie.org/private/gtbgwgtkll8wyccuqghkg
[13:14:58] <steerio> this means I can only query safely as { "_id.s": "ABC", "_id.b": 1 }, which does not use the _id index
[13:18:03] <steerio> in most languages I cannot guarantee the order in which keys of a map (hash, object) will show up in BSON/JSON
[13:19:52] <steerio> I have a compound ID and I can only query it if the query doc has the keys in the same order as the ID had when it was inserted
[13:21:33] <steerio> I see, but the databases are usually queried from some kind of other environment
[13:22:20] <cheeser> yes, it could've been implemented differently but it wasn't. this is the world as it is.
[13:24:48] <android6011> I need to create a query that grabs records for 2 days and on both days where the time is between 3:00PM and 5:00PM, if $hour and $minute the best way to do this?
[13:26:43] <android6011> the user can select minutes as well so should i just project out like hhmm ?
[13:28:15] <android6011> ok thanks. also im on ubuntu using the mongo apt repo but when i try to set a 2D index i get unkown index plugin 2D
[13:31:21] <android6011> on the record there is a location prop that looks like location: { "lng": xxx, "lat": yyy }
[13:35:08] <android6011> is there a recommended way of migrating data from postgresql into mongo? right now i have just been writing code to transform the data
[13:36:04] <Derick> android6011: that's probably the best way, but do realize that you often need to change your data schema too, to make optimal use of mongodb
[13:38:06] <cheeser> that's what i ended up doing. thankfully my data was mostly flat and not terribly relational
[13:38:37] <android6011> also for things that we had before like "reference types" a table with type values like event_types, the text on those is very small so I am assuming its best just to put the type on each doc rather than have to requery by id to pull the type, especially when a doc might have 4 or 5 ref values on it?
[13:43:23] <d0x> Any ideas why com.mongodb.DBObject isn't available on my emr cluster after putting the dependencies into the lib folder (like described on the mongo-hadoop connector emr example?) I put this question also to stackoverflow: http://stackoverflow.com/questions/28998333/connect-hadoophive-with-mongodb-on-aws-emr-class-not-found-com-mongodb-dbobjec
[13:44:01] <jokke> is it possible to make a query with the $in operator and preserve the order of the given array?
[13:45:17] <jokke> as in db.books.find({_id: {$in: [ObjectId('someid'), ObjectId('someotherid'), ObjectId('thirdid')]}})
[14:15:20] <dbclk> folks I use the rs.reconfig to remove a replic set from my primary that's complaining it can't connect to primary over port 27017
[14:16:01] <dbclk> problem is now, on the secondary machine, when I try to reconnect to primary it's still complaining about the port but, when I go inside the mongo terminal, it's marked as rs0:remove
[14:43:13] <spuz> Does the getN() method in the Java api ever return anything other than 0 after a document is inserted? http://api.mongodb.org/java/2.6/com/mongodb/WriteResult.html#getN%28%29
[14:53:30] <Diegao> is there a way to run a distinct wich returns a list of documents instead of the list of ids? I'm using mongoengine for python
[15:10:50] <pamp> hi, when querying with find, its possible using statments like IF for projection ?
[15:15:16] <GothAlice> pamp: Alas, $cond, which sounds like what you want, is an aggregate operator. Ref: http://docs.mongodb.org/manual/reference/operator/aggregation/cond/
[17:04:05] <boutell> Hi. I am inserting multiple documents with a single insert() call. There is a lot of confusing information about the limitations. Am I restricted to no more than 1,000 documents per insert? What is the maximum combined size of the documents? Right now my code just starts a new batch if the total raw BSON size of the documents exceeds 16MB, but I’m getting complaints that writes are being refused for batches near that
[17:04:24] <boutell> I suppose inserting *one* 16MB document *must* be allowed, because that is the per-document limit.
[17:05:15] <boutell> I am guessing that there is overhead per each document in the “batch,” which is pushing me just over 16MB if, for instance, there are four documents that are exactly 4MB. But I don’t know how to account for the overhead correctly. Any thoughts?
[17:08:30] <technoid_> rockmongo.com appears to be down, PHPMoAdmin just had a vulnerability found...what web based gui for managing Mongo would anyone recommend?
[17:38:37] <schu> hi there. how can I install mongodb 2.4.9 on a debian system via apt? (I need this special release but apt always tries to force me to use 3.0.0)
[17:46:03] <Boomtime> schu: try this: http://docs.mongodb.org/v2.4/tutorial/install-mongodb-on-debian/#manage-installed-versions
[17:47:13] <ndb> hi, I have a string field '00:02:04.234', is there some way to apply a function on all rows using a find?
[17:47:50] <ndb> I want to compare it with $lt, $gt seconds as numbers (360 for example), but need to convert it first
[17:48:06] <GothAlice> ndb: You're somewhat pooched by virtue of the fact that you're storing those as strings.
[17:49:11] <GothAlice> Storing them as proper date objects, just ignoring the date component, would be both more efficient than storing them as strings like that, but also allow you to use http://docs.mongodb.org/manual/reference/operator/aggregation/#date-operators
[17:53:50] <wc-> hi all, currently in the process of upgrading to mongo 3, i am using user authorization and having some issues with connecting to the mongod v3 from a mongo client v2.0.4
[17:54:29] <wc-> were there some authorization-related changes in mongod v3 that require clients, code libraries etc to be updated?
[17:55:30] <GothAlice> wc-: http://docs.mongodb.org/manual/core/authentication/#authentication-mechanisms
[17:55:51] <GothAlice> Very specifically, this section: http://docs.mongodb.org/manual/core/authentication/#mongodb-cr-authentication
[17:57:08] <GothAlice> Also: http://docs.mongodb.org/manual/release-notes/3.0-compatibility/#compatibility-security-changes
[18:15:33] <wc-> for anyone else that might hit this auth with old clients problem, this jira ticket has helpful info and a temporary solution: https://jira.mongodb.org/browse/SERVER-17459
[18:32:15] <wc-> im trying to completely remove all traces of any user accounts imight have on this mongod instance
[18:33:10] <wc-> its my understanding if i log in via localhost with no users, i should have full access until i create a user?
[18:33:57] <wc-> i know, not the recommended way to go about things, just getting a feel for what all needs to change
[18:37:11] <wc-> when mongod starts up i can see enableLocalhostAuthBypass is set to 1 in the params
[18:53:15] <claygorman_> what has been peoples experience in upgrading from 2.6.8 to 3.0 using PHP driver?
[18:59:36] <claygorman_> i know there was a change in the auth scheme for 3.0 that I think broke my application when I upgraded, so i rolled back until i could figure it out
[19:00:24] <claygorman_> yeah kinda weird... i even tested the 1.6 driver and 1.7(dev) driver for PHP
[19:02:20] <wc-> sort of following the exampel at the bottom of this jira ticket: https://jira.mongodb.org/browse/SERVER-17459
[19:02:25] <claygorman_> i tried mongo 3 with auth scheme 2.6 and that didn't work either so kinda weird
[19:02:48] <wc-> ah thats my plan im setting schema.currentversion to 3 then adding users and hoping it will work?
[19:03:11] <claygorman_> didn't work for me... i didn't wanna upgrade the auth version in the MMS manager because i thought it might break haha
[19:03:25] <wc-> it also seemed like mms monitor / automation wouldnt connect with the auth enabled either
[19:03:45] <GothAlice> To test 3.0 out I asked MMS to spin up some mongod on some of my nodes, configured them for 2.6 default authentication, added the users I needed, waited a minute, then bulk loaded my data back. Didn't have a single hiccup with authentication at the application level after "pip install -R pymongo" to bump that package version.
[19:03:47] <claygorman_> i tried the 2.6 style auth schema version with mongo 3 hoping that would work
[19:04:51] <wc-> unfortunately theres no way im gonna get management to allow this web automation stuff to manage our prod instances until its proven
[19:05:32] <phutchins> Anyone know much about BSON? I know this isn't really directly mongo related (although i'm pulling the bson id's from mongo). I'm trying to decode the Mongo ObjectId ( in ruby ) and convert the object id string to binary. Anyone have any ideas on how?
[19:05:58] <wc-> when the automation first arrived we hooked up an aws account to it, it got stuck in some spin cycle spinning up instances and shutting them down
[19:06:21] <claygorman_> sometimes i have to kill stuff manually even using the mms automation on my own boxes
[19:08:22] <phutchins> GothAlice: it does... but I'm workign toward using the general bson library to decode it... Hoping to find someone who knows it a bit better than i do to get in the right direction
[19:08:47] <phutchins> GothAlice: I.e. I can do things like decode the date from the object id, etc... But I've got a more in depth task i'm working on
[19:08:50] <wc-> claygorman_: i had to do the following commands in a script, then start adding users, then enable auth and restart mongo:
[19:11:23] <GothAlice> phutchins: There isn't to_binary, but there certainly is an array of bytes: see the "data" instance attribute.
[19:12:43] <phutchins> GothAlice: ah, It wasn't clear to me that that was what that was... that' helpful. thanks
[19:40:58] <phutchins> GothAlice: so I get that I now have an array of bytes but i'm still unclear on how to get this to binary or hex or somethign that I can decode. Have any idea?
[19:43:03] <GothAlice> phutchins: And define "decode". It sounds like you have everything you need and a general Ruby question about how to manipulate those things. Unfortunately, I don't Ruby, but #ruby may be able to help.
[19:43:42] <phutchins> GothAlice: true, I'm not forming my questions clearly :). This is definitely #ruby territory now... I appreciate the help!
[22:47:36] <daidoji> but when I set it back up to test a recovery from backup, it takes quite a while for the journal to recover
[23:11:30] <joannac> at which point we pre-allocate 3 journal files at the start and then keep reusing them
[23:13:38] <daidoji> joannac: does that preallocation time go down if I lock the DB before taking the snapshot?
[23:13:41] <daidoji> http://docs.mongodb.org/ecosystem/tutorial/backup-and-restore-mongodb-on-amazon-ec2/#verify-the-backup
[23:13:53] <daidoji> it wasn't really clear from the directions above if I should do that or not when journaling
[23:14:01] <joannac> daidoji: http://docs.mongodb.org/manual/tutorial/manage-journaling/#journaling-avoid-preallocation-lag
[23:16:18] <daidoji> joannac: ahhh I see, out of curiosity, why aren't these files captured in a snapshot?
[23:18:20] <daidoji> joannac: no, I spun up a new instance to make sure the backup process would work and attached my snapshotted volumes to that new instance
[23:19:09] <daidoji> joannac: I did not as it was unclear from those directions whether I should or not
[23:19:47] <joannac> "If the dbpath is mapped to multiple EBS volumes, then in order to guarantee the stability of the file-system you will need to Flush and Lock the Database."
[23:21:33] <daidoji> joannac: hmm, not really. The data is all on one volume, the journal and log are on another
[23:22:18] <joannac> daidoji: the previous paragraph is "The journal file allows for roll forward recovery. The journal files are located in the dbpath directory so will be snapshotted at the same time as the database files."
[23:22:54] <joannac> so yes. journal is inside dbpath. journal is on a different volume. therefore dbpath is on multiple volumes. therefor you need to lock the database
[23:24:01] <daidoji> joannac: oh I'm not sure. It might have just been my read of it. (no offense taken)
[23:25:56] <joannac> when mongodb starts up, it replays what's in the journal, then does some tests and goes "oh crap these disks suck. preallocate time!"
[23:27:40] <joannac> there's not really anything practical you can do at the snapshot time to prevent that. I guess if you knew that in advance, you should lock the database (which flushes changes from journal to data, and data to disk)
[23:28:03] <joannac> then you could just copy the data files and create your own journal files like in that docs page
[23:31:48] <GothAlice> When our DB was on AWS, the inexpensive instances were still given just under 18GB of RAM. So we decided to roll with no persistent storage at all on our DB nodes.
[23:33:03] <GothAlice> (Fully self-configuring on startup, ephemeral VMs, with occasional snapshots and streamed archiving of the oplog to S3 in the event of catastrophic failure.)
[23:36:04] <GothAlice> Well, until that dom0 reboot at the beginning of this week, I had three years continuous uptime.
[23:38:56] <GothAlice> But hey, the cluster coming back up was beautiful to watch on our monitoring dashboard. ^_^