Log file Viewer
#mongodb logs for Thursday the 5th of March, 2015
[01:06:40] <GothAlice> Ooh, initial benchmarks show a 15% speed improvement (on the average query in one app) and a reduction from 318MiB to 116MiB in terms of data storage for the app by switching to wiredTiger.
[01:25:02] <NoOutlet> However, the search results I'm finding on Google are mainly suggesting ways to avoid silly things in the future.
[01:29:46] <NoOutlet> The fixes are not ways to get fileB back, but are instead aliases for `mv` which make it so that it doesn't overwrite files.
[01:33:58] <NoOutlet> Yeah. I agree. I'll just need to transfer it again. I was just hoping there might be something.
[01:42:31] <NoOutlet> I wasn't freaking out or anything. I knew it wasn't a big deal, but if I could get it back quick without re-transferring and learn a new trick, that would be cool.
[01:43:50] <NoOutlet> I was thinking something more along the lines of it being in something along the lines of /tmp
[02:32:21] <Dexx1_> Anyone use MongoDB with CakePHP? I am trying to set it up on Mac OS X (10.6.8 SL), would love some hints/tips if anyone has any experience of getting it going
[02:55:14] <Tark> hey guys! I cannot start mongodb 3.0, after setting up packet mongodb-org-server start fails
[02:58:44] <Dexx1_> joannac, I am trying to do something similar. I got (apparently) a mongoDB plugin for cakePHP but I have no idea how to start it
[02:59:18] <joannac> okay Tark, so the service is failing. what OS? How are you starting the service?
[02:59:38] <joannac> Dexx1_: sorry, i don't knw anything about cakePHP (or much about PHP in general)
[03:00:24] <Dexx1_> joannac, no worries. I have a MongoDB directory though. Is there a standard command I use to start and administer it? Also, is there a GUI/easier way to administer the MongoDB ?
[03:02:51] <joannac> Dexx1_: the same thing I just told Tark: either mongod --options --moreoptions, or mongod --config myconfig.conf (with options in myconfig.conf), or starting via a service
[03:03:39] <joannac> Tark: http://docs.mongodb.org/manual/tutorial/install-mongodb-on-debian/#start-mongodb
[03:07:26] <Dexx1_> joannac, in the process. I think the plugin in cakePHP has to be some sort of a wrapper/interface
[03:11:54] <joannac> Dexx1_: yeah probably. it's likely to be a wrapper around the office PHP driver.
[03:15:10] <Tark> joannac, aw, I follow this steps: http://docs.mongodb.org/manual/release-notes/3.0-upgrade/
[03:16:18] <joannac> Tark: um, when I ask you "what did you type", I actually want to know what you typed.
[03:17:20] <Dexx1_> joannac, upon trying to start mongo, I get: dyld: Library not loaded: /usr/lib/libc++.1.dylib (I am on 10.6.8 OS X)
[03:17:36] <Tark> after then echo "deb http://repo.mongodb.org/apt/debian "$(lsb_release -sc)"/mongodb-org/3.0 main" | tee /etc/apt/sources.list.d/mongodb-org-3.0.list
[03:23:26] <Freman> mongodb 2.6.5 libraries should connect to mongodb 3 ok right? minimal of issues?
[03:48:26] <Freman> so, very fresh install of mongodb-node - "TypeError: Cannot read property 'done' of null"
[05:35:01] <dcuadrado> if I have mongodb 2.6.8, is it ok to add new replicas using mongodb 3.0+wiredtiger and then kill the 2.6 instances?
[05:46:45] <dcuadrado> joannac: we are moving some bigger instances and I was thinking about taking the opportunity to upgrade now
[06:15:19] <Dexx1_> hey guys. New to monogo, I am reading this: http://docs.mongodb.org/manual/core/crud-introduction/ --- is that good enough for a noob? I got thrown into a project that's already running mongo and I want to know the best course of action as far as getting familiar with mongo
[06:29:16] <divyun> hello.. if I use an object( {asd: 'asd' , sdf: ['qwe', 'qwe']} ) as an _id .. how much of a performance issue would that be?
[06:31:02] <divyun> I wanna keep track of conversations on the basis of who started that convo and when was it started along with some tags
[06:36:08] <divyun> It doesn't have to be... just wondering why not? and those combined would be used to identify an conversation.
[06:39:33] <divyun> I'll have that object available .. so if that's the id.. I'll query it right away. Otherwise, they'll have to passed as parameters.
[06:44:53] <divyun> I'll have to confirm if meteor's 'observe' driver takes care of the order and type.
[06:51:19] <divyun> if I convert that object into a string to store it as _id .. query similarly .. it should work .. ??
[06:54:30] <divyun> order of the object you were talking about... I think I can preserve that during generating queries.
[06:56:53] <divyun> @jonnac ..btw, I tried this at http://try.mongodb.org .. > db.asd.save({_id: {asd: 'asd', qwe: 'qwe'}}); > db.asd.find({_id: {asd: 'asd', qwe: 'qwe'}}); >db.asd.find({_id: {qwe: 'qwe', asd: 'asd'}}); ... seems to work
[07:23:36] <Gottox> hi! I'm getting an error when I try to compile mongodb with system v8: KeyError: 'LIBDEPS_v8_SYSLIBDEP':
[08:20:16] <QGuLL_> hi, i'm confused with mongodb built-in roles specified on http://docs.mongodb.org/manual/reference/built-in-roles/ : did they changed ? i'm on mongodb-10gen 2.4.6 and doesn't observe right what it is said in the doc
[08:50:55] <QGuLL_> "You can authenticate as only one user for a given database. If you authenticate to a database as one user and later authenticate on the same database as a different user, the second authentication invalidates the first. "
[08:51:37] <QGuLL_> does that mean that my superAdmin user would never can auth into any db, but only admin db ? (and after do a use SomeDb)
[10:03:54] <QGuLL_> can't we update user's role on mongo 2.4 ? i don't find the updateUser documentation on mongodb website for version 2.4, but for 3.0 i do
[10:20:26] <boolman> I'm trying to create a 3 node cluster with all new installations. 1st node, rs.initilize() ; rs.add("node02:27017"), but the node02 is stuck in STARTUP and the 1st node is in SECONDARY
[12:21:30] <esko> Hi!, can someone tell me how to get just the matching title/url in my result?, or is my thinking wrong.
[12:23:05] <kali> esko: you need $elemMatch, look there : http://docs.mongodb.org/manual/reference/operator/projection/elemMatch/
[12:57:11] <esko> eg: db.balls.find({"links": {"$elemMatch": { "title": "more"} } },{links:1,_id:0})
[12:58:46] <kali> esko: the first argument of the find filters documents, the second filters stuff inside matched documents
[13:05:57] <boolman> if I have a hidden replica on server x, and removed it as member and used it as a standalone to do some lab-testing. What happens when I readd it ?
[13:27:57] <esko> perhaps im better suited to have a own collection for this rather than have it in an array
[14:20:02] <skot> fhain: how did you remove it and add it back? Through a reconfig, or just by stopping the process and starting on a different port?
[14:20:49] <skot> It will make a difference with what happens to the data changes while it wasn't actively in the replica set.
[14:45:17] <zotherstupidguy> i am using mongoid, and i find it difficult to create an index for usernames
[14:46:30] <zotherstupidguy> still i can add duplicates to the document! which isnot what i want, as i aim for unique!!!!
[15:08:53] <Cygn> Is there a way to tell the count function in a $group (aggregation) to return 0 if there is no value instead of just ignoring the specific group in the result?
[15:12:36] <thewanderer1> Hi. How do I reliably open a file for writing in GridFS, but only if it did not exist before? I'm using the Node.js driver (1.4).
[15:13:16] <thewanderer1> There is the GridStore.exist(), but this seems prone to race conditions between the check and the write.
[15:50:33] <Acidnerd> how can i configure mongoddb-org-server PRIOR to install it ? does not make any sense this error message
[15:54:36] <boolman> installation process for me on ubuntu 12.04 was step 1 wget, step 2 tar xfv file, step 3 copy contents of the bin-dir to /usr/bin/, step 4 /usr/bin/mongod &
[15:55:20] <Acidnerd> http://docs.mongodb.org/manual/tutorial/install-mongodb-on-debian/#install-mongodb
[15:59:03] <boolman> https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-debian71-3.0.0.tgzCOPY LINK
[16:03:28] <boolman> node.vm.provision :shell, inline: "cp /vagrant/mongodb-linux-x86_64-ubuntu1204-3.0.0/bin/* /usr/bin/ && mkdir -p /data/db && /usr/bin/mongod --syslog --replSet 'rs0' &"
[16:06:47] <Acidnerd> are journal files needed ? i want to log, nothing of this sort, can journalisation be disabled ?
[16:07:20] <Acidnerd> also now that install was done manually, where are my mongod.conf ? same place as install from repo ?
[16:18:49] <Acidnerd> ok thanks for the help, i'm back to mysql for that tool, this is just not working after trying 3 different methods
[16:31:48] <milind> Hi, I have a n00b question here. When i run mongodump with --oplog , I see following under the output directory - http://pastie.org/10002497
[16:32:40] <milind> As you see, there is a zero byte oplog.bson at root of the directory as well as one under dump directory -- why do i have two of them?
[18:08:21] <anumpatel> Is there any alternative of creating a model except " mongoose.model(modelName, schema) " ?
[18:14:48] <girb> I have a 1 replicaset live running on prod and have setup other 3 replica sets .. can I join them live and make shard ?
[18:31:48] <GothAlice> girb: http://docs.mongodb.org/manual/tutorial/convert-replica-set-to-replicated-shard-cluster/
[18:32:05] <girb> flyingkiwi: so I need to shutdown the cluster until the whole data distribution takes place ?
[18:32:26] <GothAlice> girb: You've already done step one, create a replica set. You want to continue from "add initial replica set as a shard".
[18:33:10] <GothAlice> During promotion to a sharded replica set, your data won't initially be sharded. You must then enable sharding on the collections you want to spread amongst the clusters (by adding a sharding key).
[18:33:53] <GothAlice> E.g. sh.enableSharding("mydatabase"); db.test_collection.createIndex( { number : 1 } ); sh.shardCollection( "test.test_collection", { "number" : 1 } )
[18:34:18] <girb> GothAlice: yes that I know .. so my collections are larger of arround 600GB .. so do I need to wait untill all the collections are distributed ?
[18:35:02] <GothAlice> No, though until a shard key is set and balancing finishes up you won't gain any real benefit from the shards.
[18:35:14] <GothAlice> (But at no point should it lose track of your data; queries should still work A-OK during balancing.)
[18:35:53] <girb> GothAlice: ok .."should still work A-OK during balancing" << this was what I needed
[18:36:35] <GothAlice> girb: http://docs.mongodb.org/manual/reference/method/db.printShardingStatus/#db.printShardingStatus will be your friend (and db.stats()) while everything is processing.
[18:37:00] <GothAlice> See: http://docs.mongodb.org/manual/tutorial/convert-replica-set-to-replicated-shard-cluster/#confirm-the-shard-is-balancing
[18:38:17] <GothAlice> Performance may be degraded (similar to background indexing) while things are being bulk moved around, but otherwise yes, this type of thing should be quite safe.
[18:43:45] <GothAlice> girb: http://docs.mongodb.org/manual/tutorial/manage-sharded-cluster-balancer/ may be of interest.
[18:44:01] <GothAlice> Covers scheduling balancing so as to not have performance degradation during critical times. :)
[19:28:14] <Dexx1_> need help starting mongodb guys. I am getting ERROR: listen(): bind() failed errno:98 Address already in use for socket: 0.0.0.0:27017
[19:30:10] <GothAlice> Dexx1_: That error indicates that some other process has requested exclusive access for listening on that port (the default MongoDB port). The PS line there will let you know if it's another copy of mongod running.
[19:32:11] <GothAlice> Dexx1_: Just type "mongo" (no d on the end) and press enter to connect, by default. (If you've enabled authentication you'll need to authenticate either on the command line or after connecting.)
[19:38:36] <Dexx1_> GothAlice, the issue i have is that I have already ssh'ed into a VM and I since I need two shells open for mongo, it won't let me ssh in again
[19:41:51] <GothAlice> Dexx1_: Well, then the "connection information" you're looking for is basically just the address of that VM. Since it's bound to 0.0.0.0 (all interfaces) MongoDB within the VM will be accessible from outside the VM without issue.
[19:42:34] <GothAlice> Also, if you're hosting a VM locally for the singular purpose of a development environment, the VM is likely a rather large waste of your system's resources. ;)
[19:44:43] <GothAlice> Dexx1_: As a note, one doesn't need any "shells open" to host MongoDB. It's a network service, not a serial terminal.
[19:59:46] <GothAlice> Argh! My beautiful 1000+ day VM uptimes, ruined by a Rackspace dom0 reboot. T_T Watching the cluster re-mesh and elect was kinda pretty, though.
[20:01:02] <GothAlice> 1084 days on my former primary. ^_^ And yeah, we've got a VM on the old infrastructure they're wanting to bump. Luckily Rackspace have a seamless transition process (i.e. after is identical to before, just running on a newer dom0).
[20:04:47] <GothAlice> Also, for as much love as I give MMS, I'm terrible for finding obscure bugs, edge cases, and downright silliness in systems. My test MMS cluster is "up", but MMS reports every node as in-progress applying changes. What changes? No changes, apparently, if I cancel and attempt to review, the list is empty. T_T
[20:09:48] <GothAlice> Zelest: One hour of downtime in 1084 days = 0.9999% uptime (four nines) over nearly 3 years, or, around a 297% single-year uptime. XD
[20:11:45] <Dexx1_> GothAlice, so here's the issue I am having. (I want to connect mongoDB with cakePHP) I am running a VM through my os x terminal and I can run mongod, but once it's running how do I interact and do other stuff? It just idles and waits
[20:12:19] <GothAlice> Dexx1_: Well, that tells me you're running mongod in the foreground rather than as a system service in the VM. Simply starting the VM should start MongoDB, and no further interaction is needed at all.
[20:12:33] <GothAlice> Dexx1_: In your app, just connect the MongoDB driver to the IP address of the VM.
[20:13:06] <GothAlice> However, developing on Mac OS X, I just install the homebrew mongodb (I need SSL) and run it as a launchd system service, without a VM. The VM is overhead that isn't needed.
[20:14:32] <GothAlice> Similarly, any command-line tools you want to use (mongo, mongodump, etc.) can be run outside the VM. Just point them at the -h <ip of vm> and they should work. (Or get rid of the VM, it's not helping you in any way, and connect to your app to localhost.)
[20:15:48] <GothAlice> Zelest: Indeed yes. Worst-case in the life of that primary were some slow (and I mean _slow_) requests. No outright failures.
[20:20:32] <GothAlice> Dexx1_: The "mongod" process is a network service; it has no terminal UI. Your description of "idles and waits" is exactly what it does; it's waiting for a connection from a MongoDB client like your application. (It also has a --fork option which tells it to run in the background instead of foreground.) However you really should use the platform (Linux init.d, Mac launchd, etc.) system for keeping services running.
[21:13:00] <GothAlice> hahuang61: WT and MMAPv1 are completely different back-end implementations, with very little in common.
[21:13:34] <hahuang61> Yup, but what are the implications on the front-end or why would you use 1 vs the other?
[21:17:31] <GothAlice> hahuang61: Minor implications on the front-end in terms of differing output for certain commands, e.g. db.stats().
[21:17:54] <GothAlice> My application (excluding a bump in version for PyMongo) was blissfully unaware that I upgraded it from 2.6 to 3.0 and changed engines.
[21:27:41] <GothAlice> hahuang61: Switching to WT reduced data storage to one third the original, and increased average query performance 15%. But that's for one application. As with any comparative measure, measure using your own data for real results.
[21:31:10] <GothAlice> fewknow: Wasn't me, it t'was a prior support incident I helped resolve. (Retrying with _id $gt offsets was the preferred solution.)
[21:34:14] <GothAlice> Heh, skip is the reason when I click "show more" at the bottom of my YouTube subscription stream, and someone has posted something new, I get duplicates of previously seen entries. _Bad design, Google!_
[21:35:24] <GothAlice> (Twitter does the right thing, OTOH. They have a "X new" prepended to the top, and the infinite scroller retries rather than skipping.
[21:41:44] <FiftyFivePlus> How difficult is mongodb to admin for a sys admin noob? (why are hosted/cloud solutions so expensive?)
[21:51:30] <GothAlice> FiftyFivePlus: Cloud, for the most part, is a fancy business-to-business (B2B) marketing term to help extract money from gullible companies.
[21:53:00] <GothAlice> FiftyFivePlus: If you can download a compressed file, extract it, and run the contents, you can run a MongoDB cluster using MMS.
[21:54:09] <GothAlice> FiftyFivePlus: Someone with zero prior sysadmin experience can "admin" a MongoDB cluster.
[21:54:59] <fewknow> FiftyFivePlus: it is harder to understand the concept of replicaSets, Sharding, mongos, sharkey, etc
[21:57:40] <GothAlice> I run a darknet for my server-to-server database access. (I.e. a private VLAN.)
[22:00:31] <fewknow> I would start with inti scripts and automating spinning up a replica set locally
[22:01:01] <GothAlice> fewknow: Well, for giggles I set up a simple firewalled mongod on a VM, then did some Python packet injection magic to convince it to become a replica set by faking the packets as being from a trusted source (authorized app server) on an already-established TCP connection (no syn/ack connection handshake to worry about not getting ;) Now I make sure to enable authentication, even if I think remote access is impossible. XP
[22:01:55] <fewknow> FiftyFivePlus: one thing mongodb does really wewll is the documentation....well atleast better than other databases
[22:02:31] <GothAlice> There are a few sections that lean a bit towards the ambiguous. For example, don't expect the mongo shell to speak the MongoDB-enhanced JSON dialect.
[22:06:00] <FiftyFivePlus> while 8 servers seems a lot, does it not get pricey if used on top of commodity VPS like digitalocean or is DO my first wrong move ?
[22:14:37] <GothAlice> (I've attempted to use them in the past, intermittent Gentoo support, and have about a dozen _public_ support tickets with large vote counts open for three or more years…)
[22:17:02] <GothAlice> The minimum Rackspace setup for a non-high-availability cluster w/ < 1GB of data is ~$100/mo.
[22:18:12] <GothAlice> Something more bare-metal and less managed would improve on that level of efficiency. ;)
[22:22:18] <FiftyFivePlus> now, how difficult is mongodb to manage *without* MMS ? and if it's still simple to admin then why is MMS so expensive beyond 8 servers or is it really more full server than VPS oriented?
[22:22:52] <GothAlice> Or what is your expected growth? (Depends on where you are in development. ;)
[22:25:49] <GothAlice> FiftyFivePlus: As a note, running a database service at all is more of a full server thing, to do properly. You want low latencies and the highest possible data streaming rates for disk, RAM, and network, and any form of abstraction _will_ get in the way. The amount of in-the-way-ness that is acceptable will vary from project to project.
[22:26:09] <FiftyFivePlus> no idea - if whatever i am doing gets traction then it could be single digit petabytes ... so, i am trying to figure out what my revenue model can afford in case i succeed ......... you know the problems with these things - in order to succeed we could always turn up the richness of the service - if hosting was free ;)
[22:26:38] <GothAlice> Well, if you're potentially storing petabytes make sure your revenue rates cover the infrastructure requirements.
[22:26:40] <Dexx1_> Whats the simplest way to start a mongodb database with a specific username/password on localhost?
[22:27:18] <GothAlice> Dexx1_: "With a specific username/password" — please clarify what you mean by this. Username/password required for MongoDB client applications (like a PHP app) to connect with? Or "user to run mongod as"?
[22:27:32] <FiftyFivePlus> i was saying the same thing as in, i am trying to figure out the level of functionality that i can afford
[22:31:17] <GothAlice> morenoh149: ObjectId is a BSON type, representing a 12-byte packed binary structure. See: http://docs.mongodb.org/manual/reference/object-id/ Like any field, you can index them. This makes queries (esp. range quries, which are effectively _dates_!) efficient.
[22:31:53] <GothAlice> morenoh149: Depending on the back-end, how indexes are stored will vary. MMAPv1 uses B-Trees, AFIK. Haven't checked the new engine yet. There's a third-party engine that uses fractal trees.
[22:36:32] <GothAlice> There's all sorts of missing from your statement, and it's throwing me for a loop. No, you can't modify existing indexes. (Madness would ensue, and possibly a division by zero followed by the destruction of the Earth.) "different name" gives me pause for concern. I'd need to see just what you're doing to understand. (Gist/pastebin your commands?)
[22:52:45] <Dexx1_> GothAlice, I dumped the VM. I am on MAMP now, much faster on my old machine. The way I run mongoDB is via terminal .. is that ok? I want to see whats happening
[22:53:42] <GothAlice> Run it as a service, like the proper daemon it is, and a) tell it to output to, and then watch a log file for server activity, and b) for client activity, become familiar with the oplog.
[22:55:35] <GothAlice> ln -s /usr/local/opt/mongodb/homebrew.mxcl.mongodb.plist ~/Library/LaunchAgents/; launchctl load ~/Library/LaunchAgents/homebrew.mxcl.mongodb.plist
[22:56:14] <GothAlice> After those two commands, again, if using homebrew to install MongoDB, mongod will be running and will start automatically when you log in. To stop it, re-run the second of those two commands with "unload" instead of "load" as the first argument.
[22:57:49] <GothAlice> Dexx1_: https://gist.github.com/amcgregor/46f54e47bc2b4eef5c12 is the contents of that plist file; you can save this to the same name and path, update the ProgramArguments array to match what you need, then do the same launchctl load on it.
[22:59:34] <GothAlice> Dexx1_: Lastly, out of the box MongoDB runs without authentication. This means the client drivers operate fine without a username/password, too.
[22:59:58] <GothAlice> http://docs.mongodb.org/manual/tutorial/enable-authentication/ are the instructions on enabling authentication, however.