Log file Viewer
#mongodb logs for Thursday the 14th of May, 2015
[04:00:08] <alexband> hi, all, I started a secondary node, want to sync with primary (both mongodb 3.0), but I always get socket timeout error
[05:40:15] <sabrehagen> for two databases (e.g. localhost:27017/db1 and localhost:27017/db2), does inserting data in db1 put a write lock on db2?
[07:15:11] <amitprakash> Hi, I can connect and authenticate to a database as db = MongoClient('host:port')['database'].authenticate(user, pass) and execute queries as db[collection].find().count()
[07:15:49] <amitprakash> However, when I try to authenticate via mongo client as mongo host:port/database -u user -p pass it throws { ok: 0.0, errmsg: "auth failed", code: 18 }
[09:15:41] <abyss> To set range for sharding mostly I should use id key for my data in mongodb? So: sh.shardCollection(database.foo,id) should be enough?
[11:24:10] <amitprakash> Hi, I can connect and authenticate to a database as db = MongoClient('host:port')['database'].authenticate(user, pass) and execute queries as db[collection].find().count()
[11:24:12] <amitprakash> However, when I try to authenticate via mongo client as mongo host:port/database -u user -p pass it throws { ok: 0.0, errmsg: "auth failed", code: 18 }
[11:30:19] <sabrehagen> my mongod service is only listening on 127.0.0.1. how do i get it to listen externally?
[13:24:56] <jecran> Good morning mongo land. I am having issues with an import command from the command line: mongoimport --db test --collection restaurants --drop --file primer-dataset.json . Pulled straight from the site, and I keep getting an Unexpected identifier error. Quotes maybe?
[13:26:39] <StephenLynx> goldstar from what I know, you can get the insert date from the default _id object.
[13:27:39] <StephenLynx> so you could use in the match block for removal this date with the operator lt and a date of the current date - a month
[13:27:41] <jecran> Yeah I'm just starting the tutorial and I cant get past the first line. Download the file straight from the site too
[13:28:04] <StephenLynx> you don't need to download, you can add the repository to your package manager.
[13:32:20] <jecran> Nothing being "imported" I can do a regular database, add a collection, then save and it works fine. Just the import I am stuck on. And yes, on the command line
[13:34:20] <joannac> jecran: the doc you linked says "In the system shell or command prompt" - is that confusing?
[13:53:03] <amitprakash> Hi, I can connect and authenticate to a database as db = MongoClient('host:port')['database'].authenticate(user, pass) and execute queries as db[collection].find().count()
[13:53:04] <amitprakash> However, when I try to authenticate via mongo client as mongo host:port/database -u user -p pass it throws { ok: 0.0, errmsg: "auth failed", code: 18 }
[13:54:43] <pamp> I already remove the collection, but when I run sh.status() the collection still appears
[14:38:47] <GothAlice> abyss: Sharding on _id is perfectly acceptable… if what you want is pseudo-random (but even) distribution of your data across shards.
[17:00:18] <alobodig> Hi. Is using the positional projection operator ($) supported in the ‘fields’ part of a findAndModify command? e.g.: db.col.findAndModify({ query: { ’array.val’: 0 }, update: { $set: { ‘array.$.val’: 1 } }, fields: { ‘array.$’: true } }); It seems like it interprets the $ as the name of a field in array, but I might be overlooking something.
[17:28:43] <StephenLynx> http://docs.mongodb.org/manual/reference/command/cleanupOrphaned/ it seems so.
[17:36:05] <godzirra> Using the node mongo driver, how do I see the query I created? I'm trying to do a query by date, but in node it's not working, so I'm pretty sure I'm doing somethign wrong, just not sure what, and not sure how to view the query.
[17:37:22] <StephenLynx> I don't remember seeing a way of getting the query the driver formed with the objects and array you provided and I have been using that driver for almost a year now.
[17:38:41] <godzirra> StephenLynx: The code I wrote looks correct to me, but it's obviously not. :) Here's the code I'm using: https://gist.github.com/slooker/0417d60841c2f70c99d7
[17:40:05] <StephenLynx> 1- format the code, it makes much easier to understanding it. 2- I am pretty sure you must pass a callback to find to get the pointer you convert to array.
[17:41:48] <godzirra> I'm not really suer what you mean about formatting the code. the reply is indented, I just forgot the closing brace to the collection.
[17:42:11] <godzirra> StephenLynx: And this code works exactly as is if I remove the "dateCreated" section from the query.
[17:43:45] <StephenLynx> https://gitlab.com/mrseth/lynxchan/blob/master/src/be/public/boardDetails.js#L61 godzirra
[17:44:30] <StephenLynx> https://gitlab.com/mrseth/lynxchan/blob/master/src/be/public/boardDetails.js#L168 another query, without the block declared on a separate place.
[17:44:53] <StephenLynx> when you declare json objects completely lined it becomes really hard to understand their structure
[17:49:36] <girb1> StephenLynx: yes .. I recently upgraded from 2.7 to 3.0 .. there seems to be orphaned data when I check it through db.colection.getShardDistribution() … the data is less here when compared to "show dbs"
[18:00:57] <vzp_> Can I have different mongo versions in a replica set? In my case I want to have 2.6 and 3.0
[18:21:25] <GothAlice> It's acceptable; depending on data usage there may be more optimal ways to organize your data.
[18:21:42] <GothAlice> (If you can avoid queries needing to merge responses from multiple shards, you win.)
[18:38:03] <StephenLynx> GothAlice when you open a gridfs file on write and close it, when you open and write again will it first remove all previous data or keep writing from where it stopped since you last opened it?
[18:38:56] <GothAlice> StephenLynx: MongoDB is effectively stateless, so no, if you re-open the "file" and start writing again, it'll have a local file pointer of zero, i.e. the beginning of the "file".
[18:44:35] <GothAlice> Always fun to run an emergency audit and code cleanup on a 16-year-old e-commerce site that was broken into with multiple exploits. :/
[18:45:05] <GothAlice> (The "mysqli" upgrade for it was… painful. I'm pretty sure this is PHP 3 code.)
[18:46:06] <StephenLynx> it didn't worked with an updated version of a dependency because of a hack of the developers.
[18:46:07] <faux___> i need help designing a database. nosql databases are an option. my data are lisp expressions, eg "(f (g 1) (g 2))". when evaluating the previous expression, i'll need to hit the database for the definitions of "f" and "g". in general, if an expression references other expressions, it should be easy to get to those other expressions. any advice is greatly appreciated
[18:48:01] <greyTEO> I have been wanting to get into python for a while and Im trying every excuse to use it at work
[18:50:03] <GothAlice> StephenLynx: I think I've found a convert, using my MongoDB-as-distributed-task-queue solution as bait. ;)
[18:53:02] <ngl> Hello. Somebody please answer this question: How do I use mongofiles with a bucket that is not fs? For example: mongofiles -d myDB -c templates get somezip.zip
[18:53:12] <GothAlice> StephenLynx: The end goal of transparently bridging function calls to a distributed cluster, including transparently bridging Python generator functions (iterables that "yield" multiple values before returning), is kinda boss.
[18:53:48] <StephenLynx> I remember sawing the implementation of generators and the yield keyboard on javascript. I gagged.
[18:54:04] <ngl> Or, for that matter, even just listing another bucket? This lists fs no problem: mongofiles -d myDB list
[18:56:30] <GothAlice> Ref: http://docs.mongodb.org/manual/reference/program/mongofiles/#cmdoption--prefix
[18:56:33] <ngl> Bows to GothAlice! Thank you so much, I couldn't seem to find that in the man page. Cheers!
[18:56:49] <GothAlice> It's… not made particularly obvious. Doesn't seem to be in the man page, but is in the online doc.
[18:57:37] <greyTEO> StephenLynx, just moving away from php, I like the idea of a better interpreted language
[18:58:30] <GothAlice> Heh. I'll take a step back from language advocacy… not quite the right channel for it.
[18:58:41] <StephenLynx> plus it is non io-blocking, making it great for heavy concurrency scenarios.
[19:00:35] <StephenLynx> GothAlice I didn't even figured out what was that code intended to do. why did he went on a loop to append that $?
[19:00:35] <GothAlice> I'm shocked that there wasn't already a ##argument channel. ("I came here for an argument!" "No you didn't.")
[19:02:19] <GothAlice> StephenLynx: It was a method of slurping in form data and populating a str_replace with all of that data, keys prefixed with $, for replacement in a template string. The security flaw is having a "from" address that includes newlines. >:D
[19:06:21] <girb1> I had one replica set at the beginning and I later added 3 replica set and made a sharded cluster …. now the cluster is imbalanced
[19:06:22] <girb1> i.e the first replica set contains more data (about 60%) of data. Balances ir running and its not balancing properly
[19:06:33] <girb1> Is there a way I can stop mongos insterting data into into my first replica set ?
[19:12:31] <ngl> GothAlice, is there a particular version of mongofiles I need to have to use the --prefix flag? I'm running 2.6.9 and am told the flag is unkown.
[19:13:19] <ngl> Dang, this is our standard dockerized mongo test container. Versions... Thanks a bunch!
[20:14:00] <godzirra> StephenLynx: Meeting hell. Sorry. I reformatted the data and posted it here: https://gist.github.com/slooker/0417d60841c2f70c99d7
[20:14:54] <StephenLynx> yeah, I am almost sure you need to get the result from the callback on the find.
[20:18:33] <GothAlice> StephenLynx: Find is lazy in every driver I've gone source-diving through. I.e. the result of .find() actually doesn't do much. It's iteration that commits the query and runs it, prior to that other methods can adjust the parameters of the query in a nice, chainable way.
[20:18:54] <GothAlice> koboldthegreat: One of the route registrations you have added is badly formatted in some way.
[20:19:09] <StephenLynx> yeah, I really thought it was inconvenient to have to run callback from callback from callback
[20:19:41] <StephenLynx> so you can just do a find().sort().limit().toArray instead of using an aggregate and still keeping it using just a single callback?
[20:20:37] <StephenLynx> will make sure of that. the main reason I used aggregate for everything is that I thought I would have to use one callback for each operation.
[20:20:48] <GothAlice> (In a similar way that find().sort().limit().forEach(callback) is a single callback.)
[20:21:13] <GothAlice> find() returns a cursor. .sort() returns a new, mutated copy of that cursor. .limit() repeats this.
[20:21:31] <GothAlice> Only actual iteration (forEach, toArray, etc.) will execute the query defined by that cursor object.
[20:22:23] <GothAlice> Also: it's important to stress that these chain methods (sort, limit, etc.) return _copies_. They _do not_ modify the original. This makes partial chaining and conditional chaining a viable thing.
[20:23:30] <StephenLynx> nah, I use the official driver, I am pretty sure it allows for that, checking my last project right now.
[20:24:39] <godzirra> StephenLynx: It's literally just teh dateCreated section that I add that breaks it. I can only assume i'm setting up dates wrong or something?
[20:24:47] <GothAlice> StephenLynx: http://mongodb.github.io/node-mongodb-native/2.0/api/Collection.html#find You're not using the official driver, you're using an async fork.
[20:26:55] <StephenLynx> and the documentation fails to inform you can use the callback so just informs the returning cursor.
[20:27:19] <godzirra> I suppose so? I just need to know how I'm failing at adding the dateCreated line.
[20:27:20] <GothAlice> findOne* methods obviously terminate the expression, those have documented callbacks.
[20:28:11] <StephenLynx> well, I am using the very module that doesn't specify the callback on its documentation.
[20:28:45] <GothAlice> StephenLynx: Then you're doing it wrong and should fix your code to not use a callback on find(). As you noticed, the entire concept of having a callback there is kinda silly, since find() returns the cursor anyway.
[20:29:16] <StephenLynx> the point is that the code is working despite the documentation not specifying you can use a callback.
[20:29:46] <GothAlice> Undocumented features should not be relied upon. There is no promise that it'll work in any future (or past, for that matter) version.
[20:30:58] <godzirra> I updated my gist to show what works in the mongodb console vs what doesn't work in my node.js code.
[20:34:17] <godzirra> StephenLynx: https://gist.github.com/slooker/0417d60841c2f70c99d7 Line 26 is the data. this is in the mongo console. Line 10 is the results of console logging the start date.
[20:35:57] <StephenLynx> I don't see a field dateCreated on your docs, is that how you query for the creation date on the _id? I am not familiar with it.
[20:37:29] <StephenLynx> and you are using "dateReported" on the console and "dateCreated" on the application.
[20:49:06] <godzirra> StephenLynx: Asl ong as you have a standard you can work with, i don't think that's bad.
[20:55:15] <GothAlice> Storing dates as strings loses the ability to use date projections. But StephenLynx isn't here right now. XP
[20:58:11] <GothAlice> Also… it wastes (takes an additional) 15 bytes for each date using his format.
[21:07:27] <okanck> hello, have a sharded server which is set up. when i wanna sort the query answers so slow. even on indexed columns.
[21:08:58] <okanck> my query is: db.column.find({"PlayerID":2}).limit(100).sort({"Time":-1}).explain();
[22:13:03] <grazfather> is there a way to upsert where the spec for the search is written without having to rewrite? e.g. db.test.update({"ip":1,"host":2},{"field":"test"}, upsert=true) but the result contains ip 1 host 2 in addition to the field:test?
[22:26:30] <joannac> grazfather: http://docs.mongodb.org/manual/reference/method/db.collection.update/#upsert-behavior
[23:31:59] <sabrehagen> Derick: /etc/mongodb.conf only specifies bind_ip as 127.0.0.1 in mongodb 3 install. is --bind_ip 0.0.0.0 persistent?
[23:34:12] <sabrehagen> GothAlice: i'm using mongodb as a service so i'm not sure how to supply a command line parameter
[23:35:33] <GothAlice> If you're using a service, how do you ordinarily connect? It must be listening openly to accept connections…
[23:52:16] <sabrehagen> as mentioned before, mongodb.conf has localhost listening by default so this is how i'm connecting on a fresh install