Log file Viewer
#mongodb logs for Thursday the 18th of July, 2013
[00:56:46] <sharondio> Hi all…anybody around? Just need to confirm that our schema isn't going to work before going through the trouble of changing it.
[02:50:15] <leekaiwei> hi guys, i'm getting the error class 'mongoclient' not found. i've scoured the internet, updated things and edited php.ini but still not working. doesn't really warrant a pastebin but here it is http://pastebin.com/kpDLyFuC
[03:13:10] <ranman> leekaiwei: I think that probably means you don't have the library installed correctly ?
[04:40:04] <pplcf> I want to sort after limit after sort, something like db.col.find().sort(timestamp: -1).limit(10).sort(timestamp: 1)
[07:29:34] <rbento> For instance, I have an Author collection. I shouldn't be able to delete an author if it is referenced say, in a Book collection.
[07:33:06] <kali> rbento: you need to implement this behavious in your app, mongodb will not enforce this kind of constraint
[07:35:19] <rbento> kali Thanks. I see that. I was thinking about having any method that helps in this case. Like collection.stats() etc...
[07:37:35] <kali> rbento: as for dbref, it's just a convention. MongoDB does not do anything clever with it
[07:37:40] <rbento> kali Sure, I know that. If I cannot find anything built-in I'll surely do it myself
[07:39:56] <jbackus> Hey everyone. my friend and I are trying to figure out if MongoDB would be the right way to solve a database problem in our application. Would you all mind if I gave you a few sentence description of the problem so I can get your input(s) on whether Mongo would be the right way to go?
[07:40:10] <Nodex> referencing Authors in books is also not the greatest idea .... how often does an authors name change?
[07:41:21] <jbackus> Yeah I thought so, just wanted to check since I sometimes join programming IRC channels and somehow get yelled at in the first minute
[07:49:40] <jbackus> so our product is designed so that companies can verify customer identities. The verification criteria differs depending on the type of customer (i.e. U.S. Citizen vs. Non-U.S.). Some verification variables have multiple components (address->street,city,etc), some verification methods can be done one of many ways (identity through SSN, TIN, or License), and if a customer updates a variable (like address) we want to track the revision history for that.
[07:49:53] <jbackus> So basically the model in question (call it CustomerVerification) doesn't have any constant keys, some of the verification variables have multiple components (like a python dictionary), and some variables only need one verification for a array of options to pass. Does this sound right for Mongo to you guys?
[07:50:22] <jbackus> (My friend thinks Single table inheritance or multi table inheritance is the way to go)
[07:51:51] <puppeh> and I want to update a document by inserting some key-value pairs into that hash field
[07:55:10] <puppeh> let's say the value of that field is { "13": "abc" }. I want to make it { "13": "abc", "14": "ddd" }
[07:55:48] <jbackus> I have to ask though: Seeing as this is the mongodb room, you don't just say that for every case, right? :P
[07:56:02] <Nodex> you really shouldnt use numbers as keynames but if you must then ...{$set : { "foo.14" : "ddd" }}
[07:57:06] <Nodex> jbackus : I use mongo for everything from Job boards to CRM, CMS and Social networks. There is nothing I cannot make it do - admittedly sometimes wiht the help of external tools but mongo IS my primary data store
[08:00:35] <Nodex> why use relational when you dont need to and why use non relational when you dont need to :)
[08:00:37] <jbackus> Alright, I'll probably stick around for a bit as I think I'll try and port some of my SQL join-insanity to Mongo and see how it feels
[08:01:55] <jbackus> Yeah… You would probably cry at the weird multi table abstraction I have solving the problem now then
[09:16:03] <puppeh> is there the possibility to increment multiple values in keys of a hash field with one query?
[09:25:42] <puppeh> and I want to increment the "a" and "b" values by 1, is this possible to be done in a single operation?
[09:38:59] <rspijker> that will increase the a and b fields in the action field for _all_ documents in the collection called collection
[09:53:31] <remonvv> Hola. Anyway know what to do with { "ok" : 0, "errmsg" : "clone failed" } when invoking movePrimary?
[09:57:46] <rspijker> there is a jura issue for this in a sharding+replication situation. But it's fairly old...
[09:58:46] <remonvv> Usually. People need to learn that error messages need to be descriptive. Make it a page long if you have to.
[10:00:49] <remonvv> Well, with the possible exception of error messages like "Gee, something went wrong here. Let me explain. Back in the day when Is started this project there was a null check here but I removed it hence this NullPointerException. Sorry."
[10:01:40] <remonvv> Derick, any idea which ticket this is? Can't find it and we're currently having a $25k/month cluster refusing to scale down :s
[10:03:24] <remonvv> Yeah that doesn't seem it. There are no errors in the logs of either the current owner or the intended new primary owner
[10:04:03] <rspijker> I presume you have the preconditions listed here satisfied? http://docs.mongodb.org/manual/reference/command/movePrimary/
[10:05:16] <remonvv> It should, it's part of the automated downscale process. Maybe something went wrong with draining. Let me check.
[10:11:00] <remonvv> I just checked and it has data for a non-sharded collection on both the source and target shard.
[10:11:59] <remonvv> I know, but it does. It shouldn't have that. During the upscale we automatically distribute primaries across shards.
[11:40:42] <merpnderp> I just moved my dev to my local mac to use webstorm, but just realized I'll need to run mongo locally. I really don't want to do that. Has anyone had luck using mongo through an ssh tunnel?
[11:46:28] <rspijker> merpnderp: I haven't tried but I see absolutely no reason why it shouldn't work
[11:49:54] <merpnderp> rspijker: using the same -L ssh flag that works for mysql I'm getting a can't connect error on 27017….bizarre
[11:50:14] <Avish> I'm trying to use Aggregation Framework from C#. I know LINQ support for it is not yet ready but I'd like to use something similar to Query<T> and the other builders to be able to use MemberExpressions on my document type instead of hard-coded strings.
[11:52:35] <Avish> e.g. I want to do something like collection.Aggregate(Match<T>.Eq(x => x.SomeProperty, "someValue"), Group<T>.By(x => x.GroupProp).Value("count", Group<T>.Sum(x => 1))
[11:53:34] <Avish> Or even simpler, I'd like a simple way to translate a member expression like `x => x.SomeProperty.SomeInnerProp` to its named equivalent `someProperty.someInnerProp` using the mapping defined on T. Then I'll build the aggregation operator docs from that. Is this possible?
[11:55:45] <merpnderp> rspijker: I'm sshing in to the mongo host so connecting to 127.0.0.1:27017 should just be tunneled to mongohost:27017. Shouldn't it look like a localhost connection?
[11:56:34] <Avish> Basically, I want to do what QueryBuilder<T>.EQ does when it translates the expression to an element name. Anyone>
[11:57:43] <rspijker> you can use whatever port numbers, it's just that I have a mongos running locally for testing, so 27017bwas out of the quesiton
[13:12:50] <remonvv> Nope. It ended up in a state that wouldn't allow it. Didn't have time to figure out why but I'm assuming an ill-timed mongos restart or something.
[13:18:24] <rspijker> Derick: I completely agree that it should not be done, just trying to explain to this "dude" why it could happen :)
[13:19:16] <remonvv> shmoon, that's because the query validation is somewhat inconsistent between the "criteria" and "update" parameters of the update operation.
[13:21:13] <remonvv> If you want my advice; all fieldnames should be predictably cased (lower, camel, etc. but be consistent) and have no spaces or other non alphanumeric characters.
[13:21:43] <remonvv> Righto. That's why it's not allowed and why you shouldn't get a period in there even if your driver/import allows it.
[13:21:58] <shmoon> but i am kind of solving a complex problem and needed to have the app up asap, basically all field names are defined by the user (comes from html form)
[13:22:39] <remonvv> shmoon, I understand but you'll never be able to get this schema to work. Your user input -> field name conversion should convert it to something that follows the rules mentioned above.
[13:23:32] <remonvv> And frankly allowing end users to determine your field names is on the dodgy end of things as well. You should probably go for a (field:"No. of pages", value:"64"} sort of schema then
[13:23:33] <shmoon> actually lets discuss this a bit, maybe i can find another way to solve my problem (requirement) - 1 sec
[13:25:33] <shmoon> remonvv: so I have tables like this http://puu.sh/3FRz9.png - user can put in keys (top row) and then values, add rows/columns. I want to save it in a proper way and then be able to properly search later and do further operations like filter/comparison, etc. stuff. such tables belong to each product, and each product can have multiple such tables
[13:26:35] <Derick> shmoon: those are values for something, not keys. Keys should never be arbitrary (as they are now,as the user can use whatever they want)
[13:34:37] <kali> +1 for remonv suggestions. never use variable key names in mongodb if you want to do anything with the data
[14:21:08] <Michael_> i see a very weird behaviour on our servers that chunks are no longer split automatically
[14:24:32] <Michael_> but unfortunately, the second top most chunk is again getting more documents and not being split
[14:53:42] <remonvv> And dump the verbose version of your sharding status in a pastie as well please
[16:08:03] <novice35> Hi all, I am new to mongodb and I tried to follow the tutorial "http://docs.mongodb.org/manual/tutorial/deploy-replica-set/" but when I tired to execute the command rs.add("vm_name") the following error occured : "exception: set name does not match the set name host vm_name:27017 expects
[16:44:43] <FuZion755> hi all, i'm trying to reformat a date within an aggregate... when I try to concat the different component it yells at me because some of them are integers.. is there a way to cast integers to strings? I can't for the life of me find any way to do that in an aggregate
[17:00:19] <Chrishas> hi, the following gives me an error about argument 5: mongo_create_index(conn, "testdb.Users", key, 0, out);
[17:01:00] <Chrishas> although it's the same as the C Mongodb api tutorial, and it also complains about too few arguments
[17:26:57] <JeremyKendall> I have a question about updating multiple mongo collections in the same js file if anyone has a minute
[17:27:49] <Guest5537> I need to construct a query that only matches documents that have an array field whose first element in the array matches X. Is it possible to do positional stuff like this with queries?
[17:29:01] <JeremyKendall> I have a js file with multiple commands. I run it like 'mongo dbname file.js'
[17:30:56] <Chrishas> algernon: thx but the system I'm trying to run it on has an older version of automake tools and autoreconf shows an error, is there another way to compile this?
[17:31:10] <kali> JeremyKendall: http://docs.mongodb.org/manual/core/server-side-javascript/#running-js-scripts-in-mongo-on-mongod-host maybe that will help
[17:31:55] <sharondio> Guest5537: It is possible to do positional stuff in arrays when you know the index of the item you're looking for.
[17:32:15] <JeremyKendall> kal1: Thanks for the link. I'm not actually having an issue running the js files. My problem is with the commands *in* the files :-)
[17:32:39] <sharondio> Guest5537: If you don't know the item, you can do it with $ variables, but only one level deep. No arrays in arrays. #askmehowIknow
[17:33:27] <algernon> Chrishas: you can run autoreconf -i on a newer system, copy the results over and compile away.
[17:34:23] <JeremyKendall> Here's the file: http://pastie.org/private/27g3umsrxvbymyni6xaata When I run it via the command line, the remove() works as expected, and so does the first update(). The rest of the commands don't seem to run.
[17:34:41] <JeremyKendall> If I run the file again, the second update() runs, but again the rest of them don't seem to run.
[17:35:35] <JeremyKendall> (I'm basing "don't seem to run" on the results of counting the collections to see if the fields I unset are actually unset)
[17:35:35] <kali> JeremyKendall: can you try with $unset : { <blah>: true } instead of these empty strings ?
[17:36:49] <Guest5537> sharondio: I do know the index I'm interested in. Is it as simple as doing a regular query with the first part of the array? {'array[0].field': {$eq: 'value'}}
[17:37:34] <sharondio> JeremyKendall: Is it possible you have an asynchronous issue? I'm still pretty new to server-side stuff, but finding async.js has changed my life.
[17:38:02] <JeremyKendall> sharondio: That's entirely possible, but I woulnd't know how to diagnose that.
[17:38:29] <sharondio> Guest5537: http://docs.mongodb.org/manual/core/update/#update-arrays The first code example seems to support that, but I haven't tried it.
[17:39:32] <sharondio> JeremyKendall: Well, when I find myself bashing my head against the wall with JS not behaving like I expect, I've learned to suspect Asynchronous processes. It's also nice to have async.js just be able to tell me when everything has processed, or run an error function if something is misbehaving.
[17:40:08] <sharondio> JeremyKendall: The way your stuff is laid out, there is no error capture at all so you wouldn't even know if something were bombing.
[17:40:35] <JeremyKendall> kal1: Here are the counts I'm running http://pastie.org/private/kaoys9qe927i0oejhqina I snipped two of them because they're pretty much all the same
[17:41:40] <sharondio> JeremyKendall: And if you run each command in the command-line individually in the same order, they work?
[17:43:46] <sharondio> JeremyKendall: I'd probably put a simple callback on each call, logging that it ran just to see where it's actually stopping.
[17:45:09] <Guest5537> sharondio: It looks like the dot syntax works for arrays in queries. {'array.0.field': 'value'} -- works as expected.
[17:45:09] <JeremyKendall> kal1: I replaced the "" with true for each $unset. The behavior remains unchanged.
[17:48:12] <JeremyKendall> kal1: If it makes a difference, I'm running 2.0.7. Still waiting on sysadmin to upgrade mongo on our dev vms
[17:48:59] <sharondio> kali: Even synchronously, one of these calls is stopping everything. It might help to see which ones are actually finishing. Is there some kind of command to make the shell more verbose? I only do test queries in the shell. I do everything else in node (obviously).
[17:49:41] <kali> JeremyKendall: add print statements after each op to see if it's running till the end
[17:51:27] <sharondio> JeremyKendall: Check this out: http://stackoverflow.com/questions/9457368/inserting-data-to-mongodb-no-error-no-insert
[17:56:22] <sharondio> JeremyKendall: http://docs.mongodb.org/manual/reference/command/getLastError/ This might be useful.
[18:04:26] <JeremyKendall> kal1: I added print statements after all of the updates. It made it to each of them, but the script behaves the same as before.
[18:08:11] <kali> JeremyKendall: can you show me what you do in the interactive session that does work ?
[18:10:07] <kali> JeremyKendall: can you add a print(db.c1.count()); at the top of you file, to check if we are really working on the right db ?
[18:11:35] <JeremyKendall> kal1: OK, this is weird. I added printjson(db.runCommand({ getLastError: 1, w: 1, wtimeout: 5000 })); at the bottom of the script (inside the try) and it worked the first time through.
[18:15:36] <JeremyKendall> I'm adding that getLastError after each command just to see what happenes. I'll let you know.
[18:18:01] <JeremyKendall> kal1 && sharondio: Added getLastError after each command, got no errors from any of the commands, and everything worked as expected.
[18:18:57] <JeremyKendall> Without those checks, the script runs in milliseconds. If it were actually doing the work, it would take much longer (there's a lot to delete).
[18:21:10] <sharondio> JeremyKendall: I'm not sure leaving in those logs is the real answer, even if it does work.
[18:31:01] <pwelch> so I want to do what I currently do with mysql. I have a master and a slave. when I do a switch over I make a new master and connect it to the old slave to update the data
[18:31:48] <pwelch> can I start a mongodb instance as a master and slave so I can just point to the box?
[18:34:52] <JeremyKendall> kali && sharondio: Wild goose chase. Turns out those commands were all running. It takes 10-ish seconds for them all to finish. The js file was just queuing them up. I was counting too quickly.
[18:35:10] <JeremyKendall> This time I waited about 15 secs, ran my count script, and everything that should have been zeroed out was.
[18:36:43] <Gargoyle> pwelch: Not sure what mongodb master/slave is if you are not talking about a RS.
[18:37:33] <pwelch> same as MySQL master/slave. read/writes go to master and they get sent to the slave. You can "promote" the slave by pointing the writes to it
[18:39:30] <pwelch> form the docs mongodb master/slave is deprecated. Im trying to use it for when I build new nodes and need to point to the new one
[18:40:03] <pwelch> just one. however, I want to chain them to keep the new one updated and then swap it out
[18:40:40] <eka> hi all, mongo is crashing with this message "warning: DR102 too much data written uncommitted 315.318MB" my setup is 4 mongodb in shard with 3 configs… it was running for a very long time, today started doing this
[18:41:54] <Gargoyle> pwelch: I'd use the replica set instead of depreciated stuff. Just run an additional mongos process on one box as an arbitrator.
[18:42:54] <pwelch> and I would have to reboot my apps so the client lib (driver) knows about the new node
[18:42:58] <Gargoyle> pwelch: You just step down the Primary - the secondary will take over within a few seconds.
[18:43:09] <sharondio> JeremyKendall: It happens. I learned a new shell command out of it, so that's cool. :-)
[18:45:11] <Gargoyle> pwelch: Yeah, you can add more than one node address to your connection params. to make connecting easier when you don't know which is the master from a "cold start", but the client should also be able to figure out the other nodes once it has connected to one.
[18:47:03] <pwelch> Gargoyle: sorry, I dont fully understand how the client lib knows about all nodes. does it connect to the one you give it and then pull in info about the entire cluster?
[18:47:47] <pwelch> if I add 3 more nodes but only told the client app about a single IP/FQDN it queries the info from that single node and learns about the others?
[18:48:22] <Gargoyle> Obviously, if the node you have configured is down when the app starts, it fails. So I think you would normally specify 2 or 3 in your connection params, and let the rest be discovered.
[18:49:40] <pwelch> Gargoyle: ok, I will. Was trying to get some info to deploy a solution today but this helps. thx
[18:53:55] <artdaw> I already have mongodb in prod but today it slows down completely and I'm really in stuck
[19:11:42] <eka> artdaw: I meant the faults count, that means that mongo reads from the Disk and that makes it slow
[19:17:02] <Ontological> So, am I not allowed to use the collection name of 'auth'? If not, where might I find a list of all such reserved names?
[19:29:32] <leander> Hi. When exactly the primary of a replication set becomes unavailable? For example if a sector of a harddisk becomes corrupt, mongodb will make the machine unavailable?
[20:24:43] <kevino> any idea why i can connect to a node with the mongo client but in pymongo MongoReplicaSetClient it doesn't connect at all?
[20:32:35] <eka> I think it's the way to go, with the MongoClient api… why you use MongoReplicaSetClient?
[20:33:05] <kevino> uh because it says to? http://api.mongodb.org/python/current/examples/high_availability.html#mongoreplicasetclient
[20:34:08] <eka> I had the problem of many socket connections in the kernels… do you have many connections or not?
[20:36:13] <alexr2> i'm trying to remove all items from an array of objects (product_limits) using an array of ids... any suggestion on how to get this working? -- something like $pullAll: {"product_limits._id": id_array}
[20:37:18] <kevino> eka: no, everything is working fine otherwise. all of my clients from the web servers connect without a problem
[20:39:35] <eka> yes… lol… but I can't find something like it… I will have to wait for rethinkdb to get serious on sharing and speed