Log file Viewer
#mongodb logs for Thursday the 25th of July, 2013
[00:01:03] <LoneSoldier728> http://stackoverflow.com/questions/17846822/how-to-get-objectid-from-another-mongoose-schema
[00:02:01] <LoneSoldier728> Basically I am creating the new mongoose schema but I want to take the userId
[00:02:44] <LoneSoldier728> right, but how do I set a new primary key for the new schema aka collections
[00:04:32] <caitp> you can simply have an indexed field called userId, and make it refer to the User schema
[00:09:12] <LoneSoldier728> ok so does this work userId: {type: ObjectId, ref: "User", index: true, unique: true }
[01:20:32] <resting> i'd created an admin user with mongo --eval "db.getSiblingDB('admin').addUser( { user:'admin', pwd:'123', roles:[ 'userAdminAnyDatabase' ] } )"
[01:36:26] <LoneSoldier728> anyone know when I make a new mongoose schema how I can reference the objectID and have it created when a new user is created or how it works?
[01:36:40] <LoneSoldier728> http://stackoverflow.com/questions/17846822/how-to-get-objectid-from-another-mongoose-schema
[01:54:49] <LoneSoldier728> the second one is the register Post that I am trying to create a new friend user account too
[01:55:10] <LoneSoldier728> but I am pretty sure I might be doing the .pre part wrong and .save wrong in the second one
[01:56:01] <caitp> well you need to call next() regardless of what happens in your code, otherwise you might end up with an endless loop
[02:03:31] <caitp> I haven't looked at your route file, but I assume it's working -- if it's not (and assuming you're using express), you can put `app.use(express.logger())` to see requests and status codes logged in the terminal,
[02:07:03] <LoneSoldier728> http://stackoverflow.com/questions/17847773/how-to-create-a-new-account-in-another-collection-through-mongoose
[02:08:03] <maslowbeer> hello. hitting my head against the wall with slaveOK=false messages even though the connection URI has in the options slaveOk is true. Noted in docs that rs.slaveoK must be run on primary "once per session" what constitutes a "session" in this context/
[02:10:07] <maslowbeer> how do I set slaveOK permanently on all my mongo nodes? running as a service
[02:22:18] <LoneSoldier728> http://stackoverflow.com/questions/17847773/how-to-create-a-new-account-in-another-collection-through-mongoose anyone?
[02:35:50] <LoneSoldier728> http://stackoverflow.com/questions/17847773/how-to-create-a-new-account-in-another-collection-through-mongoose
[03:01:06] <LoneSoldier728> http://stackoverflow.com/questions/17847773/how-to-create-a-new-account-in-another-collection-through-mongoose
[03:18:17] <LoneSoldier728> I am crying because an account is not being created in my second collection : /
[03:23:26] <gx> i'm having trouble here- I have one server instance running mongodb, and i'm trying to connect to it from another server through codeigniter
[03:23:54] <gx> i've allowed port 27017 through iptables on both servers... i can connect to the db remotely using MongoVue for windows, but on the 2nd server, the connection times out
[04:32:14] <LoneSoldier728> http://stackoverflow.com/questions/17847773/how-to-create-a-new-account-in-another-collection-through-mongoose anyone?
[04:32:57] <LoneSoldier728> for some reason the account in the other collection is not being created?
[04:56:31] <rickharrison> In Mongoose, on a findAndModify command, is there a way to find out if a document was inserted *and* get the new document back
[05:04:25] <LoneSoldier728> ok so your code right now is taking the user id, and then where is user coming from
[05:05:00] <LoneSoldier728> and the doc in your function should store the information from what was queried
[05:05:16] <LoneSoldier728> so idk where user is coming from but what does console.log(doc) print out
[05:14:41] <LoneSoldier728> if you want to query it before it is being updated and then query it after calling update on it you can see if it was updated
[05:15:54] <LoneSoldier728> but I am still unclear as to what you are doing honestly... you want to know if something was inserted as oppose to updated can you explain a real life situation so I can understand you better
[05:16:15] <LoneSoldier728> You can see if something was inserted as well by just noticing if the count changed for the amount of users in your collection
[05:16:56] <LoneSoldier728> and see if it changed, if it changes then it was inserted, if it was updated the count should be the same
[05:17:33] <LoneSoldier728> if you want to see the original info before it was updated call a query on it first without the update
[06:39:44] <kevino> when you stop the mongo service via an init script, it doesn't shut down cleanly?
[06:40:10] <kevino> is there a way to make sure that won't happen? like completely flush everything to the data file and not journals?
[08:44:30] <avner> First visit. hi, would that be a good place to seek for help regarding NodeJS and mongo behavior?
[08:50:34] <avner> 10x. Trying to upsert a document and than update it's sub documents in the call back. I create a gist of how it works: https://gist.github.com/AvnerCohen/6077932
[08:51:04] <avner> I'm trying to understand why the first example does not work as expected. only the last document exists, even though i use {$set}.
[08:57:52] <Nodex> collection.update({conv_hash: '711_34401'}, base_doc, {upsert: true }, function() {
[09:00:41] <Nodex> you also don't need to do multiple $set statements.... one will suffice ... db.foo.update({},{$set:{'foo':1,'bar':2,'baz':3}})
[09:01:07] <avner> 10x. I understand it makes no sense. however, this is just used as an an example for a much more complex code. where these are not really called in a single flow.
[09:01:40] <avner> I'm trying to understand why one example works and the other not, I'm not looking for improvements for the code itself as this is just a sample.
[09:04:41] <Nodex> db.bar.update({id:1},{$setOnInsert:{a:1,b:1,c:1}},{upsert:true}); yields ... { "_id" : ObjectId("51f0ea0a8e9d0cace856ca9a"), "a" : 1, "b" : 1, "c" : 1, "id" : 1 }
[11:32:49] <solars> hi there, quick question: if I have a couple of referenced objects, like Author, Post - and I want to do join like queries like All authors where a post contains 'fu' - I suppose mongodb is not the way to go, right? I guess it's better to use sql for such kind of data? (I've got a lot more objects that are cross referenced of course in real life)
[11:54:02] <remonvv> solars: MongoDB is not a relational database so you are correct that a JOIN is not possible. Things like that always require 2 seperate queries. Whether or not MongoDB is a good choice depends on how often you have to do relational things versus things MongoDB is good at.
[11:58:11] <solars> for this purpose it's needed quite often, so I think for this app we cannot use mongo but will stick to sql ..
[11:59:09] <Nodex> also remember just because it might make sense to join tables for representations it often doens't for performance
[11:59:35] <hyperb> What's the default behavior for mongoimport without --upsert? Does it update matched _id's?
[12:00:37] <remonvv> hyperb: It upserts on _id. Meaning it overwrites existing ones and adds non-existing ones.
[12:01:33] <hyperb> Hrm... I ran an import yesterday on some test data and I got a bunch of 'duplicate _id' messages. I added --upsert and everything went fine
[12:02:02] <hyperb> Basically, I ran mongoexport, ran some sed changes on the data, and wanted to reimport
[12:02:04] <remonvv> hyperb: Ah, I misunderstood your question. If you remove it completely it will skip existing ones.
[12:03:00] <hyperb> In my scenario, there shouldn't be any unmatched _id's. But is there a way to --update?
[12:03:21] <remonvv> --upsertFields a,b,c is basically equiv to update({_id, a, b, c}, {...}, true, false)
[12:04:29] <remonvv> hyperb: Alright. So yes, without --upsert it'll skip documents with matching _id values.
[12:05:04] <hyperb> I had conjured up a hypothetical scenario though, where I had run mongoexport from live data, then I wanted to import it on top of older test data, but only updating witout insert
[12:06:27] <remonvv> hyperb: Don't think that is supported. To be fair it doesn't make a huge amount of sense. That scenario is only relevant if you removed data from the live environment that still exists in the exported data.
[12:06:59] <remonvv> hyperb: It can't always guarantee type information. It's rather like doing something like db.col.save(db.col.find().next()) in the shell
[12:09:59] <hyperb> so I ran a mongoexport, did the renames in the export file, and I'm about to run mongoimport to apply the changes.
[12:18:30] <remonvv> hyperb: If your tests were successful there shouldn't be an issue. What are you renaming?
[12:25:11] <remonvv> Options options ;) If you have the option to do something with operators rather than JS or export->modify->import you should.
[12:31:01] <hyperb> If I want to run through my 'streams' coll and for every "title" that starts with "OLD", I want to change it to "NEW"...
[12:31:49] <hyperb> db.streams.find({"title":"/^OLD/"}).update({$set:{"title":"NEW---- no, that won't work.
[12:38:43] <remonvv> hyperb: By the way, seems like a slightly odd operation. It's likely not an option for you but that looks like something that can use a schema improvement.
[12:40:32] <hyperb> I just want to do a batch rename. It probably is slightly odd. Not something that should come up very often, I'd imagine.
[12:41:11] <remonvv> hyperb: Anyway, probably easiest to do that directly in shell. Grab a cursor, iterate over it, apply the rename, save it, done.
[12:41:43] <hyperb> In any case, graylog is still in heavy development. Batch rename would be more of a 'spit & polish' feature.
[12:42:50] <hyperb> db.streams.find({"title":"/^OLD/"}) will give me a cursor containing the docs that match the regex, right?
[12:42:51] <remonvv> hyperb: Sure -> c = db.col.find(); while(c.hasNext()) {d = c.next(); d.title = <yourmagic>; db.col.save(d);}
[12:44:50] <remonvv> Okay, well when you get the above sorted tell me your experiences. I'm looking at log aggregation that isn't shit.
[12:46:04] <hyperb> I'm a Linux admin supporting numerous dev environments for a major internal webapp.
[12:47:16] <hyperb> My predecessor had a cron job that scp'd the scripts all to one box and some real simple php display scripts.
[12:48:31] <hyperb> I'm actually using the logstash client piece to format the data and send it to graylog
[12:54:56] <hyperb> Yeah. Once we're running full-time, I'm gonna look at building out a cluster with queueing to aggregate everything.
[12:56:15] <hyperb> remonvv: So can you help with the <yourmagic> part? I basically just want s/old/new with substrings
[13:00:31] <remonvv> hyperb: I'm no JS expert. Alternatively you can try "NEW" + old.substr("OLD".length, old.length)
[13:11:56] <remonvv> For you it's basically db.col.find().forEach(function(d) { d.title = d.title.replace("old", "new"); db.col.save(d);}
[13:14:00] <remonvv> Yes, it will invoke the toString() method of the object automatically due to string concatenation
[13:17:54] <remonvv> Is it? In Java it's implemented as a compiler step unique to string concatenation where toString() invocations are inserted at compile time. Afaik it was the same in JS but I could be wrong. JS isn't my thing ;)
[13:18:46] <Nodex> sounds kinda cool though... in the javascript quirks concatting always casts to a string
[13:47:23] <intellix> anyone know of a good Mongo GUI for OSX? It seems that 90% of the apps I try just do not work at all… MongoHUB seems to crash every time. Umongo doesn't work
[13:48:30] <cheeser> there's this: http://docs.mongodb.org/ecosystem/tools/administration-interfaces/
[13:52:26] <remonvv> intellix: Yeah it's written by a 10gen employee iirc. Used to be JMongoBrowser or something
[13:53:40] <cheeser> remonvv: i haven't yet because i've been busy trying to fix javabot and get my javaone presentations done. but i'm angling to get back on that soon.
[13:56:06] <remonvv> cheeser: I see. Most production environments use local mongos instances so it's not a huge deal I suppose.
[13:57:08] <cheeser> right. i know that's what we did where I worked when I wrote that so it wasn't urgent.
[14:03:14] <remonvv> If it takes you twice as long to do the result will be twice as good. It's only due to practical difficulties that I do not write code using postal pigeons.
[14:03:39] <remonvv> On a slightly more serious note; there aren't any really smooth GUIs for MongoDB that I'm aware of. I've tried quite a few.
[14:05:26] <intellix> so much easier to just scroll down a list of rows, double click, edit, double click, edit, double click, edit, save
[14:06:46] <intellix> you can't deny that all of the parens and brackets are a complete mindfuck when writing on a single line
[14:09:43] <spacemonkey> Derick can you point me to where I can setup mongo.so as a session handler?
[14:09:49] <remonvv> intellix: Nobody's disagreeing but it takes a healthy amount of effort to make a GUI that does all that and keep it up to date with mongo's still evolving syntax/operators
[14:10:14] <Nodex> spacemonkey : yu probably have to do that yourself by registering a session handler
[14:10:37] <spacemonkey> The extension by default does not register itself as a valid handler candidate though.
[14:11:05] <spacemonkey> http://www.php.net/manual/en/session.configuration.php#ini.session.save-handler
[14:11:14] <Nodex> a quick google http://www.nathandavison.com/posts/view/12/php-sessions-stored-in-a-mongo-database
[14:11:41] <spacemonkey> Yeah, that's not done through the extension itself though, was just checking there as the primary source.
[14:13:39] <spacemonkey> But wanted to check if someone else was already on it, to avoid duplicate effort.
[14:44:08] <remonvv> hyperb: If you're trying to update multiple documents at once make sure you pass multiple=true
[14:51:44] <remonvv> cheeser: We're still looking for the one person that thinks it's a good default.
[14:52:10] <cheeser> i think i know what that was so i probably shouldn't rant too much (publicly) :D
[14:53:56] <cheeser> that and collections can grow to be *quite* large and when sharded, updating everything that matches can be quite expensive.
[14:54:15] <cheeser> so it makes a bit of sense but usually ends up biting people in the ass because they expect the other behavior.
[14:54:15] <remonvv> cheeser: Yes but that's not an excuse for a bad default though. Too large -> optimize
[14:55:08] <remonvv> It really is. If you find yourself in a situation where a multi-update takes too much time then that's something you'll have to solve.
[14:55:27] <remonvv> I think it has little bearing on whether or not the default should be the unexpected but faster approach.
[14:55:59] <remonvv> I think it was an honest mistake at some point and they failed to correct it prior to 0.9
[14:56:22] <remonvv> And now everyone agrees it's a bad default (including the original authors) but it's a backward compatibility issue now.
[15:03:51] <remonvv> hyperb: Okay, let me rephrase; collections are naturally ordered and it's not easy to manipulate natural ordering reliably.
[15:04:54] <remonvv> hyperb: Only collection type where natural ordering is somewhat relevant is capped collections.
[15:09:13] <aquarius> http://docs.mongodb.org/manual/core/document/#timestamps suggests that timestamps in the oplog are a 64 bit number where the first 32 bits are seconds since the epoch. When I query mongo from code, I get weird values back, like: "ts": "5904541388648742913". What's that? (I can't test in the shell, because the shell knows about Timestamp types and automatically decodes...)
[15:28:52] <LoneSoldier728> if I have two separate arrays one that holds the friend id and another that holds the date it was received on
[15:32:02] <Nodex> it will involve the $ postional operator, not sure how you use that on an index key
[15:33:09] <LoneSoldier728> is it worth doing another query for it? or storing the id with the date in the array for date
[15:33:49] <Nodex> I would re-model the data tbh and store the _id, timestamp, name etc of the friend and pluck that whole thing from the array
[15:33:50] <LoneSoldier728> positional operators do not work in mongoose from what I read at least... I just reshaped my whole db because of it actually (adding a second collections for friendships
[15:34:01] <bbommarito> Good morning/afternoon all. I have a quick question: Is it possible that using something other than an ObjectID as the _id for a document to cause a huge slow down in read/write time on a given document?
[15:35:51] <bbommarito> Nodex: Okay, that's the answer I needed. I am working for a startup using MongoDB as a backend, and our outsourced team has basically rebuilt 30% of the app to replace our current _id structure with ObjectID, claiming it was the cause of slowdown (It's not, it's bad code).
[15:36:31] <Nodex> LoneSoldier728 : you can do it in 2 queries I imagine... first query to update the array member setting somehting to 1 (deleted:1) .... second query to $pull all foo.deleted:1
[15:37:18] <LoneSoldier728> Yeah I imagined that is a waste though right to do both queries? Does mongo timestamp when things are added to the document anyways?
[15:37:47] <Nodex> LoneSoldier728 : you don't need to do the delete query every time, you can do it once a day or w/e
[15:38:33] <LoneSoldier728> well if a friendship needs to be removed I would want to delete it right away
[15:38:47] <drag> Hi there. How do you delete with MongoOperations? It doesn't seem to have a delete() method anymore. Is it find and remove now?
[15:43:38] <bbommarito> Nodex: They come from a country that...is a part of the Bible, and I will go no further
[15:45:07] <LoneSoldier728> Nodex any clue how I can loop through to find a position in the array does something like this work find(users) then loop and state the array position for the user
[15:48:06] <LoneSoldier728> then how do I figure out the array, like I know I can loop through for the user or, do I just figure out what i equals at that point
[15:50:45] <Nodex> you're gonna have to make do with what you can then. tbh if Mongoose cannot do positional operator you should get rid of it
[15:50:54] <LoneSoldier728> small meaning for learning purposes but thinking of scalability along the way
[15:51:32] <remonvv> scalable graphs such as friend networks are...and here comes the understatement of the day...tricky.
[15:51:34] <Nodex> I wrote a large Adult networking site with MongoDB, friends was a problem, I ended up using redis to store them and when they're online etc
[15:54:17] <LoneSoldier728> I am trying to remove a date_added from the date_added array when removing a friendId from the friend Array ( so two diff arrays with the same position in each)
[15:54:53] <LoneSoldier728> The reason I have two arrays because there tends to be problems with $pull / remove when there is an array in an array
[15:55:32] <remonvv> And yeah, there are. As a general rule never do nested arrays until they fix the multi-$
[15:56:17] <remonvv> but why isn't your schema (roughly) db.users{friends:[{friendId:.., dateAdded:..}, {..}]}
[15:56:51] <remonvv> LoneSoldier728: No, you have a single top level array with objects as elements.
[15:58:51] <Nodex> db.foo.update({"_id":ObjectId("..."),"friends.friendId":1},{$pull:{friends.$:1}});
[16:00:24] <remonvv> LoneSoldier728: update({..}, {$pull:{friends:{$elemMatch:{friendId:<yourId>}}})
[16:01:20] <LoneSoldier728> would pull remove the whole object containing the id, request date, etc?
[16:01:40] <Derick> LoneSoldier728: no, you need to give it the full array value - if it's an object, it wants all keys
[16:05:11] <LoneSoldier728> let me gist my code and just show you what I am doing and then lets see the critique
[16:06:20] <remonvv> I had the inconsistent syntax discussion so often in here I jumped on the wrong side of the "correct" line on that one :p
[16:06:27] <LoneSoldier728> ok so that includes the new schema I made for friends, I actually want to set the userId as the objectId I have yet to do that
[16:06:44] <LoneSoldier728> and the code for accepting a contact should be adding a person to friends while removing them from requests
[16:07:57] <remonvv> Derick: I think I once argued that $elemMatch should be part of $pull syntax rather than it being implied and that an $pull without it should do exact object matching.
[16:09:31] <remonvv> LoneSoldier728: You can't do the second step in one go unless you have the pending requests array in the same document
[16:10:04] <remonvv> LoneSoldier728: Can't. A mapper might allow you to map it from and to _id but _id is a mandatory field in a document
[16:11:26] <remonvv> LoneSoldier728: I mean that removing a pending friend request and adding a friend to the friends array in the user document can only happen in a single update if the pending friends requests array is in your user document. Which isn't a terribly bad idea mind you.
[16:11:52] <remonvv> The add request would simply add an entry to the pending friends request array of the invitee
[16:12:48] <LoneSoldier728> I have both in the Friend's schema and I am adding to set the new friends id while removing the id from the requests
[16:13:35] <LoneSoldier728> I have a received request and pending request so this way I can show recieved requests in notifications
[16:14:05] <LoneSoldier728> and this way they are attached to the users that are performing the actions
[16:14:33] <LoneSoldier728> the only thing I am having a problem with is the date's associated with the actions
[16:15:02] <LoneSoldier728> I was actually wondering if mongo already has timestamps built in so I do not even have to collect that data
[16:20:14] <remonvv> LoneSoldier728: Not in a way you should count on but ObjectId's have a timestamp element.
[16:22:41] <remonvv> Store a timestamp field, and use it. It's not very complicated unless I misunderstand your requirement.
[16:25:35] <remonvv> LoneSoldier728: Basically wrap all your seperate arrays into one and each array element should contain all those fields (so dateRequested, dateReceived, friendId, and so on}
[16:26:59] <remonvv> ron: Why should I hire you to get your personal project started. It's a pretty good idea. Go for it ;)
[16:27:23] <remonvv> LoneSoldier728: Yes, you can use array position in your query. So friends.1.friendId
[16:28:51] <remonvv> ron: So your plan is to ask an e-buddy on the other side of the world to ask for a part time job huh ;)
[16:31:05] <remonvv> The former works because it's supposed to and the latter works because I make sure it does.
[16:31:43] <remonvv> If you're not 100% on MongoDB syntax I'd start with doing it directly on the JS driver.
[16:31:50] <LoneSoldier728> the good thing about mongoose is the setup of a schema even tho mongo is schema-less
[16:32:47] <remonvv> LoneSoldier728: I'm not going to learn mongoose to help you out kind sir ;) There are limits to what you can expect from a support channel :)
[16:36:24] <LoneSoldier728> my code works beautifully right now except for the date removal, I will deal with that later tho
[16:40:52] <remonvv> LoneSoldier728: It's because he wants to work for someone that walks that fine line between brilliant and beautiful
[16:50:53] <ron> remonvv: see, I managed to add LoneSoldier728 on linkedin without his email. you must be an idiot.
[16:51:53] <remonvv> ron: I'll have you know my LI is rather wonky. It currently says I should upgrade to premium to do X, I'm upgraded.
[18:02:41] <_jo_> Ohai Mongodb. Any guesses why my query collection.find({'poll':{'$ne':null}}) returns nothing even though I have some entries with poll = [1, 2, 3]?
[18:10:40] <_jo_> I want to select only those where (1)the field exists, and (2)the value is not none.
[18:15:22] <was85> Hi there, I'm using mongo db for the first time, but I'm unable to connect to remote db :( Can someone tell me how to do it?
[18:17:09] <was85> Nodex: I'm able to connect using the command line and gui tool, but unable to connect using the code
[18:20:07] <was85> yeah should've mentioned this in the first place, I'm having problem with connection string
[18:56:45] <WDC> Hello all. I have sent a text index on a collection and do a db.col.runCommand("text", {search: "text search", limit: 10}) and get the following error: { "ok" : 0, "errmsg" : "can't find ns" }
[19:55:06] <aleszoulek> Could you help me with replicaset a bit? I had three instances lets say m1 (primary), m2 and m3. m1 and m3 died. So I have just m2, which is sencondary
[19:55:24] <aleszoulek> When I try to reconfig replica set from m2, it says, that I need to call it on primary
[19:55:45] <aleszoulek> "errmsg" : "replSetReconfig command must be sent to the current replica set primary.",
[20:30:51] <drag> What exactly is the purpose of the WriteResult? Is it just to hold any errors or concerns that occur during write?
[20:37:08] <drag> cheeser, so after a write/update, if I call the getError() method on the returned WriteResuilt and it's null, no errors occurred, right?
[20:44:22] <drag> cheeser, let me ask it this way, can the getErrors() method ever return a non-null value if that everything went fine and the writeConcern was set to 1?
[20:45:59] <cheeser> i can only speak to the java driver behavior but i *believe* that WriteResult.getResult() will always be non-null with non-zero error codes.
[20:46:23] <cheeser> isOk() determines the error state generically and then you can get the actual code and/or message
[20:47:40] <jcromartie> is there a way to sort by one field and THEN by another? in a specific order?
[20:50:05] <drag> cheeser, I'm using the java driver as well, bu tI don't see a getOk() method in the code or javadocs
[20:53:38] <drag> cheeser, ok, so I need to get the last commandResult with getLastError() or getLastError(WriteConcern) and then call ok() on that? If true, then all good?
[22:04:52] <wc-> i can do service mongodb stop and they die, but if i do a start all of them come back
[22:05:04] <wc-> should there be multiple mongodb's with the commnad line /usr/bin/mongod --config /etc/mongodb.conf
[22:10:40] <ranman> do you think it could be this: http://unix.stackexchange.com/questions/10362/why-does-htop-show-more-process-than-ps