PMXBOT Log file Viewer

Help | Karma | Search:

#mongodb logs for Wednesday the 24th of July, 2013

(Back to #mongodb overview) (Back to channel listing) (Animate logs)
[00:21:51] <VarunVijayarghav> Had a question regarding MMS. I just finished setting it up for our mongo hosts. There is a column called "Cluster" which is not getting filled. Presumably, there is an option somewhere in my configuration that I am not setting up?
[00:21:51] <VarunVijayarghav> Just curious to know what that is.
[01:09:06] <caitp_> question: could I use $explain to provide insight into everything $pull is doing?
[01:09:33] <caitp_> and if so, what would that look like?
[01:10:05] <caitp_> would just db.collections.update({{}, {$pull: {…}}}).explain() be enough?
[01:22:38] <meLon_> Any benefit from indexing on 0,1,2,3 for a value vs a single char, such as I,F,T,P ?
[01:28:44] <caitp_> ignore my question, I got it to work
[07:45:36] <[AD]Turbo> hola
[07:48:20] <rspijker> hey
[09:53:21] <remonvv> \o
[09:53:43] <Nodex> o/
[09:54:10] <ron> /o\
[09:54:34] <Nodex> go make us a drink ron
[09:54:45] <ron> filled with poison, for you.
[09:54:57] <Nodex> yum yum
[13:07:46] <remonvv> Would anyone agree if I said MongoDB should not allow me to drop the shard key index? Currently there no error or warning.
[13:08:42] <ron> heh.
[13:10:04] <remonvv> Hm, both sides can be argued. The "well don't do anything stupid" argument has some merit. On the other hand, who would possibly want to break their cluster that way on purpose?
[13:10:35] <ron> remonvv: Nodex probably.
[13:11:31] <Nodex> die
[13:12:11] <remonvv> I want to be the best man on the wedding between you two.
[13:12:52] <Nodex> I'm to good for ron
[13:13:01] <ron> and you can't spell.
[13:13:23] <Nodex> he needs someone ugly who will love hi no matter what he does for a living, even if he's just a tea boy
[13:13:27] <Nodex> him*
[13:19:22] <remonvv> Well even the most loving couples have their less pleasant moments, clearly.
[13:19:52] <Nodex> haha
[13:41:44] <pithagorians> hey guys, i have a 3 members replica set. i kill the mongod process to test the failover and when i start it back mongo creates about 25 files of 2 GB each
[13:41:56] <pithagorians> are they kind of snapshot in time?
[13:43:18] <remonvv> I would assume it's pre-alloc for your data.
[13:43:38] <quattr8> I have a collection of user sessions which i want to shard, every user gets a http cookie with the documents mongoid and websiteid, everytime this document is retrieved with the _id and the websiteid, should i set a shardkey on _id and websiteid?
[13:45:02] <remonvv> why do you need more than _id? it's unique.
[13:46:38] <remonvv> It's rather like wanting to mail something to Petey and Petey says "I live at this address and also I like jeans"
[13:46:41] <remonvv> You just need the address.
[13:46:57] <Nodex> but what if you're mailing him jeans?
[13:47:00] <Nodex> :P
[13:47:31] <remonvv> Ain't nobody got time for that!
[13:48:37] <quattr8> so I should combine all the data in one document and leave out the websiteid as shardkey
[13:50:39] <remonvv> Isn't all the data in one document to begin with?
[13:51:59] <quattr8> No i thought to seperate the documents by websiteid, else documents might get too large to work with for me
[13:52:41] <remonvv> They can be seperate documents but they're in the same collection no?
[13:52:57] <remonvv> so db.sessions {_id:...., websiteId:..., and so on}
[13:53:15] <Nodex> please don't tell me you're saving multiple users sessions in one document?
[13:53:21] <quattr8> no lol
[13:53:26] <Nodex> phew
[13:53:55] <quattr8> I build a simple adserver for my site and the tracking is now done by a seperate attribute trackingid and websiteid but the collection is getting a bit too large (88 million documents)
[13:54:17] <quattr8> then i thought i could just use the _id as trackingid and save a lot of space
[13:54:49] <remonvv> quattr8: What I am saying is this; if all documents are in one collection an _id value will ALWAYS uniquely identify any document in that collection. You say you have the _id value at lookup time which means you do not have to query on or put an index on "websiteId"
[13:54:54] <remonvv> quattr8: Make sense?
[13:56:28] <quattr8> hmmm yeah i think i have to think of something else
[13:57:05] <remonvv> Why? it's a perfect solution.
[13:57:51] <remonvv> Stick your session key in _id, shard key is {_id:1} and you're done. websiteId is a property of the session.
[14:00:25] <pl2> /msg NickServ identify masterkard
[14:00:32] <pl2> lol shit
[14:00:34] <Number6> :-|
[14:00:56] <remonvv> Nice pw bro :)
[14:01:22] <pl2> lol thanks
[14:01:30] <cheeser> nice
[14:02:32] <remonvv> http://bash.org/?244321
[14:03:14] <Nodex> lmfao
[14:03:29] <Nodex> remonds me, I need to id
[14:03:38] <pithagorians> remonvv: they are in format /var/lib/mongodb/local.1
[14:03:41] <Nodex> msg nickserv identify ronisabiggirlsblouse
[14:03:45] <Nodex> goddamit
[14:03:54] <Nodex> forgot the /
[14:04:22] <remonvv> pithagorians: local is where the oplog is, so makes sense that that is created. Why it's pre-alloced to 50Gb I'm not sure though. Is that the size of your dataset roughly?
[14:04:34] <pl2> /msg nickserv set password NodexHasAnUglyGirlfriend
[14:05:16] <Nodex> evidently that's a lie lol
[14:05:32] <Nodex> never seen an ugly underwear model
[14:05:45] <Derick> you scared meghan away
[14:05:54] <Nodex> doh!
[14:06:01] <pl2> ok now that I changed my password!! ;) I was wondering why my databases are being capped at 0.5GB? I'm on a 32bit install, but my first database stopped taking inserts at 2.5gb, now every other database maxes out at 0.5? What's going on here?
[14:06:50] <Nodex> db's are capped at 2gb on 32bit, are you sure it's 2.5?
[14:07:27] <remonvv> pl2; never use 32-bit anything
[14:07:40] <Nodex> 33 bit ftw
[14:08:08] <remonvv> We need to start keeping track of NodeX's useful contributions.
[14:08:12] <pl2> check it: local 0.03125GB,... test(empty),... db1 2.4990234375GB,... db2 0.5GB, db3 0.5GB
[14:08:31] <Nodex> I have a whole collection sharded for my useful contrib
[14:08:34] <remonvv> pl2, use db1; db.stats()
[14:08:35] <pl2> My host is 32-bit unfortunately :'(
[14:08:58] <remonvv> pl2: I'd advise against MongoDB then if that's not changing. Having a hard limit on DB size is too risky.
[14:09:30] <remonvv> pl2: It's a consequence of the storage engine being implemented on top of mmapped files. Other DBs do not have that limitation usually.
[14:09:59] <pl2> remonvv, yeah, it's been a bit problematic
[14:10:09] <remonvv> pl2: Understatement :)
[14:10:52] <pl2> remonvv, output from stats(): http://pastebin.com/8JdzMqji
[14:11:43] <remonvv> pl2: Right, the sum of dataSize, indexSize and nsSizeMB can't be > 2^31
[14:11:58] <pl2> I plan on moving from this host, I was a bit shocked when they told me the server was 32bit.
[14:12:07] <Nodex> which host is it?
[14:12:09] <remonvv> pl2: It's very 2003
[14:12:27] <Nodex> let's email them and laugh at thier lack of 64bit awesomeness
[14:12:41] <pl2> HipsterHost Vintage servers for Ironic programmers
[14:12:58] <Nodex> HVI ?
[14:15:56] <pl2> Hudson Valley Internet? jesus that's amazing.
[14:18:20] <Nodex> never heard of them
[14:18:56] <Nodex> holy cow, they don't care about their website why should they care about thier servers
[14:20:53] <pl2> One thing I still don't get about the size issue.. if there's a 2gb limit, why is it that the databases are now being capped at 0.5gb?
[14:21:20] <Nodex> that is strange
[14:21:26] <cheeser> use linode
[14:21:27] <Nodex> which mongodb version?
[14:21:29] <pl2> Nodex, oh didn't see you were asking about the host. It's KnownHost, btw
[14:22:01] <pl2> 2.4.5
[14:22:54] <Nodex> I've not seen that happen before, perhaps it's a new thing in 2.4.5
[14:25:42] <LoneSoldier728> hey guys, how do I query an array without specifying the value (meaning I want it all)
[14:26:07] <Nodex> throw it the array
[14:26:18] <Nodex> i/e .. foo.find({a:[1,2,3]});
[14:26:54] <LoneSoldier728> 1,2,3? but let's say 1,2,3, is x meaning I do not know how big the length is
[14:27:15] <Nodex> eh?
[14:27:24] <Nodex> you just said you want to query on the entire array
[14:27:24] <pl2> Nodex, yeah it's strange, the first db went to: 2.5gb, the next two > 0.5gb and now another one I just tried stopped at > 0.25gb. I think mongo is trying to tell me something..
[14:27:48] <LoneSoldier728> Ok let me explain the situation im in
[14:28:19] <LoneSoldier728> I am querying a user's id that is stored in the cookie, so I want to find the person's signed in array for friendRequest
[14:28:37] <LoneSoldier728> I want to grab all there friend requests and then display it on their notifications page
[14:28:47] <LoneSoldier728> so I have this code so far
[14:28:54] <E1ven> Is there any place that old versions of the manual (2.0 or 1.8) are posted?
[14:29:03] <LoneSoldier728> User.findById(req.signedCookies.userid, {notifications}
[14:29:22] <LoneSoldier728> but notifications is incorrect as it does not query
[14:29:42] <Nodex> LoneSoldier728 : you will have to post things in Mongodb shell language, I don't know what language that is you're using so I don't know how to read it
[14:29:50] <LoneSoldier728> oh ok sorry
[14:29:57] <LoneSoldier728> Ya I am using mongoose
[14:29:59] <LoneSoldier728> so figure
[14:30:13] <Nodex> are you trying ot get a list of who else is only from an array of user id's ?
[14:30:15] <Nodex> to*
[14:30:29] <LoneSoldier728> right a list of all the user id's in the notification array
[14:30:37] <Nodex> i/e find anyone who is in [1,2,3,4,5] (online)
[14:30:52] <Nodex> you can use $in and throw it an array - it will select any id that's in it
[14:30:53] <LoneSoldier728> yes find anyone in this array notifications[]
[14:31:17] <Nodex> foo.find({online:{$in:[1,2,4,5,6,7]}});
[14:31:27] <Nodex> that will get 1 or 2 or 4 or 6 ......
[14:31:33] <LoneSoldier728> so $in {arrayName:}
[14:31:37] <LoneSoldier728> can I do that
[14:31:46] <LoneSoldier728> or $in {notifications}?
[14:31:51] <Nodex> no it's array name : {$in : Array}
[14:32:06] <Nodex> "online" = the key of the document
[14:32:12] <Nodex> i your case it might be "_id"
[14:32:31] <Nodex> db.users.find({uid:{$in:['list','of','friends']}})
[14:33:05] <LoneSoldier728> ok thanks that makes sense
[14:33:08] <LoneSoldier728> going to try it out now
[14:33:13] <Nodex> no probs
[14:34:18] <quattr8> projecting/expluding fields from a result set performance wise how costly is this operation?
[14:34:37] <LoneSoldier728> it is saying notifications is undefined...
[14:34:39] <LoneSoldier728> db.users.find({_id: {$in: [notifications]}})
[14:34:57] <LoneSoldier728> or does that look wrong?
[14:35:12] <Nodex> notifications has to be an array
[14:35:22] <LoneSoldier728> it is
[14:35:28] <Nodex> [ObjectId("...."), ObjectId("....")]
[14:36:05] <LoneSoldier728> ok notifications is an array that has a field called friendId in it
[14:36:07] <Nodex> you probably want just ntoifications without the []
[14:36:19] <Nodex> *notifications
[14:36:30] <LoneSoldier728> ya not working either
[14:36:37] <pl2> Thanks for the help guys and thanks for not using my login credentials for illicit activity ;)
[14:36:39] <Nodex> you need to pastebin a typical document
[14:36:49] <Nodex> pl2 lol
[14:37:36] <LoneSoldier728> sorry i dont understand pastebin
[14:37:49] <kali> oO
[14:37:57] <LoneSoldier728> like temp storing?
[14:38:12] <Nodex> pstebin.com
[14:38:15] <Nodex> pastebin.com
[14:38:59] <LoneSoldier728> like to share code?
[14:39:10] <LoneSoldier728> is it like gist, jsfiddle, etc?
[14:39:26] <kali> yes.
[14:39:58] <LoneSoldier728> Oh ok, what code do you want me to include it is just a query line?
[14:40:11] <Nodex> a typical document (mwithout sensitive information)
[14:41:48] <LoneSoldier728> http://pastebin.com/BJ0qfSsz
[14:41:58] <LoneSoldier728> ok does that help, I tried to explain the mongoose part
[14:42:15] <Nodex> no.
[14:42:30] <Nodex> can you goto the shell and do db.users.findOne(); <-- pastebin the output
[14:43:41] <LoneSoldier728> oh ok sorry
[14:45:21] <LoneSoldier728> http://pastebin.com/PG9SCz5L
[14:45:36] <LoneSoldier728> sorry let me find a user with the request tho
[14:46:15] <Nodex> ok you're gonna have to somehow loop your "friendRequest" array and pluck out every _id and put it into another array
[14:46:20] <Nodex> that has to be done serverside
[14:46:24] <LoneSoldier728> http://pastebin.com/dRrdEcyM
[14:46:59] <LoneSoldier728> ok so I have to loop it
[14:47:06] <LoneSoldier728> I cannot just pull out the whole list of arrays
[14:47:07] <Nodex> to generate the array yes
[14:47:36] <LoneSoldier728> and once I loop through all the userId's in the array
[14:47:51] <Nodex> make sure they're objectId's then do what you did earlier
[14:47:52] <LoneSoldier728> can I use just the user Id's to pull out each individuals name
[14:48:14] <LoneSoldier728> or do I have to query that separately
[14:48:20] <Nodex> once you have an array that looks somehthing like ... [ObjectId("..."), ObjectId("...")....]
[14:48:24] <Nodex> you can use an $in
[14:48:30] <Nodex> example....
[14:48:53] <Nodex> var friends = [ObjectId("..."), ObjectId("...")....]; db.users.find({_id:{$in:friends}});
[14:50:12] <LoneSoldier728> so I have to stick to first querying just the user logged in then loop through the notifications.receivedRequest array
[14:50:32] <LoneSoldier728> then after the loop, loop through each user in the array
[14:50:38] <Nodex> but you will; have to make friends by looping the friendRequest array from the user's original document... (psuedo) foreach(friendRequests as i) { friends.push(i.friendId)}
[14:53:17] <LoneSoldier728> k thans
[14:53:18] <LoneSoldier728> thanks
[14:54:07] <DanWilson> Any idea why the $eq operator isn't supported in the aggregation framework?
[14:55:05] <MilosStanic> hi guys, is it possible that PHP has trouble with MongoId objects array?
[14:55:17] <MilosStanic> I'm having a weird error on this code: http://pastebin.com/XGwPpWJ6
[14:56:49] <Nodex> pastebin is slooooow today
[14:58:02] <Nodex> MilosStanic : you need foreach($foo as $key=>$val)
[14:58:08] <Nodex> not just $val
[14:59:03] <MilosStanic> Nodex, can you please translate that foreach to my example?
[14:59:49] <Nodex> don't be lazy
[15:00:06] <Nodex> foreach ($haystack as $lazy=> $hay){
[15:00:07] <MilosStanic> foreach ($haystack as $hay=>$idobj) ?
[15:00:14] <Nodex> corret
[15:00:15] <harenson> Nodex: xD
[15:00:15] <Nodex> +c
[15:00:37] <MilosStanic> these two are not same
[15:00:42] <MilosStanic> yours and mine
[15:00:54] <MilosStanic> anyway, I'll test and see, thanks a lot Nodex
[15:00:57] <Nodex> key = 0,1,2,3,4....
[15:01:04] <Nodex> $val = the actual array member
[15:01:11] <Nodex> (in your case $hay)
[15:01:27] <MilosStanic> ok, I see, thanks
[15:01:51] <Nodex> there is a reason you can't just foreach as you were but I can't remember what it is
[15:02:06] <Nodex> something to do with array to itterator iirc
[15:02:10] <Nodex> or vv
[15:03:28] <LoneSoldier728> Nodex:
[15:03:29] <LoneSoldier728> http://pastebin.com/6QaqZVXJ
[15:03:46] <LoneSoldier728> I know it is in mongoose, but was wondering if you understood the js part
[15:04:00] <LoneSoldier728> because I am not getting an error but the array stays empty
[15:04:50] <LoneSoldier728> and userX has the correct info
[15:05:00] <LoneSoldier728> for the user who sent the friend request
[15:05:08] <LoneSoldier728> (I console.log -ed it)
[15:05:09] <Nodex> "notifications" : [ ], <--- empty in your document you pastebinned
[15:05:32] <Nodex> does "notifications" have array members?
[15:06:00] <LoneSoldier728> notifications has two arrays in them (1 is receivedRequest, 2 is newNotifications)
[15:06:50] <LoneSoldier728> http://pastebin.com/vNFFGmDb
[15:06:52] <LoneSoldier728> like so
[15:08:55] <MilosStanic> Nodex: I get the same error, I think PHP doesn't like the array it's getting into foreach
[15:10:24] <Nodex> can you tell me the type (gettype) please
[15:12:20] <LoneSoldier728> who me?
[15:12:49] <Nodex> MilosStanic
[15:13:02] <MilosStanic> ok, will do
[15:13:13] <MilosStanic> gettype of $haystack?
[15:13:45] <Nodex> yes
[15:13:54] <Nodex> LoneSoldier728 : User.findById(user.notifications[i].receivedRequest, function(err, userX){ .... wat is that for?
[15:14:25] <LoneSoldier728> I am querying the id that is in the friends array, so I can get the entire user's info and throw it in the array
[15:14:33] <MilosStanic> nodex, it's "array"
[15:16:01] <LoneSoldier728> basically I am looping through the notifications array, and for each id [i] I am calling the receivedRequest that is nested in notifications then I am querying it by Id that is being given back to me and storing that in userX then pushing userX to the usersRequest array
[15:16:29] <Nodex> why are you querying them - it's a waste of time
[15:16:42] <Nodex> just add the id to an array and use $in at the end
[15:17:04] <Nodex> MilosStanic : can you var_dump($haystack)
[15:17:23] <MilosStanic> sec...
[15:19:11] <MilosStanic> Nodex, here's the vardump: http://pastebin.com/Rk79R8vV
[15:25:14] <LoneSoldier728> I am confused as how to use the $in - I threw all the id's in a usersRequest array
[15:25:25] <LoneSoldier728> find({lastName:{ $in: [usersRequest]}}
[15:25:31] <LoneSoldier728> amd I suppose to query it like that?
[15:25:43] <Derick> are they values of lastName ?
[15:26:23] <spuz> hi, I have an item in a collection with an array field. How do I get the number of items in that array from the mongo shell?
[15:26:24] <LoneSoldier728> no they are ids ok
[15:26:25] <Nodex> LoneSoldier728 : yes, it works the same as SQL IN()
[15:26:28] <benza89> Hi Its is the same for me for the $in operator ,How I am suppose to write it In the C driver
[15:26:31] <benza89> ?
[15:27:19] <LoneSoldier728> find({_id:{ $in: [usersRequest]}}
[15:27:25] <Nodex> correct
[15:27:42] <Derick> what's usersRequest ?
[15:29:15] <LoneSoldier728> it is an array
[15:29:20] <LoneSoldier728> that I pushed all the id's in
[15:29:28] <LoneSoldier728> http://pastebin.com/6QaqZVXJ
[15:29:33] <LoneSoldier728> http://pastebin.com/vNFFGmDb
[15:29:38] <LoneSoldier728> the first one is my node.js code
[15:29:45] <LoneSoldier728> second one is my db results
[15:30:06] <Nodex> so it's working?
[15:30:52] <LoneSoldier728> well I am not getting an error but the page is in forever loading mode
[15:30:59] <rspijker> so, if usersRequest is already an array, why are you writing [usersRequest]?
[15:31:34] <LoneSoldier728> good question, took out the []
[15:31:44] <LoneSoldier728> but still same loading issue where it never loads
[15:32:01] <Nodex> console.log() the length of the array
[15:33:08] <LoneSoldier728> it says 1
[15:33:11] <LoneSoldier728> let me update the pastebin
[15:33:12] <xenophon> I am trying to use pulp, which a program that manages package repositories, and uses mongodb as a backend. However, it has been causing mongodb to segfault. https://bugzilla.redhat.com/show_bug.cgi?id=986070 http://pastebin.com/VgfHZj0q (the file just contains "db.queued_calls.drop();" ). The second link in particular seems to point the finger at mongo
[15:33:27] <Nodex> can you console.log the array and pasatebin the output
[15:33:30] <LoneSoldier728> http://pastebin.com/c2c23f7R
[15:33:41] <LoneSoldier728> and let me put up the new results
[15:34:21] <LoneSoldier728> http://pastebin.com/tWg1SXkZ
[15:34:25] <LoneSoldier728> console.log responses
[15:34:56] <Nodex> err you're pushing the WHOLE user document into the array
[15:35:02] <Nodex> just push the ObjectId
[15:35:13] <LoneSoldier728> I did but I want to get the user's name afterwards
[15:35:22] <LoneSoldier728> and if the id is only in there I cannot pull it out no?
[15:35:29] <Nodex> eh?
[15:35:40] <LoneSoldier728> i tried it and i was not getting the names out
[15:35:41] <Nodex> THE array is what gets queried on
[15:35:44] <LoneSoldier728> with just the ids in it
[15:36:15] <LoneSoldier728> I am doing a second query on the array
[15:36:30] <MilosStanic> Nodex, did you take a look at my vardump? http://pastebin.com/Rk79R8vV
[15:36:35] <LoneSoldier728> and calling each one as userX?
[15:37:53] <LoneSoldier728> nvm got it
[15:38:00] <LoneSoldier728> I had to make an array for the second list?
[15:38:07] <LoneSoldier728> does that make sense or am I doing too much work
[15:38:11] <LoneSoldier728> for one query
[15:38:16] <LoneSoldier728> well technically 2
[15:38:32] <Nodex> MilosStanic : I don't understand it sorry
[15:38:48] <LoneSoldier728> http://pastebin.com/cESEyU2N it works now
[15:39:13] <Nodex> LoneSoldier728 : QUery 1... get your USER who logged in and loop his notifications array to build a list of USERS to query by _id using $in
[15:39:29] <LoneSoldier728> but is it bad the way I am querying the signed in user and throwing the friendRequest list into an array then querying each id in the array and throwing their whole docs in the another array
[15:39:30] <Nodex> Query 2. Execute the ABOVE query on $in and loop those results
[15:39:47] <Nodex> that makes 2 queries
[15:40:01] <LoneSoldier728> yeah ok, so that makes sense the way I did it
[15:40:10] <LoneSoldier728> is it considered scalable?
[15:40:49] <Nodex> if the way you did it was a query FOR EACH user then NO
[15:41:22] <LoneSoldier728> I did it exactly the way you just said it
[15:41:44] <LoneSoldier728> thanks btw
[15:41:47] <Nodex> so you execute a total of 2 queroes?
[15:41:58] <Nodex> 1 for the original user and one that uses $in
[15:42:03] <LoneSoldier728> yep
[15:42:11] <LoneSoldier728> I thought it is considered multiple queries
[15:42:11] <Nodex> then yeh scalable
[15:42:14] <LoneSoldier728> if it loops through it
[15:42:22] <Nodex> no it's one query just like SQL IN()
[15:42:24] <LoneSoldier728> each time it loops is it not a query
[15:42:35] <LoneSoldier728> ok
[15:42:40] <Nodex> the loop on the $in is a cursor not a query
[15:42:49] <LoneSoldier728> k makes sense
[16:09:47] <LoneSoldier728> quick question how do you remove an array from an array
[16:10:09] <LoneSoldier728> db.users.remove({array1: {array2}})
[16:12:38] <rspijker> db.x.remove(query) removes a document which matches the query
[16:12:40] <Nodex> $unset
[16:12:49] <rspijker> if you want to remove a part of a document you need to update it
[16:13:08] <Nodex> db.foo.update({query},{$unset:{field:1}});
[16:13:25] <LoneSoldier728> can I do $addToSet and $unSet at the same time?
[16:13:38] <Nodex> yeh
[16:13:38] <LoneSoldier728> in one query?
[16:14:05] <LoneSoldier728> is a {$un},{$add}
[16:14:09] <LoneSoldier728> is that good enough
[16:14:38] <LoneSoldier728> meaning just a curly brace and a comma separating
[16:14:39] <LoneSoldier728> the 2
[16:14:47] <Nodex> db.foo.update({query},{$set:{},$addToSet:{},$unset{}});
[16:15:05] <Nodex> $unset : {}
[16:16:38] <LoneSoldier728> kk
[16:16:40] <LoneSoldier728> thanks
[16:18:07] <OliverJAsh> i'm using MongoDB in my node application, and i need to write a function that manipulates data when it is received from the database. however, i'm not sure how to design the interface of the function – to receive one object (collection#findOne) or an array of objects (collection#find).
[16:18:56] <OliverJAsh> if i design it to work with arrays, then it will be optimised for that. however if i design it for just receiving an object, it won't be optimized. so, when i do call it with an array, this work it has to do will be slower.
[16:19:14] <rspijker> OliverJAsh: find does not return an array
[16:19:30] <rspijker> it returns a cursor over the results
[16:19:48] <OliverJAsh> the JavaScript adapter i'm using does (it has to i guess)
[16:19:55] <OliverJAsh> for Node, sorry
[16:21:06] <rspijker> that will probably mean the driver 'pre-iterates' the cursor and passes the result on as an array
[16:21:28] <rspijker> I'm not sure what you question is btw :)
[16:31:58] <LoneSoldier728> $addToSet: {friend: {friendId: mongoose.Types.ObjectId(req.body.friend), date_added: new Date()}}, $unset: {notifications: {receivedRequest: mongoose.Types.ObjectId(req.signedCookies.userid), newNotification: 0}}
[16:32:24] <LoneSoldier728> the unset should just be removing the 1 id array and not all the id's in this query right?
[16:39:20] <dougb> if I'm doing a search that looks in field1, field2 and field3. Do I need to index these fields individually and create a combined index with all fields, or just create a combined index with all fields?
[16:39:53] <rspijker> for that search you would just need a compound index
[16:40:46] <dougb> rspijker: ok, so if I have 3 different types of searches, it's more efficient to create 3 compound indexes?
[16:41:16] <rspijker> this all very strongly depends on the scenario dougb
[16:42:37] <dougb> ok
[16:45:29] <rspijker> you should probably read up a little bit on indices and then decide what is best in your case dougb
[16:45:50] <rspijker> it's fairly simple, really
[16:46:38] <dougb> i would think compound indexes would be more efficient, i heard that having too many indexes can slow things down so I don't want to index fields individually when most of my searches require more than one field
[16:47:05] <rspijker> dougb: ordering is important though...
[16:47:23] <rspijker> a compound index on fields a,b and c (in that order
[16:47:32] <rspijker> won't do anything for a query on b and c
[16:48:03] <dougb> yes, i understand, so what I'm planning on doing is setting up two, one for a, b and c and one for b and c
[16:49:17] <rspijker> setting up a lot of indices is really only a problem when you start hitting RAM limits
[16:49:58] <rspijker> there are some tools as well which can analyse your query behaviour and make index recommendations based on that
[16:50:13] <rspijker> dex for instance: http://blog.mongolab.com/2012/06/introducing-dex-the-index-bot/
[16:50:21] <rspijker> might be worth a look :)
[16:50:45] <leifw> dougb: covering indexes are a lot more useful to a query than non-covering indexes
[16:51:08] <LoneSoldier728> hm ok so now my query $unset: {notifications: {receivedRequest: mongoose.Types.ObjectId(req.signedCookies.userid)}}
[16:51:08] <LoneSoldier728> }, function(err) {
[16:51:10] <LoneSoldier728> oops
[16:51:34] <LoneSoldier728> $unset: {notifications: {receivedRequest: mongoose.Types.ObjectId(req.signedCookies.userid)}}
[16:51:52] <LoneSoldier728> is this going to remove everything inside the notifications array?
[16:53:33] <dougb> rspijker: i'm not hitting ram limits, i'm hitting CPU limits
[16:53:53] <rspijker> then more indices are fine
[17:08:38] <LoneSoldier728> anyone know how to unset a sub array without unset - ing the whole array
[17:09:11] <LoneSoldier728> I know $pull does a specific field but I want to get rid of a nested array without unsetting the parent array
[17:17:02] <LoneSoldier728> http://stackoverflow.com/questions/17840723/how-to-unset-a-sub-array-mongodb
[17:17:02] <LoneSoldier728> anyone know the answer to that simple question
[17:19:04] <Derick> notifications: {receivedRequest -> 'notifications.receivedRequest'
[17:20:21] <LoneSoldier728> thanks!
[17:21:20] <Derick> LoneSoldier728: updated it on SO too
[17:22:05] <LoneSoldier728> errors right away?
[17:22:18] <LoneSoldier728> saying the . is unexepted
[17:22:24] <LoneSoldier728> unexpected*
[17:22:48] <LoneSoldier728> $unset: {notifications.receivedRequest: mongoose.Types.ObjectId(req.signedCookies.userid)}
[17:23:15] <Derick> LoneSoldier728: i did add " " around it...
[17:23:17] <LoneSoldier728> Syntax Error on the period - unexpected token
[17:23:19] <Derick> or rather, ' '
[17:23:19] <LoneSoldier728> oh
[17:23:43] <Derick> LoneSoldier728: see my reply on StackOverflow
[17:24:06] <LoneSoldier728> oh ok
[17:24:26] <LoneSoldier728> in mongoose quotes are included
[17:24:31] <Derick> ?
[17:24:34] <LoneSoldier728> I mean I didn't know quotes had to be included
[17:24:45] <LoneSoldier728> when calling it through mongoose but I am not getting the period error anymore
[17:24:46] <Derick> otherwise it's invalid code...
[17:26:04] <Derick> LoneSoldier728: if it works, accept the answer ;-)
[17:26:37] <LoneSoldier728> it doesn't work : /
[17:27:49] <Derick> "it doesn't work" is a bad bug report
[17:27:52] <LoneSoldier728> It doesn't unset anything
[17:27:59] <Derick> show the code again
[17:28:07] <LoneSoldier728> I will paste bin it
[17:28:22] <Derick> also, try it on the shell
[17:28:56] <LoneSoldier728> ok let me see on the shell then right now
[17:34:30] <LoneSoldier728> db.users.findOne({'email':'test12@gmail.com'}, {$unset: {'friendRequest.friend
[17:34:30] <LoneSoldier728> d': "51f006977a48e82412000003"}})
[17:34:48] <LoneSoldier728> db.users.findOne({'email':'test12@gmail.com'}, {$unset: {'friendRequest.friendId': "51f006977a48e82412000003"}})
[17:35:25] <LoneSoldier728> that errors in the shell Unsupported projection option friendRequest.friendId
[17:41:18] <DanWilson> do you need to put a dollar sign on it?
[17:41:28] <DanWilson> db.users.findOne({'email':'test12@gmail.com'}, {$unset: {'$friendRequest.friendId': "51f006977a48e82412000003"}})
[17:42:03] <LoneSoldier728> someone said it is not possible is this true?
[17:42:12] <LoneSoldier728> http://stackoverflow.com/questions/17840723/how-to-unset-a-sub-array-mongodb/17840776?noredirect=1#17840776
[17:42:58] <LoneSoldier728> ya for the $ signn tho
[17:52:04] <LoneSoldier728> is it possible to do a $pull on each sub-array element?
[18:09:36] <hyperb> So I basically want to do a db.coll.find() > export.txt
[18:09:40] <hyperb> Any way to do that?
[18:10:43] <DanWilson> hyperb: like http://docs.mongodb.org/manual/reference/program/mongoexport/ ?
[18:11:50] <hyperb> Looks like the trick.
[18:12:18] <DanWilson> read this : Note Do not use mongoimport and mongoexport for full-scale backups because they may not reliably capture data type information. Use mongodump and mongorestore as described in “Backup Strategies for MongoDB Systems” for this kind of functionality.
[18:12:32] <hyperb> Yeah, I saw that.
[18:12:53] <hyperb> I mainly just want to have a good reference point before I go renaming some fields.
[18:13:06] <hyperb> Not an actual 'backup.'
[18:13:45] <DanWilson> right...
[18:21:38] <LoneSoldier728> http://stackoverflow.com/questions/17840723/how-to-unset-a-sub-array-mongodb/17840776?noredirect=1#17840776
[18:21:43] <LoneSoldier728> damn anyone have a solution for this
[18:22:18] <DanWilson> Can;t you pull out the old array, change it and reset it?
[18:23:29] <LoneSoldier728> how so?
[18:23:47] <LoneSoldier728> how do I pull it out
[18:23:55] <LoneSoldier728> I just switched $unset to $pull
[18:24:00] <LoneSoldier728> and it isnt doing anything
[18:24:12] <DanWilson> retrieve the document using whatever language you use, change it in your language, then save the document again
[18:24:51] <LoneSoldier728> oh like completely update it... isn't that an unscalable approach?
[18:25:18] <LoneSoldier728> if for every request I loop through the user and extract everything minus the request then resave it
[18:25:34] <DanWilson> it's more scalable than a solution that doesn't work
[18:26:05] <LoneSoldier728> understood, but is there really no other approach... I am pretty sure mongodb has some way to ease friendships?
[18:27:35] <LoneSoldier728> there always needs to be one thing screwing up my flow! : /
[18:27:44] <DanWilson> that is why we get paid the big bucks :)
[18:27:54] <DanWilson> if you find a workable solution, post it back here.
[18:28:05] <LoneSoldier728> Will do
[18:28:36] <LoneSoldier728> btw if you want to add me on skype lplior is my name on it
[18:37:11] <hyperb> Great. I think I broke my test mongodb
[18:37:26] <ron> WOO HOO!
[18:37:26] <hyperb> > show collections
[18:37:30] <hyperb> > --NOTHING--
[18:37:37] <ron> maybe there are no collections
[18:37:46] <ron> did you remember to 'use' the proper db?
[18:38:33] <hyperb> whew.
[18:38:52] <hyperb> I hadn't connected to the db after all.
[18:39:01] <ron> well, that was easy.
[18:39:17] <hyperb> I had just done an import without --upsert and got a bunch of duplucate id messages.
[18:39:27] <hyperb> It's a test box, but still...
[18:42:43] <Dreamer3> why does gridio on ruby always think the text i'm reading is US-ASCII?
[18:42:43] <Dreamer3> that doesn't sound right
[18:43:37] <LoneSoldier728> hey anyone here want to take a crack at this, new mongodb gurus here
[18:43:40] <LoneSoldier728> http://stackoverflow.com/questions/17840723/how-to-unset-a-sub-array-mongodb/17840776?noredirect=1#17840776
[18:58:26] <xenophon> I am trying to use pulp, which a program that manages package repositories, and uses mongodb as a backend. However, it has been causing mongodb to segfault. https://bugzilla.redhat.com/show_bug.cgi?id=986070 http://pastebin.com/VgfHZj0q (the file just contains "db.queued_calls.drop();" ). The second link in particular seems to point the finger at mongo
[20:11:41] <xenophon> I am trying to use pulp, which a program that manages package repositories, and uses mongodb as a backend. However, it has been causing mongodb to segfault. https://bugzilla.redhat.com/show_bug.cgi?id=986070 http://pastebin.com/VgfHZj0q (the file just contains "db.queued_calls.drop();" ). The second link in particular seems to point the finger at mongo
[20:15:19] <xenophon> here is the tail end of the mongodb.log, not burried in the bug report: http://pastebin.com/WMsnnTSs
[20:34:11] <LoneSoldier728> I think the solution to my problem is to just change the data type to an object and then nest subarrays in that object
[21:30:09] <czajkowski> aloha
[21:30:17] <czajkowski> Derick: you about ?
[21:38:30] <bjori> I bet he is sleeping
[21:38:40] <bjori> czajkowski: anything I can help you with?
[21:41:06] <czajkowski> bjori: ah it's ok I can ping in the morning cheers
[21:58:13] <caitp> okay, I'm not quite sure how to write this query (this seems to happen a lot): I can write a date filter fairly simply, but occasionally I need to filter not by a complete date, but only by the time portion of it
[21:58:25] <caitp> and I'm not sure how that would look
[22:00:17] <LoneSoldier728> Damn cannot get this to work still
[22:00:41] <LoneSoldier728> Anyone know how to remove an array nested down one
[22:01:31] <caitp> $pull maybe, but i'm not sure i parsed the question correctly
[22:01:50] <LoneSoldier728> http://stackoverflow.com/questions/17840723/how-to-unset-a-sub-array-mongodb/17840776?noredirect=1#17840776
[22:02:02] <LoneSoldier728> that is the problem I am having
[22:03:27] <caitp> have you tried the suggested solution?
[22:07:33] <caitp> sigh :( i just want to compare the time components of timestamps ;-;
[22:09:39] <LoneSoldier728> yeah
[22:09:58] <LoneSoldier728> does not work
[22:10:20] <LoneSoldier728> I am trying to avoid writing a loop though as it is very unscalable
[22:15:15] <caitp> maybe I should just store date-less times and time-less dates as indexed auxilliary fields for better performance
[22:15:18] <caitp> but that sucks
[22:15:19] <caitp> :u
[22:15:46] <caitp> or does it, maybe it's not so bad
[22:15:54] <caitp> mm
[22:17:14] <caitp> how small are dates in the mongo backend anyways
[22:17:19] <caitp> 32bits?
[22:17:27] <caitp> 64?
[22:17:41] <ranman> caitp: http://bsonspec.org/#/specification
[22:17:44] <ranman> 64
[22:18:23] <caitp> hmm, so almost 400 bytes per record just storing dates, if I were to do it that way
[22:18:45] <caitp> but maybe that's not such a big deal
[22:18:45] <xenophon> I am trying to use pulp, which a program that manages package repositories, and uses mongodb as a backend. However, it has been causing mongodb to segfault. https://bugzilla.redhat.com/show_bug.cgi?id=986070 http://pastebin.com/VgfHZj0q (the file just contains "db.queued_calls.drop();" ). The second link in particular seems to point the finger at mongo
[22:18:47] <xenophon> here is the tail end of the mongodb.log, not burried in the bug report: http://pastebin.com/WMsnnTSs
[22:19:53] <xenophon> I want to emphasise that API access is causing mongodb to SEGFAULT
[22:20:24] <caitp> that's always a good sign
[22:27:21] <caitp> mongo folks: does that even make sense? for a web app running on cheap cloud hosting, could I get away with storing 256 bytes of dates just so that I could index searches on the time components?
[22:27:48] <caitp> I am not really sure what my disk/memory constraints are, but I can't imagine they're very big
[22:34:11] <cheeser> what's the problem exactly?
[22:36:03] <caitp> I need to occasionally query for documents where a time value fits within a given range (eg, any date between the hours of 6 and 12)
[22:36:40] <caitp> I'm not sure how I could do that efficiently without having an auxilliary field containing only the date component
[22:36:45] <caitp> er
[22:36:46] <caitp> the time component
[22:38:21] <caitp> but, if there is an efficient way to do that without special "please don't overwrite without permission" fields, then I'd love to hear about it
[22:38:32] <cheeser> what's the problem with it?
[22:40:42] <caitp> are you asking what the problem is with querying components of a date?
[22:41:44] <cheeser> i'm asking what problem you're trying to deal with
[22:42:17] <caitp> requirement to filter events based on /time/, not including /date/
[22:42:29] <cheeser> oh, i see.
[22:43:25] <cheeser> yeah, i don't think you can do that on a date in mongo. there isn't, e.g., an hour() function i'm aware of
[22:43:52] <caitp> hmm, seems like a patch is in order:p
[22:43:55] <caitp> but I guess for now aux fields are ok
[22:44:22] <cheeser> there is in the aggregation framework, though.
[22:44:24] <cheeser> http://docs.mongodb.org/manual/reference/aggregation/operator-date/
[22:46:11] <LoneSoldier728> can we talk about mongoose here?
[22:46:25] <LoneSoldier728> I am trying to figure out how to set up a new schema possibly for just friendships
[22:46:46] <caitp> what do you mean by friendships?
[22:46:51] <caitp> i'm kind of reading "bridge table" from that
[22:46:56] <caitp> which you shouldn't need
[22:48:38] <caitp> if you're talking about friendship in like a social media sort of way, would it not make sense to have an array of subdocuments per user, with the identities of their friends, and maybe some other values too?
[22:49:33] <cheeser> hrm. did that last bit about a marathon post here, too? my client is being a bit wonky.
[22:51:19] <caitp> no, not here
[22:51:54] <cheeser> great
[22:52:07] <Argorok> Hello. Im new to MongoDB and NoSQL databases... When I upsert a document, there's a way to work with "PREVIOUS VALUES" and "NEW VALUES" like I do in TRIGGERS on other databases? For example: I have a field called: times updated... I want to everytime that I update this field, set this field to FIELD VALUE + 1
[22:53:29] <cheeser> not yet, no
[22:54:06] <Argorok> cheeser: so the only way to do it is do a manual find and then update it manually too? (Instead of use upsert)
[22:54:15] <cheeser> yeah
[22:54:22] <LoneSoldier728> yeah caitp I have done that but that causes a lot of problems
[22:54:32] <LoneSoldier728> because of the nested array
[22:54:44] <LoneSoldier728> and the lack of ability to use the positional operator with mongoose
[22:55:10] <LoneSoldier728> so I am thinking of starting a new schema that has the userID in it as well
[22:55:22] <LoneSoldier728> and then everything pertaining to notifications and friendships
[22:55:24] <caitp> why would you need positional operators?
[22:55:33] <LoneSoldier728> to select a sub document for removal
[22:55:45] <caitp> you wouldn't use a positional operator for that
[22:55:52] <caitp> you'd use a unique field within the subdocument
[22:56:00] <LoneSoldier728> http://stackoverflow.com/questions/17840723/how-to-unset-a-sub-array-mongodb
[22:56:05] <LoneSoldier728> that is the problem I was having
[22:56:17] <LoneSoldier728> how would I remove the element...
[22:56:23] <LoneSoldier728> without a positional operator
[22:57:04] <Argorok> cheeser: thank you, then
[22:58:02] <LoneSoldier728> side note, how do I set a collection to a mongoose schema? I have no idea where the first one is being created from
[22:58:13] <LoneSoldier728> and for some reason it uses the users collection
[22:59:50] <caitp> you mean how do you set a schema to a particular collection?
[23:00:03] <maslowbeer> hello. having issues with my mongo cluster config and php. When i get the exception Fatal error: Uncaught exception 'MongoCursorException' with message '10.176.97.204:27017: not master and slaveOk=false' - is that a mongo server config issue (slaveOK) or something in the php connection/string/driver that is having the issue?
[23:00:04] <caitp> you can do it with a parameter of the Schema constructor
[23:00:10] <LoneSoldier728> yes
[23:00:34] <LoneSoldier728> how so?
[23:00:41] <LoneSoldier728> Mongoose#Collection()
[23:01:05] <LoneSoldier728> I see this but I dont get where that gets thrown on and the documentation doesnt show anything
[23:02:21] <caitp> Schema({fields....}, {collection: "collectionstring"})
[23:02:51] <LoneSoldier728> ah ok
[23:02:55] <LoneSoldier728> is it better to do that or this
[23:02:57] <caitp> https://github.com/LearnBoost/mongoose/blob/3.6.15/lib/schema.js#L26 all of the schema options shown there
[23:02:59] <LoneSoldier728> mongoose.model('UserInfo', UserInfo, 'UserInfo');
[23:03:03] <LoneSoldier728> the third
[23:03:05] <caitp> I don't think it really matters
[23:03:06] <LoneSoldier728> being the collection name
[23:03:08] <LoneSoldier728> ok
[23:03:27] <LoneSoldier728> but you know what I am confused about, how is it pointing to the collection users right now in my db
[23:03:34] <LoneSoldier728> or is it pointing to that because it is the only collection
[23:03:43] <caitp> if you don't specify a collection, it tries to figure it out for itself
[23:04:00] <caitp> similar to what that rails thing does
[23:04:10] <LoneSoldier728> {collection: "users"});
[23:04:18] <LoneSoldier728> this should point to users collection correct?
[23:04:31] <caitp> if you have a schema/model called User, I think it will default to collection "users"
[23:04:42] <caitp> but i'm not sure if it pluralizes on its own
[23:04:45] <caitp> can't remember ^^
[23:04:53] <LoneSoldier728> ok but is this correct {collection: "users"});
[23:04:57] <LoneSoldier728> for pointing to users
[23:05:00] <caitp> yes
[23:05:04] <LoneSoldier728> kk thanks alot
[23:05:25] <LoneSoldier728> Now you don't think I should make a new schema for friendRequest/notifications
[23:05:30] <LoneSoldier728> pointing to another collection?
[23:05:39] <LoneSoldier728> and including the userid?
[23:05:41] <caitp> I don't think it necessarily makes sense
[23:06:00] <LoneSoldier728> you saw the stackoverflow problem I was having
[23:06:12] <caitp> i mean, it's a user has many, friendship has one
[23:06:15] <caitp> kind of thing
[23:06:29] <caitp> so it makes sense to model it as an array of subdocuments, i think
[23:06:44] <LoneSoldier728> right the only problem is and I have been reading everywhere
[23:07:01] <LoneSoldier728> is when you have to remove an array's sub documented array
[23:07:12] <LoneSoldier728> it becomes a problem
[23:36:55] <LoneSoldier728> anyone know how to call a userId from another schema
[23:37:00] <LoneSoldier728> to a new schema
[23:37:21] <LoneSoldier728> te userId is the objectId for each user and is created on mongo's end
[23:37:26] <LoneSoldier728> (this is for mongoose)
[23:53:06] <caitp> http://i.imgur.com/CiWWD29.jpg how do i data
[23:58:59] <LoneSoldier728> http://stackoverflow.com/questions/17846822/how-to-get-objectid-from-another-mongoose-schema
[23:59:03] <LoneSoldier728> anyone able to answer that