PMXBOT Log file Viewer

Help | Karma | Search:

#mongodb logs for Wednesday the 24th of October, 2012

(Back to #mongodb overview) (Back to channel listing) (Animate logs)
[01:43:52] <Nerp> do you guys happen to know the projected final release of 2.2.1?
[03:18:18] <zivester> Hi, I have a "Product" that has many "Attributes" .. what's the correct way to search products that have a specific attribute, lets say keyed by "name".. I currently only have products reference attributes
[03:40:42] <IAD> zivester: .find({'key_name' : {'$exists' : true} }
[03:44:30] <zivester> err.. I'm trying to find all products that have an attribute 'shoes' ... my Product has: attributes : [ { type : Schema.Types.ObjectId, ref: 'Attribute' } ]... I'm guessing this has to be a many to many relationship and I have to store Products on attributes as well?
[03:46:43] <IAD> zivester: pastebin please ".[your_collection].findone().pretty()"
[03:49:43] <IAD> * without pretty()
[03:50:03] <zivester> IAD: http://pastie.org/5106967
[03:53:48] <zivester> just reading about embedding data, or linking (which I assume is what I'm doing in that version)... I'm guessing if I want to search by attribute.. I either need to embed all attributes directly into the product... or reference all products from attributes and perform the find() on those instead
[03:55:37] <IAD> zivester: at first you can find _id of db.attributes : $a = db.attributes.findone('name' : 'shoues') . and at second $products = db.products.find('attributes' : $a['_id'])
[03:56:12] <IAD> or yes, you can store attributes inside of products. it's also good way
[03:59:21] <IAD> it will be difficult to change an attribute name (in two collections) but more faster =)
[04:03:47] <zivester> thanks IAD.. now trying to figure out why my successive saves are creating new attributes even if I supply the same name :)
[06:22:38] <andoriyu> is it possible to put $limit inside of $group?
[07:45:23] <[AD]Turbo> hi there
[07:48:52] <|RicharD|> hello
[07:49:15] <|RicharD|> i don't know if my solution at my problem is mongodb/nosql
[07:51:26] <ron> well, we don't know either.
[07:51:47] <ron> of course, you can share your use case, and then we may be able to assist, but otherwise, we can just guess.
[07:54:52] <|RicharD|> sorry i was on phone :(
[07:54:55] <|RicharD|> anyway yes
[07:55:16] <|RicharD|> i want a table where i don't know what column i need�example: table: objects
[07:55:36] <|RicharD|> i insert a pen with attributes: name,desc,color
[07:55:51] <|RicharD|> and then i insert in the same table a car with attribtu
[07:56:11] <|RicharD|> attributes: name,desc,weight,height,color,speed_max
[07:57:08] <|RicharD|> in myswl/postgresql i should use Open Schema, but maybe is better use a nosql db like mongodb ?
[07:58:07] <ron> well, yeah, a schemaless db would be easier
[07:58:26] <ron> however, don't think about 'how' you want to store. the real question is what queries you want to run on your data.
[07:59:05] <NodeX> +1
[07:59:10] <|RicharD|> why ?
[07:59:41] <IAD> I spent a lot of time to figure it out =)
[07:59:53] <ron> because storing the data is nice, but you don't store it for the sake of storing
[08:00:03] <ron> you want your queries to be fast (mostly)
[08:00:11] <ron> it all depends on your usage and your needs
[08:01:15] <NodeX> the first question most "nosql-ers" ask themselves is howcan I query my data in the fastest and most efficient manner
[08:01:25] <NodeX> how can*
[08:02:13] <|RicharD|> umh
[08:02:23] <|RicharD|> will be a search engine
[08:02:52] <|RicharD|> so i put some values, and the data in db that match my values are good for search engine
[08:03:15] <NodeX> define search engine
[08:03:30] <NodeX> are you hoping for mysql style "LIKE" queries?
[08:05:58] <|RicharD|> no
[08:06:05] <|RicharD|> is something like:
[08:06:16] <|RicharD|> color = my value, speed=my value
[08:06:58] <NodeX> ok cool
[08:08:51] <|RicharD|> is
[08:08:56] <|RicharD|> good nosql for me ?
[08:09:57] <NodeX> it will suit your need yes
[08:10:45] <|RicharD|> i know that i don't should ask...
[08:10:54] <|RicharD|> mongodb is good or better other nosql ?
[08:11:05] <LesTR> :D
[08:11:20] <|RicharD|> :D
[08:11:51] <LesTR> its question like "its better audi or bmw"
[08:12:06] <IAD> it's different =)
[08:13:41] <|RicharD|> audi anyway thx :P
[08:13:53] <|RicharD|> someone use mongo with ruby ?
[08:14:09] <|RicharD|> i tried mongoid, but it want a scheme definite in model
[08:17:16] <NodeX> so you want an open schema then to lock it down?
[08:19:43] <|RicharD|> yes
[08:20:12] <NodeX> :/
[08:20:22] <IAD> because there is no need to create columns?
[08:20:38] <|RicharD|> exact :| the columns
[08:20:40] <|RicharD|> are dynamic
[08:21:00] <|RicharD|> the schemes can change day to day
[08:21:10] <NodeX> so why do you want to lock it down then>
[08:21:11] <NodeX> ?
[08:21:32] <|RicharD|> because a user can add a new column
[08:21:39] <|RicharD|> a new "value"
[08:23:34] <shingara> Hi
[08:23:48] <shingara> There are some plan about a 3.0 mongodb version or not ?
[08:24:23] <shingara> I never heard about it. The next will be the 2.4 but I don't know if there are some plan about 3.0 version on MongoDB :)
[08:24:28] <NodeX> you do understand that locking it down means "fixing the schema"
[08:24:44] <NodeX> shingara : LOl, are you serious
[08:24:45] <NodeX> ?
[08:25:36] <IAD> shingara: we are thinking about mongodb 4.0
[08:25:41] <|RicharD|> no
[08:25:45] <|RicharD|> i don't know
[08:33:51] <shingara> NodeX: Yes
[08:34:26] <shingara> it's someone ask me if there are some plan a major version
[08:34:40] <shingara> I just check if it's plan or not, not I want it :)
[08:36:22] <kali> shingara: i must say it's hard from a technical / feature / operations pov to make the difference between minor and major version of mongo
[08:36:31] <kali> shingara: 2.2 brought more stuff than 2.0 for instance
[08:36:44] <kali> shingara: so i honestly think this is mostly communication stuff
[08:38:01] <kali> shingara: same, in the 1.x era, there was a version bump (1.2 ? 1.4 ?) that required a data migration... something you usually expect for major version only
[08:38:58] <shingara> kali: agree with you but it's not to a technical person my answer. so only the number count :)
[08:39:04] <shingara> thanks for you reply :)
[09:59:34] <Gargoyle_> If I am iterrating over a cursor and need to update every document, is there a recommended way?
[10:00:34] <NodeX> dont make a new connection each time
[10:00:40] <vnico> hi there, how can I increase the RAM usage in MapReduce operations to reduce hard disk usage?
[10:00:51] <Gargoyle_> I've noticed in the past, that changing the data while looping over the cursor will cause it to do more loops than items! It this OK or should I be doing something else? like writing somewhere else?
[10:00:53] <ron> isn't that automatic?
[10:01:30] <NodeX> Gargoyle_ : if you want speed and your update is not index bound then create an index first then update
[10:01:33] <NodeX> then drop the idnex
[10:01:36] <NodeX> index*
[10:01:44] <Gargoyle_> NodeX: Not worried about speed.
[10:02:07] <NodeX> then a simple set, update on each itteration will be fine
[10:02:32] <Gargoyle_> Just that say I have 10,000 documents, and I loop over the cursor with a foreach and modify and save() them, then the loop will often run well over 10,000 times.
[10:03:08] <NodeX> I dont see who it can, if it's fed 10k items it can only foreach() 10k times
[10:03:20] <Gargoyle_> NodeX: nope!
[10:03:33] <NodeX> who -> how *
[10:03:59] <NodeX> I would say that's a code problem then because I have never had that happen to me lol
[10:08:08] <idank> when doing a find({..}).count(), does it matter if I filter out some columns or is mongo smart enough to see there's a count() there?
[10:08:41] <ron> I'm fairly certain it's sequential.
[10:09:15] <idank> so if my documents are big I should filter them to only fetch _id?
[10:09:43] <ron> I'm not entirely sure, but you can run a simple benchmark to test it out.
[10:11:23] <idank> my collection is ~650gb in size, with 700k rows where one of the fields is an array
[10:11:47] <ron> my comment stands :)
[10:11:58] <idank> when I try find({arr : {$size:1}}).count() to get the number of documents where the array has a single element
[10:12:04] <idank> it runs for hours
[10:12:15] <idank> does that make sense?
[10:12:28] <ron> if there's no relevant index, sure.
[10:12:47] <idank> why? even if it does a table scan
[10:12:55] <idank> how can it take _hours_?
[10:14:02] <idank> k, I should probably add a size field and index it
[10:14:06] <idank> like the docs suggest
[10:14:20] <ron> nah, never trust the docs. they lie.
[10:14:33] <idank> ? :)
[10:14:45] <ron> I was being sarcastic.
[10:14:55] <idank> thought so ;)
[10:15:03] <ron> you good, you.
[10:16:58] <NodeX> count() with find() and a query is slow
[10:17:36] <NodeX> db.foo.find({bar:1}).count() = slow .... db.foo.count() = fast
[10:17:55] <NodeX> count() with no query params use mongodb's internal counters
[11:27:09] <Gargoyle> NodeX: You still around?
[11:31:48] <NodeX> yeh
[11:32:09] <Gargoyle> A basic update loop:- https://gist.github.com/3945549
[11:32:52] <Gargoyle> Don't worry about Place_handler, it just allows static config of the db connection details.
[11:33:15] <Gargoyle> Output from that function:- "Processed 568 of 516 places."
[11:34:30] <NodeX> and the count is definately right?
[11:35:06] <Gargoyle> yup. The collection only has 516 items.
[11:35:29] <NodeX> 1 second let me build a quick test
[11:35:40] <Gargoyle> If I omit only the line that calls save(), then the loop runs 516 times.
[11:36:15] <Gargoyle> I've built a lot of these scripts over the last few months - and while its a little un-nerving. They all seem to produce the expected results.
[11:36:21] <NodeX> is there a difference if you call $unset instead of save?
[11:37:39] <NodeX> All I can think is that either an APC cache (if you're running one) is messing up $count or the variable is being overwritten globaly
[11:37:52] <Gargoyle> In this case, yes. It only loops the expected number of times. But IIRC previous scripts that have used $set showed the same behaviour (I may have used save() calls though - cant quite remember)
[11:38:25] <Gargoyle> I don't think I have APC enabled on this version.
[11:39:35] <Gargoyle> I have come across PHP "funny business" when updating an array that you are looping over - but the cursor isn't actually an array is it?
[11:44:36] <Gargoyle> NodeX: Running the script again (With no more entries requiring the unset) does not do any extra loops.
[11:44:52] <NodeX> https://gist.github.com/3945637
[11:45:53] <NodeX> I thought save() was just a helper method to update()
[11:48:07] <NodeX> I also get the same number from an upsert
[11:48:16] <NodeX> which is close to a save()
[11:49:12] <Gargoyle> I have a bit more to add to my script, and then I need to remove the flags subdoc. This time I am going to output the count and the id being saved for each iteration
[11:49:56] <NodeX> I get also expected counts for an upsert
[11:50:06] <NodeX> let me add save() to my wrapper a minute and test
[11:52:13] <NodeX> I get the same expected results with save();
[11:53:06] <NodeX> https://gist.github.com/3945669 <--- yields 10k updates and 10k itterations
[11:54:33] <Gargoyle> now I am only getting the expected number of loops!
[11:54:39] <NodeX> LOLOL
[11:55:07] <Gargoyle> I will eventually restore my db and retest my scripts - so I'll keep the extra output for now.
[12:14:59] <nopz> Is it possible to have a condition when using $group in the aggregation framework ?
[12:15:24] <nopz> db.docs.aggregate({$group: {_id: "$doc.has_attr", total: {$sum:1}}}) filtered by "doc.date": null
[12:18:24] <nopz> something like db.docs.find({'doc.date': null}, {'doc.has_attr': true})
[12:19:30] <Gargoyle> nopz: Don't you need to have a match condition or something?
[12:20:14] <nopz> Yeah I need to match only documents with date: null
[12:20:20] <NodeX> you need a $match to do that
[12:20:46] <idank> is it possible to inspect a querys progress?
[12:21:05] <idank> I have a count query that is running for >2 hours already
[12:21:07] <kali> idank: look for db.currentOp() in the shell
[12:21:18] <nopz> Ho yes ! thank you!
[12:21:31] <kali> idank: but not all request are showing progress information
[12:22:11] <idank> kali: I can see the query in there but yeah looks like there's no progress meter
[12:23:31] <idank> I'm running a .count({arr:{$size:1}}) query on a collection with 700k documents, where arr max size is 50
[12:23:43] <idank> the entire db is 650gb in size
[12:24:03] <idank> why is it taking hours to do this query?
[12:32:23] <NodeX> idank : I already explained that
[12:32:35] <NodeX> count() with find() and a query is not efficient
[12:32:49] <idank> I saw what you wrote
[12:32:52] <idank> I took the .find off
[12:33:03] <idank> now I only have count with the selector
[12:33:29] <overflow> what are the toughest problems in databasing
[12:34:55] <NodeX> you're missing the point
[12:35:03] <NodeX> a query + count is not efficient
[12:35:51] <idank> so count is only good if it's directly on a collection without a query?
[12:36:58] <NodeX> correct
[12:37:26] <idank> what if I have an index on the query?
[12:40:10] <NodeX> doesn't mtter
[12:40:12] <NodeX> matter*
[12:40:18] <NodeX> it's still not efficient
[12:41:15] <idank> so basically I should add a size field to each document?
[12:52:12] <NodeX> it wont help you count all docs with Size=1 for example
[12:52:16] <NodeX> it will always take a long time
[12:53:40] <idank> all I really want is to sum the lengths of the array in all the documents
[12:53:53] <idank> surely that shouldn't be as hard as it seems
[12:54:00] <idank> db.scraper.find().limit(50000).forEach(function(r) { sum += r.resources.length })
[12:54:31] <idank> after about 5m of mongoshell thinking about this line I'm getting a "Killed"
[13:08:58] <MongoDBIdiot> hi boys
[13:09:24] <Gargoyle> Hi idiot!
[13:09:30] <MongoDBIdiot> be my guest
[13:12:20] <idank> NodeX: any idea why that query is getting killed?
[13:21:08] <sirious> if i have documents which contain a key which is a list of dictionaries, how do i query for the existence of a specific dictionary in that list based on multiple k/v pairs in that one dictionary?
[13:22:31] <sirious> basically want to do db.a.find({'a': {'a': 'foo', 'b': 'bar'}})
[13:24:40] <kali> sirious: you're looking for $elemMatch
[13:25:14] <sirious> kali: you're a star :)
[13:25:18] <sirious> thanks!
[13:27:06] <NodeX> idank : possible ran out of memory
[13:28:38] <sirious> kali: can you use $elemMatch for determining which dictionary in an array to $pull?
[13:28:53] <kali> sirious: yes, that should work
[13:28:57] <sirious> sweet
[13:29:31] <idank> NodeX: it keeps the entire result set in memory?
[13:29:40] <idank> even though I'm using forEach?
[13:33:59] <NodeX> yeh, there is nowhere else for it togo
[13:34:19] <NodeX> sum += r.resources.length <--- that part is definately in memory
[13:34:38] <sirious> $elemMatch requires the keys in the dictionary to be in the same order as how they are stored in the db???
[13:34:42] <NodeX> you might be able to do this in aggrgation idank
[13:34:57] <kali> sirious: nope
[13:35:27] <idank> NodeX: that part sure but then it throws it away
[13:36:06] <sirious> kali: cool. someone mentioned it and i flipped since python doesn't enforce order in dictionaries
[13:36:29] <kali> sirious: don't listen to what people say
[13:37:05] <idank> is .skip(n) complexity O(n)?
[13:37:11] <kali> idank: it is
[13:37:17] <idank> ouch
[13:37:43] <idank> then what's the correct way to get a 'sample' of documents in a collection?
[13:38:33] <kali> add a random float between 0 and 1 as a field and index it
[13:38:42] <kali> then query for $gt values, with limit(1)
[13:38:50] <kali> for instance
[13:39:07] <idank> do I have to touch my data?
[13:39:11] <idank> I'd rather not change it
[13:39:46] <kali> idank: it's often you have to touch the data when you want a new way to query it
[13:40:15] <idank> perhaps, but this is a one time query to gather stats on my data
[13:40:27] <idank> I gave up on running it on all of it
[13:40:37] <idank> and now even on a sample seems hard
[13:57:46] <Freso> Is there a comparison somewhere of MongoAlchemy vs. MongoKit vs. "raw" pyMongo (vs. other drivers?) somewhere? I'm just starting out with Mongo, so just trying to find my way around. :)
[13:59:19] <MongoDBIdiot> http://lmgtfy.com/?q=mongokit+mongoalchemy
[14:04:41] <NodeX> ah, MongoDBIdiot you're back
[14:04:50] <MongoDBIdiot> my dear!
[14:04:56] <NodeX> are you ready to tell me why you're so angry yet?
[14:05:57] <MongoDBIdiot> i only talk to talented people, not to you
[14:05:59] <MongoDBIdiot> *g*
[14:24:19] <NodeX> it seems you talk to yourself and you have no talent
[14:26:11] <MongoDBIdiot> who let you out of the slurry?
[14:32:22] <NodeX> the slurry master
[14:52:18] <MongoDBIdiot> is it always so dead hear?
[14:52:32] <MongoDBIdiot> it smells like here
[14:52:48] <MongoDBIdiot> it smells like dead IT shit here
[14:53:16] <ron> like many professional irc channel, it's active when needed.
[14:53:37] <NodeX> or full of trolls with nothing better to do
[14:54:39] <NodeX> you work with vdeo right ron?
[14:54:45] <NodeX> or your team does?
[14:54:48] <NodeX> video*
[14:55:12] <MongoDBIdiot> mongodb idiot get IT jobs?
[14:55:13] <MongoDBIdiot> how come?
[14:59:09] <ron> NodeX: sortof.. not quite.
[14:59:36] <NodeX> live streaming / rtmp?
[15:00:12] <ron> we don't develop that part, but yes.
[15:14:01] <rio{> I get a duplicate key error on a key that doesn't exist in the collection (it used too) how can I fix this?
[15:14:37] <rio{> I get it every time i try to add something since its null i presume
[15:16:16] <MongoDBIdiot> if you get an error then it does exist
[15:16:39] <rio{> I fixed it by dropping the collection all together
[15:16:48] <rio{> :)
[15:17:44] <rio{> which is fine since I don't have any data at this stage. But is there a better way to do this if I would find myself in that position again. Of course you shouln't get there but if.
[15:22:39] <NodeX> fora unique index you have to make sure it's unique
[15:22:45] <NodeX> for a *
[15:23:06] <NodeX> i/e if there is a null value that's seen as "null is now a taken key so no more are allowed with null"
[15:27:59] <ron> NodeX: needed anything specific re streaming?
[15:28:43] <NodeX> I was wondering if you guys streamed live stuff and if so if you recorded the streams for archive and if so what was used
[15:30:36] <ron> NodeX: right now we don't offer catch up tv, though we will in the future.
[15:49:09] <MongoDBIdiot> back again
[15:50:58] <solussd> is it possible to, without actually updating in the database, see the result of an update (so I can validate the document) and then commit the update?
[15:51:48] <MongoDBIdiot> no transactions, no commits
[15:51:56] <MongoDBIdiot> no rollback
[15:53:25] <solussd> no speculative updates? :)
[15:53:49] <MongoDBIdiot> NO NO NO
[15:54:04] <MongoDBIdiot> only database idiots like you can ask such a stupid question
[15:54:34] <IAD1> solussd: you can mark first document as "busy" and do what you want
[15:55:32] <solussd> MongoDBIdiot: please don't attempt to answer my questions, you lack the social skills necessary to actually be helpful.
[15:55:36] <solussd> IAD1: thanks
[15:55:50] <MongoDBIdiot> you lack every kind of database skills
[15:56:16] <MongoDBIdiot> like all MongoDB so-called "developers"
[15:56:27] <solussd> i think you're just going to turn people off to mongodb with that attitude. :D
[15:56:42] <MongoDBIdiot> the worst of IT can be found in the MongoDB world :-= LOL
[15:56:54] <algernon> I liked our past trolls better.
[15:57:35] <IAD1> MongoDBIdiot: you mean that? http://lenta.iadlab.ru/wp-content/uploads/2012/10/4.jpg
[16:02:24] <NodeX> solussd : you can roll your own transactions but you'll need an MQ in the middle
[16:03:03] <MongoDBIdiot> MQ as a workaround for crap database technology...perfect
[16:03:16] <NodeX> was I talking to you?
[16:03:19] <MongoDBIdiot> and you wrap another layer of duct tape around your app in order to make it work
[16:03:45] <NodeX> do I care about your lack of knowledge with all things IT related?
[16:03:46] <MongoDBIdiot> mongodb, the database from idiots for idiots
[16:04:00] <wiseguysonly> MongoDBIdiot: try that duct tape on your mitss - then you can't type
[16:04:01] <NodeX> how has this guy not been banned yet?
[16:04:04] <MongoDBIdiot> could someone please kick me?
[16:04:10] <wiseguysonly> He's fun. Wait
[16:04:14] <MongoDBIdiot> you should be kicked :)
[16:04:18] <solussd> I'm loving mongodb- makes so much more sense for 99% of what I used to use SQL DBs for. :)
[16:04:31] <MongoDBIdiot> mongodb is like sqlite…a database for jerks
[16:04:40] <IAD1> =)
[16:04:40] <wiseguysonly> MongoDBIdiot: you know that the nick MongoDBNob is available
[16:04:46] <wiseguysonly> you should upgrade
[16:05:15] <MongoDBIdiot> real developers use a rdbms
[16:05:24] <MongoDBIdiot> you and your webscale scrap
[16:05:38] <MongoDBIdiot> 10gen laughs at you idiots using their smelly crap
[16:05:50] <MongoDBIdiot> you test their software crap and they make the money :)
[16:05:52] <wiseguysonly> MongoDBIdiot: I think your mum just called - bedtime
[16:05:53] <MongoDBIdiot> that's how it works :)
[16:06:39] <MongoDBIdiot> mum gave me an extra hour
[16:06:50] <wiseguysonly> I wouldn't stay up. I will make you cry
[16:06:57] <MongoDBIdiot> make me cry
[16:07:26] <wiseguysonly> do you have a mirror close by
[16:08:06] <MongoDBIdiot> it broke
[16:08:10] <NodeX> wiseguysonly : the guy has mental issues, he lives to try and annoy people
[16:08:23] <NodeX> he's been banned once, now he is ban evading
[16:09:03] <wiseguysonly> ahhh - makes sense. well, where I come from we don't call them trolls - we call them nobbers
[16:09:48] <NodeX> I feel sorry for him, no friends or no girlfriend to talk to
[16:10:00] <NodeX> so he trolls the internet being annoying
[16:10:35] <wiseguysonly> true. I guess it stops him thinking about how futile it all is and doing something a bit rash
[16:11:25] <MongoDBIdiot> let's assume i could be gay or bisexual or asexual
[16:11:39] <wiseguysonly> or a nob
[16:11:39] <MongoDBIdiot> hm……good question
[16:11:48] <MongoDBIdiot> nob is a sexual orientation?
[16:12:46] <wiseguysonly> MongoDBIdiot: name your goal in life
[16:13:00] <MongoDBIdiot> you're my therapist?
[16:13:28] <wiseguysonly> your goal is for me to be your therapist?
[16:14:57] <NodeX> lol
[16:15:04] <NodeX> aim a little higher MongoDBIdiot
[16:15:52] <MongoDBIdiot> you're just a bare incompetent idiot
[16:15:58] <NodeX> I know thta
[16:16:02] <NodeX> that*
[16:16:14] <NodeX> but you want to be a therapist and you're in a database help chan
[16:16:16] <MongoDBIdiot> then shut up bitch :-)
[16:16:22] <NodeX> does that not compute?
[16:16:43] <MongoDBIdiot> 42
[16:16:47] <NodeX> perhaps jimmy savville got to you also?
[16:16:52] <algernon> NodeX: we're failing so badly at our job, that we need therapists to cope with that amount of fail.
[16:17:03] <algernon> NodeX: it is therefore safe to assume that we have an online therapist on the channel
[16:17:06] <NodeX> algernon : +1
[16:17:45] <wiseguysonly> haha NodeX - kudos on the JS reference!
[16:17:57] <NodeX> ;)
[16:17:58] <wiseguysonly> Now then, Now then MongoDBIdiot
[16:18:14] <MongoDBIdiot> entertain me .)
[16:18:16] <NodeX> he's not a very good therapist, I want a refund
[16:18:41] <wiseguysonly> MongoDBIdiot: Tell me about your mother
[16:19:00] <MongoDBIdiot> my mother fucked with a pig and NodeX is the result
[16:19:02] <MongoDBIdiot> LOL
[16:19:11] <ron> MongoDBIdiot: dude. leave.
[16:19:11] <NodeX> correct
[16:19:15] <wiseguysonly> ooh that is so kickable
[16:19:20] <NodeX> oink oink
[16:19:41] <MongoDBIdiot> ron: why?
[16:19:53] <algernon> life would be so awesome!
[16:19:54] <wiseguysonly> Pigs rock
[16:20:07] <NodeX> pigs rule the world
[16:20:09] <wiseguysonly> Clean, intelligent.
[16:20:16] <NodeX> without them there is no bacon
[16:20:22] <MongoDBIdiot> dumb americans can't talk about sex?
[16:20:45] <NodeX> I am not american
[16:20:48] <wiseguysonly> MongoDBIdiot: show us where he touched you on this doll
[16:20:51] <MongoDBIdiot> India?
[16:20:53] <NodeX> no
[16:21:01] <NodeX> the place that didn't let you win the war
[16:21:08] <wiseguysonly> haha
[16:21:20] <NodeX> the country that bent hitler over your mom and rammed home
[16:21:24] <wiseguysonly> Is he from Argentina ;)
[16:21:36] <NodeX> MongoDBIdiot - aka MacYET is german
[16:21:52] <NodeX> or at least is from Germany
[16:22:09] <wiseguysonly> Oh best be quiet - I live in Spain and I'll be getting some of your tax euros soon MongoDBIdiot
[16:22:22] <NodeX> Lol, I want some too
[16:22:30] <wiseguysonly> I'll be sure to thank you for the bailout
[16:22:55] <MongoDBIdiot> mum called me, time for bed and night drugs
[16:22:58] <MongoDBIdiot> cu tomorrow
[16:23:25] <wiseguysonly> c u next tuesday
[16:23:35] <MongoDBIdiot> by darling
[16:24:05] <astropirate> Is mongodb eventually consistent or always consistent
[16:24:47] <kali> astropirate: you're in control of consistency on every write
[16:25:12] <astropirate> I see
[16:25:27] <ron> you are in control!
[16:26:10] <remonvv> yes, depends on how you write.
[16:26:11] <NodeX> Eventual consistency across replica's though
[16:26:23] <remonvv> By default writes are consistent to all but replica sets unless you write with w > 1
[16:26:27] <kali> NodeX: not if you ask for write > 1
[16:26:29] <NodeX> you can determine number of nodes
[16:26:33] <remonvv> replica set members that aren't primary, that is
[16:27:08] <remonvv> true, but then I don't think anyone should do writes where w = number of repset members
[16:27:26] <astropirate> Another questions
[16:27:39] <remonvv> so even if you do w = "majority" or w > 1 you have to consider the reads eventually consistent because some members might not have the data
[16:27:52] <NodeX> ;)
[16:27:59] <NodeX> I am in pedantic mode today
[16:28:02] <remonvv> unless you always read from the primary only
[16:28:13] <NodeX> remonvv : you work with video right?
[16:28:14] <remonvv> You don't really have any other mode kind sir.
[16:29:01] <astropirate> how is mongo's map reduce performance compared to others, my primary use of the database is, write some data, and make a whole lot of map reduces very often
[16:29:01] <remonvv> NodeX, not really. We provide second screen services for TV shows and films.
[16:29:14] <NodeX> second screen?
[16:29:19] <remonvv> astropirate, it's not great. I'd go for Hadoop or something like that
[16:29:21] <kali> astropirate: not good
[16:29:33] <astropirate> ahh ok
[16:29:48] <astropirate> Hadoop is an option but it seems to require multiple nodes
[16:29:54] <NodeX> aggregation is faster - if your app can do it in aggregation
[16:29:55] <remonvv> NodeX, software that provides additional info/interaction during live events. E.g. you're watching sports and real-time data or comments are shown on your iPad or you can play along with Weekend Millionaires from home, etc.
[16:30:11] <NodeX> ah ok, so you dont use rtmp for that then?
[16:30:27] <remonvv> No we have in-house tech for most of it.
[16:30:41] <kali> astropirate: hadoop can run standalone
[16:30:57] <astropirate> kali, have you had any dealings with riak?
[16:31:04] <astropirate> thats my second choice after mongo
[16:31:08] <remonvv> In order to keep it cost efficient we need to push content to a lot of clients effectively and RTMP wasn't a very clean fit at the time.
[16:31:20] <remonvv> astropirate, never do m/r on databases directly.
[16:31:38] <astropirate> remonvv, howcome?
[16:31:41] <ron> remonvv!
[16:31:57] <kali> astropirate: well, m/r ops are full table scans, you need oriented for bandwidth. db like riak and mongodb (and sql) are optimized for latency
[16:32:04] <kali> +db
[16:32:19] <astropirate> I see
[16:32:24] <astropirate> so i really do need to go the hadoop route
[16:32:39] <astropirate> alright, thank you very much for the advice!
[16:32:41] <remonvv> Exactly, you don't want to hog db node resources with m/r, you need it to guarantee throughput
[16:32:42] <kali> astropirate: unless you have a small dataset, in which case anything will work
[16:32:49] <remonvv> mongodb -> hadoop works well
[16:33:25] <astropirate> I have a whole lot of learnign to do then :)
[16:33:33] <remonvv> ron!
[16:33:45] <remonvv> astropirate, isn't it fun? ;)
[16:33:55] <remonvv> MongoDB really is nice tech and it's maturing pretty quickly
[16:34:04] <astropirate> remonvv, dude..... i've been learning so much i'm about to throw up
[16:34:06] <remonvv> Not quite there yet but hey.
[16:34:11] <remonvv> astropirate, welcome to IT
[16:34:19] <astropirate> i'm dealing with databases, GPU processing, machine learning
[16:34:19] <remonvv> where every single thing changes every other month
[16:34:20] <astropirate> lol
[16:34:24] <astropirate> exciting but tiresome
[16:34:38] <remonvv> just divide the problems into chunks and focus on every single chunk seperately
[16:34:47] <astropirate> that is the plan
[16:34:49] <remonvv> after you decided how the different chunks are going to work together that is
[16:34:52] <astropirate> but i must first get a big picture
[16:34:55] <remonvv> well then, divide and conquer
[16:35:11] <remonvv> right, so define your problems, decide on an interface between your solutions and work on the individual solutions
[16:35:18] <remonvv> and make sure you use something for what it does best
[16:35:24] <astropirate> :D thats the plan
[16:35:30] <remonvv> e.g. use a db for the db-y things and not m/r and such
[16:35:44] <remonvv> btw MongoDB has a native aggregation framework that is actually a lot better than the m/r in terms of performance
[16:35:51] <remonvv> if that fits your need it *might* be an option
[16:35:56] <remonvv> i'd still go for dedicated m/r services
[16:36:04] <ron> remonvv: I got news, but now's not a good time.
[16:36:16] <remonvv> pregnant?
[16:36:51] <remonvv> and okay, well feel free to drop me an e-mail or catch me tomorrow
[16:37:18] <thrashr888> hi guys
[16:38:37] <remonvv> hi
[16:57:59] <mnaumann> i'm using mongodump (mongodb runs with --auth) and i'm using mongodump for the first time, trying to backup multiple databses where i have created the same non-admin user with the same password.
[16:58:23] <mnaumann> i'm using the --user and --pasword options with mongodump and it says that authentication fails.
[16:58:53] <mnaumann> i'm using the --username and --pasword options with mongodump and it says that authentication fails. << correction
[16:59:32] <mnaumann> i'm passing multiple databses to backup as --db="first_database second_database"
[16:59:51] <mnaumann> is something about this wrong? do i need to use an admin user to backup?
[17:01:05] <jedir0x> I have an entity collection, and a collection of comments that are associated with those entities. given a collection of entity ids, how might i pull the latest 3 comments (dateCreated) for each of the ids? This needs to be a realtime query, i can't use MapReduce.
[17:06:01] <Dennis-> use the aggregation framework
[17:07:59] <kali> it will be expensive
[17:08:06] <kali> less than m/r, but still
[17:08:37] <jedir0x> yeah, i was looking into the aggregation framework
[17:08:49] <jedir0x> the list of ids will never be greater than 50 or so
[17:09:46] <kali> you need the comments collection to fit in memory
[17:10:11] <jedir0x> the entire collection?
[17:10:30] <jedir0x> or just the comments that match my entity ids
[17:10:56] <kali> just these, but they'll be all over the place
[17:20:40] <psychowico> hello all
[17:26:42] <meghan> hi psychowico
[17:29:19] <ron> remonvv left? that lazy ass.
[17:29:28] <psychowico> I just start learn mongodb. I preparing to use it in my project and I think this can be good choice for this one.
[17:29:45] <ron> \o/
[17:30:26] <psychowico> I guess some of you have done projects on it. I thinking about using Doctrine ODM (I writing in php)
[17:31:22] <psychowico> but I just read than working with ODM is "not schemaless". what do you think about this? some of you use doctrine odm or some other ODM in php?
[17:32:30] <NodeX> avoid things that fix your schema
[17:35:15] <IAD> psychowico: my ODM uses __set and __get to get schema less
[17:35:37] <NodeX> also avoid unnecessary bloat in your ap
[17:35:39] <NodeX> app*
[17:35:41] <NodeX> especialy in php
[17:36:04] <psychowico> what exacly you mean by 'bloat'?
[17:36:24] <psychowico> IAD, what odm you use?
[17:36:28] <NodeX> large libraries and frameworks
[17:36:45] <NodeX> symphony, doctrine, smarty etc etc
[17:37:08] <IAD> psychowico: I wrote it for myself
[17:37:18] <psychowico> ah, I understand
[17:38:20] <IAD> to use arrays is also a good solution
[17:39:14] <psychowico> Can anyone recommend some light support library for mongo in php?
[17:39:40] <NodeX> the php driver is enough
[17:39:46] <IAD> psychowico: pecl install mongo =)
[17:39:51] <NodeX> if you wantt o extend it just write a wrapper
[17:40:23] <psychowico> mhhh.. thx for this opinion.
[17:41:18] <jedir0x> theres http://code.google.com/p/mongo-java-orm/ for java :D
[17:41:27] <jedir0x> shameless plug
[17:41:32] <psychowico> I have some noob question about nosql working too. I had standard relation sql experience before and I thinking about some scenario:
[17:42:40] <psychowico> mhhh.. or maybe not. I just realize answer by myself :P sorry,sometimes it happens to me :P
[17:44:17] <jedir0x> at some point i'll add aggregation support to MJORM
[17:44:47] <jedir0x> and update the documentation, it supports any type for ids now, as well as polymorphic mapping.
[17:45:04] <jedir0x> aggregation via MQL would be nice probably too
[18:07:34] <patrickod> I have a large TTL collection on which I need to change the timeout
[18:07:46] <patrickod> is the only way to do this to delete the first index and then re-index?
[18:10:34] <jedir0x> ok, with the aggregate framework how would i limit the number of "commentIds" returned on the document to 3: http://pastie.org/5110428 ?
[18:37:31] <psychowico> maybe someone of you can show me your lightweight wrapper for php \Mongo using, if u have any opensource project that have something like this?
[18:38:31] <Dennis-> show you what exactly?
[18:39:11] <jedir0x> is there a way to do a limit on a group with the aggregation framework?
[18:43:01] <NodeX> jedir0x : I dont think so
[18:49:58] <sirious> anyone else have issues doing a $pull with $elemMatch which contains an ISODate value?
[18:51:18] <Freso> psychowico: Why not try the default/standard PHP driver?
[18:51:32] <Freso> psychowico: Who knows? It might just turn out to be all you needed.
[18:52:30] <Freso> psychowico: And if you, after having tried it out, realise you need more than that, you'll most likely know *what* it is you need more, and you'll be able to pick more selectively among the wrappers out there - or make your own.
[19:24:13] <__ls> if i call find() on a collection, does the resulting cursor always return results in the same order?
[19:25:35] <Dennis-> if you depend on ordering, then use sort()
[19:26:28] <__ls> so the answer is no?
[19:26:59] <Dennis-> read between the lines
[19:28:17] <__ls> http://www.mongodb.org/display/DOCS/Sorting+and+Natural+Order suggests otherwise
[19:29:02] <Dennis-> only fools depends on implicit behaviour
[19:29:11] <__ls> you are so nice today
[19:30:10] <Dennis-> you are a serious developer or a tinkerer?
[19:31:11] <yawning> i did a hard reset without shutting down properly and now my db is missing all of its collections. i just see system.indexes and session_table
[19:31:23] <yawning> I do have a backup i could pull down. is there anything else I could do instead?
[19:31:59] <Dennis-> --repair?
[19:32:27] <yawning> didn't seem to help
[19:33:15] <Dennis-> complete data loss through unclean shutdown? what a mess
[21:11:05] <sinisa> hi all
[21:12:34] <sinisa> is aggregation framework blocking , anyone
[21:24:56] <kian> i'm setting up a sharded cluster; is it possible to, in addition, have a node which is never queried but contains the entire data set? (for simplifying backups, for example)
[21:39:21] <camonz_> is there a way to update each record with a value within that record in a mass update ?
[21:42:51] <camonz_> something like referencing a value of an embedded field in the iterated object when specifiying the value to be updated
[21:54:20] <jgornick> Hey guys, looking at this gist https://gist.github.com/61b7c0266e88347d8968, why doesn't the $pushAll on the AtomicIssue cause a unique index violation if a task with the same title already exists?
[21:54:51] <jgornick> I'm assuming a pushAll would also cause unique index violations?
[22:53:51] <jgornick> Hey guys, looking at this gist https://gist.github.com/61b7c0266e88347d8968, why doesn't the $pushAll on the AtomicIssue cause a unique index violation if a task with the same title already exists?
[22:53:53] <jgornick> I'm assuming a pushAll would also cause unique index violations?
[23:13:04] <jgornick> Hey guys, is there a reason why $pushAll does not trigger a unique index violation?
[23:13:24] <Gargoyle_> I seem to be failing to connect to a replica set with php5.4 and driver 1.3.0beta2. Im getting "not master and slaveOk=false".
[23:13:56] <Gargoyle_> What's the correct way to specify slaveOk in the new driver (as setSlaveOk() is dep'd)
[23:16:16] <Gargoyle_> Bit confused, as our live servers running php 5.3 and 1.3.0beta2 are working. Is this something I have overlooked in my new php-fpm config? Any hints would be helpful.
[23:29:50] <jgornick> Am I wrong in saying that the update command in 00-mongo.js should cause a unique index violation? https://gist.github.com/61b7c0266e88347d8968
[23:35:00] <jgornick> Well, this sucks: https://jira.mongodb.org/browse/SERVER-1068
[23:36:57] <Gargoyle_> hmm.