PMXBOT Log file Viewer

Help | Karma | Search:

#mongodb logs for Friday the 7th of September, 2012

(Back to #mongodb overview) (Back to channel listing) (Animate logs)
[00:03:05] <fg3> mongo is a beast
[00:10:43] <fg3> I'm wondering when this irc channel lights up
[00:11:50] <crudson> when euro/EST people are working
[00:11:58] <crudson> have a question?
[00:27:08] <fg3> crudson, yes
[00:28:23] <fg3> data = categories: [ { name: 'foo', items: [ {name: 'item1'} ] } ]
[00:28:41] <fg3> crudson, how can I update item1?
[00:48:29] <crudson> fg3: I presume you can have multiple categories and multiple items in each category
[00:48:36] <fg3> yes
[00:49:46] <fg3> crudson, that is correct
[00:49:54] <crudson> fg3: to update a specific matched item in an array use the http://www.mongodb.org/display/DOCS/Updating#Updating-The%24positionaloperator
[00:50:36] <crudson> fg3: although I am not sure you can use it in an array nested within an array, that would be something to experiment
[00:51:05] <fg3> I heard you cannot
[00:51:31] <fg3> the $ is the position in the array but when there is more than one array...
[00:52:28] <fg3> I lost the stack-exchange page where they chat about this -- I'm wondering if I have to change my structure
[00:54:43] <Guest55811> whats the cookie cutter use case example to choose mongodb over like mysql or postgresql
[01:00:46] <crudson> fg3: yeah, either fetch, change, save, or do in a very small db.eval() - both of course should be considered v carefully depending on the mutability/load of your db. Otherwise document restructure should be considered to make querying and updating easy, fast and atomic.
[01:01:05] <crudson> fg3: search the JIRA for nested positionals, I bet there is a case for it.
[01:08:50] <fg3> crudson, thanks
[01:20:39] <SpanishMagician> hello
[01:21:48] <SpanishMagician> is it possible to sort result based on an element inside an embedded document?
[01:22:53] <SpanishMagician> ex I have a colletion of users and inside of each user there's an embedded document comments
[01:23:12] <SpanishMagician> is it possible to sort based on commets.date?
[01:23:43] <SpanishMagician> find(....).sort('comment.date':1);
[01:23:49] <SpanishMagician> something like that?
[03:00:50] <phatduckk> hey all - quick question. In the logs does "locks(micros) r:63" mean LOCKED for 63 micros or WAITED on a lock for 63 micros?
[03:01:18] <phatduckk> i get some nice (aka high) lock % and am trying to track down the cause(s)
[05:22:42] <_Tristan> Hello. How would I go about removing all documents from a collection where "expires" (a date objectything) is before now?
[05:24:24] <_Tristan> er, would that be db.foo.remove({expires: {'$lt': new Date()}})?
[05:29:32] <skot> yes
[05:29:51] <skot> you can run the query using find instead of remove to check the query
[06:26:37] <ron> eew.
[06:47:31] <NodeX> it's way to early for panty sniffing
[06:48:19] <Gargoyle> NodeX: what!?
[06:49:43] <NodeX> [07:23:49] * ron sniffs kali
[06:49:44] <NodeX> [07:24:05] * emocakes sniffs kali's panties
[06:49:44] <NodeX> [07:25:32] <ron> eew.
[06:50:03] <emocakes> never too early for panty sniffing NodeX
[06:50:09] <emocakes> its nearly 5pm here
[06:50:09] <Gargoyle> Get a room!!! (not this one) ;)
[07:02:15] <abhishek__> can someone explain me reason for this : https://groups.google.com/forum/?fromgroups=#!topic/mongodb-user/JddGQ8ooUcw
[07:23:52] <kali> oO
[07:35:55] <[AD]Turbo> hi there
[07:48:26] <NodeX> Derick : any update on the socket connect from the PHP driver?
[08:00:41] <SpanishMagician> is it possible to sort result based on an element inside an embedded document?
[08:00:57] <SpanishMagician> ex I have a collection of users and inside of each user there's an embedded document comments
[08:01:08] <SpanishMagician> is it possible to sort based on commets.date?
[08:01:15] <SpanishMagician> find(....).sort('comment.date':1);
[08:01:25] <SpanishMagician> something like that?
[08:01:27] <ron> did you try what you suggested?
[08:01:40] <SpanishMagician> yes, didn't work
[08:04:08] <ron> gave you an error or unexpected results?
[08:19:26] <samurai2> hi
[08:20:17] <samurai2> what's actually sh.setBalancerState(false) or disable the balancer do actually? thanks :)
[08:28:18] <gigo1980> hi, i get an timeout in my mongo cluster some times, what can be the reasons ? i dont compact or repair the collections during this time
[08:33:36] <Axsuul> Not sure if I recall correctly, but was mongodb's global lock addressed recently?
[08:34:06] <kali> Axsuul: since 2.2 the write lock is database per database
[08:34:15] <kali> Axsuul: but the js interpreter lock stays as before
[08:38:59] <Axsuul> kali: thanks, how do you guys get around heavy read/write applications involving mongo?
[08:42:29] <Gargoyle> Axsuul: Write to master, read from slaves
[08:42:33] <Gargoyle> ?
[09:24:39] <Vile> Hi guys. is it possible to return multiple attributes from map/reduce?
[09:26:04] <Vile> i.e. not { _id, value } but http://pastebin.com/SxVySJNQ
[09:30:09] <Albeneth> Hi. How do i do this (http://pastebin.com/R5RXXVZ3) in mongodb (with the same collections and fields names) ?
[09:40:20] <Gizmo_x> anyone have expirience with doctrinemongoODM reference query http://pastie.org/4678732 - self reference many to many
[09:41:20] <Gargoyle> Albeneth: Mongo doesn't have the concept of JOINs and for GROUP BY, you'll be wanting to checkout the new aggregation stuff in 2.2.
[09:47:46] <Albeneth> Gargoyle, thanks
[09:48:45] <kali> Gargoyle: true for JOIN, but the AF is all about GROUP BY
[09:49:15] <Gargoyle> punctuation fail!
[09:49:31] <Gargoyle> s/JOINs and/JOINs. And
[09:49:43] <Gizmo_x> anyone have expirience with doctrinemongoODM reference query http://pastie.org/4678732 - self reference many to many
[09:50:18] <kali> Gargoyle: parsing failed on my side too, actually
[09:53:27] <Vile> Is it possible at all to modify collection field?
[09:53:40] <Vile> i.e. update based on its current value
[09:53:42] <Gizmo_x> need help here http://stackoverflow.com/questions/12243929/mongodb-odm-docrine-self-reference-many-to-many-get-specific-data
[09:54:28] <NodeX> Vile: not in one query
[09:54:42] <kali> Vile: not efficiently. you can do it with application code, or by server side javascript (avoid that in prod)
[09:55:18] <Gargoyle> Vile: You mean update({someProp: 'foo'}, {$set: {someProp: 'bar'}}, false, true) type of thing. Don't think that's allowed.
[09:55:28] <Vile> basically i have a problem. i have large database with field format as http://pastebin.com/SxVySJNQ and I need to update _id.t fields
[09:55:56] <kali> Vile: I don't think you can change _id
[09:56:19] <Vile> kali: why not? same field as any other
[09:56:59] <Vile> ah, i see. it would not allow it
[09:57:17] <kali> The _id field is immutable. If you try to change its value on an update you will get a "Mod on _id not allowed" error from getLastError.
[09:57:18] <Vile> can i do find() into collection?
[09:57:22] <kali> http://www.mongodb.org/display/DOCS/Object+IDs
[09:59:51] <Vile> or no matter what i will have to rely on external program?
[10:00:07] <Vile> :(
[10:01:00] <Gargoyle> Vile: Not sure what you're asking!
[10:02:26] <Vile> Gargoyle: basically, i need to take huge collection of records as in the paste and modify _id.t field
[10:02:55] <Vile> I can afford to copy collection once into separate collection
[10:03:43] <Gargoyle> Vile: You'll have to re-insert the while doc and delete the old ones. And have a think about how you have come to need to change your _id's !
[10:04:13] <stevie-bash> Hello, I want to run logrotate on the router, since kill -USR1 has a deadlock bug.
[10:04:14] <stevie-bash> http://www.mongodb.org/display/DOCS/Logging
[10:04:38] <stevie-bash> can I send the echo also the the mongos process? Will it logrotate only the router?
[10:04:39] <Vile> there was a bug in external program that was reporting wrong values for _id.t field
[10:14:00] <NodeX> Pretty sure _id.t is not allowed as it can't be an object or array as kali says
[10:30:54] <emocakes> hai kali
[10:34:21] <jwilliams> for insert with continuous on error, is it required to have driver also support it? Or it would be default on at server side regardless of driver specifying "continuous on error" or not?
[10:50:18] <Vile> Guess I'll write a function that does find() and then iterates over cursor doing inserts into other collection
[10:51:51] <kali> emocakes: :P
[12:25:31] <Gizmo_x> need help here http://stackoverflow.com/questions/12243929/mongodb-odm-docrine-self-reference-many-to-many-get-specific-data
[13:07:25] <Gargoyle> Ping Derick
[13:09:58] <gigo1980> hi all, is it posible to run only one map reduce action per collection at a time ?
[13:09:58] <Derick> Gargoyle: Hi
[13:10:48] <jmar777> gigo1980: it's impossible to run more than one at a time
[13:11:12] <jmar777> gigo1980: m/r in mongodb runs on a single javascript thread
[13:11:29] <gigo1980> what happens with that task ?
[13:11:30] <gigo1980> https://jira.mongodb.org/browse/SERVER-4258
[13:11:39] <Gargoyle> Derick: Soz, phone just rang….
[13:11:44] <Gargoyle> http://pastie.org/private/1vipwtzdsfbb66f90yn56q
[13:11:52] <Gargoyle> http://pastie.org/private/hut3kjxopd8fms7oejm3w
[13:12:32] <Derick> for the 2nd valgrind to be useful, could you also export "export ZEND_DONT_UNLOAD_MODULES=1" ?
[13:12:59] <Derick> i don't see a crash with valgrind though, just a memleak
[13:15:38] <gigo1980> @jmar777 so the thing is i have an 24 core system where my mongo db is running
[13:16:08] <gigo1980> so will it be an boost to run one physical machine more than one shard ?
[13:18:45] <Gargoyle> Sorry Derick, Phone went just as you pinged back. I noticed still getting some segfaults again in my apache! :(
[13:18:54] <Derick> hmm
[13:19:03] <Derick> it's possible that there is another bug
[13:19:34] <Derick> can you redo the valgrind with the ZEND_DONT_UNLOAD_MODULES=1?
[13:20:53] <jmar777> gigo1980: hmm... i would think yes, but only with regards to m/r, since that can operate over individual shards in parallel
[13:21:03] <Gargoyle> Derick: http://pastie.org/private/lwzmyiwghhii42eytqbkbw - Is that berrer? Looks the same to me! ;-)
[13:21:12] <jmar777> gigo1980: for all other queries, you'd have to test it, but mongo already utilizes multiple cores pretty well
[13:25:09] <gigo1980> yes but i have some concurrency map/reduce request ...
[13:25:22] <gigo1980> so about 20 at a time
[13:27:56] <Vile> how can i check progress for the command like db.eval("function () { db.histories.find().forEach( function(v) { v._id.t = new Date( v._id.t.getTime() - 28800000 ); db.histories2.insert( v ); })}"); ?
[13:28:18] <Derick> Gargoyle: which php version is that?
[13:28:36] <Gargoyle> PHP 5.3.10-1ubuntu3.2 with Suhosin-Patch (cli) (built: Jun 13 2012 17:19:58)
[13:32:46] <Derick> meh, that probably doesn't have that flag :-/
[13:35:58] <Derick> hmm
[13:36:20] <Gargoyle> I'm getting much less segfaults, so I'll see if I can narrow it down.
[13:36:28] <Derick> Gargoyle: all it now tells me there *is* a memleak - but not in what
[13:36:38] <Derick> could be adifferent extension alltogether
[13:37:08] <Gargoyle> I'll see if It happens locally.
[13:37:11] <NodeX> Derick : while you're here, any update on the .sock php driver connect thing?
[13:37:47] <NodeX> Gargoyle : do you run APC ?
[13:37:54] <Gargoyle> yup
[13:38:02] <NodeX> certain PHP versions segfault alot with that on
[13:38:24] <NodeX> scratch that, certain PHP versions and apache
[13:38:33] <Gargoyle> NodeX: I turned it off yesterday, and it made no difference.
[13:38:46] <NodeX> what else you got loaded?
[13:38:50] <Gargoyle> NodeX: Was it you asking me about using Apache alltogether the other day.
[13:38:52] <Gargoyle> ?
[13:38:56] <NodeX> (just trying to narrow down what it could be)
[13:38:57] <NodeX> yes
[13:39:11] <Derick> NodeX: nothing yet
[13:39:12] <NodeX> I was wondering why you used it
[13:39:42] <NodeX> Ok Derick thanks
[13:39:43] <Gargoyle> NodeX: http://pastie.org/4680011
[13:39:55] <Gargoyle> In all honesty, probably a bit of lazyness!
[13:40:01] <NodeX> Kudos on the new connection handling btw Derick : it's alot better
[13:40:08] <NodeX> my logs are not so full anymore!
[13:40:16] <Derick> what were they full of?
[13:40:29] <NodeX> connects/disconnects!
[13:40:35] <Derick> oh
[13:40:43] <NodeX> Gargoyle : php -m > foo.txt
[13:40:58] <Gargoyle> never been in the situation where I needed to wring every last bit of performance, and apache has always been stable! (And damn easy to install!)
[13:41:29] <NodeX> It's just weird because aot of people use Mongo for the speed, then throttle it with apache
[13:41:38] <Gargoyle> NodeX: Same paste, refresh it.
[13:42:16] <Gargoyle> NodeX: Yeah, I'm aware that other webservers have been gaining ground over the last few years, not kept up with the progress properly.
[13:42:21] <Derick> NodeX: what used to be your connection string with the socket?
[13:43:05] <NodeX> Derick, one second
[13:43:19] <NodeX> public function __construct($connect="mongodb:///tmp/mongodb-27017.sock") {
[13:44:01] <NodeX> Gargoyle : what other things does the app do at the time of segfault ... i/e are you creating images or anything?
[13:44:15] <NodeX> (trying to narrow what modules might be in play at that point)
[13:44:30] <Gargoyle> Just looking to see if I can narrow it down to a single request like I managed to the other day.
[13:44:58] <NodeX> if your php up to date
[13:45:34] <Gargoyle> NodeX: Not "fully"
[13:45:53] <Gargoyle> Running ubuntu 12.04LTS.
[13:46:08] <NodeX> you running mod_php or cgi under fpm or anythign
[13:46:11] <NodeX> are you*
[13:46:26] <Gargoyle> So it's a trade-off between stability and being up-to-date.
[13:46:40] <Derick> NodeX: he runs modphp5
[13:46:41] <Gargoyle> NodeX: Just mod_php under apache, and some command line batch stuff
[13:48:21] <Gargoyle> Slightly concerning that 5.3.12/13/14 are security updates, and 3 months down the line ubuntu is still not got them!
[13:48:36] <NodeX> :S
[13:49:33] <Gargoyle> Might schedule myself some time away from clients and do some digging and messing with other distros.
[13:50:09] <Gargoyle> Kept with ubuntu since 06.04 ish, just because of the sheer ease of installation.
[13:50:51] <NodeX> LTS implies they'll fix things lo
[13:50:55] <NodeX> lol*
[13:50:58] <Gargoyle> Slightly off-topic, what linux distros do people here use for desktop dev work?
[13:51:06] <ppetermann> ubuntu
[13:51:16] <NodeX> debian
[13:51:21] <dominikzogg> ubuntu
[13:51:26] <ppetermann> with gnome instead of whatsthatshit
[13:51:30] <NodeX> winblows atm
[13:51:38] <dominikzogg> with whatsthat shit
[13:51:48] <ppetermann> NodeX: winblows is no linux distro (yet)
[13:52:02] <NodeX> debian normaly, winblows at the moment
[13:52:27] <Gargoyle> Again, been a while since I had a good play with a desktop linux. What's Mesa3D ?
[13:53:21] <Derick> been fighting with windows all day today :-(
[13:53:49] <ppetermann> :(
[13:53:57] <ppetermann> i'm fighting linux atm
[13:55:58] <Gargoyle> Just that latest vmware finally adds hardware support for 3D under linux, so it could yeild a more useable dektop under VM.
[13:56:21] <NodeX> I always fight windows
[13:56:38] <NodeX> tbf winblows 7 isn't as bad as I thought it would be
[14:44:23] <Gizmo_x> need help here http://stackoverflow.com/questions/12243929/mongodb-odm-docrine-self-reference-many-to-many-get-specific-data
[14:44:38] <Derick> jmikola: that one is for you
[14:46:07] <Gizmo_x> Derick: how can i query a reference many to many as here: https://groups.google.com/forum/?fromgroups=#!topic/doctrine-user/tG_jHF8XdEw
[14:46:56] <Derick> Gizmo_x: i've no idea about it...
[16:08:16] <Gargoyle> NodeX: This could be the longest drawn out conversation ever! But what do you use with php instead of apache?
[16:30:17] <veesahni> from the docs: "You may use sort() to return data in order without an index if the data set to be returned is small" .. this doesn't sound right.. limit() creates a small returned data set, but sorting needs to happen on the pre-limited() potentially large set.. am I missing something?
[16:31:09] <kali> veesahni: sort happens before limit
[16:31:23] <kali> veesahni: or else there is no way to get a predictable result
[16:33:03] <veesahni> kali: that's what I thought.. thanks
[16:34:05] <joshua> I have a shard cluster built, now wheres the best docs to start a 3 server configserver cluster... do you just start up 3 instances and point mongos to it, or is there more than that?
[16:34:34] <joshua> I wish they would just get a support contract with 10gen. heh
[16:36:03] <jmikola> Derick: taking a look now
[16:43:18] <ggreenha> Hey guys whats going on have a question about grouping
[16:43:38] <ggreenha> how do you return one record per say customer
[16:43:45] <ggreenha> in a group
[16:43:56] <ggreenha> group appears to only return the grouped by field and a counter
[16:44:06] <jY> map/reduce
[16:44:26] <ggreenha> does that kil the tmp data after the request
[16:46:10] <ggreenha> seems compicated to simple do a group by profile in sql
[16:46:25] <ggreenha> sorry type that all fucked up
[16:46:51] <ggreenha> in sql I want to do say select * from images group by customer
[16:47:15] <ggreenha> how would you do it in map reduce
[18:52:37] <boll> Has anyone actually deployed 2.2.0 to production, and if so, how's it working out?
[18:56:42] <dstorrs> boll: I was actually coming here to ask about the deploy process. I am deploying it to prod right now. :>
[18:57:42] <boll> I deployed to staging the day it came out. It was a simple matter of upgrading the replica set one replica at a time
[18:57:48] <boll> it basically "just worked"
[18:58:26] <boll> I am inherently suspicious about deploying any .0-releases to production though
[18:58:53] <boll> One caveat though, the connection parameters have changed for the php-driver
[18:59:02] <boll> (1.3 beta)
[18:59:05] <dstorrs> I don't use it, so no worries.
[18:59:18] <dstorrs> Are you on RHEL / CentOS / Fedora ?
[18:59:23] <boll> Ubuntu.
[18:59:29] <dstorrs> probably smart.
[18:59:35] <boll> I like to think so :)
[19:00:13] <dstorrs> Ubuntu was what I wanted, but our contracted sysadmin talked me into CentOS
[19:00:26] <boll> Is your application read or write intensive?
[19:01:05] <boll> We have quite a lot of writing going on, so I am quite excited about the global lock mostly being gone on writes.
[19:01:22] <dstorrs> we do about 11M writes per day
[19:01:27] <dstorrs> Far fewer reads
[19:03:46] <boll> Let me know how it goes
[19:11:27] <dstorrs> boll: when you deployed it, did you just do a 'apt-get install ' ? or did you do manual installs?
[19:11:55] <boll> apt-get update && apt-get upgrade
[19:12:42] <boll> If memory serves, 2.2.0 successfully joined the 2.0.x instances in the running replica set
[19:14:21] <dstorrs> yes, unless you're using sharding
[19:14:31] <dstorrs> in which case, it Fs everything sideways
[19:14:42] <boll> yeah, we don't
[19:16:42] <boll> that's a little nasty actually
[19:20:52] <camonz> hi, is it posible to store and query a complex object, such an array of hashes inside a document? ex. https://gist.github.com/f16f02b496ce10b99ef3#file_new_format.json
[19:21:26] <camonz> also, i'm changing the keys to be information by itself instead of simply an accesor to a value
[23:09:29] <toddnine> Hi guys
[23:09:37] <toddnine> I'm working on emulating the mongo protocol
[23:09:54] <toddnine> What's the response to an insert operation? Wireshark doesn't show any response at all
[23:10:06] <toddnine> I've looked on the wiki but I can't find the documentation anywhere
[23:11:54] <wereHamster> toddnine: maybe there is none
[23:12:13] <wereHamster> if you want to make sure the write made it to disk you have to use getLastError
[23:12:15] <toddnine> wereHamster: let me back up
[23:12:25] <toddnine> I'm no mongo expert, and I'm using MongoHub to test
[23:12:47] <toddnine> Is there a way in MongoHub to set the writes to WriteConcern.SAFE?
[23:12:56] <toddnine> I want to see if there's a response when that's set
[23:13:06] <wereHamster> use the shell to test it
[23:13:09] <toddnine> I assume the server asks the client somehow during an insert if that's set
[23:14:00] <wereHamster> toddnine: http://www.mongodb.org/display/DOCS/getLastError+Command
[23:14:44] <wereHamster> it's the clients responsibility to make sure the write succeeded. The server doesn't care.
[23:14:49] <toddnine> ahh, so it's a 2 call then?
[23:14:51] <toddnine> a put
[23:14:55] <toddnine> then a getLastError?
[23:15:05] <wereHamster> yes
[23:15:19] <toddnine> awesome! Thanks man, that greatly simplifies things!