PMXBOT Log file Viewer

Help | Karma | Search:

#mongodb logs for Wednesday the 26th of September, 2012

(Back to #mongodb overview) (Back to channel listing) (Animate logs)
[00:24:51] <aboudreault> any user of mongoengine? I got this warning: FutureWarning: ReferenceFields will default to using ObjectId strings in 0.8
[00:25:09] <aboudreault> which look like an important change...
[00:27:26] <aboudreault> ok, will modify my code to that behavior now.
[01:21:51] <aboudreault> using mongoengine, ReferenceField with an embeddedDocument is not supported?
[01:26:18] <_m> aboudreault: #mongoengine might be better able to answer those questions.
[01:26:45] <aboudreault> _m, ok, wasn't sure, there was no topic there. thanks
[01:31:16] <_m> No worries. Hope they can get you rolling.
[01:55:17] <timeturner> is there a way to check if a doc exists
[01:55:24] <timeturner> just the doc exists
[01:55:28] <timeturner> true/false
[01:56:08] <_m> You have the doc's ID?
[01:56:12] <timeturner> yea
[01:56:20] <timeturner> but I don't want to keep checking for null doc
[01:56:44] <timeturner> I'm trying to find out if there is a way to just get a simple true/false straight from mongo
[01:56:58] <_m> Second. Let me test to make sure.
[01:57:02] <timeturner> ok
[02:00:04] <_m> db.foo.find({"_id" : ObjectId("asdf123asd123")}).count() > 0
[02:00:17] <_m> Should true/false
[02:01:07] <timeturner> ah
[02:01:11] <timeturner> yeah that's a better way
[02:01:14] <timeturner> thanks
[02:01:32] <_m> No problem.
[05:49:55] <jcims> quick question, i've got a csv with ~16M rows. eacy row has lat/long, and i'd like to merge them into a location field for geospatial searches. Is it possible to run an upsert or simlar query after the import to copy the separate fields into one
[06:03:33] <crudson> jcims_: still there?
[06:27:21] <owen1> my collection have documents like this: {'chains':[{'id': 1}, {'id': 2}]}. i want to know if a given integer exist among all the ids. what's the best way to do that?
[06:28:07] <owen1> so in this case 1 will return true. but 3 false. since 3 doesn't exist in any document.
[06:28:22] <owen1> (there is only 1 document in my example)
[06:32:27] <crudson> owen1: .find({'chains.id':1}) will match your example document
[06:43:20] <owen1> crudson: oh. so simple. thanks
[07:25:34] <crudson> cool :)
[07:38:19] <[AD]Turbo> hola
[07:48:44] <crudson> hello
[10:02:04] <gyre007> what is the acceptable replication lag in mong
[10:02:05] <gyre007> o
[10:03:00] <gyre007> we have a monitoring set up and we are getting loads of alerts ...like intermittent alerts for replication lag...at the moment we are checking for the number of seconds....30s and we are raising an alert...but then in next check the alert is resolved...really annoying :)
[10:15:04] <oskie> hello, if Mongo's sharding is automatic, why do you need to specify the key to shard on?
[10:35:57] <NodeX> oskie : because it doesn't make assumptions on your data
[10:36:02] <NodeX> what's the problem?
[10:50:52] <oskie> NodeX: the problem is that I don't know how to shard :)
[10:51:04] <oskie> I mean, based on what key. That is what I'm trying to figure out
[10:51:25] <NodeX> it depends why you're sharding
[10:51:31] <NodeX> and what you're trying to achieve
[10:51:39] <oskie> write scaling
[10:52:11] <NodeX> so you want to split evenly across shards?
[10:52:31] <NodeX> or do you want replica sets
[10:54:25] <oskie> we have replica sets.. but the server's are not powerful enough any longer
[10:55:21] <MongoDBIdiot> /mode * b
[10:56:36] <NodeX> then I suggest you shard on an even key like perhaps _id
[10:57:04] <NodeX> that should give even writes amongst X shards
[10:57:36] <oskie> NodeX: ok, like a hash table would distribute keys?
[10:58:09] <oskie> - that was what I was thinking. To add some field that is evenly random
[10:58:45] <NodeX> ObjectId's are tiemstamp based so you'll get odd/even
[11:00:46] <oskie> hmm.. unfortuntely it seems not all collections have _id e using ObjectIds
[11:01:00] <oskie> err.... _id's of type ObjectId
[11:01:34] <NodeX> perhaps add field then
[11:58:09] <oskie> are mongos intended to be placed in the web servers (assuming that is where the data is accessed)?
[12:05:50] <oskie> ah found the answer
[12:06:25] <oskie> but one other question: where does the shard balancing occur? in mongod processes i presume? but what process initiates the balancing?
[12:06:38] <NodeX> mongos
[12:12:39] <oskie> NodeX: hmm, so mongos is not so lightweight during actual balancing?
[12:13:02] <oskie> I mean, the balanced data passed through the mongos process?
[12:13:12] <oskie> passeS
[12:21:45] <oskie> oh, it is described in sharding-internals, Chunk Migration
[12:26:56] <NodeX> afaik it just routes the data
[12:32:38] <oskie> yep
[14:01:09] <listerine> Hi, is there a command to dump a document and recursively it's references/dependencies?
[14:01:59] <wereHamster> listerine: that depends on the driver, The driver resolves the references, not mongodb
[14:05:54] <listerine> wereHamster, got it .. I was thinking to do it using mongodump or something like that, but instead I must do using some mongo driver .. is there a scripting language that you recommend to sync just a given document in two environments?
[14:07:17] <listerine> wereHamster, got it .. I was thinking to do it using mongodump or something like that, but instead I must do using some mongo driver .. is there a scripting language that you recommend to sync just a given document in two environments?
[14:07:22] <wereHamster> get a driver for your favourite language and code it yourself
[14:12:06] <listerine> wereHamster, but what I need is some kind of a dump because I need to transfer that document between environments .. if I do that in my favourite language I'll have the document in-memory after querying and need to send it to the target environment. Would I do some service to receive this document on the target environment and perform some kind of merge operation with the coming document and the document in the target environment?
[14:12:57] <listerine> sorry if the question was confusing, I can reformulate it heh
[14:13:43] <wereHamster> open a conneciton to the first database, read the documents from it, open a connection to the second database, write the documents there.
[14:14:27] <wereHamster> you will have to write the merge algorithm yourself anyway, mongodb can not know how to do that.
[15:11:37] <remonvv> People need to stop with blogs like these; http://t.co/869JZ7TV
[15:12:27] <remonvv> "Use indexes" isn't a design and performance tip. It's like saying "Add Wheels" is a performance tip for making cars fast.
[15:12:33] <remonvv> Bit more in depth discussion would be good.
[15:12:51] <NodeX> lol
[15:12:55] <NodeX> +1
[15:14:33] <remonvv> I was looking forward to my end of the day blog read but @MongoDB failed me by not filtering content properly.
[15:14:48] <NodeX> this is the trouble with a product being both simple and good
[15:15:09] <NodeX> to many idiots think "Oh Perhaps I can do XYZ with mongo" and then these posts pop up everywhere
[15:16:04] <NodeX> I am sure *SQL was fast at one point until the majority got it's hands on it and asked for foo, bar, baz to be implemented or John hancock wrote a blog post about how to do things (which he really didn't know about)
[15:19:48] <remonvv> Well I certainly don't mind blogs about MongoDB indexes, but this is sort of "Yeah it's faster now. Not explaining why that is, not explaining how to make appropriate indexes for specific use cases, not explaining why you want to right balance your indexes.....but there it is. Indexes are awesome!"
[15:20:23] <remonvv> It would even be okay if this was part 1 in a multi part blog on indexes in MongoDB.
[15:20:28] <remonvv> Oh well..../endrant
[15:20:33] <NodeX> comment on it then !!
[15:20:36] <Aartsie> hmm why you must build mongodb with scons ? and not just make ?
[15:21:15] <remonvv> Nah. I think the problem is @MongoDB tweeting about non helpful blogs rather than the blogs being written ;)\
[15:21:37] <NodeX> 10gen seem to be on the hunt for blogs at the moment
[15:21:48] <NodeX> syndication and so on
[15:22:00] <NodeX> they approached my company yesterday for it
[15:22:18] <remonvv> I'm going to start my own blog series. I'll call it "Actually Interesting MongoDB Tips"
[15:22:31] <remonvv> Oh?
[15:22:37] <remonvv> Hm, I'd be interested.
[15:23:34] <NodeX> apparently (frmo what they told my business partner) they've been watching my progress on a project for a long time and would like me to blog about the trials and tribulations
[15:23:46] <remonvv> Awesome, so why don't you?
[15:23:52] <NodeX> I guess I';ll have to learn hows to spell lol
[15:24:03] <NodeX> I'm not really the blogging type but I'll give it a go
[15:24:08] <remonvv> Spelling iz an detail son fo sho
[15:24:28] <NodeX> I'm a "matter of fact" person rather than a natural writer
[15:24:33] <NodeX> more a matter of fact*
[15:24:39] <remonvv> I'm not a writer by any stretch of the imagination.
[15:25:06] <NodeX> I would certainly like to put some blogs out there with the "right" way to do things and the reasons for them
[15:25:47] <remonvv> But it's the small things that nobody blogs about. You know, in what order does MongoDB put documents if you sort them on field A where values for A are [1,2,8] and [0,1,9] if you sort ascending and descending.
[15:26:24] <NodeX> the internals
[15:26:28] <remonvv> Or what are good production settings for drivers. Or how do you make sure only small parts of your indexes have to be kept in memory.
[15:26:42] <remonvv> In other words; "why isn't my shit working the way i expect it to"
[15:26:46] <NodeX> I'm not an "internals" person, I try a few ways and see which one works fastest - I dont often care why it does lol
[15:27:15] <NodeX> these are all good topics that should be addressed
[15:27:21] <remonvv> Fair enough but you'd be surprised how many SO questions are due to a lack of understanding of how b-trees work, or binary compares work, or how sharding works internally, etc.
[15:27:40] <remonvv> And then there's schema design
[15:27:40] <NodeX> perhaps if people had a primer on Mongo like that then alot of their initial design could be thought out properly
[15:28:00] <remonvv> Why shouldn't you use values as field names, when to embed, when to denormalize, etc.
[15:28:12] <remonvv> Nobody goes beyond "Well say you're creating a blogging website"
[15:28:12] <NodeX> most -Mongo- related problems can be solved with better schema design imo
[15:28:18] <remonvv> Yes yes we know...blogging is awesome in MongoDB.
[15:28:26] <remonvv> Right, I agree.
[15:28:49] <remonvv> most "This isn't very practical, MongoDB sucks!" problem too.
[15:29:01] <NodeX> yes, that annoys me alot
[15:29:14] <NodeX> people who blame software because it doesn't -fit- their application
[15:29:31] <NodeX> or "Mongo doesnt transact - it sucks man"
[15:29:42] <remonvv> Yeah "This Ferrari is broken because I can't fit 17 suitcases in the trunk"
[15:29:51] <NodeX> LOL
[15:30:54] <remonvv> I'm gonna write a blog and I'll dedicate it to you.
[15:31:20] <remonvv> "To Boldly Go Where No MongoDB Blogger Has Gone Before - Inspired by NodeX"
[15:31:43] <NodeX> haha
[15:31:56] <NodeX> I would read it
[15:32:37] <remonvv> I'd expect you to.
[15:33:15] <remonvv> The only blog I was actually close to writing was about RavenDB
[15:33:33] <remonvv> The consistent MongoDB bashing of the owner/creator of that database was getting on my nerves.
[15:33:58] <NodeX> they seem to have died out a bit now, they were the -buzz- for a day or two then nothing
[15:34:04] <remonvv> Especially since he's not exactly...what's the word....using facts.
[15:34:17] <remonvv> It's still gaining some traction in C# world.
[15:34:29] <NodeX> C# LOL
[15:34:48] <NodeX> I think I've seen perhaps 5 C# users in here in 2 years
[15:35:06] <remonvv> Hm, well, it's an okay platform. It's RavenDB that's sucking.
[15:35:44] <remonvv> I checked it out and there's very little documentation, the installer crashed, when I finally got it working it was stupidly slow and basically the worst of both worlds (NoSQL <-> RDBMS).
[15:36:03] <NodeX> :S
[15:36:31] <ralphholzmann> is anyone aware of a way to "monitor" changes to a collection?
[15:36:51] <ralphholzmann> looking for a way to push changes to a client from a collection in node.js
[15:36:54] <ralphholzmann> without polling
[15:37:39] <NodeX> ralphholzmann : I have seen a couple of rivers in python
[15:37:46] <NodeX> they monitor the oplog iirc
[15:37:59] <ralphholzmann> ah interesting NodeX
[15:39:44] <NodeX> https://github.com/richardwilly98/elasticsearch-river-mongodb <---- for es
[15:42:30] <remonvv> That's actually an interesting point. I'm working on a 100% MongoDB message queue but I was having some issues with multiple tailable cursors on the same collection. I'm assuming the solution above has to do that as well.
[16:33:18] <t0ms> hi, is there any way how to build HA cluster of mongodb on 2 physical nodes?
[16:34:42] <W0rmDrink> hi
[16:34:49] <W0rmDrink> where can I find references on functions like print
[16:34:57] <Gargoyle> t0ms: Yes and No.
[16:35:23] <Gargoyle> t0ms: You could also run an arbiter on one, to give you your min of 3 votes.
[16:35:59] <Gargoyle> t0ms: However, if the machine with the arbiter is the one that goes down, the remaining node will not promote itself to primary.
[16:36:33] <NodeX> W0rmDrink : javascript ;)
[16:36:51] <W0rmDrink> well - I have been looking - but most reference sites kinda sux
[16:37:04] <W0rmDrink> JDN only knows about window.print as far as I can see
[16:37:41] <W0rmDrink> would like something like this: http://en.cppreference.com/w/
[16:37:47] <W0rmDrink> I mean thats clear and concise
[16:37:47] <t0ms> Gargoyle: that's what I thought.. I need somehow to make sure that I can access the data from mongodb :)
[16:38:37] <Gargoyle> t0ms: You can force the single node online, but it has to be done manually.
[16:39:16] <t0ms> Gargoyle: how can I do that? I can't find any info how to "force" node to become primary
[16:45:25] <Gargoyle> t0ms: 1 sec
[16:45:45] <jcims> if i'm going to be running a 10gb database from a system with just a gig of memory, does it make any sense to move indexed attributes to a separate collection and link them back to the main collection by id?
[16:46:08] <Gargoyle> t0ms: http://www.mongodb.org/display/DOCS/Reconfiguring+a+replica+set+when+members+are+down
[16:46:36] <NodeX> jcims : you're mad running it on 1gb of ram
[16:46:42] <RexGibson> Gargoyle: could you just run 3 arbiters?
[16:46:44] <jcims> well, i gots what i gots
[16:47:55] <RexGibson> 2 on one and 1 on the other?
[16:47:55] <NodeX> it wont make a difference
[16:47:55] <jcims> ok
[16:47:55] <NodeX> the kernel will swap in and out what it feels is MRU
[16:47:55] <t0ms> Gargoyle: thank you!
[16:47:55] <Gargoyle> RexGibson: 2 out of 5 is not a majority!
[16:47:55] <jcims> maybe move the docs to disk and just indexed attrs in mongo then?
[16:48:04] <NodeX> jcims : again, the kernel will swap it in if it needs to so it wont make a difference
[16:48:54] <Gargoyle> jcims: You have your logic wrong!
[16:49:03] <jcims> thats why i'm asking :P
[16:49:20] <Gargoyle> jcims: We've just deployed a 10G database on 96G RAM
[16:49:20] <jcims> i'm storing the entire FCC license dataset, ~11GB, 16 million rows
[16:49:55] <jcims> basically there will only be one query, geospatial against lat/long
[16:50:10] <jcims> *that* attribute set is much smaller, <<1GB
[16:50:34] <Gargoyle> jcims: That's nice. Get 32GB RAM!
[16:50:48] <jcims> i'm trying to avoid a colo
[16:51:03] <jcims> and don't want to spend $500/month
[16:51:18] <Gargoyle> jcims: Then don't use a memory mapped database!
[16:51:57] <Gargoyle> jcims: But I wouldn't have thought you'd get much performance searching 11Gs of data on a system with 1G RAM from any other DB.
[16:53:06] <jcims> this is the first one i've tried :)
[16:53:24] <jcims> i started with amazon simpledb but it's a bit too simple
[16:53:41] <jcims> this is the first DIY db i guess i should say
[16:54:13] <Gargoyle> jcims: If you are *ONLY* searching via lat/long then you might have some success if you only store lat, long, filename
[16:54:19] <jcims> so maybe a hybrid of just lat/long in the database and the rest of the records in a simple disk-backed getup
[16:54:23] <jcims> right
[16:54:27] <jcims> going to try that approach
[16:54:34] <jcims> thanks for the help folks.
[16:54:47] <Gargoyle> thus producing a list of files you could then go and read, but I guess that depends on the final result set size.
[16:54:59] <jcims> right, they are pretty small.
[16:55:26] <jcims> and b/c they are geographically clustered i could probably do some 'tiling' of data and let the client ignore what it doesn't need
[16:55:30] <Gargoyle> jcims: Even then, as NodeX said, when you use memory for something else, it *WILL* get swapped out.
[16:55:37] <jcims> sure
[16:55:47] <jcims> it's got to be better than what i'm getting now :)
[17:04:05] <Almindor> how do you specify date query to mongodump?
[17:04:05] <Almindor> I always used var s = new Date in the console, but how do you do it for mongodump?
[17:07:13] <shadow_fox> cant seem to start mongodb from my linuxmint OS
[17:07:26] <shadow_fox> help needed someone please guide me
[17:08:18] <shadow_fox> i have downloaded the 32bit linux tgz file and extracted it and renamed to mongodb and moved the whole extracted package to /usr/local/bin/
[17:08:34] <shadow_fox> also created /data/db/ and set it to user read write
[17:08:42] <wereHamster> and then?
[17:09:16] <shadow_fox> whenever i try to run mongod from terminal it shows its nt install please use the apt-get to download mongodb-server
[17:09:34] <wereHamster> jcims: why would 32G ram cost 500$/month? I have a box with 64G and pay only 100EUR
[17:10:32] <shadow_fox> wereHamster: can you please tell what else do i need to do to get the mongodb started ?
[17:10:47] <wereHamster> shadow_fox: /usr/local/bin/../mongod
[17:10:48] <NodeX> wereHamster : what host is thaqt with ?
[17:10:57] <wereHamster> NodeX: hetzner
[17:10:58] <NodeX> (the 64gb box for 10EUR)
[17:11:03] <NodeX> nice
[17:11:20] <shadow_fox> wereHamster: its in /usr/local/bin/mongo/
[17:11:30] <wereHamster> NodeX: 109 to be precise
[17:14:17] <shadow_fox> well?
[17:14:58] <wereHamster> shadow_fox: well what. Use the full path to the binary when starting mongod
[17:15:00] <jcims> wereHamster: amazon ec2 , high memory double extra large is 34gb and >$2hr, rackspace wants $1.80/hr
[17:15:26] <wereHamster> jcims: EC2 never was the cheapest option. They are quite expensive if you compare them to the alternatives.
[17:15:51] <wereHamster> by my last calculations about 10x as expensive as the servers I get at hetzner
[17:16:56] <jcims> this is just a little hobby project, and on the low end ec2 is pretty cheap
[17:17:04] <jcims> <$20us/mo
[17:17:24] <shadow_fox> wereHamster: that worked, now its up and running, could you please tell me how to include the whole path so that i won't have to write the whole path again and again if you don't mind
[17:17:42] <jcims> i do have rack space available for fairly cheap, i'd probably just colocate it if i wanted to crank up the memory
[17:17:46] <NodeX> wereHamster : how do you rate the hardware from them ? ... we're a bit dubious due to it not being enterprise level
[17:18:38] <wereHamster> NodeX: rate how?
[17:19:04] <wereHamster> you mean reliability of the components? Speed?
[17:20:15] <wereHamster> performance?
[17:20:15] <NodeX> all of the above
[17:20:15] <NodeX> or overall
[17:20:39] <wereHamster> well, you get very detailed specification of the hardware. The exact CPU model, whether the memory is ECC or not, whether the disk is enterprise class or not. And you can always do your own benchmark.
[17:20:58] <NodeX> In your experience with them are they reliable?
[17:22:03] <wereHamster> I can't tell. Disks always go bad, even on EC2 machines. But there you don't see it because it's all managed by amazon.
[17:22:22] <NodeX> I would never use amazon in production
[17:23:00] <wereHamster> I've had a few disks go bad on my OVH machine. But a support ticket was all that was needed to have the disk changed.
[17:23:41] <NodeX> we use redstation currently but their support is terrible
[17:39:19] <bindr> So I managed to put up upgrading to 2.2.0 until yesterday and immediately became afflicted by https://jira.mongodb.org/browse/SERVER-6909 which appears to be impacting one of my applications. I see from https://jira.mongodb.org/browse/SERVER/fixforversion/11494 that 2.2.1 with the fix for that was to go out yesterday. Is there an updated ETA for release 2.2.1? Is there a workaround I can put in place until then short of having the devs adapt their
[17:39:19] <bindr> code to better handle the error or not provoke it?
[17:39:29] <bindr> s/put up/put off/
[17:56:48] <pingboy> can someone tell me how to make bson_ext ( presumably ) or the mongo ruby driver not convert my long int into a NumberLong() when an insert takes place?
[17:57:12] <pingboy> mongo ruby driver is 1.7.0 and bson_ext is 1.7.0
[17:57:28] <pingboy> my nodejs app writes longs into the same database and they appear like:
[17:57:35] <pingboy> "created_at" : 1337366101971,
[17:57:41] <pingboy> but my ruby app creates:
[17:57:51] <pingboy> "created_at" : NumberLong("1348629145000"),
[17:57:58] <pingboy> which makes me want to go out to the pet store and buy a puppy so i can kick it in the balls
[18:06:51] <hillct> Good afternoon all. I'm trying to work out the proper syntax for $addToSet where I have an array element in my target document photographers.checkins=[12,5,24] but I can't seem to get the syntax correct. Can someone advise me as to the proper syntax?
[18:08:45] <awc7371> whats the best db viewer for linux?
[18:08:45] <hillct> where this is an update being performed using the findAndModify method
[18:29:16] <Aartsie> Hi guys
[18:29:27] <pingboy> hey
[18:29:36] <pingboy> my kingdom for an answer to this NumberLong() issue.
[18:29:36] <Aartsie> how is it possible that my clean MongoDB is 3,2 GB ?
[18:41:26] <Aartsie> does anyone know why my bluid is 3,2 GB ?
[18:51:27] <awc7371> anyone use an elasticsearch river?
[18:52:07] <_m> awc7371: We do not as it indexes more than we require. But maybe I can help, what's up?
[18:52:38] <awc7371> just wondering if this is a stable / supported solution: https://github.com/richardwilly98/elasticsearch-river-mongodb
[18:53:16] <awc7371> i'm looking at using Lithium PHP, because of native mongo support, and the doc mentions an ES plugin, but I don't see one
[18:53:47] <awc7371> I ask if its stable / supported / a good idea, because I work at a large company and it's going to be a tough sell to my boss
[18:54:18] <_m> That's the one we tried. Seemed stable enough, but it didn't really fit our use-case
[18:54:54] <_m> As for the Lithium PHP… I have no idea.
[18:55:16] <awc7371> do you just use mongo to search? and were you basically duplicating data stored in mongo and es?
[18:57:15] <_m> Our entire dataset is Mongo. However, we only wanted to index specific fields/etc. Rolled our own minimal driver and interface to said driver
[18:57:15] <_m> We also tried tire, which hurt more than it should have.
[18:57:15] <awc7371> lol
[18:57:15] <awc7371> yeah ill probably keep it all php/js
[18:57:15] <awc7371> i'm tempted to use couchDB because of the 'official' river
[18:57:15] <_m> Oh god
[18:57:46] <_m> I've never been a huge fan of CouchDB.
[18:58:00] <awc7371> haha i don't have experience with couch or mongo, so i wouldn't know
[18:58:04] <Aartsie> _m: did you build the last version ? MongoDB 2.2 ?
[18:58:28] <_m> Aartsie: Our systems guy did
[18:58:57] <_m> Aartsie: We build an internal deb based on the official ubuntu package scripts.
[18:59:00] <awc7371> _m: so the mongo river worked well, just wasn't your preference?
[18:59:18] <Aartsie> _m: hmmm my build only is 3,2 GB
[18:59:28] <_m> It worked well enough, yeah. We weren't stoked on how the indexing with said river operated and opted for another route.
[19:02:20] <awc7371> is your app php _m?
[19:03:38] <_m> Absolutely not.
[19:03:52] <_m> Ruby.
[19:04:20] <awc7371> haha
[19:06:35] <_m> PHP is just… no.
[19:07:36] <_m> At Grooveshark, we used sphinx and the mongodb official php driver. Beyond that though, I have no idea which PHP extensions would be bets.
[19:07:38] <_m> *best
[19:08:24] <awc7371> well there is Elastica, a very popular PHP es integration
[19:08:32] <awc7371> i wonder if it would be better to use that than a river
[19:10:45] <pingboy> anyone know how to get the mongo ruby driver and bson_ext to *not* convert stuff to NumberLong()
[19:11:07] <pingboy> like a raw mode or something?
[19:13:26] <_m> awc7371: What ended up being our "best solution" was to insert/update indexes on record updates via a simple driver. This allowed more flexibility and less overheard for indexing in general.
[19:13:34] <awc7371> so you are using es _m?
[19:13:56] <_m> awc7371: Correct. ES, elasticsearch-ruby, elasticized, and mongodb
[19:14:15] <awc7371> ah ok. I misunderstood and thought you ditched ES, but you just ditched the river
[19:14:16] <_m> Not sure if we've open-sources elasticized yet
[19:14:28] <awc7371> so that is your driver
[19:14:29] <_m> Yeah. Just the river. It was overreaching for our requirements.
[19:15:18] <awc7371> so by your method, it sounds like the PHP client might be better for my case, and just manually adding indexes
[19:15:21] <awc7371> is deleting a problem?
[19:15:57] <_m> The driver handles that as we remove records.
[19:16:56] <awc7371> nice, Elastica probably would provide all that for me
[19:17:05] <awc7371> with a river, i imagine deleting WOULD be a problem
[19:18:40] <awc7371> so you pretty much just CRUD everything twice?
[19:28:24] <hillct> What goes on here? https://jira.mongodb.org/browse/SERVER-831 Major issues go unassigned for two years?
[20:06:40] <oskie> hillct: looks like a major *feature request* to me
[20:07:53] <hillct> oskie: ok, you could look at it that way, or recognize that this sort of query is an obvious usage so the lack of the feature is itself a bug
[20:08:26] <hillct> It's like saying "SQL doesn't support spending strings fields"
[20:39:27] <Guest__> How can I be sure that an application is connecting to my mongodb database? I've got "mongod" up and running, but never see any text output to it, even after failing to login to the application (I would assume it would show failed logged in attempts, since they access the database)
[20:45:21] <_m> awc7371: Essentially, yeah.
[20:46:03] <Guest__> Hmm, cause the app is configured to use pow.cx, so I guess it isn't connecting then? I'm so confused, haha.
[20:48:36] <_m> Guest__: Write a very simple endpoint to test this. If you're using rails, fire up the console and try to select/save/update a record.
[20:49:56] <AndreKR> Can anyone explain this to me...
[20:50:08] <AndreKR> On http://www.mongodb.org/display/DOCS/Schema+Design it says: "If finding the most recent posts is a very frequent query, we would then shard on the when field."
[20:50:52] <AndreKR> Doesn't that mean that all the "recent posts" queries end up on one shard, so all the load is on one node?
[20:51:21] <AndreKR> Why is "when" a good shard key then?
[20:53:14] <_m> Sort order/indexing for time-based selections
[21:48:56] <owen1> Error: couldn't connect to server 127.0.0.1:27017 src/mongo/shell/mongo.js:91
[21:48:57] <owen1> exception: connect failed
[21:49:01] <owen1> (centos)
[21:49:15] <owen1> (i created /data/db folder, just case)
[22:11:26] <Bilge> >centos
[22:11:27] <Bilge> :(
[22:11:36] <Bilge> Why would you do that
[22:23:55] <owen1> Bilge: not my call ):
[22:24:09] <owen1> my heart is made of debian
[22:24:24] <Bilge> Mine also
[22:24:32] <owen1> but it's for work
[22:29:10] <_m> owen1: I know this is simple, but you're certain the daemon is actually running?
[22:29:24] <owen1> _m: it's not
[22:29:43] <_m> What happens when you run it interactively? There's generally a wealth of information in the crash logs
[22:30:27] <owen1> _m: i believe that the way to run it interactively is 'mongo'
[22:30:46] <owen1> that's where i see the error: connection failed
[22:32:43] <bindr> That's not right.
[22:33:04] <bindr> mongod is the mongo daemon and mongo is a CLI client to talk to mongod
[22:34:30] <owen1> bindr: true. so 'mongo' is a CLI client. that's the mongo shell, right?
[22:34:35] <Bilge> I want to be the mongod
[22:35:18] <bindr> mongo won't make you happy unless mongod is running
[22:35:31] <owen1> typing it shows me: MongoDB shell version: 2.2.0
[22:35:33] <owen1> connecting to: test
[22:35:35] <owen1> Wed Sep 26 22:28:23 Error: couldn't connect to server 127.0.0.1:27017 src/mongo/shell/mongo.js:91
[22:35:37] <owen1> exception: connect failed
[22:35:52] <Bilge> WELL STOP BEING A DERP
[22:36:03] <Bilge> RUN NETSTAT -PLNT | GREP 27017
[22:36:06] <Bilge> IF YOU SEE NOTHING
[22:36:09] <Bilge> YOU ARE BEING A DERP
[22:37:25] <owen1> Bilge: yeah. derp
[22:38:59] <owen1> Bilge: so it's not running. how do i run it? on the mongo's website it say: service mongod start
[22:39:13] <owen1> i don't have the service command
[22:40:21] <mgriffin> owen1: which os/distro?
[22:40:27] <owen1> mgriffin: centos
[22:40:32] <owen1> http://docs.mongodb.org/manual/tutorial/install-mongodb-on-redhat-centos-or-fedora-linux/
[22:40:44] <owen1> that's where it say to use service
[22:40:45] <mgriffin> owen1: you should have a service command, did you first "su -"
[22:41:03] <mgriffin> you may have done "su" which did not use the super user paths
[22:41:52] <owen1> mgriffin: are u asking if i run 'su -' and after that 'service mongod start' ?
[22:42:00] <mgriffin> no.
[22:42:04] <mgriffin> before
[22:42:32] <owen1> i never used the su - command since i can't. i can use sudo.
[22:42:37] <mgriffin> sudo su -
[22:43:14] <owen1> mgriffin: ok. now i see #root
[22:43:21] <mgriffin> and the service command now exists?
[22:43:50] <owen1> mgriffin: no. command not found
[22:44:07] <mgriffin> use the init script instead then
[22:44:20] <mgriffin> /etc/init.d/mongod start
[22:44:35] <mgriffin> maybe upstart got rid of /sbin/service or something.
[22:45:00] <bindr> Centos has upstart?
[22:45:07] <owen1> mgriffin: i don't think my centos uses upstart btw, i think it's sysinit-v
[22:45:17] <owen1> bindr: they have it in 6.x i think
[22:45:24] <owen1> but mine is 5.x
[22:45:31] <bindr> Ah. Good to know.
[22:46:51] <mgriffin> # rpm -qf /sbin/service
[22:46:52] <mgriffin> initscripts-9.03.31-2.el6.centos.x86_64
[22:49:44] <owen1> mgriffin: i don't see mongod inside init.d
[22:50:16] <owen1> what is that initscript-9.03.31-2 ...
[22:56:00] <mgriffin> btw, the box i ran that on is using upstart and does have the service command..
[22:56:13] <chiel> hi all. i'm using the ruby driver to query mongodb and i was wondering how i would filter based on a property that's not at root level. i assume this is possible?
[22:56:16] <mgriffin> owen1: you seem to have a strange configuration.
[22:56:33] <mgriffin> owen1: i would perhaps suggest asking #centos for help
[22:57:16] <owen1> mgriffin: ok. thanks!
[22:57:26] <owen1> they actualy don't deal with mongo
[22:57:35] <mgriffin> you are not asking mongo questions
[22:58:02] <mgriffin> another repo they might be more familiar with is EPEL http://dl.fedoraproject.org/pub/epel/6/x86_64/repoview/mongodb-server.html
[22:58:10] <mgriffin> but i dont think the repo is your problem.
[22:59:46] <mgriffin> owen1: actually, do you have a /etc/rc.d/init.d/mongod ?
[23:00:05] <mgriffin> if so, then it was just not added to chkconfig
[23:00:21] <mgriffin> so chkconfig --add mongod should create /etc/init.d/mongod
[23:00:29] <mgriffin> as to the missing service command, no idea.
[23:01:10] <owen1> mgriffin: i don't have the mongod file
[23:01:28] <mgriffin> then you didnt install the packages?
[23:02:27] <mgriffin> im surprised mongo docs dont use something simpler like rpm -Uvh http://downloads-distro.mongodb.org/repo/redhat/os/mongo10gen-release.rpm && yum install mongo-10gen mongo-10gen-server
[23:03:00] <mgriffin> owen1: do rpm -q mongo-10gen-server
[23:03:13] <mgriffin> what is the output?
[23:03:20] <owen1> mgriffin: 1 sec
[23:04:39] <owen1> package mongo-10gen-server is not installed
[23:05:15] <mgriffin> told you.
[23:05:22] <mgriffin> rpm -qa | grep -i ^mongo
[23:05:43] <_m> chiel: Same as in the mongo shell. db.foo.find({'bar.baz.gaz' : "asdf" })
[23:06:04] <owen1> mgriffin: ongo-10gen-2.2.0-mongodb_1
[23:06:19] <chiel> _m: i didn't know how to do it in the shell either, hehe. :p
[23:06:21] <chiel> thanks
[23:06:25] <_m> No worries.
[23:07:11] <owen1> mgriffin: that's what i did previously: yum install mongo-10gen mongo-10gen-server
[23:07:16] <owen1> that's what the docs say.
[23:07:41] <owen1> mgriffin: should i give it another try?
[23:09:23] <mgriffin> looks like client but not server installed.
[23:09:51] <_m> # which mongod
[23:10:05] <_m> (assuming it's on your path)
[23:10:57] <owen1> _m: /usr/bin/which: no mongod in (/home/t/bin:/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin)
[23:11:22] <owen1> this box is suppose to use tpkg but i installed stuff using yum
[23:11:49] <owen1> that's why u see /home/t/bin
[23:12:07] <owen1> whenever i install with tpkg it install it there
[23:12:10] <_m> Doesn't matter to me what your path is. Could include /dev/null for all that matters. =P
[23:13:14] <nickmbailey> nice
[23:13:21] <nickmbailey> oops, wrong window
[23:14:24] <_m> Ensure your packages installed correctly and give it another go.
[23:14:34] <_m> According to the docs you linked, the init script should be at
[23:14:43] <_m> : /etc/rc.d/init.d/mongod
[23:15:21] <owen1> nothing there. i'll just use one of our tpkgs..
[23:15:24] <owen1> thanks a lot!
[23:28:03] <_m> You're welcome! Hopefully that points you in the right direction.
[23:42:25] <kisielk_> does mongodb handle lots of connections coming and going well? or is it better to write a service that connects to it once and send data there?
[23:44:38] <bindr> So in the unlikely event anyone noticed my earlier question about 2.2.1, I resolved my issue with the findAndModify error in 2.2.0 by swapping in the 2.2.1-rc0 binaries from dl.mongodb.org and that seems to have been fine. That should hold me until there's an actual release of 2.2.1
[23:46:51] <_m> kisielk_: I've found it better to use a persistent connection.
[23:47:36] <kisielk_> I have a batch processing cluster, it can run about 400 jobs simultaneously, and each one will need to send its data to mongo
[23:47:47] <kisielk_> so I guess it would be better to funnel them through some central point?
[23:49:53] <_m> We use one connection per machine.
[23:53:47] <kisielk_> _m: that's a good idea actually, I can run a daemon on each node that will forward the files
[23:54:33] <_m> Glad I could help. =)