PMXBOT Log file Viewer

Help | Karma | Search:

#mongodb logs for Monday the 28th of October, 2013

(Back to #mongodb overview) (Back to channel listing) (Animate logs)
[00:20:42] <defunctzombie> is there a way to force disconnect unauthorized users after a specific time?
[00:20:54] <defunctzombie> so they are not allowed to maintain a connection to the server?
[04:08:45] <safani> hello
[04:08:47] <safani> I want to sort products based on each category..this means a product may have a different sort order for each of the categories it is in
[04:09:00] <safani> what might be the best way to implement this
[10:21:32] <quattr8> I have a shared collection with a hashed _id shardkey does this mean I can't add datacenter aware sharding iwth a shardkey?
[10:22:03] <quattr8> *with a shardtag
[11:44:57] <sQVe> I would love some advice on how to handle search in this case: I have a very large database with movies. The titles of the movies need to be searchable. My question is if this should be done with regex, full text search or another method that I do not know of?
[11:45:32] <sQVe> Any ideas? The movie DB is around > 200k records.
[12:23:34] <Nodex> sQVe : mongodb is probably not the right tool for that
[12:24:39] <sQVe> Nodex: What would you recommend then?
[12:25:24] <Nodex> something more suited to full text searching
[12:25:33] <Nodex> solr / elastic search maybe
[12:26:21] <sQVe> Yeah, I've looked at those options. It just seems a bit overkill when I only need the title to be searchable, and nothing else.
[12:27:16] <Nodex> Mongo can and will do it, it's just not the best tool for it and won't be as relevant as somehting dedicated
[12:28:35] <sQVe> Allright, I guess I'll have to look into elasticsearch and check it out.
[12:29:21] <Nodex> personaly I would recommend solr due to it's longevity, I found ES to be a little green when I last looked at it
[12:30:11] <sQVe> I might skip the movie DB all together and work with the TMDB API instead.
[12:30:24] <sQVe> I just dont like the limitations.
[12:30:41] <lineux_007> hey! quick qn. if i delete a capped collection and create a new capped collection (of the same size), will a new data file be created or will the space be reused?
[12:35:25] <cym3try> I have configured my first replica set on a testing environment. If the primary goes down (localhost:27017), and one of the secondary takes over (localhost:27018), how can i make sure that the failover is transparent to the app? can I use haproxy?
[12:39:06] <cym3try> ...my app is written in php
[12:41:48] <Nodex> pretty sure you can configure that in your connection string
[12:49:23] <lineux_007> any thoughts? if i delete a capped collection and create a new capped collection (of the same size), will a new data file be created or will the space be reused? do i have to keep doing repair db?
[12:54:38] <poff> Hi, i wanted to do something like this: db.PlayerRelation.aggregate( { $group: { _id: { $concat: [ "$who.getId().toString()"," ","$user.getId().toString()" ] } } } ); where $who and $user are both a DbRef, but it isn't properlly converting the DbRef to a String
[12:54:57] <poff> any ideia how to fix this?
[12:55:41] <kali> wow.
[12:55:47] <Nodex> I don't think you have access to those kind of commands
[12:55:51] <kali> aggregate will not do javascript
[12:55:55] <Nodex> (in aggregation framework)
[12:56:27] <kali> and certaintly not resolve dbref
[12:57:21] <poff> what i wanted to do was a group by of the concat of those 2 ids
[12:57:58] <poff> and if i do the $concat of $who and $user i get a warning saying that it must be a string
[12:58:20] <poff> so what i need is a way to convert a DbRef to string inside the aggregate
[12:59:09] <Nodex> you can't
[13:00:22] <poff> okay thanks
[13:01:34] <Nodex> dbRef's are a convention, they have no special meaning in anything except some ORM/ODM's
[13:01:48] <Nodex> probably better to just store the id as a string
[13:03:23] <poff> but if a do something like this on the comand line it works: DBRef("Player", ObjectId("51a5e867d80041e429000000")).getId().toString();
[13:03:39] <poff> it gives me: ObjectId("51a5e867d80041e429000000")
[13:04:39] <poff> maybe i can accomplish that using map reduce than
[13:04:40] <Nodex> aggregation framework has no javascript methods available
[13:05:40] <poff> yup, thanks for the help
[13:59:48] <hyperboreean> hey, I have the following structure: 'key1': {'key2': []} ... I want to know if key2 exists and doing {'key1.key2': {'$exists': True}} doesn't seem enough (using pymongo here)
[13:59:51] <hyperboreean> any ideas ?
[14:00:04] <elux> g`morning
[14:00:16] <elux> on http://docs.mongodb.org/manual/core/index-text/ .. it says to now use text search in production...? lol.. is that a joke?
[14:00:30] <elux> so text search in mongodb shouldnt be used?
[14:00:49] <Derick> elux: it has been released as a technology preview.
[14:01:53] <elux> ok cool
[14:14:50] <kurtis> Hey guys, when doing large read/write operations, I notice the database gets heavily grid-locked. Is there a way to "de-prioritize" certain operations? Especially in the context of using an external (or even internal) map-reduce operation?
[14:19:17] <eldub> mongod dead but pid file exists -- how do I avoid this?
[14:20:24] <Nodex> did somehting kill the operation?
[14:20:50] <eldub> Fresh boot, just have mongod on chkconfig
[14:20:57] <eldub> so not that I know of.
[14:21:13] <cheeser> sounds like an dirty shutdown
[14:21:56] <ron> you like a dirty shutdown, don't you cheeser? ;) :*
[14:22:08] <cheeser> when it gets cold enough
[14:22:23] <eldub> I don't think it's a dirty shutdown... I'll chkconfig off reboot and then just start mongod I guess... see if that does it
[14:43:06] <eldub> I just installed mongodb and started it... as soon as I status the service... it's already in a mongod dead but pid file exists....
[14:43:26] <cheeser> how do you start it?
[14:43:33] <eldub> service mongod start
[14:43:57] <cheeser> ok. stop it just so 'service' doesn't have it listed as running and remove that pid.
[14:44:15] <eldub> done.
[14:44:30] <cheeser> then start it manually with "mongod" and see what errors you get.
[14:44:53] <ron> you should work as support for mongodb
[14:44:57] <t0th_-> hi
[14:45:10] <t0th_-> i am trying use db.lnp.runCommand( "text", { search : "apple" } );
[14:45:11] <cheeser> ron: funny you should mention that
[14:45:17] <ron> ;)
[14:45:20] <t0th_-> but i have this error: "errmsg" : "no such cmd: text",
[14:45:22] <t0th_-> how i can solve?
[14:45:29] <cheeser> are you on 2.4?
[14:45:41] <t0th_-> me?
[14:45:44] <eldub> mongod
[14:45:44] <eldub> mongod --help for help and startup options
[14:45:44] <eldub> mongod: symbol lookup error: mongod: undefined symbol: _ZN7pcrecpp2RE4InitEPKcPKNS_10RE_OptionsE
[14:45:48] <cheeser> t0th_-: yes
[14:46:00] <cheeser> eldub: um. wow. what?
[14:46:12] <eldub> That's what I get when I try starting with just mongod
[14:46:30] <cheeser> looks like a bad build. is this a nightly?
[14:46:34] <t0th_-> https://gist.github.com/anonymous/0444ff4ca16334255a09 ?
[14:46:50] <cheeser> t0th_-: you need at least 2.4 for text search
[14:46:55] <t0th_-> hmmmm
[14:47:04] <cheeser> http://docs.mongodb.org/manual/core/text-search/
[14:47:14] <t0th_-> what is the diference text search and find with regex?
[14:47:24] <cheeser> indexing for starters.
[14:48:12] <t0th_-> ?
[14:49:44] <cheeser> regular expressions can't really use indexes.
[14:50:02] <cheeser> with text search you get more advanced indexes suitable for searching
[14:50:29] <t0th_-> with text search can i use full text search?
[14:50:40] <t0th_-> now i my project i am using sphinx search
[14:50:41] <ron> forget support, you should do marketing!
[14:50:49] <cheeser> for some value of "full." that link has more info.
[14:51:43] <cheeser> ron: i'm spending most of next month on the road doing that.
[14:51:54] <ron> money well spent.
[14:59:25] <t0th_-> can i change my sphinx full text search with mongodb ?
[15:02:31] <whattata> I've got a question: If I have a document that contains and array, and I want to be able to reference any given object within that array directly, is there a way to have mongo assign an id to each member of the array?
[15:14:46] <Nodex> not automagically
[15:16:10] <t0th_-> can i change my sphinx full text search with mongodb ?
[15:16:43] <Nodex> eh?
[16:28:51] <t0th_-> hi
[16:33:05] <t0th_-> can i use wild card in text search ?
[16:42:50] <t0th_-> https://jira.mongodb.org/browse/SERVER-10413?jql=text%20~%20%22wild%22
[16:50:46] <Nodex> t0th_- : if you;re system really needs good text search then you should use the right tool for that
[16:52:14] <eydaimon> What's the better architecture if my application is the host of multiple other applications: a) use a separate database for each application I'm hosting, or b) use one database and use collection prefix to keep the collections separate for each app.
[16:52:16] <eydaimon> ?
[16:55:09] <cheeser> at my previous job, we'd just tag each entity with a customer specific ID and include that in all our queries
[16:56:06] <eydaimon> We need to have some separation in order to be able to bill for diskspace used etc
[16:56:19] <eydaimon> It seemed easier to separate it out by collection
[16:56:34] <eydaimon> Would there be any performance concern with doing a or b?
[16:56:47] <cheeser> that's probably true in that case. i might consider separate DBs at that point
[16:57:25] <cheeser> then your application code needn't be aware of the customer. you'd just have to connect to the correct database and then everything is simple.
[16:58:15] <eydaimon> Is there any concern with having thousands of databases then? We may host a lot of apps
[16:58:31] <cheeser> i wouldn't think so but I don't know 100% on that one
[16:58:55] <eydaimon> I also figured that if we do per database, there may be some internal sharding logic that is actually in favor of using 'a' as opposed to 'b'
[16:59:04] <cheeser> probably, yeah
[17:04:00] <Nodex> we tag everything with a client id which tracks all over the system
[17:04:46] <eydaimon> Nodex: are you able to see how much resources a client uses then?
[17:04:53] <eydaimon> That's a requirement for me
[17:06:56] <smw_> Is there a good guide on master/slave vs replica sets?
[17:07:09] <Nodex> eydaimon : in terms of space in collections?
[17:11:02] <eydaimon> Nodex: yes
[17:12:31] <kali> smw_: replica set is the way to go except for a handful highly specific scenario (if you need more than ~10 read slaves for instance)
[17:13:20] <cheeser> master/slave is deprecated, no?
[17:14:48] <kali> cheeser: i'm not sure it's "officialy deprecated", as it helps a few of these very borderline setups
[17:15:38] <kali> cheeser: http://docs.mongodb.org/manual/core/master-slave/ it does not say "deprecated" :)
[17:16:34] <Nodex> eydaimon : We can but don't, it's easy to aggregate
[17:17:07] <eydaimon> Nodex: SO you can aggregate each document easily?
[17:17:11] <eydaimon> Nodex: like each doc size?
[17:18:12] <Nodex> not in the AF (as far as I know) but a simple loop can gather document size
[17:19:25] <eydaimon> Nodex: So do you think I should add a c) to my question and c is the correct answer?
[17:20:28] <smw_> kali, I have two nodes. If one goes down, will both stop working?
[17:21:02] <smw_> kali, stop working = stop accepting writes. I am sure they will accept reads
[17:22:00] <smw_> kali, ah! I neex an arbiter
[17:40:20] <Nodex> eydaimon: I missed what "c)" was
[17:41:41] <eydaimon> Nodex: what you're doing. one database, and using client id to separate data
[17:41:57] <Nodex> I don;t know your use case
[17:44:01] <smw_> how can I add a role to an existing user?
[17:44:13] <smw_> or modify their roles in any way
[17:44:25] <smw_> so far I have only been able to learn how to ad a user :-\
[17:44:52] <Nodex> eydaimon : in our setup we have a database per client (for their own stuff) and a master database that holds all data (multi tenant system) if that makes sense
[17:45:47] <smw_> I want to give my user readWriteAnyDatabase, userAdminAnyDatabase, dbAdminAnyDatabase and clusterAdmin which should make me all powerful
[17:48:52] <eydaimon> Nodex: yeah, so it sounds like you're doing a) then. How many databases do you have?
[17:49:04] <eydaimon> Nodex: and is there a limit that I shoudl worry about?
[17:51:09] <Nodex> per machine we limit it to 50 per box but that;s purely for managability and redundancy
[17:51:50] <eydaimon> ok, thanks
[18:20:47] <Sodu> I'm trying to add an index to a collection {tags: [ {name:'php', id:100}, {name:'java', id:101} ]}
[18:21:12] <Sodu> can it be done so it can search by tag name using an index?
[18:29:34] <cym3try> how do i cap the oplog size from the mongod.conf file directly?
[18:29:51] <cym3try> I tried the oplogSize option but it's still writing big oplogs
[18:31:27] <cym3try> ahh nevermind...it's in MB
[19:02:18] <eldub> vi /etc/mongodb.conf
[19:02:20] <eldub> oops
[19:12:21] <eldub> Is there a guide anywhere that describes setting up a 3 node replica set from scratch?
[19:12:32] <eldub> I'm currently following http://docs.mongodb.org/manual/tutorial/deploy-replica-set/
[19:12:54] <eldub> but when I type 'mongo' in... keeps erroring out about 'couldn't connect to server 127.0.0.1
[19:13:00] <tystr> then your mongod isn't running
[19:13:28] <eldub> tystr it says it's running -
[19:13:38] <tystr> different ports maybe?
[19:13:49] <eldub> I think there's confusion. in my mongodb.conf I specified an IP address, and when I type mongo it appears to be looking for 127.0.0.1
[19:13:54] <eldub> when I already changed that in the mongodb.conf
[19:14:09] <tystr> well if edit mongodb.conf you need to bounce mongod
[19:14:15] <eldub> I did
[19:14:33] <tystr> mongo will try 127.0.0.1 unless you give it a host
[19:14:36] <tystr> mongo some.host.net
[19:14:56] <eldub> so should I leave the mongodb.conf bind_ip to 127 to avoid this?
[19:22:27] <eldub> tystr when I put the IP bind back to 127.0.0.1 it starts... but when I put the IP of that machine 10.x.x.x it doesnt.
[19:22:35] <eldub> still looks for the 127.0.0.1 it seems.
[19:23:25] <tystr> b/c it's listening on 10.x.x.x
[19:23:39] <eldub> But that's where I want it to listen to...
[19:23:50] <tystr> then do mongo 10.x.x.x and it should connect
[19:23:53] <JakePee> If a mongocursortimeoutexception is encountered in the php libs, does the connection survive?
[19:23:54] <tystr> if your network settings are correct
[19:25:25] <eldub> I always get 'mongod dead but subsys locked'
[19:28:49] <tystr> then check your mongod.log
[19:28:56] <tystr> sounds like it's not starting up correctly
[19:28:59] <eldub> im checking now
[19:29:02] <tystr> or something is happening
[20:46:52] <eydaimon> Nodex: hey, when you said you have 50 clients per box, do you mean just that you add new hosts if you get more clients, or what exactly?
[21:22:50] <eldub> Does anyone have a guide for setting up a replica set with 3 nodes?
[21:33:20] <cheeser> eldub: http://docs.mongodb.org/manual/administration/replica-sets/
[21:37:10] <eldub> I'll try that again. When I made my config file that, it wouldnt allow me to start mongo. Had some sort of dead locks
[21:38:17] <rafaelhbarros> u
[21:38:27] <rafaelhbarros> frak, wrong window
[21:51:31] <IrishGringo> is anyone doing week 4 of the 10Gen class for mongo administrators...
[21:51:32] <IrishGringo> ?
[21:52:09] <retran> sounds likea good idea
[21:52:36] <JakePee> Is there a specific reason in PHP driver, if a find() doesn't have any results, trying to iterate over the cursor throws an exception instead of acting like a 0 element iterator (and just breaking iteration)
[21:53:18] <bjori> what
[21:53:21] <bjori> no
[21:53:33] <bjori> JakePee: what version are you using?
[21:54:02] <bjori> JakePee: ..and what exactly is the exception msg?
[21:58:15] <JakePee> Warning: Invalid argument supplied for foreach() in /home/jake/sandbox/app/bin/scrm/test.php on line 5
[22:01:15] <bjori> JakePee: you are doing something wrong
[22:01:15] <JakePee> Oh wait, my fault
[22:01:26] <bjori> :)
[22:01:44] <JakePee> Yeah, passed the wrong object as the first element in the foreach
[22:03:32] <bjori> so.. we all good? :)
[22:07:56] <eldub> I found out why I was getting subsys locks on my mongodb every so often (mainly on reboots)
[22:08:10] <eldub> It's because the log file is owned by root:root and mongodb can't write to the log file...
[23:36:01] <JakePee> bjori: yup, thanks :)
[23:56:31] <livinded> When setting up a replica set do I need to run rs.initiate() on any other nodes/arbiters I'm going to add from the primary?
[23:56:43] <joannac> no
[23:56:58] <joannac> only run rs.initiate() once
[23:57:49] <livinded> I added an arbiter and a second node and the arbiter has stateStr of "UNKNOWN" and the second node "(not reachable/healthy)"
[23:58:13] <joannac> What does the mongod log say?
[23:58:48] <livinded> failed authentication
[23:59:38] <livinded> the docs didn't mention anything about authentication. Is there somewhere I need to configure that?
[23:59:44] <joannac> Did you start your mongods with --auth ?