pmxbot IRC Log Viewer

[00:19:09] <aknewhope> hello

[00:39:02] <w3pm> hi

[00:39:10] <Zelest> o/

[00:41:56] <owen1> hi

[03:02:14] <JoeyJoeJo> How can I select partial matches? For example, if a document is {abcd:efghi}, how can I select it by just searching for db.collection.find({abcd:efg})?

[03:16:47] <IAD> JoeyJoeJo: db.collection.find({'abcd': {'$regex' :'efg'}})

[03:17:40] <IAD> JoeyJoeJo: http://docs.mongodb.org/manual/reference/operator/regex/

[03:33:35] <JoeyJoeJo> IAD: Thanks, I didn't know about $regex

[03:36:41] <toothr> does a full scan unless you use a case sensitive rooted (eg: using ^) regex

[03:39:22] <JoeyJoeJo> One more question - I have 4 sharded servers and a collection with about 183 million documents. The servers are each 32 cpu cores, 256GB of RAM and raided SSDs (not sure which level). I did a query which resulted in 3304 results, but the query took about 20 seconds. How can I speed that up?

[03:40:10] <JoeyJoeJo> The query was db.data.find({"date":"2012-12-02 10:27:45"}).count()

[03:40:24] <toothrot> you have an index?

[03:40:29] <JoeyJoeJo> No

[03:41:40] <JoeyJoeJo> I can create an index with ensureIndex(), right?

[03:41:56] <JoeyJoeJo> db.data.ensureIndex('date')

[03:41:58] <toothrot> yes

[03:42:04] <toothrot> also, what version of mongodb are you using?

[03:42:15] <JoeyJoeJo> 2.2.2

[03:42:26] <toothrot> https://jira.mongodb.org/browse/SERVER-1752

[03:43:11] <toothrot> also, you may consider precalculating that as you go using $inc, etc

[04:03:46] <sirious> JoeyJoeJo: is every document guaranteed to have 'date'?

[04:09:31] <JoeyJoeJo> sirious: yes, every document has a date

[04:10:23] <sirious> ok, no need to suggest sparse then :)

[08:34:51] <[AD]Turbo> hola

[10:38:15] <remonvv> \o

[10:43:09] <ragsagar> How can i mention the year of a datetime field 'processed_time' in aggregation framework? '$processed_time.year' ?

[10:46:40] <kali> { $year: "$processed_time" }

[10:46:52] <kali> (i think)

[10:52:57] <ragsagar> kali, In which operator can I use it like that? I want the count of objects processed in a particular year.

[10:53:20] <kali> ragsagar: $project the year, then $group on the value

[10:53:47] <ragsagar> kali, thanks, let me try out.

[10:55:58] <ragsagar> kali, can I use '$year': '$processed_time' in $project because I am getting error $expressions are not allowed at the top level of $project.

[10:56:53] <kali> ragsagar: try { $project : { y : { $year: "$process_time "} }

[10:59:49] <ragsagar> kali, Thanks, that worked!

[11:11:56] <jwilliams__> i follow the doc at http://docs.mongodb.org/manual/tutorial/convert-replica-set-to-replicated-shard-cluster/

[11:12:57] <jwilliams__> and setup a replica set by issuing db.runCommand({"replSetInitiate" : {"_id" : "first", "members" ...});

[11:13:12] <jwilliams__> it works as described in the doc.

[11:13:27] <jwilliams__> but how can i publish an existing collection to that replocation set?

[11:14:15] <jwilliams__> the doc descirbes creating a new collection. but what i need is replcating an existing collection.

[11:18:32] <ragsagar> kali, Is it possible to use $match operator to filter based on an object id

[11:33:40] <oskie> hello, i've done a snapshot backup (LVM), and now I want to run mongodump to make smaller backups. MongoDB will think it has shut down incorrectly. Can I just run 'mongodump' with --dbpath on the offline instance data?

[11:46:12] <oskie> I guess I could just do 'mongodump' on the live database instead of going LVM route, but IIRC, that would take ages which is why I ended up with --forceTableScan which is bad.

[11:46:40] <ragsagar> Is is possible to concatenate two variables given by $project operator inside $group in aggregation framework?

[12:28:45] <remonvv> ragsagar, $concat is supported in 2.3+ iirc

[12:45:01] <ragsagar> remonvv, thanks

[14:36:03] <mansoor-s> Who would you guys recommend as a managed MongoDB host?

[14:36:08] <mansoor-s> or not* recommend

[14:36:22] <NodeX> saas?

[14:36:24] <ron> I wouldn't recommend NodeX.

[14:36:35] <NodeX> I wouldn't recommend ron because he is dumb

[14:36:42] <ron> that's true.

[14:36:53] <mansoor-s> o_O

[14:37:18] <mansoor-s> I dont kknow if I should go fully managed

[14:37:27] <mansoor-s> or I should set it up on my EC2 instance

[14:37:38] <NodeX> if you have the know how then save the money

[14:37:49] <mansoor-s> not really..

[14:37:51] <NodeX> the majority of these hosted solutions I've seen are very expensive

[14:37:59] <mansoor-s> thats what i've been seeing

[14:38:09] <mansoor-s> I'm doing this for a client, and I won't be around to mintain it

[14:38:20] <mansoor-s> so i want a setup and forget solution

[14:38:52] <NodeX> if the client is paying then get the most expensive LOL

[14:39:42] <mansoor-s> well :p

[14:39:50] <mansoor-s> they aren't exactly made of money

[14:39:57] <mansoor-s> its a startup

[14:40:00] <mansoor-s> non-funded

[14:40:20] <NodeX> https://www.mongohq.com/home

[14:40:25] <NodeX> and there's mongolab

[14:40:37] <mansoor-s> mongohosting.com ohhh

[14:40:40] <mansoor-s> this looks nice

[14:40:42] <NodeX> but very expensive

[14:41:54] <NodeX> 5gb for $50 a month LOL, thse guys must me minted

[14:45:03] <mansoor-s> i read that as 50 :(

[14:46:12] <NodeX> one of my clients data would cost $800 a month just in mongo hosting :/

[14:47:03] <mansoor-s> http://hostedmongo.com/

[14:47:09] <mansoor-s> now this is competative

[14:48:57] <revoohc> anyone have any metrics on how much overhead there is in a shared environment vs just a replica set? i.e. how much does running through mongos cost?

[14:54:03] <NodeX> mansoor-s : I would avoid companies that can't even create a working website!

[14:54:12] <mansoor-s> lol

[14:54:52] <NodeX> really doesn't say much for their other more complex operations

[14:54:59] <mansoor-s> indeed

[14:55:05] <Zelest> does MongoDB run on a ARM processor?

[14:55:10] <Zelest> a/an

[14:55:42] <kali> there was a guy around here last week, he was looking for help in fixing ARM compilation

[14:55:47] <kali> so my guess is "no"

[14:55:56] <Zelest> Ah

[14:56:13] <Zelest> Thought of dev'ing on my raspberry ;)

[14:56:30] <Zelest> Perfect platform to code on.. if it runs smooth there, it sure as hell runs awesomely on a "normal" computer :D

[14:56:53] <kali> next step is arduino :P

[14:57:00] <kali> 2k ram

[14:57:01] <NodeX> Raspberry pi ftw

[14:57:05] <kali> this is a challenge.

[14:57:15] <NodeX> XBMC + Rpi ;D

[14:57:29] <Zelest> :-D

[15:01:57] <tom0815> hello, i have a question related to replicaset: in my setup i have 3 mongod nodes + 1 mongod in another data center on a different country. the three nodes are in one data center and they are syncing fine, but the node in the other data center has not sync at all. i checked the network with ping (latency ~ 100ms) and nmap (ports are opened in both directions). can someone give me a hint what goes wrong and how to go on? mongod version is v2.2.2

[15:02:24] <karptonite> if I am doing a db.collectionname.find().count(), is there any increase in speed if I specify in the find query a list of fields with just _id in it, since I'm not really retrieving any records at all? or should speed either way?

[15:03:17] <NodeX> it will save the default first cusor size over the wire

[15:03:28] <NodeX> so it will be faster as it's not transferring that data

[15:03:34] <NodeX> cursor *

[15:04:35] <revoohc> anyone know how you can calculate your active working data set size?

[15:05:41] <NodeX> http://docs.mongodb.org/manual/faq/diagnostics/#faq-memory

[15:11:00] <karptonite> NodeX: not sure I understand. Are you saying that it will be faster if I DO include the query limiting the fields?

[15:13:42] <NodeX> yes because it's sending less data on the initial return

[15:14:01] <NodeX> that's what "so it will be faster as it's not transferring that data" means ;)

[15:15:53] <karptonite> NodeX: thanks--Since adding the query means transferring a bit more on the way TO the server, I wasn't sure. Is this a substantial amount of data? It sounds like something that would be negligible, but if it could be substantial, i could easily add it to my counts.

[15:24:09] <NodeX> it depends how much data is in your documents

[15:24:47] <NodeX> some people say that over-optimisation is a bad idea, I say there is no such thing as over optimisation and every call and query should be as fast as it can be

[15:25:59] <kali> and all general statements are wrong.

[15:26:37] <NodeX> which is kinda the point or the concept to most nosql problems...

[15:26:42] <NodeX> each case is different

[16:26:50] <schlitzer|work> hey folks

[16:27:09] <schlitzer|work> is there a way to see how big each collection is?

[16:27:22] <kali> db.foo.stats()

[16:27:24] <schlitzer|work> i think there was a way, but i cannot remember

[16:27:34] <schlitzer|work> ahh, thank you

[16:31:12] <Gargoyle> Is there a way to pipe the restore information into "mongorestore"? (trying to write a 1 line wget > tgz > mongorestore command)

[16:54:48] <NodeX> Gargoyle : you can pipe anything into anything on the command line

[16:55:05] <NodeX> in bash do.... echo "My arguments" | mongorestore

[16:55:21] <Gargoyle> I think it was the fact that I don't have wget! ;)

[16:55:37] <Gargoyle> mongorestore was correctly barfing on "Command not found"

[16:55:39] <Gargoyle> :P

[16:55:43] <UForgotten> 15936 Creating collection x failed Errmsg: exception: specify size:<n> when capped is true

[16:55:43] <NodeX> ah

[16:55:43] <UForgotten> db.collection.isCapped()

[16:55:43] <UForgotten> false

[16:55:44] <UForgotten> wtf

[16:55:51] <NodeX> pastebin

[17:57:12] <owen1> let's say one host in my replica died. what is the procedure of bringing him back? rm -r /data/db and start mongod?

[18:05:21] <JakePee> trying to write a script that makes updates, and then waits for the oplogs to catch up. Is 'sleep(ms)' database related or does it only apply to the script?

[18:06:20] <JakePee> that is, if i add `sleep(1000)` to my script, does it impact the db

[18:06:37] <sneakyness> try it and see

[18:07:26] <NodeX> any sleep() calls in scripting languages will only imapct the script

[18:07:37] <JakePee> thank you

[18:07:47] <JakePee> just wanted to make sure it wasn't calling this http://docs.mongodb.org/manual/reference/command/sleep/

[18:22:36] <CrawfordComeaux> Quick poll: Is this something you'd fund to get to play with and if so, how much would you put in? If not, what could be offered to change your mind - I want to setup a crowdfunding campaign for setting up a system that lets devs play with all social media data related to the super bowl or geotagged as coming from New Orleans from 1/26-2/4. Devs would be able to submit their own data processing components (for www.storm-project.org) or dashboard

[18:22:36] <CrawfordComeaux> widgets w/custom queries attached to them.

[18:36:55] <Xat`> which mecanism/server provides routing between members of replicaset ?

[18:41:53] <Xat`> I think it's purely sharding

[18:43:33] <Xat`> in fact, is it possible to use a rs without sharding ?

[19:03:00] <Xat`> can't understand how to query a rs while failover

[19:04:07] <Xat`> without sharding

[19:14:16] <kali> Xat`: the idea is to give the list of the replica nodes when you start the client, instead of just one node

[19:14:38] <kali> Xat`: the client is responsible for finding a working node

[19:17:18] <bean> i'm also doing stuff with replicasets right now -- I have one DB that is the current master, but we want to add some redundancy to it. Is the best option to just let it connect to the node and have the client figure it out? One of the guys here wants to use mongos in front of the RS, but we're not sharding at all so I don't know how much sense that makes.

[19:20:53] <linsys> Bean: you need to define the nodes in the client driver, all of them

[19:21:14] <linsys> bean: that way the driver can figure out which is the master and which node has the least latency when using SlaveOK

[19:21:45] <linsys> bean: using mongos could be important if you plan to shard one day... but I agree it isn't necessary yet..

[19:22:00] <bean> linsys: he's saying he doesn't want to make code changes

[19:22:26] <bean> so

[19:22:28] <bean> mongos it is

[19:22:31] <linsys> Well tecnically it isn't code... it should be a configured value in a config file.. but I guess it all depends on how you guys write code...

[19:22:45] <bean> yeah, heh

[19:23:12] <linsys> You should never have to make code changes when changing mongodb servers, like if you have a dev, pre prod and prod environment you want to have the flexability to change the nodes without changing code.. but that is just my opinion :)

[19:24:57] <bean> yeah

[19:25:20] <bean> linsys: we dont have to change code, but it might have to change code for replicaset -> not replicaset

[19:25:21] <bean> idk

[19:25:28] <bean> i could be way wrong

[19:25:46] <linsys> Just be aware you will need to setup config servers if you want to use mongos

[19:26:06] <linsys> I don't think the Jira ticket has gone though which allows mongos to connect to a single replica set, you still need to setup config servers

[19:26:33] <linsys> You can track the activity here https://jira.mongodb.org/browse/SERVER-1594

[19:26:49] <linsys> The good thing, if you go this route you will be in a position to shard quickly

[19:27:40] <kali> mmmm

[19:28:03] <kali> i have been using a sharded system with on single replica set in prod for years

[19:28:13] <kali> i can't see what this is about

[19:28:17] <linsys> With config servers right/

[19:28:21] <linsys> and mongos?

[19:28:32] <linsys> or are your mongos connecting directly to the replica set/

[19:28:58] <kali> linsys: sure, with config server

[19:29:03] <kali> but one single shard

[19:29:13] <linsys> Right, I'm just explaining to bean he will need to setup config servers as well

[19:29:34] <kali> yeah. i'm just puzzled about SERVER-1594

[19:29:53] <linsys> It says "no config servers"

[19:30:01] <bean> i'll probably just run the config server on my main node.

[19:30:07] <linsys> so I take this to mean your app connects to your local mongos, and mongos is able to connect to a single replica set.

[19:30:11] <kali> ha ! ok :)

[19:30:30] <kali> this is... well, why not.

[19:31:05] <kali> i prefer 10gen guys to spend their times on useful stuff :)

[19:32:51] <kali> bean: seriously, once your app configuration is done properly, it's *really* easy to have code compatible with whatever setup variant you'll have

[19:33:00] <bean> no, i know.

[19:33:13] <bean> Just working with a stubborn developer :P

[19:33:23] <kali> okay :)

[19:36:20] <linsys> just tell him "Hard part in devops is make money from wheel after you are reinvent it. "

[20:06:46] <revoohc> if a slave in a replica set dies, does mongo do a new master vote?

[20:11:16] <linsys> revoohc: it might

[20:36:04] <lineux_007> hey, i'm trying to recover space from a mongodb installation. dont have enough space to run repairDB. can i copy the db files of the db i want repaired to another instance, repair it there and copy and replace it back on the original instance?

[20:42:58] <crudson> I think that should work safely with directoryperdb. Shutdown first of course (and keep copy of original to be safe).

[20:47:44] <lineux_007> crudson: okay. thanks!

[20:49:59] <lineux_007> crudson: would having journaling enabled have any issues with copying directory per db?

[20:56:52] <crudson> lineux_007: a clean shutdown should result in no journal files

[20:57:39] <Zelest> what OS/version does people mostly run MongoDB in?

[20:57:58] <lineux_007> crudson: got it. thanks!

[21:56:40] <JakePee> what's the difference btwn these two queries

[21:56:50] <JakePee> {'groups': {$ne: {}, $exists: true}}

[21:56:50] <JakePee> {'groups': {$ne: {}}, 'groups': {$exists: true}}

[21:57:30] <Derick> the second one isn't valid json

[21:57:35] <Derick> can't have two keys with the same name

[21:58:17] <JakePee> ah yes, should have caught that

[21:58:24] <JakePee> mongo does accept it

[22:03:27] <JakePee> which is somewhat curious

[22:12:17] <crudson> I think you'll find that the document won't get all the way to mongo like that.

[22:13:19] <JakePee> as a query param it does

[22:13:25] <crudson> e.g. in mongo shell: d = {a:1,a:2} => { "a" : 2 }

[22:15:04] <JakePee> i stand corrected :)

[22:15:04] <JakePee> > query = {'groups': {$ne: {}}, 'groups': {$exists: true}}

[22:15:04] <JakePee> { "groups" : { "$exists" : true } }

[22:16:24] <crudson> right, whatever language you're using should not allow duplicate hash keys

[22:52:00] <mgallardo> Hi, does anyone know if mongodb's shell is the same than mongodb's version?

[22:54:06] <steverabouin> mongo shell will show the shell version, not the server version

[22:54:55] <sambomart> hi all, has anyone got mongodb running on debian?

[22:55:08] <sambomart> (as a deamon)

[22:55:16] <steverabouin> db.version() will tell you the server version

[22:57:07] <mgallardo> thanks @steverabouin... I am trying to update mongodb's server version to 2.0.0 with no success (using port), do you have any idea how can I do that?

[22:57:54] <mgallardo> if I do 'sudo port installed mongodb' I get 'mongodb @2.2.2_0 (active)'... and if I do db.version I get '1.4.0'

[22:58:20] <steverabouin> have you restarted the daemon?

[22:58:37] <mgallardo> no, I haven't

[22:59:17] <steverabouin> i'm not sure if port stops/starts after copying the new binaries, perhaps you just need to restart it

[23:00:53] <mgallardo> ok, let my try that... thanks

[23:51:44] <mgallardo> which shell version should be the one for mongodb version 2.2?

Log file Viewer

Help | Karma | Search:

#mongodb logs for Monday the 21st of January, 2013