pmxbot IRC Log Viewer

[06:25:28] <gsamat> I have articles and photos json objects in same collection, what do I say to .find() to get photos only?

[06:37:22] <pomke> Hello :)

[06:38:00] <pomke> looking at http://docs.mongodb.org/manual/tutorial/enable-text-search/ it says 'do not enable on production systems'

[06:38:08] <pomke> is there any reason for this other than it's beta?

[06:38:10] <pomke> I mean

[06:38:19] <pomke> Is it known to corrupt data?

[06:39:54] <preaction> if it's beta, it probably can't be known not to corrupt data, or cause the server to go unresponsive, or any number of other unknowns

[06:43:52] <pomke> ty

[07:40:53] <[AD]Turbo> hola

[07:52:42] <lqez> aloha-

[08:27:16] <beepbeep_> I've always worked with relational db's (mysql) and want to experiment with mongodb. Any must read articles about developing applications that use mongodb?

[08:29:12] <Nodex> tbh the best thing you can do is do the opposite to what you would do in a relational DB

[08:29:21] <TeTeT> hi, a question on mongojs: i try to count the number of objects in a collection, basically doing: mongo pwo --quiet -eval "db.User.count()"

[08:29:25] <Nodex> and it's more or less nearly the right way to do things

[08:30:06] <TeTeT> I try http://pastebin.ubuntu.com/5615193/ but no result is returned

[08:30:41] <TeTeT> looks like the function in docs.count is never triggered

[08:30:48] <TeTeT> any ideas?

[08:30:50] <Nodex> why are you trying to execute that when you can get the result from your d

[08:30:52] <Nodex> driver *

[08:31:07] <TeTeT> Nodex: for use it in a script

[10:03:49] <d0ugal> So - my database seems to be corrupted and the replica stopped updating at the time of the corruption. This all happened silently.

[10:03:54] <d0ugal> Any ideas why this would happen?

[10:08:24] <ron> possibly machine rage against you.

[10:15:36] <lqez> Is there any bulkUpsert or similar feature?

[10:15:58] <lqez> Or, how can I make multiple upserts faster?

[10:22:02] <Nodex> turn off the safe writes if possible

[10:26:42] <lxsameer> does embedded documents have object_id ?

[10:32:51] <Nodex> no

[10:32:57] <Nodex> unless you make it have one :)

[11:44:33] <jon_r> Hi, does anyone know how to stop the mongodb shell from evaluating documents it returns? gives me weird errors rather than just printing the document

[11:44:49] <jon_r> used to work fine but i think 2.4 has introduced this weirdness

[11:47:00] <jon_r> ah nm, answering my own question, it's a bug that might be fixed in 2.4.2

[13:11:33] <DinMamma> Hello yall, I have a 4 shard cluster with 3 replicas in each shard.

[13:12:33] <DinMamma> One of the replica in each shard runs on one machine in our office as hidden, just so that we have an onsite 'backup'. Today I noticed that 3 of the 4 mongos in the replicasets was ~1 day behind in the oplog, but one of them was up to date.

[13:12:40] <DinMamma> How to start figure out what is going on?

[13:21:52] <oskie> how can I show what collections are sharded and using what key? when I do sh.status() it gives me only a database list

[13:22:53] <DinMamma> You need to enable sharding in a collection aswell.

[13:23:33] <oskie> so you are saying that if I only see databases in the sh.status() output and no collections, it means that no collections have been sharded yet?

[13:23:35] <DinMamma> sh.status() shows me collection-level info for the collections I have enabled sharding for.

[13:23:41] <DinMamma> Correct.

[13:23:46] <DinMamma> Atleast with 2.2 and 2.4

[13:24:34] <oskie> it's strange, because for some databases it says primary: "shard0000", and for some "shard0001"

[13:25:13] <oskie> but there are no collections listed at all

[13:25:15] <DinMamma> Yes, every database has a primary shard.

[13:25:25] <DinMamma> Even if you havnt enabled sharding for it.

[13:25:30] <oskie> oh, ok, I see. thanks.

[13:25:38] <DinMamma> Since the database need to live on a server :)

[13:29:00] <oskie> DinMamma: btw, I was unable to set up a sharded cluster with replica sets with hidden members

[13:29:59] <DinMamma> Strange..

[13:30:24] <DinMamma> I created my initial sharded cluster with two nodes in each replica-set, then added the hidden one directly to the replica-sets.

[13:30:27] <DinMamma> Worked for me.

[13:31:24] <oskie> I don't remember what the exact issue was

[13:33:26] <oskie> DinMamma: btw, when things go out of sync, the log will usually give some clue

[13:33:57] <oskie> i've seen secondaries go out of sync when rebuilding some huge index

[13:34:30] <DinMamma> Yeah, Ive had a look. It doesnt say anything of interest really.

[13:34:45] <DinMamma> I added a new shard to the cluster last friday actually, I wonder if this is realted.

[13:35:10] <DinMamma> Since alot of chunks have been moved away from these replica-members.

[13:35:20] <DinMamma> And the member that is up to date is the new one!

[14:13:07] <_{Dark}_> helo

[14:14:10] <DinMamma> Hello

[14:15:07] <_{Dark}_> I'm looking for the best solution to use mongodb on my available servers

[14:15:58] <DinMamma> Ok.

[14:16:01] <_{Dark}_> I have 3 machine to use, searching for best configuration

[14:16:24] <_{Dark}_> I'm thinking something like this:

[14:16:44] <_{Dark}_> mongos run on webapplication server

[14:16:50] <Nodex> do you need performance or integrity?

[14:17:01] <Nodex> read or write performance

[14:17:03] <Nodex> 42 or 56

[14:17:31] <_{Dark}_> read performance and if possible wirth 3 machine also a replica

[14:17:38] <_{Dark}_> let me explain my idea:

[14:17:57] <_{Dark}_> Application servers: run mongos (so not using my dedicated 3 servers)

[14:18:06] <_{Dark}_> Server A: shard1

[14:18:14] <_{Dark}_> Server B: shard 2

[14:18:37] <Nodex> shards or replicas?

[14:18:46] <Nodex> you can get read performance without sharding

[14:18:49] <_{Dark}_> Server C: Mongoconfig server + replica of Server A/B

[14:18:51] <_{Dark}_> make sense?

[14:19:18] <DinMamma> You want to run 3 configservers in production.

[14:20:13] <DinMamma> If you dont have a lot of data(>TBs, to big for SSDs) you should probably be able to just run a replica-set.

[14:20:44] <jiffe98> anyone know how to recover from 'can't find a shard to put new db on' errors when I try to access databases via mongos?

[14:20:54] <jiffe98> I can access the replica sets directly just not through mongos

[14:20:57] <_{Dark}_> Probably the data will increase a lot so the idea is to configure also Shard from the beginning

[14:21:23] <DinMamma> What do you think the increase will be per day?

[14:29:56] <Nodex> _{Dark}_ please don't PM without asking, it's rude and annoying

[14:31:10] <_{Dark}_> no problem

[14:34:38] <jiffe98> can I just recreate the config servers and go through the steps of enabling sharding on an existing replica set to fix sharding

[14:51:03] <jiffe98> seems that worked, probably because I have only a single shard

[15:46:32] <Mech0z> is there a shell command to see all collections in the db I am in?

[15:56:22] <jiffe98> Mech0z: show collections

[16:00:13] <Mech0z> i just dont get why db.system.profile.find() returns nothing

[16:22:21] <harenson> Mech0z: what returns "show collections"?

[16:23:02] <Mech0z> nothing, but I am not sure there is any collections to be sure, its just a sysprofile json

[16:23:23] <Mech0z> that I imported

[16:23:42] <harenson> Mech0z: how do you imported it?

[16:23:51] <Mech0z> mongoimport -d m101 -c profile < sysprofile.json

[16:24:08] <Mech0z> shoudnt that show under db.system.profile.find()

[16:24:09] <harenson> Mech0z: do this:

[16:24:18] <harenson> use m101 -> enter

[16:24:30] <harenson> db.profile.find() -> enter

[16:24:45] <harenson> there is your data

[16:24:59] <Mech0z> :S thoguht I had done that

[16:25:01] <Mech0z> but thanks

[16:25:30] <harenson> o.O

[16:27:32] <harenson> Mech0z: with "db.system.profile.find()", you're querying for the subcollection "profile" within collection "system" within the db you selected before

[16:28:07] <harenson> and your mongoimport importing to the db "m101" in the collection "profile"

[16:28:41] <Mech0z> that makes sense

[16:29:13] <harenson> Mech0z: that's why I told you that follow the steps

[18:21:19] <mcilrain> hi, I'm using a query that selects documents based on items it has in an array and then sorting based on a timestamp (20 most recent posts with tags foo+bar), what sort of indexes should I be using for such a query?

[19:02:02] <Scrivener> Heya guys. I was having this particular problem and wanted to see if there was any other solution to it than the one posted in this stackexchange answer: http://askubuntu.com/questions/180319/mongod-fails-to-start-with-error-mongod-symbol-lookup-error-mongod-undefined

[19:02:19] <Scrivener> Is it a known issue? I could find almost nothing about it through googling.

[19:02:46] <Scrivener> In my case, Amazon Linux, yum-installed mongodb instance, mongod itself starts okay, and things seem to work.

[19:02:59] <Scrivener> Until one uses the mongo prompt. When you enter in anything it gives that error and quits.

[19:03:36] <Scrivener> Using the precompiled binaries sounds like a really ugly solution.

[19:04:14] <Scrivener> I would think that non-working MongoDB packages would be something of a larger concern than they seem to be. I'm not sure why it's happening.

[19:05:41] <kali> Scrivener: this is with the ubuntu maintained packages, right ?

[19:06:01] <Scrivener> I'm on Amazon Linux, so this'll be EPEL

[19:06:07] <Scrivener> But I've got the same problem.

[19:06:30] <Scrivener> Seeing that that stackexchange post was made last year, I expected to find a bit more of a fuss about it at least surrounding Ubuntu, but I can't seem to.

[19:07:29] <kali> Scrivener: i don't know what epel is... but as 10gen maintains packages for most distribution in the right form in their own repository, i think there is less pressure around distribution-maintained packages

[19:08:15] <Scrivener> Kali - It's a Fedora/CentOS/Redhat repo.

[19:08:21] <Scrivener> https://fedoraproject.org/wiki/EPEL

[19:08:44] <Scrivener> Interesting, hold on.

[19:10:15] <Scrivener> I will give this a go and report back -- thanks already.

[19:10:58] <ehershey> Scrivener: 2.0.4 is fairly old

[19:11:15] <kali> yeah, that too. 2.0.4 is ancient

[19:11:44] <Scrivener> I'm not using 2.0.4 -- it's not me in this particular post, but I'm having the same kind of issue. I believe the version I had was 2.2.x or 2.4.x

[19:11:55] <Scrivener> Whichever is in the EPEL repo.

[19:12:17] <ehershey> ah

[19:12:35] <ehershey> probably still old, but not as bad as 2.0.4

[19:12:44] <Scrivener> 2.2.3-4

[19:12:46] <Scrivener> Hmm.

[19:12:57] <ehershey> as far as I know, the 10gen repos are the only linux package source of mongodb that is reliably up to date

[19:13:23] <ehershey> that sounds like a package building problem though

[19:13:46] <ehershey> you could try getting the source package

[19:13:48] <ehershey> and building yourself

[19:14:45] <bjori> brew is usually very up2date

[19:14:49] <ehershey> true

[19:14:56] <ehershey> but not linux packages :)

[19:15:10] <bjori> :)

[19:16:40] <Scrivener> I would guess depending on distro.. arch is probably up to date :P

[19:16:46] <Scrivener> But who runs arch on prod.

[19:17:04] <Scrivener> RHEL-family distros and Debian are notoriously out of date.

[19:21:46] <Scrivener> kali, works like a charm :)

[19:22:07] <kali> Scrivener: great.

[19:22:11] <Scrivener> I feel kinda stupid for not seeing that, but glad you pointed it out.

[19:22:18] <Scrivener> Thanks a lot :)

[19:23:57] <GRMrGecko> Can someone translate this SQL query to mongo? UPDATE t1 SET b=b*2 WHERE a>=0 AND a<10; The problem I'm having thinking about is the B=B*2

[19:24:27] <kali> GRMrGecko: you just can't

[19:25:08] <GRMrGecko> kali: So, MongoDB doesn't have that functionality?

[19:25:17] <kali> GRMrGecko: correct

[19:25:45] <GRMrGecko> kali: Can I maybe have a function in mongo do this?

[19:26:11] <GRMrGecko> like set b=function(doc) {return doc['b']*2;}

[19:26:25] <kali> you can maybe achieve it with server side javascript execution, but it's bad practise

[19:27:08] <GRMrGecko> ok

[19:36:10] <ixti> hi all!

[19:36:52] <ixti> probably stupid question, can somebody point me to the article explaining why docs in collection should be with as similar schemas as possible

[19:37:05] <ixti> i'm nearly 100% sure that i read something like this

[19:37:16] <ixti> and nearly 75% sure it was on official web site :D

[19:37:44] <ixti> or probably it was in m101 or m102... :D

[19:40:13] <BobFunk> Is there any way to stream a binary field from a document to a file without loading the whole field in memory with either the ruby or java mongodb driver?

[19:40:39] <wereHamster> ixti: they should be of similar size, but I don't think mongodb cares about the actual content

[19:41:14] <wereHamster> also, if you have non-sparse indexes on fields that are only in selected docs, the index will be inefficient

[19:41:43] <kali> also, it just makes sense

[19:46:31] <ixti> :D thanks guys

[19:46:52] <ixti> i guess what i read was about indexes indeed

[19:49:29] <ixti> yeah, now i'm nearly 100% positive it was a page with index types explanation: http://docs.mongodb.org/manual/core/indexes/

[19:49:44] <ixti> wereHamster: kali: thanks :D

[20:13:25] <Mech0z> when using the mongodb driver for C# how do I get mongodb to autoincrement my ID field when I have created the ID field myself

[20:15:27] <ron> you don't?

[20:15:49] <ron> and that's just a guess as I've never used the mongodb C# driver, it's just that your question doesn't make sense at all.

[20:17:21] <Mech0z> but then how do I use the ID field if I cant access it on my model

[20:17:59] <Mech0z> mongodb will autocreate a _id field, but that is "hidden" because my c# model wont know about it

[20:18:43] <ron> well, then make it know about it?

[20:18:59] <ron> if it won't know about it, how do you expect updates to work?

[20:20:46] <the_lord> Hello, I'm trying to make some indexes, I'm using mongoid, and it's generating the query like 'collection.find( { "$query" : { query params }, "$orderby": { sort params}})', that query does not make use of an index I have for that query, instead, if I use collection.fing({ query params }).sort({sort params}) it does, any ideas on how to make mongo use the index in the first query?

[20:25:09] <ixti> the_lord: you want to pass query and sort into find?

[20:25:22] <the_lord> ixti, I don't want to, mongoid does that

[20:26:22] <the_lord> ixti, I suppose I could make a bug in mongoid and see the reply, but I just want to make sure there's no way of making that work

[20:26:43] <ixti> the_lord: are you sure. i think mongoid executes command

[20:26:46] <ixti> anyway

[20:26:50] <ixti> you can use hint

[20:27:35] <ixti> Queryable#hint

[20:27:42] <ixti> http://mongoid.org/en/origin/docs/options.html

[20:30:52] <ixti> all: DOCS says each index takes minimum 8KB what does it means?

[20:31:02] <ixti> min 8KB per document?

[20:31:07] <ixti> or per field?

[20:31:20] <ixti> or in general

[20:32:24] <ron> yes.

[20:33:04] <ixti> o_O

[20:33:12] <ixti> maybe

[20:33:30] <ixti> O_o

[20:34:03] <ron> yeah, was trolling there, couldn't help myself, sorry. no idea here.

[20:35:03] <ixti> :D

[20:36:00] <the_lord> ixti, it _seems_ a bug in explain()

[20:36:25] <ixti> the_lord: might be :D

[20:38:01] <andre_fffff> Hi there

[20:38:11] <andre_fffff> I am 5 hours

[20:38:20] <andre_fffff> trying to initiate a replica set

[20:38:26] <andre_fffff> i already did this very easy

[20:38:30] <andre_fffff> but i have allways a problem

[20:38:35] <andre_fffff> errmsg" : "couldn't initiate : need all members up to initiate

[20:38:49] <andre_fffff> i already set in the mongodb.conf the replica set name

[20:39:08] <andre_fffff> but whern i do the rs.inititate

[20:39:15] <andre_fffff> and pass as parameter the configuration

[20:39:21] <andre_fffff> the mongo allways says errmsg" : "couldn't initiate : need all members up to initiate

[20:39:35] <andre_fffff> and o used ping to verify that they are up

[20:39:39] <andre_fffff> and running mongodb

[20:43:45] <ixti> andre_fffff: try watching this http://www.youtube.com/watch?v=iEnioS8n254&list=PLUgynT_08TC6cpVKbGq1PnpaXrh7-jqhy

[20:43:53] <ixti> and review if you did all correctly

[20:44:28] <ixti> usually replicaset setup should not be a problematic. unless you screwed rs config upon init

[20:44:38] <ixti> if so - you'll need reconfigure replicaset

[20:44:46] <the_lord> ixti, indeed, http://pastebin.com/UqKuzXZE

[20:44:56] <andre_fffff> I know, is very simple

[20:44:59] <andre_fffff> i did the mongodb developer course

[20:45:09] <andre_fffff> and tested a lot of replication sets configurations

[20:45:16] <the_lord> ixti, the first query log is with an index for opponent_id and challenger_id

[20:45:18] <andre_fffff> this is being a complete nightmare

[20:45:23] <the_lord> ixti, the second one has no index

[20:45:27] <the_lord> same query for both

[20:46:47] <the_lord> even though the explain for the query states that is not using any index: http://pastebin.com/DUweRzaa

[20:47:23] <ixti> the_lord: wow

[20:47:33] <andre_fffff> Can anyone help me in replica set ? :/

[20:47:44] <andre_fffff> i am 5 hours trying to do this

[20:48:07] <andre_fffff> :(

[20:48:22] <ixti> two exactly same queries, but one scanned 2 docs and second 39606

[20:48:28] <ixti> that's weird

[20:48:45] <the_lord> ixti, no, that's ok, one did a full scan, the other one used an index

[20:48:56] <the_lord> the collection has 39606 documents

[20:49:38] <ixti> the_lord: snippet on pastebin queries in both cases same query

[20:49:54] <ixti> i mean both queries have conditions

[20:49:55] <bean__> random question: i have 3 nodes in a replicaset. Can one assume that the slaves will be pretty quickly consistent

[20:49:55] <bean__> or

[20:50:11] <bean__> how long will it take to sync them, assuming there is no configured slave delay

[20:50:15] <the_lord> ixti, yes, but one was without an index in the database, and the other had an index

[20:51:06] <ixti> the_lord: they both use same collection

[20:51:21] <the_lord> ixti, yes

[20:51:32] <ixti> looks like one of the queries passed hint to use natural index o_O

[20:51:40] <ixti> instead of optimal one

[20:51:47] <andre_fffff> Is normal having this error while initiating the replica set configuration? "couldn't initiate : need all members up to initiate,

[20:51:50] <ixti> andre_fffff: i really don't know how to help you

[20:52:08] <bean__> andre_fffff: are all of the members that you've defined up?

[20:52:23] <the_lord> ixti, this was my test, I created the collection, with no special indexes except the _id one

[20:52:27] <ixti> your rs config could tell little bit more than "i have problems"

[20:52:37] <ixti> the_lord: ahhh

[20:52:52] <the_lord> ixti, made the query, and the explain was http://pastebin.com/DUweRzaa

[20:53:05] <the_lord> ixti, created this index: db.david_app.matches.ensureIndex({challenger_id: 1, opponent_id:1})

[20:53:13] <ehershey> I just signed up for "M102 MongoDB for DBAs"

[20:53:20] <the_lord> ixti, made the same query again, and then 2 scans

[20:53:24] <andre_fffff> My members are all up

[20:53:30] <the_lord> ixti, but the explain keeps being the same

[20:53:35] <bean__> andre_fffff: what does your rs.config() look like

[20:53:37] <andre_fffff> i already put this replica set working today

[20:53:44] <andre_fffff> 1 sec

[20:54:01] <ixti> ehershey: good for you :D

[20:54:03] <andre_fffff> rs.conf() is null

[20:54:14] <the_lord> ixti, mongo is using the index, but the explain doesn't tell you that

[20:54:23] <bean__> andre_fffff: no, rs.config()

[20:54:37] <andre_fffff> yes, the same rs.config is null

[20:54:45] <andre_fffff> actually rs.conf is an alias to rs.config

[20:54:47] <andre_fffff> hummm

[20:55:00] <andre_fffff> is because this is a 32 bits development servers?

[20:55:06] <andre_fffff> Because they are openvz?

[20:55:12] <andre_fffff> so this is unstable?

[20:55:20] <andre_fffff> I have no logic explanation for this

[20:56:00] <the_lord> ixti, http://stackoverflow.com/questions/14608642/mongodb-query-operator-ignores-index

[20:56:46] <bean__> andre_fffff: you can try initiaiting the RS on one node, then doing rs.add("<hostname>:port")

[20:57:19] <andre_fffff> yes i allready did that

[20:57:30] <andre_fffff> when i do that, that node start as secondary

[20:58:07] <bean__> andre_fffff: as in, the main starts as secondary?

[20:58:22] <andre_fffff> yes

[20:58:32] <andre_fffff> very odd

[20:58:37] <ixti> the_lord: wow! good cacth

[20:58:40] <ixti> *catch

[20:58:45] <andre_fffff> i restarted my 3 servers to see if works now xD

[20:58:55] <andre_fffff> i mean, i believe in everything now

[20:59:30] <andre_fffff> nop, restart didn't worked

[21:27:26] <GRMrGecko> Is this possible in MongoDB? INSERT INTO t1 SELECT b,a,c FROM t2;

[21:27:41] <GRMrGecko> (I'm replicating https://www.sqlite.org/speed.html incase you're wondering.

[21:30:50] <GRMrGecko> It's basically cloning one table into another while swapping fields.

[21:32:10] <GRMrGecko> db.people.find().forEach( function(x){db.user.insert(x)} );

[21:32:37] <GRMrGecko> I think I can handle it now

[23:06:29] <bobbyjayblack> what is a pay per month flat rate service available for a decent price?

[23:06:41] <bobbyjayblack> mongod hosting*

[23:07:56] <xenoxaos> MongoLab?

[23:19:13] <dcholth_> Hello. I have a newbie question... _id is generated by Mongo. I have 3 object collections, and I want to have my own ID thats unique to each of them as well that I track.

[23:19:54] <dcholth_> Where I want Question.id, Category.id, Zone.id... But I feel its dirty to have both _id and id... So I started doing Question.qid, Category.cid... Is that a valid concern or should I just go with id and _id ?

[23:39:21] <Raynos> I'm running into 'Error: connection closed due to parseError' coming out of the node driver

[23:39:38] <Raynos> mainly because the protocol header says this packet contains more then 16mb

[23:43:01] <bartzy> Derick: I tested the remove() question I had yesterday

Log file Viewer

Help | Karma | Search:

#mongodb logs for Monday the 29th of April, 2013