pmxbot IRC Log Viewer

[01:22:39] <fbjork> I get the following error from the ruby mongo driver when I query a 2.0 database: BSON::InvalidDocument: Document too large: This BSON documents is limited to 4194304 bytes.

[01:22:41] <fbjork> any ideas?

[01:27:29] <LesTR> create smaller document : )

[01:27:41] <LesTR> you have 4M

[01:30:30] <LesTR> but limit from mongdb are 16M, so maybe you need update driver. I don't know, i don't use ruby with mongo

[01:50:03] <Auz> anyone know what version the db.isMaster().me was added?

[02:07:27] <fbjork> LesTR: I have the latest 1.8 driver

[02:07:29] <fbjork> makes no sense

[02:31:06] <Guest_1448> I have a collection with documents with a "type" field. How would I get max 10 documents for each type using aggregate/group/whatever?

[02:32:54] <Guest_1448> I could for example do .collection.aggregate({ $group: {_id: "$type", count: {$sum: 1}} }) to get a list of types and their counts but I want the actual documents (with max 10 from each type)

[05:00:50] <flyinprogramer> i know this isn't the mongoid group, but, any chance someone knows what i'm doing wrong: https://gist.github.com/4490672

[05:03:44] <flyinprogramer> semicolons

[05:04:33] <binroot> Hey, I have a collection called books that looks like {type:"fiction", author:"bob"}

[05:04:40] <binroot> how can i find all unique "type"s

[05:04:41] <binroot> ?

[05:04:55] <binroot> fictions, non-fiction, etc

[05:05:09] <binroot> I just want a list of unique types

[05:06:40] <Guest_1448> distinct()

[05:07:23] <Guest_1448> http://docs.mongodb.org/manual/reference/javascript/#db.collection.distinct

[05:08:59] <binroot> thank you Guest_1448 !

[06:18:03] <synchrone> hi everyone

[06:18:15] <synchrone> So I was wondering, what happens if in my .NET application I try to use MongoDBFileStream after IDisposable RequestStartResult object was disposed ?

[06:31:11] <scumcentral> hi

[06:32:27] <mikesm> shut up scum

[06:33:22] <scumcentral> we enroach the planet

[06:47:03] <mikesm> suck some balls scumbag

[06:51:42] <zakovyrya> Hi guys. I was wondering if there any known issues with oplog in 2.2.2?

[07:22:53] <synchrone> hi there fellow-wanderer zakovyrya

[07:24:31] <owen1> I start mongoDB with this command /home/t/bin/mongod --fork --logpath /usr/local/nextgen/push/shared/logs/mongodb.log --logappend --config /etc/mongodb.conf and get: exception in initAndListen std::exception: boost::filesystem::exists: Permission denied: "/data/db/mongod.lock", terminating

[07:24:38] <owen1> btw, I have write permission with the user running it: ls -l /data => drw-rw-r-- 2 root nextgen 4096 Jan 9 07:17 db

[07:25:49] <owen1> could it be related to the bind_ip = 10.8.0.10 in my config? i am not sure what is that actually.

[07:31:59] <owen1> chmod 755 fixed that but now i see: "Wed Jan 9 07:30:16 [initandlisten] ERROR: listen(): bind() failed errno:99 Cannot assign requested address for socket: 10.8.0.10:27017"

[07:53:07] <synchrone> type `ifconfig`

[07:53:14] <synchrone> see if you actually have that ip

[08:07:42] <owen1> synchrone: i think it's ok now. i set both hosts to have 127.0.0.1

[08:07:57] <owen1> it means - don't let anyone outside myself to access me, right?

[08:08:04] <synchrone> yes

[08:08:10] <owen1> so mongo --host <my server ip> will fail

[08:08:14] <owen1> (from any machine)

[08:08:15] <synchrone> no

[08:08:28] <synchrone> you should do that in case you want other servers to get there

[08:08:37] <owen1> do what?

[08:08:47] <synchrone> get access to your mongo

[08:08:58] <synchrone> like your frontend farm, etc

[08:09:00] <owen1> i have host1 and host2 and i want to have a replica set.

[08:09:40] <synchrone> i'm not sure how that works, but i guess you should connect to one of them

[08:09:46] <owen1> so i should type ifconfig and in 'bind_ip' put whatever i see there?

[08:10:05] <synchrone> what do you need replica for ?

[08:10:47] <owen1> synchrone: redundency

[08:10:57] <owen1> in prod i'll have 4 machines

[08:11:10] <owen1> i am setting up dev enironment now. 2 hosts.

[08:11:45] <synchrone> in case you are planning different servers to connect to different synchronized mongo instances - you should set up the bind IPs to network interfaces corresponding to the networks they are meant to be accessed from

[08:12:56] <synchrone> unless the app hosts are the same as mongo hosts

[08:13:17] <synchrone> in that case 127.0.0.1 will do for the slave server

[08:13:44] <synchrone> but not the replica source server, because it should be accessed from slave

[08:14:05] <owen1> the app itself lives on each host. so i have mongo+app on each of them.

[08:14:20] <owen1> but the app will connect to the replica. not dirctly to the mongo.

[08:14:26] <synchrone> replica master should be accessed from the slave right

[08:15:28] <owen1> i am not sure i follow you. i know one db will be elected to primary and he other will be secondary, so yeah, primary should be accessed from secondariies.

[08:15:54] <owen1> "you should set up the bind IPs to network interfaces corresponding to the networks they are

[08:16:14] <owen1> meant to be accessed from" - can u explain this please?

[08:16:55] <synchrone> just a second...

[08:19:16] <synchrone> so uhm

[08:19:50] <synchrone> replica master server should bind to a globally-accessed socket

[08:20:10] <synchrone> white ip, or what do you call it

[08:20:16] <owen1> so on dev i have foo.bar.com and foo2.bar.com

[08:20:31] <synchrone> you can't bind to a hostname afaik

[08:20:55] <synchrone> it's the network interface address you need to put there

[08:20:57] <owen1> so let's say i find the ips.

[08:21:17] <owen1> dev1 ip is x and dev2 is y

[08:21:49] <synchrone> if dev1 is master then it should have bind_ip = x, and dev2 should have bind_ip = 127.0.0.1

[08:22:02] <synchrone> so that dev2 could connect to it

[08:22:14] <synchrone> also

[08:22:31] <synchrone> 10.8.0.10 -- is that your server ip ?

[08:22:37] <owen1> no

[08:22:43] <owen1> it's from the expmale online

[08:22:48] <synchrone> ah i see

[08:23:01] <owen1> i found it - 10.1.72.132

[08:23:06] <owen1> that's dev1

[08:23:18] <synchrone> it's a local network ip. be sure that both servers are in the same network

[08:23:26] <synchrone> otherwise they won't see each other

[08:24:07] <owen1> synchrone: so the test will be to ssh from one to the other?

[08:24:22] <synchrone> no ssh

[08:24:31] <synchrone> mongo servers communicate thru their ports

[08:25:43] <owen1> synchrone: ok. let me try. so i set dev1's bind_ip to his ip and dev2 to 127.0.0.1

[08:26:05] <owen1> (and than i'll have to use the console and add dev2, correct?)

[08:26:15] <owen1> rs.add("mongodb2.example.net")

[08:29:00] <synchrone> uhm

[08:30:09] <[AD]Turbo> hola

[08:30:20] <owen1> Wed Jan 9 08:27:21 [rsStart] couldn't connect to localhost:27017: couldn't connect to server localhost:27017

[08:30:39] <owen1> synchrone: i see that in the log of dev1, after i try to start mongo

[08:30:47] <synchrone> rs.add("mongodb2.example.net") ?

[08:31:00] <synchrone> is that what you typed ?

[08:31:02] <owen1> it's just copied from the example

[08:31:24] <synchrone> what makes you think this will work then ?

[08:31:42] <owen1> after changing the bind_ip of dev1, i can't run mongo. i get the error above

[08:32:49] <owen1> synchrone: i meant that i just copy pasted into the chat. that's not what i am planning to puth there. i plan to put the ip of dev2.

[08:32:53] <synchrone> rsStart is a replica log

[08:33:15] <owen1> Wed Jan 9 08:27:38 [rsMgr] replSet PRIMARY

[08:33:26] <synchrone> you probably added localhost:27017 into your replica servers set at some point

[08:33:34] <owen1> true

[08:33:44] <synchrone> and since your mongo master is not at 127.0.0.1 anymore ...

[08:33:46] <owen1> now i can't access the mongo log from dev1

[08:34:00] <synchrone> it can't connect to it %)

[08:34:13] <owen1> so how to manage my replica setting if i can't run 'mongo' and use the console?

[08:34:33] <synchrone> change the settings back, cleanup, change back to normal IP, enjoy

[08:35:04] <owen1> so change /etc/mongodb.conf, start mongo,

[08:35:25] <owen1> access the mongo console and try to clean the setting?

[08:35:52] <synchrone> and I've just realized that I misguided you at the point of slave IP setting. it should also be your network ip, because it's master who connects to it in the first place

[08:36:09] <synchrone> yes

[08:36:43] <owen1> so dev2 should be set to dev2's ip?

[08:36:53] <synchrone> yes

[08:37:23] <owen1> not sure how to cleanup the replicaset. let me first try to change my conf and start mongo

[08:38:31] <synchrone> me neither. i've never set up mongo replica :) It's more of a network-knowledge you seem to need help with

[08:39:32] <owen1> (; nice

[08:40:17] <synchrone> apparently noone else here... so i figured some help is better than no help

[08:46:48] <owen1> i am trying rs.remove(hostportstr) remove a host from the replica set (disconnects)

[08:48:18] <NodeX> why dont you remvoe teh entire set and re-initialise

[08:48:23] <NodeX> remove the *

[08:48:45] <owen1> NodeX: looking in rs.help()

[08:52:12] <owen1> "hut all of the members down, delete the

[08:52:15] <owen1> local.* files from the primary, delete all of the data files from the

[08:52:17] <owen1> other members, and start everyone back up again"

[08:52:21] <owen1> i am reading it on google group

[08:52:43] <owen1> the question was how to start from fresh

[08:55:28] <NodeX> if you already know your master then you can skip the rest as it will load out to them

[08:56:20] <owen1> NodeX: deleted the content of /data/db on both hosts.

[08:58:29] <owen1> i am setting bind_id to the ip of the host itself. so dev1 with it's own ip and the same with dev2

[09:00:52] <NodeX> I only saw half your problem because I joined late but I'm assuming you just want to re-initialise your replicaset

[09:01:44] <owen1> NodeX: true. i got dev1 and dev2 and i try to create replica set

[09:03:16] <owen1> NodeX: btw, both hosts are in a vpn of my company.

[09:03:18] <NodeX> you have 2 servers and you've deleted the data on both?

[09:04:28] <owen1> NodeX: correct. there is no data yet. it's the first time i am setting it up

[09:04:57] <NodeX> ah ok

[09:05:08] <owen1> i am reading about testing availability of the hosts. it's being done with a simple mongo --host <ip>

[09:05:26] <NodeX> each host binds to it's own IP, then one is set to the master and you setup the config as per the docs

[09:07:33] <owen1> NodeX: from some reason i can't access dev2 from dev1 and visa versa

[09:07:52] <owen1> oh. they both down (:

[09:10:32] <NodeX> check your error log

[09:11:54] <owen1> ok. both can connect! now i'll open the mongo console of dev1

[09:12:41] <owen1> this is my plan:

[09:12:50] <owen1> rs.initiate()

[09:12:54] <owen1> rs.conf()

[09:13:29] <owen1> rs.add("dev2.company.com")

[09:13:55] <owen1> (using the hostname, since i was able to do that when testing the connection with mongo --host)

[09:14:13] <owen1> NodeX: sounds ok?

[09:16:10] <NodeX> yup

[09:17:10] <owen1> NodeX: oh. i can't access mongo of dev1. here is my log: Wed Jan 9 09:10:56 [rsStart] replSet can't get local.system.replset config from self or any seed (EMPTYCONFIG)

[09:18:40] <NodeX> you added the config to it?

[09:19:15] <owen1> NodeX: that's how i run it: mongod --fork --logpath /usr/local/nextgen/push/shared/logs/mongodb.log --logappend --config /etc/mongodb.conf

[09:19:20] <Zelest> NodeX, did you see the sexy porn i linked yesterday?

[09:19:34] <Zelest> http://www.youtube.com/watch?v=d8wEvWNrIxo <3

[09:19:46] <Zelest> that truely makes me happy in the pants

[09:19:49] <NodeX> I suggest you follow this tutorial owen1 http://docs.mongodb.org/manual/tutorial/deploy-replica-set/

[09:20:06] <NodeX> I didn't Zelest, watching now

[09:20:18] <owen1> 380

[09:23:29] <owen1> NodeX: yeah. i am following this one. anyway, i am going to sleep and will be back soon. thanks all!

[09:31:03] <NodeX> Zelest : pretty cool

[09:31:22] <Zelest> Mhm, especially since it got stemming and such built-in

[09:31:23] <NodeX> I hope they mix it up with geo indexes so you can search text in places

[09:31:31] <NodeX> else it's not of use to me

[09:31:41] <NodeX> (i'm sure it will be for other things)

[09:31:44] <Zelest> hmms

[09:32:04] <Zelest> ah yeah, since you don't really use a find() now

[09:32:05] <NodeX> they're after ideas so I'll put that forward I think

[09:32:13] <Zelest> :-D

[09:32:23] <NodeX> pretty sweet feature none the less

[09:32:56] <Zelest> yeah, FTS is basically the only thing I lack in mongo today..

[09:33:06] <NodeX> yer

[09:33:13] <Zelest> sure, there are alternatives and such, but still, it feels hackish

[09:33:24] <NodeX> it's annoying to keep a second index

[09:33:33] <NodeX> purely for FTS

[09:33:37] <Zelest> mhm

[09:33:56] <Zelest> one of the best things i know with mongo is the failover thingie that's built into the drivers

[09:34:07] <NodeX> this is going to attract all kinds of noobies to mongo now

[09:34:07] <Zelest> that somewhat fails if you have to sync and setup another index..

[09:34:14] <Zelest> probably

[09:34:30] <Zelest> then all "real" DBA's will go shoot it down

[09:34:32] <NodeX> gonna need to shard my ignore list LOL

[09:34:36] <Zelest> "screw mongo! it's not relational!"

[09:34:41] <Zelest> haha

[09:35:12] <NodeX> "real" DBA's just can't think lateraly

[09:35:24] <NodeX> latteraly

[09:35:31] <NodeX> or however it's spelt lol

[09:35:47] <Zelest> the thing is, some of the points I do find somewhat valid if you go strictly for best practices..

[09:35:55] <Zelest> but those best practices were made up 30 years ago

[09:36:14] <Zelest> and doesn't really apply to todays systems

[09:36:20] <NodeX> I've come to the conclusion that there is a balance and fine line between performance and ease of use

[09:36:47] <Zelest> you have the same line for security and ease of use.

[09:36:56] <NodeX> yeh

[09:37:13] <NodeX> for most DBA's and developers ease of use comes first

[09:37:20] <NodeX> for me performance is absolute key

[09:37:44] <NodeX> I do not compramise on performance for anything

[09:37:54] <NodeX> (apart from security)

[09:38:38] <Zelest> I'm not that fuzzed about ease of use.. what's key for me is redundancy/availability..

[09:38:44] <Zelest> which I find mongo to handle absolutely fantastic

[09:38:46] <NodeX> I hope 10gen don't lose site of this as they grow

[09:39:22] <Zelest> does 10gen make any money on it's own yet? or does it still live of investors?

[09:39:57] <NodeX> they just got a new round of funding I read somewhere a few months back

[09:40:20] <NodeX> I think they make some from support and contracts

[09:40:27] <Zelest> ah

[09:40:35] <Zelest> enough to live on its own?

[09:40:58] <NodeX> probably not, they've got quite a few offices/employees

[09:41:18] <NodeX> the product itself is probably worth a lot though

[09:42:18] <Zelest> mhm

[10:25:08] <synhchrone> ok, so

[10:25:29] <synhchrone> (repeating the question in case someone who is able to answer just connected

[10:25:37] <synhchrone> I was wondering, what happens if in my .NET application I try to use MongoDBFileStream after IDisposable RequestStartResult object was disposed ?

[10:42:40] <strnadj1> jo

[10:42:42] <strnadj1> sry bad window

[10:57:03] <Zelest> is --notablescan a sane option in production use?

[10:57:15] <Zelest> like, is the idea of that command to force the usage of indexes?

[11:00:50] <NodeX> it means it will not do a table scan in the event of no index

[11:01:00] <NodeX> "sane" depends on your app

[11:02:39] <Zelest> yeah, that's what I mean

[11:02:53] <Zelest> it will force me to use indexes everywhere I want to do a query.

[11:07:30] <NodeX> correct

[11:40:10] <mighub> Hello, I am using the new version of the Ruby wrapper and now I get this kind of error when I save a doc with a file: Mongo::OperationFailure: Database command 'filemd5' failed: (errmsg: 'need an index on { files_id : 1 , n : 1 }'; ok: '0.0')

[11:40:29] <mighub> Does anybody know what is going on and how to fix this?

[12:38:57] <oskie> I've set ulimits in /etc/security/limits.d/mongodb-nofile.conf, and it works for root, but not mongodb! (at least MMS still complains)

[12:47:32] <mighub`> If somebody is interested, I fixed the problem by using the class MongoClient instead of the deprecated Connection class. I submitted the issue to the bug tracker.

[14:45:03] <aster1sk> Morning all, getting some weird behaviour on a 'proto-duction' box

[14:45:03] <aster1sk> Tue Jan 8 22:45:36 got signal 2 (Interrupt), will terminate after current cmd ends

[14:46:25] <aster1sk> We're running a migration from sql -> mongo ~700 000 000 records and this is the second time this has happened.

[14:46:36] <aster1sk> The logs aren't particularily helpful.

[14:47:48] <kali> aster1sk: have you checked the system and security logs ? you can hit some kind of limit

[14:48:59] <aster1sk> auth.log and dmesg shows nil, colud it be that a mongostat is running constantly?

[14:49:17] <kali> i don't think so

[14:49:19] <aster1sk> It almost appears as if someone ^C'd the proc. SIGINT whateven sig2 is.

[14:50:00] <aster1sk> I'm also seeing instances of a 'got signal 15' but those very well could be manual shutdowns.

[14:50:24] <kali> what get the SIGINT ? client or server ?

[14:50:50] <aster1sk> server.

[14:51:05] <aster1sk> mongod dies on occassion - this is the second time.

[15:41:19] <remonvv> \o

[15:49:29] <NodeX> o/

[15:52:30] <remonvv> Well, I have nothing else to contribute to this channel.

[15:57:05] <aster1sk> So no ideas eh?

[15:57:13] <aster1sk> sar shows no memory weirdness.

[17:14:45] <toky> I would like to know if there is a way to empty a mongodb without destroying the actual db...i basically need to empty the data in it but allow it to continue to collect data right after i empty it

[17:26:45] <eka> toky: run a query that deletes old data?

[17:28:48] <toky> eka, and how do i do that? im not a dba...i just ran the application and installed some tools that use it

[17:29:15] <eka> toky: depends on your collections… delete data with a query

[17:29:32] <eka> toky: as in any db… or run mycollection.drop

[17:29:41] <eka> that will delete the entire collection

[17:29:47] <eka> but you will lose any index

[17:30:19] <toky> yea that is fine, i just need the /var/lib/mongo dir to drop to as close to 0 as possible

[17:30:34] <toky> all it has is 1 application using it (for now)

[18:08:07] <nonent> i've got a rather unwelcome question. reading the faq i just realized that in a sharded context if i have a doc {_id: a, foo: 1, bar: 2} where the shard key is foo,bar.. and i resave it with {_id: a, foo:2, bar:3} that that could create a situation with _id: a being on 2 different shards. that's unpleasant enough, but what's the failure case if those two rows are balanced to the shame shard? indeterminate? or will one get pr

[18:10:48] <nonent> living and learning. i'm assuming the only way to resolve this is to change my shard key to something where that doesn't happen (just _id would be safe, tho less optimum in terms of inserts) and re-import my data set

[18:11:24] <nonent> but if the failure case of rebalancing is determinate, i could just correct/remove the duplicate docs

[18:19:28] <JoeyJoeJo> What's mongo's equivalent to SQL's OR? For example 'SELECT * FROM table WHERE column = 1 OR column = 2'

[18:22:11] <kchodorow_> JoeyJoeJo: {column:{$in:[1,2]}}

[18:25:01] <JoeyJoeJo> thanks

[18:31:20] <Diable> How to use FindAll function with C# driver ?

[18:51:27] <bmamone> I've got this service that stores nightclub information in a collection. Each nightclub has a photo and i`m thinking in storing this pictures into the colection (they are small-sized < 128k). You think its a good idea to store it in mongodb, and if so, in the same collection or different one?

[19:08:26] <w3pm> so a related question.. when is it a good idea to create separate collections for things?

[19:08:32] <w3pm> what are the guidelines for dbs vs collections

[19:11:05] <kali> bmamone: it's unlikely you'll manipulate the metadata and the image at the same time, so i would not put them in the same document

[19:13:18] <colinhines> Greetings mongo-ers. I've got a sharding curiousity... If I have an over 256gb replset currently, can I still shard following the mongodb.org instructions if I just change the chunksize to like 1gb?

[19:57:07] <bmamone> kali: thanks!

[20:02:40] <Guest11650> hello, I wanted to know if someone has opinion on the best mapper for Ruby?

[20:23:09] <matto_chicago> still hoping someone is around?

[20:23:59] <krawek> sup

[22:03:38] <djMax> I'm thinking this is a common pattern but can't find guidance. I have a "transactions" table that I update with +/- transactions using findAndUpdate to make sure that I don't double enter based on a unique id. BUT, I want to cache the balance in a separate collection for a given "buyer"... How does one do this in Mongo?

[22:41:59] <limpc> so i have a number of shards and a config server. I'm using php, and everything seems to work fine except if i try to insert data. I get this error: " can't create user databases on a --configsvr instance"

[22:47:46] <strnadj1> limpc: Iam not sure, but you are connect to config servers? I think you have to connect your application to shard server

[22:47:55] <strnadj1> http://docs.mongodb.org/manual/reference/mongos/#mongos

[22:48:39] <strnadj1> According to documentation Shard server is routing service for queries from application

[22:54:46] <limpc> im connected to local port 20000 which is: mongod --auth --fork --configsvr --port 20000 --dbpath=/data/mongo/db/config --logpath=/data/mongo/logs/dbconfig --rest --keyFile /mnt/sdb1/www/keys/public.key

[22:55:28] <limpc> the error is specific in that i CANT do what I want to do, from a config server connection. but idk what else im supposed to connect to

[22:58:52] <joe_p> limpc: strnadj1 already answered that - to a mongos

[23:01:21] <limpc> oh i misread.

[23:01:49] <limpc> great. that worked, thanks

[23:11:39] <jrule> hello I have a newbie question here...

[23:20:57] <jrule> is anyone here?

Log file Viewer

Help | Karma | Search:

#mongodb logs for Wednesday the 9th of January, 2013