pmxbot IRC Log Viewer

[00:19:20] <kexmex> any idea why it takes 2-3 seconds to open a connection to a local mongo instance?

[00:21:37] <Boomtime> kexmex: what does the log file timestamp say for the connection? do you use authentication?

[00:21:47] <kexmex> yea

[00:22:29] <kexmex> oh

[00:22:31] <kexmex> i know what it is

[00:22:39] <kexmex> the call to mongoServer.Ping()

[00:22:42] <kexmex> any ideas why it takes so long?

[00:24:00] <Boomtime> can you try connecting using the mongo shell

[00:24:29] <kexmex> sure

[00:24:31] <kexmex> what am i looking at?

[00:25:32] <Boomtime> do you observe the 2-3 second delay when connecting using the mongo shell?

[00:25:36] <kexmex> no

[00:25:48] <Boomtime> ok, so we've established it is not the specific server that you are connecting to

[00:25:48] <kexmex> btw, delay in my code was coming from mongoServer.Ping()

[00:25:55] <kexmex> seems like that command takes long

[00:26:08] <Boomtime> is it a replica-set?

[00:26:26] <kexmex> nope

[00:26:27] <Boomtime> also, why do you need to call .Ping()? what does it do for you?

[00:26:53] <Boomtime> what driver/version is that anyway?

[00:27:01] <kexmex> Boomtime: it's call up during some setup. To make sure credentials are ok etc. Easier to fail it right there instead of failing somewhere deep down the line

[00:27:07] <kexmex> c# 1.10

[00:27:21] <Boomtime> that will tell you nothing then

[00:27:50] <Boomtime> ping almost certainly uses isMaster or some other extremely low grade requirement cmd

[00:28:11] <kexmex> ah

[00:28:20] <kexmex> well its slow. 2+ seconds each time

[00:28:33] <Boomtime> that is odd, it is certainly surprising

[00:28:41] <kexmex> i guess i don't need it since i am dong db.getCollectionNames()

[00:28:42] <Boomtime> but it won't do what you want it for anyway

[00:29:04] <kexmex> well ping() tells me if server is up at least right

[00:29:11] <kexmex> though db.getCollectionNames() should too :)

[00:29:16] <Boomtime> indeed

[00:29:30] <kexmex> i just saved off more than 50% off my integration tests !

[00:29:36] <kexmex> shaved*

[00:29:40] <Boomtime> the only way to really know if you have the creds (and connection) to do what you want is to attempt what you want

[00:29:48] <kexmex> yeh

[00:30:13] <Boomtime> you might find, for example, that getCollectionNames works then you fail to insert - because your creds are readonly

[00:30:30] <kexmex> yea makes sense

[00:30:35] <Boomtime> alternatively, you might find that inserts are fine by getCollectionNames fails because the creds permit insert on only a single collection

[00:45:45] <crazydip> does mongodb 3.0.5 use standard or a mongodb-specific modified version of boost? is it safe to use the distributions' boost-devel when compiling mongodb, especially if it's more up to date than the boost included w/ mongodb?

[02:37:03] <brockfredin> I have a question about mongo queries. Wondering if anyone can help? :)

[02:42:14] <Boomtime> brockfredin: hi there - ask your question, perhaps someone will be able to help

[04:02:07] <asa_> May I ask if the risk of MongoDB losing data higher than relational database like MySQL? don't mean to put down MongoDB but someone warned me against using MongoDB. I have not used it before. So, personally, I don't know enough to make a judgment call.

[04:04:51] <Boomtime> asa_: why would the risk be any different?

[04:05:24] <Boomtime> a lot of people have a misconception about mongodb due to a poor choice of 'default writeconcern' in the drivers from many years ago

[04:05:30] <asa_> Boomtime: i really don't know. I've never used MongoDB. It's just what people say and what I read.

[04:05:56] <asa_> At the risk of offending the people here, here's one article I read that made me scared. http://cryto.net/~joepie91/blog/2015/07/19/why-you-should-never-ever-ever-use-mongodb/

[04:06:20] <Boomtime> and do you believe everything you read on free blogs?

[04:06:59] <Boomtime> note that guy cites the same source, from a self-described 'developer' that doesn't know what he is doing

[04:07:12] <asa_> I don't know enough to believe or not to believe. That is why I am here. To get expert, objective information from MongoDB folks

[04:07:36] <Boomtime> like any technology, if you use without bothering to understand how it works, you might be surprised by behavior it exhibits and leap to conclusions that aren't true

[04:08:19] <asa_> Boomtime: what has led to the complaints in MOngoDB? Misuse?

[04:08:26] <Boomtime> since you asked about data safety, the place to start is to understand the ways that mongodb provide levels of assuarance

[04:08:49] <Boomtime> this is called writeconcern: http://docs.mongodb.org/manual/core/write-concern/

[04:09:35] <Boomtime> asa_: here is a simple thought experiment -> let us install a mongodb and an sql on the same hardware.. now let us say the hard-disk in that machine fails - which database is safer?

[04:09:54] <asa_> cannot tell

[04:09:59] <Boomtime> right

[04:10:27] <Boomtime> they both lost data, because they were forced to store it on fallible harddisk

[04:10:46] <asa_> Boomtime: but I don't hear people making the same complaints with mysql. I was wondering why the same happened to mongodb

[04:10:50] <Boomtime> you can mitigate this risk in mongodb the same way you can in many sqls; via replicasets

[04:11:19] <Boomtime> asa_: you claim that an sql can survive a hardware failure?

[04:11:30] <asa_> no. impossible

[04:11:46] <Boomtime> ok.. good starting place, so let's get in to more subtle details

[04:12:02] <asa_> thanks for your patience

[04:12:12] <Boomtime> let us say you send a write command to an SQL and the same write to MongoDB...

[04:12:23] <Boomtime> is it safe in both cases?

[04:12:37] <Boomtime> i haven't provided enough information to say, have i?

[04:12:47] <asa_> yes, can't tell at this point

[04:12:55] <Boomtime> you need to listen for the answer from the host that you send the write command to

[04:13:27] <Boomtime> but what if you listen for the answer from sql, but don't listen for the answer from mongodb? is that fair?

[04:13:42] <asa_> unfair

[04:13:57] <Boomtime> the link you gave basically summarizes to that situation

[04:14:49] <Boomtime> there is lots more to this, and i would urge you to read about writeconcern

[04:15:00] <Boomtime> it can provide the assurance of data durability that you want

[04:15:32] <asa_> with writeconcern, data integrity is assured? Not possible to lose data?

[04:16:13] <asa_> was writeconcern introduced only recently? Is this why MonogoDB has some bad press?

[04:16:14] <Boomtime> of course not, if every host that holds data suddenly explodes then you will losoe data

[04:16:30] <Boomtime> this is true of SQL or mongodb - all you can ever do is lower the risk

[04:16:52] <asa_> i mean, barring uncontrollable situation, data will not be lost due to MongoDB's fault?

[04:17:10] <Boomtime> writeconcern allows you to control the risk, by specifying how many hosts to ensure that writes have propagated to

[04:17:22] <Boomtime> before returning an answer to the application

[04:39:09] <pylua> what is the diff netween mongodb search engine and elasticswarch engine ??

[05:02:20] <preaction> mongodb is not a search engine

[06:24:49] <jamiel> if sharding by foo: 1, bar: 1 ... will documents with the same "foo" be on the same server where chunksize permits and "bar" just decides how the "foo"'s are split?

[06:30:44] <Boomtime> jamiel: i don't believe so, there is no reason for a split point to occur only on a field boundary

[06:36:08] <jamiel> So if I have a dataset of 3 million records, foo has 3 distinct values, I have 3 shards and I want the data to be split across 3 shards 1 million of each type of foo on each server (as queries are always for one type of foo at a time) ... I have no way of doing this? Shard key can't only be foo as the chunk will be un-splittable if my shards are only big

[06:36:08] <jamiel> enough for 1 million of each type.

[06:38:46] <Boomtime> you can do that, but you need to tell the balancer how you want your data arranged

[06:39:17] <Boomtime> http://docs.mongodb.org/manual/core/tag-aware-sharding/

[06:45:17] <jamiel> Thanks

[07:09:28] <dddh> shoul I try to use mongodb on btrfs with compression enabled? autodefrag or weekly defrag?

[07:09:39] <dddh> *should

[09:15:18] <m3t4lukas> hey guys, I got a problem with the mongodb java driver, two days ago my code was still working , but today I cannot create an authenticated client anymore.

[09:16:47] <m3t4lukas> I use com.mongodb.MongoClient.MongoClient(ServerAddress addr, List<MongoCredential> credentialsList) as a constructor and it throws http://pastebin.com/1JwqrCGS

[09:17:08] <m3t4lukas> seems like the problem lies within the java driver

[09:18:58] <m3t4lukas> and I'm building with gradle using 'org.mongodb:mongodb-driver:3.0.2' as a dependency

[09:32:29] <m3t4lukas> fixed it,just switched back to 3.0.1 and it works again

[09:33:04] <m3t4lukas> guess I shoudn't go bleeding edge for production

[09:33:28] <m3t4lukas> although I thought those in gradle were stable

[10:31:50] <Gemtastic> Hello!

[10:33:25] <Gemtastic> Is there anyone who knows the difference between insertMany, writeBulk and the intitializeUnorderedBulkOp?

[18:07:05] <tzahi> hello, i was wondering if i can get help with a schema im thinking about using? im coming from a mysql background

[20:03:30] <riot> hmm, i get "ImportError: cannot import name _unicode" in "from bson.py3compat import (_unicode," - on a rather fresh debian install.. wtf?!

[20:03:43] <riot> i'm running py2.7.9

[20:17:16] <riot> hah, old installed python-bson package.. gnf

[21:11:23] <bros> Hi. All of my data is missing from my database?

[21:11:28] <bros> I have backups, but I am trying to figure out what happened.

[21:12:20] <bros> I just did a new deploy to the node.js codebase. Nothing change in terms of the mongo code. I've been working on /develop database, and now I moved it back to the production database, and it's entirely empty?

[21:14:09] <bros> I did a apt-get upgrade, and now it's all gone?

[21:15:29] <bros> are there any log files anywhere?...

[22:03:31] <bros> can somebody help me figure out why my replicated sharded cluster isn't working?

[22:12:49] <bros> 339 inactive users in the channel for the database that drops data?

[22:23:13] <iszak> bros: what

[22:23:33] <bros> can't figure out how to bring a replicated shardest cluster my co-worker set up after a reboot

[22:23:34] <bros> and neither can he

[22:23:46] <iszak> bros: it probably didn't flush before it went down?

[22:23:56] <bros> no, it's that he has it requiring it be brought up manually

[22:24:01] <bros> and we can't figure out how to bring it up

[22:25:32] <iszak> bros: bring it up manually? logs are under /var/logs/

[22:25:49] <bros> thanks. i had no idea...

[22:25:52] <bros> what do we need to do

[22:25:54] <bros> turn on query router

[22:25:56] <bros> then config server

[22:26:04] <bros> then shard instances

[22:26:08] <bros> then add instances to shard cluster?...

[22:26:12] <bros> it's like.. a sick joke

[22:29:46] <iszak> bros: i have no idea what your issue is

[22:29:51] <iszak> you've not explained it clearly

[22:30:02] <bros> my entire mongo stack isn't working and i have no idea how to get it to work

[22:30:05] <bros> what information do you need?

[22:31:07] <iszak> bros: mongo stack as in a single server? sharded?

[22:31:13] <bros> sharded

[22:31:34] <bros> i've got the config server up

[22:31:40] <bros> where do i run the query routing server

[22:31:45] <bros> on the shards, or on the config server?

[22:32:19] <bros> "errmsg" : "couldn't connect to new shard ReplicaSetMonitor no master found for set: foo"

[22:36:09] <iszak> sounds like it hasn't re-elected a master node

[22:41:04] <iszak> bros:

[22:41:09] <bros> i figured it out

[22:41:10] <bros> thanks

[22:41:12] <bros> jesus did that suck

[22:41:35] <iszak> bros: what was it?

[22:41:47] <bros> my mate combined the need for config servers and query routers into one

[22:41:52] <bros> then i had to turn all 4 of the shards on

[22:41:56] <bros> i guess i did it in the wrong order

[22:42:01] <bros> not too difficult, just didn't work the first time

[22:42:10] <iszak> bros: it sounds like you have a complicated setup, any reason why?

[22:42:19] <bros> yes. stupid fucking mate told me we needed it

[22:42:25] <bros> how long until you should scale up from 1 server?

[22:42:27] <bros> never, right? lol

[22:42:41] <bros> he's refusing to run dist-upgrade on servers :/

[22:42:42] <iszak> bros: well you can usually keep throwing hardware at it and that is sufficient.

[22:43:22] <iszak> bros: but then you have the fault tolerance is low of course

Log file Viewer

Help | Karma | Search:

#mongodb logs for Saturday the 1st of August, 2015