pmxbot IRC Log Viewer

[00:39:51] <fels_zh> hi - anyone use this wiht c#?

[00:41:05] <Boomtime> mongodb officially supports a C# driver, many people use it

[00:41:47] <Boomtime> if you have a question about the mongodb C# driver, feel free to ask it, someone may be able to help you

[00:44:47] <fels_zh> Boomtime: thx

[00:46:35] <fels_zh> http://pastebin.com/ZL8vyrjd this fails with the error: SelectMany is not supported - but as you can see I'm not using that here - I did before now I am not

[00:52:34] <Boomtime> @fels_zh: i'm having trouble finding any Mongodb related code in there, it appears your problem is with LINQ and not related to mongodb at all

[00:53:28] <Boomtime> which C# driver version are you using?

[03:11:43] <Flightless> hey guys, quick question. I'm pretty new to MongoDB here and I'm wondering if the following schema is possible/viable: My database needs to store the set of words each user knows. In SQL I'd just have a user table and a words table do a join. Since MongoDB doesn't have joins my instict is I'd be best off making knownWords a subdocument of users. If I do this would I be able to create a query that would return specific words for

[03:23:19] <Boomtime> -> "store the set of words each user knows", you mean their entire vocabulary?

[03:29:35] <Flightless> precisely, I need the database to store every word the user knows.

[03:38:36] <Flightless> The key problem is I need the ability to search for a specific word. Since the number of known words can get very high I'm going to want indexing to search through the subdocuments. Is that even possible with mongo?

[03:39:07] <Boomtime> yes, but the number of words is extreme

[03:39:37] <Boomtime> { suername: "blah", words: [ "a", "aardvark", "abercrombie" ... ] }

[03:39:48] <Boomtime> index on 'words'

[03:41:56] <Boomtime> the trouble with this is that "words" array is going to get very large, and manipulating arrays beyond a few thousand entries becomes problematic

[03:46:53] <Flightless> maybe I misunderstood something but my understanding of mongo structuring was that nesting documents is the way to get around no table joins. if I have a BSON structure like: { username: "blah", words: [{wordId: "123", definitionId: "321"}]} I'll really run into an issue at around one thousand pages?

[03:52:52] <haploid> Ok, here is some strange behavior. if I insert 50k documents letting mongo assign the _id, everything is fine. If Ido the same with a user-generated(and verified unique ) _id, it only inserts 6 of those documents, and reports no errors with safemode on

[03:52:59] <haploid> any ideas what is going on ?

[05:51:46] <LinkRage> when mongodb2.4.x is run with --auth... can I force it to NOT use authentication for specific user? I'm trying to remove the auth messages in mongod.log...

[10:49:37] <whyhankee> hi all, i'm migrating a MongoDB from 2.4.9 to 2.6.4 and i'm having trouble with a query that is returning no documents in mongo 2.6, in 2.4 the query is fine. The query is here: http://pastebin.com/ZVqsWt0R

[10:49:54] <whyhankee> is there something i'm missing?

[13:09:29] <tscanausa> Today is a scary day, I am shutting off old shard =/

[13:13:38] <jordana> tscanausa: I'm syncing 500 million documents :(

[13:14:15] <jordana> migrating away from SSD's *cries*

[13:19:20] <tscanausa> The last time I did this a portion of my data disapeared.

[13:42:05] <boo1ean> Hi everyone, I'd want to get some suggestions about which db is best to work with large statistic datasets (12 columns available for different group combinations and filtering)? Or is there some good solutions to solve that and have good performance?

[14:02:15] <jordana> Is there anyway to force a mongod repliaset node out of STARTUP2 into SECONDARY mode?

[14:02:37] <jordana> It's done with the indexes and from what I can tell it's not do anything

[14:02:46] <jordana> It's synced all the data but seems to have got stuch

[14:02:49] <jordana> stuck*

[14:03:49] <Boomtime> "from what I can tell" -> It's still in STARTUP2 for a reason

[14:04:13] <jordana> How do I find out the reason?

[14:04:22] <jordana> every command I run in the shell crashes mongo

[14:04:28] <jordana> it just hangs

[14:22:02] <saml> lols queries run 12324ms

[14:22:54] <Derick> I remember back in the days where I've had SQL queries running for an hour

[14:23:06] <Derick> I am glad to see that people now thing 12⅓ seconds is too much.

[14:23:50] <saml> i haven't seen an SQL query that queries single table taking that long

[14:23:59] <jordana> I have

[14:24:07] <saml> i'm a noob

[14:24:15] <jordana> MSSQL 2008 R2

[14:24:22] <jordana> 700 million rows

[14:24:27] <jordana> 57 minutes

[14:24:28] <jordana> to count

[14:24:30] <jordana> lol

[14:25:04] <saml> lol

[14:25:51] <jordana> Hmm I'm having serious issues with replication today, it just decided to start the resync agian

[14:25:54] <jordana> again*

[14:26:01] <jordana> 7 hours that took :(

[14:26:16] <saml> how do you resync?

[14:27:04] <saml> http://docs.mongodb.org/manual/reference/command/resync/ run it every minute

[14:27:36] <jordana> That only applies to master / slave

[14:27:38] <jordana> this is a replica set

[14:28:46] <jordana> I took a secondary down, reinstalled the OS, installed mongod and booted it with rs1 as the replSet name

[14:28:49] <saml> mongo connector?

[14:29:00] <saml> why not rs0?

[14:29:11] <jordana> I already have rs0, it's a sharded cluster

[14:29:22] <saml> so you copied rs0 to rs1

[14:29:24] <jordana> No

[14:29:49] <jordana> I have rs1 with nodes in, i took one secondary out and reinstalled the entire OS

[14:30:09] <jordana> booted it back up with replSet=rs1

[14:30:19] <jordana> mongo picked it up and it started syncing

[14:30:23] <saml> did you reformat disk?

[14:30:26] <jordana> Yeah

[14:30:45] <saml> makes sense. out of sync node joins replicaset. gets replicated

[14:31:01] <jordana> but it did sync

[14:31:08] <jordana> finished replication

[14:31:29] <saml> i don't understand difference between sync and replication

[14:31:38] <saml> replication is fast and sync is slow but it chose to do sync instaed of replication?

[14:31:42] <jordana> Sorry, I'm talking about the inital sync.

[14:32:06] <jordana> The inital sync in STARTUP2 finished copying all the collections and rebuilt all the indexes but then nothing

[14:37:31] <tscanausa> jordana: what do you mean nothing?

[14:38:49] <jordana> tscanausa: It just stayed on STARTUP2, the logs were just logging connections. Couldn't see any db stats

[14:38:54] <jordana> just hung

[14:41:12] <vrkid> hi

[14:41:26] <prbc> Is there an easy way to find docs matched with two or more different values? without making two queries and merging them?

[14:41:41] <tscanausa> jordana: give it some more time

[14:42:13] <vrkid> how do I setup ldap authentication when the OpenLDAP server I contact acts as a LDAP proxy to an AD (Windows Server 2008) LDAP?

[14:44:27] <jordana> tscanausa: well I left it over night last night this same thing happened. I rebooted the mongod instance and thinking it would sort itself out and it just started the sync again

[14:45:03] <tscanausa> oh i think the op log is too far behind… try rsyncing the data files?

[14:48:17] <jordana> Yeah, I've stopped all processing on our primaries and started the resync again. Hopefully it'll sort itself out

[14:48:39] <tscanausa> i ment rsync or scp

[14:50:18] <jordana> Yeah I know but I'll give it one more shot.

[15:27:01] <tscanausa> Woot old shards are down and no alerts or missing data

[15:42:24] <hydrajump> anyone running mongo in docker containers for production?

[17:20:38] <linuxboytoo> I am working on a legacy mongo system (2.4.5) that I adopted - looking to add a replica to the set. It is not sharded but they are using mongos on the clients to connect. If I do the rs.add on the primary mongod so I need to do anything for the configservers to get the updated hostnames for mongos?

[18:07:52] <nicolas_leonidas> I have a mongodb instance that is just too busy CPU usage is so high... Is it replication that I need or sharding?

[18:40:09] <tscanausa> nicolas_leonidas: did you get an answer?

[18:47:45] <nicolas_leonidas> tscanausa: nope

[18:49:41] <tscanausa> Replication generally only adds fail safes for data ( and will add more cpu usage ) . sharding could be a solution or a bigger server is another and possibly better option

[18:54:50] <nicolas_leonidas> is it possible to have master/master replications with mongo? so all instances can both write and read?

[18:57:41] <redblacktree> Using the mongo profiler, I'm having some trouble understanding the output. An update operation has an nscanned of 100663554. First of all, that seems like a very large number to scan in any circumstance (100 million??). Second, the collection being updated only has 25 documents. Am I reading this wrong? Can anyone explain what might be happening?

[18:59:46] <tscanausa> nicolas_leonidas: no. you would need a second shard.

[19:10:32] <Demolidor> #buildout

[19:11:41] <cheeser> #nonsequitur

[19:31:18] <Diplomat> Hey guys, can I ask a PHP & MongoDB related stuff here ?

[19:34:16] <Diplomat> Well, I'll ask.. $update = $collection->update(array('_id' => $id), array('$set' => array('hostname' => $hostname, 'ip' => $ip)));

[19:34:31] <Diplomat> But when Im print_r last error.. it doesn't show that anything was updated

[19:34:34] <Diplomat> but ID is correct

[19:34:58] <Derick> you might have to wrap it in a: new MongoID($id) ?

[19:35:13] <Derick> (and yes, you can ask those questions in here)

[19:35:40] <Diplomat> ohhhhhhhh

[19:35:46] <Diplomat> yes, totally forgot that, thank you

[19:36:53] <redblacktree> can anyone suggest ideas about why I might have very large nscanned values (>100 million) using mongo profiler on update to a very small collection (25 documents)?

[19:37:33] <redblacktree> the query isn't super slow. It took 63 ms

[19:40:38] <rafaelhbarros> how does mongodb textScore works?

[19:41:37] <Diplomat> Weird.. it still doesn't work

[19:41:40] <Diplomat> Array

[19:41:41] <Diplomat> (

[19:41:41] <Diplomat> [connectionId] => 5

[19:41:41] <Diplomat> [n] => 0

[19:41:41] <Diplomat> [syncMillis] => 0

[19:41:42] <Diplomat> [writtenTo] =>

[19:41:44] <Diplomat> [err] =>

[19:41:46] <Diplomat> [ok] => 1

[19:41:48] <Diplomat> )

[19:41:50] <Diplomat> ops

[19:41:52] <Diplomat> my ba

[19:41:55] <Diplomat> d

[19:41:56] <Diplomat> but that's the result

[19:42:07] <Diplomat> I thought it was in straight line.. not like that :/

[19:42:20] <tscanausa> I dont think mongo returns the updated upject

[19:43:02] <Diplomat> Well, it didn't update anything and yes it returns updated object usually

[19:43:18] <Diplomat> I mean after insert i think

[19:43:52] <redblacktree> http://pastebin.com/Ad110DVS

[19:44:46] <Diplomat> Here's my PHP: http://pastebin.com/uSAMACsu

[19:49:40] <Derick> it doesn;t return the updated document

[19:49:58] <Diplomat> I just found out what's wrong

[19:50:06] <Diplomat> I misplaced 1 )

[19:56:29] <hydrajump> no one running mongodb in docker containers in production who wouldn't mind sharing details?

[20:52:00] <ramfjord> god I hate this database

[20:52:17] <ramfjord> anyone know why this query has stopped using its index?

[20:52:25] <ramfjord> index on id_from_source

[20:52:52] <ramfjord> ex query: { account_id: blah, other_thing: blah, id_from_source: blah }

[20:53:05] <ramfjord> take out the account_id and other_thing, and it's using the index again

[20:53:18] <ramfjord> upgrade to 2.6 caused this problem

[20:55:16] <tscanausa> ramfjord: do you have the explain?

[20:56:46] <ramfjord> tscanausa: 1 sec

[20:58:55] <ramfjord> hmmm

[20:58:59] <ramfjord> perhaps driver issue

[20:59:10] <ramfjord> when I run the query in the shell it uses the index

[20:59:54] <tscanausa> ramfjord: if you want you can hint the index to use through most clients, ( not recommended but eh…)

[21:00:28] <tscanausa> the index changes in 2.6 are extremely nice, whihc is why i recommend not hinting.

[21:00:44] <saml> ramfjord, hint

[21:01:14] <saml> okay don't use hint

[21:01:28] <saml> ramfjord, which index do you expect it to use?

[21:01:35] <saml> and what are the values of blah ?

[21:01:45] <saml> it really depends on values of blah

[21:01:52] <saml> the way mongodb chooses which index to use

[21:02:48] <saml> i had a date field. and an index {date:1}

[21:03:49] <saml> and query was {date: SOME_DATE, ...} and it used {date:1} index when SOME_DATE did not match any doc's date field

[21:04:03] <saml> it stopped using the index when some docs had date = SOME_DATE

[21:04:11] <saml> so i had to hint

[21:04:58] <ramfjord> intersetingly enough the explain plan says it's going to use the id_from_source index, even within the driver query

[21:05:05] <ramfjord> using Moped, ruby driver for mongo

[21:05:18] <ramfjord> but it's clearly not using the index when the query is executed

[21:06:07] <tscanausa> how do you know?

[21:06:34] <ramfjord> because the driver actually has a built in explain method on the Query object

[21:07:04] <ramfjord> explain["indexBounds"] => {"id_from_source"=>[["cde0620c-6c6e-4014-9564-aa6465af808d", "cde0620c-6c6e-4014-9564-aa6465af808d"]]}

[21:07:14] <ramfjord> oh, how do I know it's not using it?

[21:07:18] <tscanausa> so you are saying the client is not returning the same explain from the explain call then the built in driver one?

[21:08:09] <ramfjord> er...

[21:08:19] <tscanausa> forget that.

[21:08:30] <ramfjord> driver is giving the same explain

[21:08:36] <tscanausa> how do you know the query through your client is not using the index?

[21:08:44] <ramfjord> but when it executes the query it is clear from the time it takes that it's not using the index

[21:08:53] <ramfjord> average time spent is over 1700 ms

[21:09:12] <ramfjord> on a collection of size 170K

[21:09:17] <ramfjord> waaaaay too high

[21:09:54] <tscanausa> sounds like a client issue. can you do a quick test with the mongodb ruby client?

[21:10:17] <ramfjord> yeah, I suspect because this gem is outdated

[21:10:37] <ramfjord> we had to switch to a different driver on another database to get a cursor back from aggregate - Moped doesn't support that

[21:10:49] <ramfjord> what mongodb ruby client?

[21:11:18] <tscanausa> the mongodb ruby client http://docs.mongodb.org/ecosystem/drivers/ruby/

[21:15:03] <ramfjord> ugh

[21:15:12] <ramfjord> actually when I use the driver directly to query the index is used

[21:15:20] <ramfjord> somewhere in the Mongoid layer

[21:16:58] <tscanausa> so ti sounds like the client or framework

[21:36:20] <freeone3000> I'm using MMS for montioring on a 8-server replicaset with 7 voting members, two of which have non-zero priority. I'm seeing around 800 connections; is this odd?

[23:08:16] <ramfjord> saml: hey, you still there?

[23:08:24] <ramfjord> tscanausa: you also?

[23:08:43] <ramfjord> Hey guys

[23:08:57] <ramfjord> I upgraded our mongo to 2.6.4, and certain queries have become very, very slow

[23:09:07] <ramfjord> > 1 second

[23:09:44] <ramfjord> raw_ad_line_item.find(account_id: blah, other_thing: blah, id_from_source: blah) is perfectly fast

[23:09:50] <ramfjord> it uses the index on id_from_source

[23:10:00] <ramfjord> and always returns 1 document

[23:10:10] <ramfjord> the problem is if it's sorted

[23:10:22] <ramfjord> raw_ad_line_item.find(account_id: blah, other_thing: blah, id_from_source: blah).sort(_id: 1)

[23:10:26] <ramfjord> takes > 1 second

[23:10:45] <ramfjord> explain generates the exact same plan, with nscannedObjects: 1, both using the index

[23:10:57] <ramfjord> the only difference is scanAndOrder

[23:19:21] <ramfjord> if I hint it to use the id_from_source index, the query is fast even with the source

[23:20:00] <ramfjord> it generates the exact same explain once again, except that nscannedObjectsAllPlans and nscannedAllPlans are both 1 instead of 15 and 18 respectively

[23:33:59] <hurdcatzDOTcom> In honor of the recent Oracle announcement, http://hurdcatz.com now points to MariaDB

[23:57:33] <ramfjord> So, we had one useless index

[23:57:49] <ramfjord> and somehow the planner was querying against it before deciding on the correct plan?

[23:58:02] <ramfjord> deleting it caused a huge speedup

Log file Viewer

Help | Karma | Search:

#mongodb logs for Thursday the 18th of September, 2014