pmxbot IRC Log Viewer

[00:16:30] <jedir0x> does the java driver automatically select slaves when reading and the master when writing?

[01:53:06] <arex\> Anyone familiar with MongoDB Replica Sets who can take a look at my planned setup (Two Replica Members and an Arbiter)? https://www.lucidchart.com/publicSegments/view/506f8d6d-4d48-4092-9d20-11940a460d7f/image.png

[01:53:20] <arex\> In particular I'm wondering if it makes sense to have 3 NICs as shown in the diagram.

[01:59:15] <arex\> kali: As you can see, I made a server dedicated to arbitration :-D

[03:03:43] <quazimodo> any plans for transactions in mongo?

[03:30:29] <LouisT> say i use db.foo.find(), how can i search for part of a string within the results?

[03:32:05] <crudson> LouisT: use a regular expression, but be aware that only matching the start of a string will use an index

[03:32:36] <crudson> http://www.mongodb.org/display/DOCS/Advanced+Queries#AdvancedQueries-RegularExpressions

[03:49:07] <LouisT> crudson: oh ok, got it, thanks

[04:03:20] <crudson> cool

[04:11:14] <tystr> hello

[04:11:33] <tystr> I've got a replica set consisting of 2 nodes + 1 arbiter node

[04:12:05] <tystr> I just had the primary randomly step down and I'm trying to figure out why that happened...

[04:12:19] <tystr> can anyone give me some direction as to what i should be looking for in the logs?

[04:49:55] <crudson> some error condition in the primary I'd expect or network interruption perhaps

[04:55:15] <tystr> ya the server status seem good (very low cpu load, iowait is < 1%, memory usage < 50%)

[04:55:27] <tystr> and I've gone through the logs and can't find anything unusual

[04:55:37] <tystr> expect the notification that the other hostname is now the primary

[04:59:31] <aster1sk> Super basic mongodb url shortening http://5kb.us/

[05:00:01] <aster1sk> I'll open it up if anyone wants, though it is throwaway code.

[05:02:41] <aster1sk> We're using it on our IRC hosts to auto-shorten urls.

[05:03:09] <aster1sk> Anyway thanks for looking.

[05:23:05] <quazimodo> I have an embedded document foo but bar.foo returns [] in rails

[05:23:13] <quazimodo> that aint meant to happen?

[05:24:38] <quazimodo> hrm this is odd

[05:32:45] <crudson> using what mapping framework?

[05:34:44] <tystr> under what network connectivity circumstances would a primary node in a replica set step down?

[05:34:49] <crudson> maybe you have it declaratively as an Array in the model but it's in fact a Hash

[05:35:55] <crudson> tystr: I wish I could offer you more but I am not an expert on replsets other than simple cases that the documentation goes over http://docs.mongodb.org/manual/core/replication/

[05:36:59] <tystr> I finally found out that amazon aws is having network connectivity issues which I assume is responsible for my primary stepping down

[05:37:11] <crudson> that's what we thought initially :)

[05:37:26] <tystr> ya me too, but they didn't update their status dashboard until a couple of min ago

[05:37:34] <crudson> right

[05:37:42] <crudson> well at least it's behaving like it should do

[05:37:45] <crudson> no mystery

[05:37:46] <tystr> ya

[05:37:54] <tystr> and I'm not crazy for not finding an error somewhere heh

[05:38:02] <crudson> exactly.

[05:38:19] <crudson> if anything it should make you reassured

[05:38:45] <tystr> yeah…now I'm just curious as to what exactly causes it to step down

[05:38:48] <tystr> oh

[05:38:59] <tystr> well maybe the secondary couldn't find the primary and promoted

[05:39:07] <crudson> that's what it should do in times of network interruption

[05:39:12] <tystr> then when the primary saw the other one it stepped down….

[05:39:16] <tystr> that makes sense

[05:39:39] <crudson> good for you for having a replset established in the first place :)

[05:43:23] <quazimodo> crodas: mnogoid mate

[05:43:24] <quazimodo> erm

[05:43:38] <quazimodo> crudson: mongoid. I'm having difficulty in my brain understanding why :/

[05:43:51] <crudson> quazimodo: what is that attribute defined as in your model?

[05:44:05] <quazimodo> class foo embeds_many :bars

[05:44:23] <quazimodo> but when I check the document, the attribute is called bar

[05:45:26] <crudson> that is if you are referencing another collection rather than just an embedded attribute

[05:45:42] <crudson> so be sure that is what you want

[05:46:04] <quazimodo> I'm not sure the way to go here

[05:46:08] <crudson> otherwise field :name, type: Hash may be what you want

[05:46:22] <tystr> I wonder if I had a secondary node in another region if it would be able to keep up with the primary...

[05:46:26] <quazimodo> well i mean

[05:46:34] <quazimodo> foo *does* end up having many bars

[05:46:45] <quazimodo> and i need to be able to foo.bars.where(conditions)

[05:47:03] <crudson> ok so bar is another collection

[05:47:12] <quazimodo> then bar should not be embedded?

[05:47:58] <quazimodo> zzz

[05:48:25] <quazimodo> this is like a schemaless, not transactional regular db so far

[05:48:39] <crudson> it depends what your document structure is. If it's just { bars: [{}, {}, {}] } then use Array field

[05:48:52] <quazimodo> each bar has its own stuff

[05:48:59] <quazimodo> but it deosnt nest

[05:49:04] <crudson> ok

[05:49:07] <quazimodo> and those bars are only related to foo

[05:49:33] <quazimodo> otherwise im gonna have a few hundred thousand bars floating around

[05:51:22] <crudson> been a while since I used mongoid, just refreshing...

[05:52:01] <crudson> can you create a paste of an example document?

[05:53:17] <quazimodo> hold up :)

[05:54:09] <crudson> sure your attribute shouldn't be 'bars' rather than 'bar'? - not sure it make assumptions about pluralisation

[05:56:55] <quazimodo> https://gist.github.com/3ee95de1c826d1404556

[05:57:44] <crudson> and an example document from your collection?

[05:58:03] <quazimodo> its at the bottom

[05:58:13] <crudson> ah yea

[05:58:35] <quazimodo> i forgot to paste in the include Mongoid::Versioning

[05:59:09] <crudson> embeds_many should imply an array, no? seems like it should be bars: [{description:'aosu'}]

[05:59:20] <quazimodo> uhuh

[05:59:35] <quazimodo> how do I actually *add* to the array?

[05:59:52] <crudson> see http://mongoid.org/en/mongoid/docs/relations.html#embeds_many

[05:59:56] <quazimodo> @foo.bars.create(description: "bla") just returns a bar object and doest add it

[06:00:04] <crudson> well it needs to be an array in the first place

[06:00:10] <quazimodo> ive read that im sure

[06:00:11] <quazimodo> oh

[06:00:12] <quazimodo> what

[06:00:30] <crudson> bar: {} is not compatible with embeds_many

[06:00:58] <quazimodo> but hold on

[06:01:03] <quazimodo> *I* didnt do any of that

[06:01:10] <quazimodo> just does it by itself :/

[06:02:08] <crudson> sure you didn't insert documents then redefine your model? embeds_one would result in that I would expect.

[06:03:51] <quazimodo> um

[06:04:00] <quazimodo> is .relations something important

[06:04:04] <quazimodo> like a keyword or method?

[06:04:43] <quazimodo> what the hell is going on lol

[06:04:46] <quazimodo> ok

[06:04:49] <crudson> I'd go over the examples here and match what you want :) mongoid.org/en/mongoid/docs/relations.html

[06:04:55] <quazimodo> hold up something nuts is going on

[06:05:06] <quazimodo> @foo returns a document like i showed you

[06:05:16] <quazimodo> @foo.bars returns all the bars I made

[06:05:17] <quazimodo> 0_0

[06:05:48] <quazimodo> where deos that data get stored hahaha

[06:06:13] <crudson> you need to have a mongo shell running to check the data in the collection

[06:06:32] <crudson> or irb and talk to the mongo ruby driver

[06:06:48] <quazimodo> ok, im retarded

[06:06:56] <crudson> verify what is happening in the db independent of what you are seeing with mongoid

[06:07:02] <quazimodo> all this time my brain didnt ermind me that irb isnt mongo

[06:07:14] <quazimodo> yep yep

[06:07:21] <quazimodo> sorry crudson, im clearly tarded

[06:07:43] <crudson> flush the collection too to avoid any old data misguiding you

[06:08:04] <crudson> no it's ok :)

[06:08:24] <quazimodo> at this point im not certain, though

[06:08:33] <quazimodo> should I embed or reference

[06:09:02] <quazimodo> my form created an attribute... instead of embedding... will have to fix that

[06:10:34] <quazimodo> Ahhhhhh i see

[06:10:39] <quazimodo> ok so embedding is a pain in the ass

[06:14:14] <crudson> :)

[06:19:12] <quazimodo> so... i dont get it

[06:19:20] <quazimodo> this doesnt feel that different from a relationaldb

[06:19:38] <quazimodo> i mean, i'm not making any schema but thats about it so far

[06:20:03] <quazimodo> maybe its because rails turns table based data into object models anyway,

[07:40:54] <tpae> hello.. i read somewhere that it's difficult to create analytics with mongodb.. is that true?

[07:50:02] <crudson> there is a great presentation on this - one sec

[07:57:46] <crudson> http://www.10gen.com/presentations/mongosf-2012/mongodb-for-analytics but also search analytics on this page http://www.10gen.com/presentations

[08:46:10] <tpae> Is there a way to "if exists, return. if not, insert" ?

[08:47:22] <algernon> that's called an upsert

[08:47:31] <tpae> upsert is mainly for updating, isn't it?

[08:47:44] <algernon> it can also insert.

[08:48:13] <tpae> i was thinking in terms of retrieval

[08:48:29] <tpae> if data exists, return the data. if it doesn't exist, insert the data.

[08:48:45] <algernon> but if you already have the data, why would you want to return it?

[08:49:57] <tpae> there's a field i need to check, which by default is 0 when inserted. if it exists, i need to check whether that field is 1 or 0.

[08:57:48] <kali> tpae: lookat http://docs.mongodb.org/manual/reference/command/findAndModify/

[09:02:01] <tpae> kali: if both new and upsert is set to true, it will return the document ? "In version 2.2, the findAndModify command returns null when upsert is true."

[09:04:34] <kali> tpae: not sure, but it's easy to try

[09:05:50] <tpae> thank you ! :D

[09:43:38] <arex\> Good morning

[09:46:02] <arex\> Anyone familiar with MongoDB Replica Sets who can take a look at my planned setup? https://www.lucidchart.com/publicSegments/view/506f8d6d-4d48-4092-9d20-11940a460d7f/image.png

[09:46:07] <arex\> In particular I'm wondering if it makes sense to have 3 NICs as shown in the diagram

[09:54:33] <kali> arex\: i would use the same NIC for arbiter and replication. the whole point of the arbiter is to diagnose faults, so it should be "in band"

[10:07:36] <arex\> kali: That makes sense.

[10:07:53] <arex\> kali: The second NIC/network may be unneccessary

[10:08:19] <arex\> For both the WSFC nodes and the MongoDB Replica Set members

[10:38:45] <arex\> http://serverfault.com/questions/435468/how-do-i-network-this-windows-failover-cluster-and-mongodb-replica-set-diagram

[10:39:14] <arex\> I posted it to serverfault if anyone is interested. I included two diagrams (proposed designs)

[13:43:28] <arex\> kali: I re-organized the diagram and added a web server. Does this still make sense to you? https://www.lucidchart.com/publicSegments/view/507008a7-db5c-41c9-ac22-652a0a7c4e7c/image.png

[13:44:56] <arex\> kali: Ops, wrong link. Here is the correct one: https://www.lucidchart.com/publicSegments/view/507034cb-98c4-4d05-b1db-3b340ad6e19c/image.png

[13:56:12] <aster1sk> Here's an updated MongoDB url shortener http://5kb.us/

[13:56:28] <aster1sk> Let me know what you think, there's no captcha and the API is super simple.

[13:56:48] <aster1sk> I know there's a ton of them but I figured I'd take a stab at it, perfect project for mongo that's for sure.

[14:05:14] <arturhoo> Hi there! I have a 15M documents collection, and they have a certain field which is indexed. The index size is 400MB and fits nicely in RAM. When I try to count all the documents which have a certain value for this field, the operation takes a long time to complete (the disk is being sequentially scanned). Shouldn't it be quick, as this information can be taken from the index? Thanks

[14:06:07] <kali> arturhoo: you're hitting this one https://jira.mongodb.org/browse/SERVER-1752

[14:12:54] <arturhoo> kali: thanks! I took a look at it, and it is sad to see that this issue hasn't been addressed. I should expect poor performance on my aggregation operations that have $match as well, right?

[14:14:18] <kali> arturhoo: i can't see the link

[14:16:52] <arturhoo> kali: no worries, thanks!

[14:19:08] <arex\> morning, kali

[14:26:35] <kali> arex\: hi

[14:37:32] <arex\> Updated :) https://www.lucidchart.com/publicSegments/view/507034cb-98c4-4d05-b1db-3b340ad6e19c/image.png

[14:43:09] <arex\> People are teling me i don't really need to have a private network for internal replica set traffic

[14:43:16] <arex\> What do you think about that, kali?

[14:43:34] <arex\> "Mongo is not as cluster-featured as MS is, so you can very well get away with only the single NIC in MongoDB"

[14:52:21] <kali> arex\: i would keep it simple too

[16:10:36] <wahben> Hi all, I am having trouble understanding how to limit the $nearSphere query with $maxDistance. What am I supposed to enter in $maxDistance? Kilometers?

[16:12:39] <wahben> Actually nevermind, I think that I figured it out.. anyways too bad I can't use maxDistance with mongoengine :P

[16:26:10] <wahben> Question I have a query as follows: db.base_ad.find( { geo_point: { $nearSphere: [45.521046, -73.585507], $maxDistance: 2/6371.0 }}) which should return all items in a radius of 2 kilometers from the given lat/ long pair. However, it returns locations up to 7 kilometers in range. Why is that?

[17:02:39] <ShiintoRyuu> hello. Ive been having this idea of a database system architecture that could be helpful for developing and rapid deployment on a live system, now Im wondering if this is anywhere near common practice. I was thinking of using replica sets between my hosted server and my home pc, so that I can turn my pc off at any time and whenever its online, all data on mongodb will be replicated.

[17:02:55] <ShiintoRyuu> does anyone have experience with that?

[17:11:44] <wahben> Does anyone know what formula Mongodb uses to filter results using nearSphere or centerSphere?

[17:31:52] <IAD> wahben: Pythagorean theorem? =)

[17:33:16] <kali> IAD: :)

[17:33:39] <kali> IAD: i think there is some kind of correction for distance to be correct on a spere :)

[17:34:42] <IAD> =)

[17:40:50] <wahben> IAD: Just found it.. it uses something called n-vector

[17:41:10] <wahben> which is less accurate than Haversine formula.. this explains the discrepancies I was getting

[19:50:42] <driscoll> hi everyone, quick question...

[19:50:56] <driscoll> part 1: can i view open cursors in the interactive shell?

[19:51:07] <driscoll> part 2: can i close open cursors manually in the interactive shell?

[19:51:26] <driscoll> i had a python script die with a cursor open set to no-timeout

[19:51:44] <driscoll> my assumption is that the cursor is still open somewhere in memory. am i right?

[19:51:48] <driscoll> thanks in advance!

[22:23:53] <dorzey> where's the best place to ask questions about the java driver?

[23:07:46] <arex\> Question about replica sets... When you connect to it, what hostname do you specify? What if that server is down? Is this handled in application code? I know SQL Server has an "Availability Group Listener Virtual Network Name" for this...

[23:08:00] <arex\> by "that server" i mean "that replica member"

[23:42:25] <arex\> To answer my own question, I guess you can specify all the replica member hostnames in the code, and it'll try each one until it finds one that works

Log file Viewer

Help | Karma | Search:

#mongodb logs for Saturday the 6th of October, 2012