pmxbot IRC Log Viewer

[02:03:42] <Guest84664> hi, is it normal for initial replication sync to build indexes a second time?

[02:09:15] <joannac> as in, it builds indexes twice during initial sync?

[02:28:36] <quackslike> hey

[02:29:28] <quackslike> looking to run mongo on aws, but doing the numbers the mongo AMI seem to run expensive for my taste

[02:31:31] <quackslike> its seems as though the piops is a large chunk of the cost, wondering if anyone has info to share about minimising the cost by lowering iops and ebs sizes, my requirements are fairly minimal as far as performance and use go.

[02:31:55] <Skaag> can I somehow trigger an action when a document in mongo is updated?

[02:32:12] <quackslike> i know there are other options but i'm currently looking at mongo on aws.

[02:34:05] <quackslike> looks like the minimal cost of the mongo ami per month is about $450 no data xfer inc.

[02:34:17] <quackslike> unless i miscalculated.

[02:35:09] <Skaag> I'll give you a VPS with MongoDB for $20 ~ $30/month

[02:35:11] <Skaag> :)

[02:36:22] <quackslike> lol

[02:36:37] <quackslike> Skaag thanks for the offer

[02:36:47] <Skaag> np, lemme know, with pleasure

[02:37:05] <Skaag> I take bitcoin too

[02:37:39] <quackslike> Given your question above, i think i'll pass. No offence ;)

[02:38:03] <quackslike> 1 bit coin would last me quite a while at that price point :D

[02:38:46] <Skaag> what question?

[02:39:02] <quackslike> I was thinking maybe just provisioning the ebs iops much lower..

[02:39:21] <Skaag> EC2 makes sense when you start scaling to 8+ machines

[02:39:22] <quackslike> but i dont know what affect that would have, ie) how much of a turd it would be.

[02:40:19] <Skaag> it depends on your actual need

[02:40:37] <Skaag> if you update a few documents once in a while, you can get a super low resources instance

[02:40:47] <quackslike> There are other options besides AWS.. but i'd like to control the infrastructure to some extent.. if you know what i mean.

[02:40:49] <Skaag> I run mongo on an ARM platform with 512mb RAM successfully

[02:41:23] <Skaag> but it isn't doing anything "heavy" by any means

[02:42:10] <quackslike> it's for a service that i'm trying to get off the ground, but i'd need to show that it's "capable" of handling growth if needed.

[02:42:38] <Skaag> to handle growth, you need to automate the creation of instances, and the addition of mongos to a replica set

[02:42:38] <quackslike> its a bit of "show" to some extent.. muscle flexing..

[02:42:54] <Skaag> (and their removal, later)

[02:43:31] <Skaag> and there are tricks to make it grow smoothly without affecting service

[02:43:53] <Skaag> my own trick is to use red, orange and green instance

[02:44:02] <Skaag> red is production, and serves queries

[02:44:14] <Skaag> orange is standby, contains a replica, but never serves requests

[02:44:32] <Skaag> when you need another instance, you create a green instance, and you make it replicate from the orange instance

[02:44:33] <quackslike> yeah, the growth wont be much more than i'd need a half decent machine with, but as i say, it's about "showing" that it's deployed on a platform that has the capability and also not controlled by some random paas provider.

[02:44:48] <Skaag> once the green instance finishes replicating, it becomes orange, and the orange becomes red

[02:45:04] <quackslike> sounds good.

[02:45:45] <Skaag> AWS gives you that buzzwordiness that bosses like, so you're covered if you use them

[02:46:38] <quackslike> thats exactly right

[02:46:53] <quackslike> 99.(11 9s) durability.

[02:46:55] <quackslike> ;)

[02:47:23] <quackslike> truth i could probably serv it from a rasperry pi

[02:55:25] <Skaag> you could

[02:55:33] <Skaag> but I wouldn't recommend it for any real work

[03:03:29] <Guest84664> joannac: sorry, yes, it looks like from logs, initial clone, build index, catch up oplog, then build index again, then catch up oplog again

[03:03:57] <Guest84664> wondering if the 2nd index build phase is normal

[03:19:33] <quackslike> no, i wouldnt use it for anything real

[03:19:54] <quackslike> i think i'll just use a hosted mongodb instead of dicking around

[03:20:22] <quackslike> the app server will live in aws so i'll pick a mongo provider that plays well with aws

[03:22:46] <Guest84664> quackslike: mongohq is pretty good, they are 2ms to aws as well

[03:43:03] <quackslike> Guest84664: thanks i intend to check them out

[03:46:43] <joannac> Guest84664: I can't reproduce that. I see different indexes being built but not the same index being built multiple times

[03:48:05] <Guest84664> joannac: sorry, it was my mistake, i realized later it was a different index. it was surprising to me though that it would build some indexes, catch up the oplog, then build other indexes - do you happen to have a pointer to some documentation that explains the process in more detail?

[03:48:52] <Guest84664> joannac: these were indexes for the same collection, it built some 2 indexes for it, then did some oplog syncing, then built another index

[03:53:50] <Skaag> ok looks like what I need is tailable cursors

[03:54:09] <Skaag> found this interesting writeup about it: http://shtylman.com/post/the-tail-of-mongodb/

[03:55:37] <Guest84664> Skaag: also check out this: https://cloudup.com/blog/introducing-mydb using similar concepts

[03:55:56] <Skaag> thank you

[03:57:18] <joannac> Guest84664: well, i only spun up a small test. But I have the index local.replset.minvalid { _id: 1 }, which can only be built once we have finished oplog sync

[03:57:22] <Skaag> beautiful use of the concept!

[03:57:54] <Guest84664> joannac: thanks for looking into it, apologies for the goose chase

[03:57:59] <joannac> Guest84664: Whereas indexes for user collections need to be created as soon as we start cloning data

[03:59:56] <Guest84664> joannac: my logs show, clone first, some indexes built, some op log syncing, then more indexes built. it eventually completed so i think everything is working as it should, I had just thought that all indexes would be built before it tried to catch up on the oplog

[04:04:38] <joannac> Guest84664: http://www.kchodorow.com/blog/2012/06/26/replica-set-internals-part-v-initial-sync/ explains why it's post-oplog sync

[04:07:35] <Guest84664> joannac: thanks

[08:36:12] <[AD]Turbo> hola

[11:12:46] <tiller> hi!

[11:12:50] <Derick> hi

[14:13:11] <remonvv> \o

[14:13:19] <ron> o/

[14:13:28] <remonvv> ron!

[14:13:35] <ron> remonvv!

[14:13:38] <remonvv> How's life?

[14:13:40] <ron> I'm coming to Amsterdam!

[14:13:54] <remonvv> Awesome. On a temporary or permanent basis?

[14:14:10] <ron> Not really, man. I'm broke.

[14:16:03] <remonvv> That's not good. Still on career pause?

[14:16:57] <ron> Yes. Let's call it that.

[14:17:20] <remonvv> I was steering away from "unemployed" intentionally ;)

[14:17:31] <ron> hehe

[14:17:57] <ron> I may have an opportunity for a part time job that would allow me to work on the project I talked with you about.. that would be cool.

[14:18:58] <remonvv> It would. Is that an "I can say yes if I want it" sort of deal or still in the process?

[14:19:33] <ron> we need to discuss payment and how much 'part time' is.

[14:20:18] <remonvv> Ah, a relatively vital part of the discussion ;)

[14:20:37] <ron> yes, but the 'fit the job' part is behind us.

[14:20:53] <remonvv> Cool. And it's a fun job?

[14:21:09] <ron> I'm excited about it. It's working on an open source project.

[14:21:25] <ron> Which could be really useful for my own project, as you can imagine.

[14:21:43] <remonvv> Indeed. What sort of project?

[14:22:06] <ron> Hmm, let's take it out of the channel :)

[14:22:20] <remonvv> Alright.

[14:35:19] <tiller> Is it possible to "$addToSet" nothing?

[14:35:35] <tiller> I mean during an upsert, I want to create an empty array if it doesn't already exists.

[14:35:53] <tiller> Or should I just use $onInsert (don't remember the name, I'll check)

[14:36:53] <tiller> huum, not in fact $setOnInsert wouldn't work, because it the entry was here without the field it won't do much

[14:37:09] <tiller> no* if*

[14:44:05] <Nodex> you dont need to create an array

[14:45:04] <tiller> I know I could just leave it "unexistant" but it won't stay empty for long, so I thought it would be "better" to create it directly

[14:49:03] <tiller> But pushAll do the trick

[14:49:28] <remonvv> tiller: It isn't better to create an empty array unless it simplifies your business logic.

[14:50:14] <tiller> it allows me not to put if ( != null) in a non-negligeable amount part of my webapp :)

[14:50:36] <remonvv> that's a good reason ;)

[14:52:21] <Nodex> or just cast it ;)

[14:52:43] <Nodex> I assuem you're using javascript with that sentence.... var m = document.a || [];

[14:52:51] <tiller> Java :)

[14:52:53] <Nodex> (where a is the array)

[14:53:06] <Nodex> surely there is that kind of logic in java though?

[14:53:37] <tiller> Don't think so. You'll need to if (tab == null) { tab = new BasicDBList(); }

[14:53:51] <Nodex> you can't do a tenery operator?

[14:54:22] <Nodex> tenary *

[14:54:23] <tiller> yes you can, but it ends the same way

[14:54:38] <tiller> tab = tab == null ? new BasicDBList() : tab;

[14:55:02] <tiller> and if you do it ealier, you'll fetch your object 2 times

[14:55:24] <tiller> tab = doc.get("orgs") == null ? new BasicDBList() : doc.get("orgs");

[14:58:18] <Nodex> can you not assign doc.get("orgs") to a variable first ?

[14:58:50] <Nodex> a = doc.get("orgs"); ....... tab=a==null ? new BasicDBList() : a;

[14:59:17] <Nodex> I don't know java at all but it seems backwards to get thigns twice... you don't have to in any other language I have used

[14:59:27] <tiller> it ends up to be exactly the same as my first proposal :) You don't nee any "a", you re-use tab

[15:19:49] <Vladimir> hello everyone

[15:19:56] <Derick> Hello Vladimir

[15:19:56] <tiller> hi

[15:20:19] <Vladimir> Could you please help me solve problem with mongodb

[15:20:31] <tiller> We can try ;o

[15:21:32] <Vladimir> I have used mongodb and it worked fine

[15:21:48] <Nodex> congrats

[15:22:30] <Vladimir> But recently I've deployed my website on heroku and mongodb crushed

[15:22:44] <tiller_> Vladimir> Can you repeat what you said before your last message please?

[15:23:02] <Nodex> tiller : he didn't say anything

[15:23:25] <Vladimir> Mongodb worked fine, untill I deployed my web app on heroku

[15:25:28] <Vladimir> Now when I'm writing in the shell: mongod --dbpath "mypath", it says: unable to create/open mongod.lock

[15:26:31] <Vladimir> and ask: is a mongod instance already running?

[15:26:44] <Nodex> rm the mongod.lock

[15:26:48] <Nodex> then restart the db

[15:27:07] <Nodex> you probably didn't shut the db down properly bfore you pushed it and it pushed the lock file

[15:27:32] <Vladimir> how to restart db proper?

[15:27:45] <Vladimir> maybe I was doing it wrong

[15:28:23] <Vladimir> mongod --restart?

[15:28:41] <Nodex> which operating system?

[15:28:49] <Vladimir> windows 7

[15:29:09] <Nodex> does heroku work on windows 7?

[15:29:15] <Nodex> I thought it was linux based :(

[15:29:37] <Vladimir> It runs well on windows 7

[15:30:20] <Derick> heh, I thought Heroku was a cloud provider?!

[15:30:30] <Vladimir> It is

[15:30:59] <Vladimir> Here is cloud service for windows Azure

[15:31:14] <Vladimir> But I like to use Heroku

[15:31:40] <Vladimir> Please help me with mongo starting

[15:33:07] <tiller> net stop mongod ?

[15:33:20] <tiller> and then net start mongod

[15:34:03] <Vladimir> will try

[15:34:04] <tiller> if you installed MongoDB as a service of course

[15:34:26] <tiller> and the wiki tells me it's MongoDB* and not mongod

[15:51:31] <Vladimir> Doesn't work

[15:52:24] <Nodex> best to ask heroku documentation how to restart it on their platform

[15:52:45] <Nodex> I would rather shoot myself in the ear than use windows on a server

[15:55:30] <Vladimir> On server side it works fine

[15:55:51] <Vladimir> It crushed on my localhost

[15:56:06] <Nodex> you didn't say that

[15:56:12] <Vladimir> sorry

[15:56:13] <Nodex> you said you're using Heroku

[15:56:59] <Vladimir> Yes, after I deployed my wep app on heroku my local db crushed

[15:57:11] <Nodex> http://docs.mongodb.org/manual/tutorial/install-mongodb-on-windows/#mongodb-as-a-windows-service

[15:57:55] <Vladimir> ok will try

[15:57:57] <Vladimir> Thanks

[15:58:31] <Vladimir> I know that windows sucks

[16:00:04] <Vladimir> But I have a lot of licensed stuff on this laptop and trying it on windows

[16:00:33] <Vladimir> Does anyone uses ubuntu?

[16:00:55] <starefossen> Vladimir: I do

[16:01:27] <Vladimir> Do you like it&

[16:01:28] <Vladimir> ?

[16:01:45] <Vladimir> It is the same as linux?

[16:01:47] <Nodex> I don;t know what that is sorry

[16:05:44] <starefossen> Vladimir: it's ok I guess

[16:06:10] <starefossen> Vladimir: personally I don't feel it's that much of a difference

[16:06:53] <Derick> Ubuntu is a variant of Linux

[16:06:57] <Derick> nobody runs just linux

[16:07:15] <Derick> There is also Debian, Fedora, Suse, Mint...

[16:07:17] <Derick> many variants

[16:09:07] <Vladimir> Great

[16:09:14] <Vladimir> I will try it out

[16:09:35] <Vladimir> Maybe after will luckily destroy my windows

[16:10:01] <Nodex> haha

[16:47:05] <dnsdds> I'm querying with a $gt and $lt on two fields; works fine with ints, but returns nothing with floats. What's wrong?

[16:49:21] <dnsdds> i.e. { location_lat: { '$gt': 19.3234, '$lt': 22.000543 }, location_lng: { '$gt': 72.7664536, '$lt': 75.543992 } } doesn't return anything, whereas location_lat: { '$gt': 19, '$lt': 22 }, location_lng: { '$gt': 72, '$lt': 75 } } does

[16:50:55] <starefossen> dnsdds: looks like you should have used a Geospatial Indexe and Query: http://docs.mongodb.org/manual/applications/geospatial-indexes/

[16:51:50] <starefossen> dnsdds: and then used a $geoWithin query: http://docs.mongodb.org/manual/reference/operator/query/geoWithin/#op._S_geoWithin

[16:52:24] <dnsdds> starefossen: Hmm... looks good, but that'd require rather a lot of rewriting on a framework core that I didn't develop... shouldn't $gt and $lt work on floats just the same?

[16:54:15] <starefossen> works for me

[16:54:25] <starefossen> db.test.insert({x: 1.1, y: 2.2})

[16:54:30] <starefossen> db.test.find({x: {$gt: 1.0, $lt: 2.0}, y: {$gt: 2.0, $lt: 3.0}})

[16:55:37] <starefossen> ?

[16:56:06] <starefossen> I'm on db.version() 2.4.8

[17:00:01] <dnsdds> starefossen: Let me check on the CLI

[17:00:10] <starefossen> -n

[17:00:18] <starefossen> s/-n//

[17:19:08] <dnsdds> starefossen: Alright, works perfectly on CLI. So probably some bug in the framework. Sorry for wasting your time and thanks for the suggestion :)

[17:19:27] <starefossen> dnsdds: glad to help :)

[19:04:27] <sikor_sxe> hello, i have the weird problem that on certain numbers the Array.sum calculation results in rounding errors: [0.8, 1.1] results in 1.9000000000000001 :/

[19:04:44] <sikor_sxe> anyone has an idea why this occurs?

[19:05:00] <kali> that's just floating number being floating numbers

[19:05:34] <sikor_sxe> wow

[19:05:36] <sikor_sxe> you're right

[19:06:18] <sikor_sxe> 0.8 + 1.1 != 1.9 in js

[19:06:23] <sikor_sxe> didn't know that

[19:06:52] <kali> sikor_sxe: same in any language, really

[19:07:03] <sikor_sxe> true

[21:08:06] <michael_mbp> hi all

[21:08:30] <barnaba> hi

[21:08:32] <michael_mbp> is there anyway to find which fs.chucks are missing a parent fs.files?

[21:08:44] <michael_mbp> *fs.chunks

[21:09:09] <michael_mbp> the document metadata for files stored in GridFS were trashed by mistake!

[23:39:54] <mst1228> Can someone assist me with some general data structuring? I'm not sure if I want a single collection for what I'm doing with lots of nested docs or if I should seperate them all to individual collections

[23:40:33] <mst1228> I'm building a little site for our office internally, we play a lot of ping-pong and foosball and general office games

[23:41:07] <mst1228> I want to do some kind of league/season/tournament management

[23:42:48] <mst1228> struggling with deciding if a league is a single collection that holds all the players/teams/matches or if I should separate those sub categories into their own collections and just refeference them from a league doc

Log file Viewer

Help | Karma | Search:

#mongodb logs for Friday the 29th of November, 2013