pmxbot IRC Log Viewer

[00:19:07] <kba> is there a reason why mongodb doesn't use sequential ids by default? I found a guide to changing it, which I was considering, since urls with a 24-length hex string isn't that nice

[00:19:31] <StephenLynx> if I am not mistaken

[00:19:44] <StephenLynx> its because the _id is like a hash of the object or something.

[00:20:01] <kba> if the object changes, I don't think the _id changes, does it?

[00:20:02] <StephenLynx> the object is more than its 24 digit string.

[00:20:06] <StephenLynx> no, it doesn't.

[00:20:21] <Razerglass> hey im using the find one function, and trying to print out the DB document it found, but im having trouble going 'deeping' paste the ID

[00:20:31] <kba> StephenLynx: so what is it a hash of?

[00:20:33] <Razerglass> how do i access specific property from my document

[00:20:49] <StephenLynx> the id object.

[00:20:51] <joannac> Razerglass: what?

[00:20:56] <StephenLynx> the id is not just a string

[00:20:56] <Razerglass> http://pastebin.com/STqUTLEy

[00:21:10] <Razerglass> its returing { _id: 556656fc0fe0c9d01bed7696, __v: 0 }

[00:21:10] <StephenLynx> it is an object with a bunch of information

[00:21:16] <kba> StephenLynx: I see. So why is it more complicated than just a string or numerical id?

[00:21:17] <Razerglass> but i want a peice of the document not the ID

[00:21:21] <joannac> kba: http://docs.mongodb.org/manual/reference/object-id/

[00:21:38] <StephenLynx> probably because mongo is designed to hold huge amounts of data.

[00:21:55] <Razerglass> in my example would i access the object with like doc[0].profileUrl or something?

[00:22:09] <StephenLynx> but I haven't looked into the beast's underbelly to know for sure.

[00:22:17] <StephenLynx> GothAlice would probably know.

[00:22:18] <joannac> Razerglass: looks like that document has no profileUrl field

[00:22:39] <Razerglass> the bottom snippet is from my DB

[00:22:57] <Razerglass> how do i access profileUrl? its in the DB and it found that document with the same ID

[00:23:16] <joannac> Razerglass: but your app isn't printing it out?

[00:23:27] <Razerglass> nope, just { _id: 556656fc0fe0c9d01bed7696, __v: 0 }

[00:23:35] <Razerglass> im missing some command to go into the document deeper i think

[00:23:55] <Razerglass> like console.log(doc[0].profileUrl) or something

[00:24:03] <joannac> Razerglass: you know that {_id: "..."} and {_id: {ObjectId("...")} are different right?

[00:24:18] <Razerglass> no i did not

[00:24:32] <joannac> ...well now you know

[00:24:40] <Razerglass> is findOne not what i need to use?

[00:25:07] <joannac> I have no idea what you want to use

[00:25:30] <joannac> I'm telling you the document you get and the document you want are 2 different documents

[00:25:40] <kba> so if I want some nice id, should I just create another field, or would it be okay to make the _id sequential?

[00:25:53] <joannac> kba: if you want to generate your own id, feel free.

[00:26:01] <joannac> but then you need to make sure they don't clash

[00:26:32] <StephenLynx> kba I prefer to create a new unique index.

[00:27:17] <kba> I was just wondering if this technique was bad http://docs.mongodb.org/manual/tutorial/create-an-auto-incrementing-field/

[00:27:33] <kba> I guess I'd lose the possibility of using getTimestamp(), but would it cause other issues down the road?

[00:28:00] <kba> I can imagine if I'd need sharding eventually, having a sequential id would be troublesome

[00:28:04] <StephenLynx> probably not bad, since its from 10gen official material, but it doesn't mean it is the most intuitive solution either.

[00:28:33] <StephenLynx> its much easier to just use findOneAndUpdate and use an $inc operator.

[00:28:43] <StephenLynx> instead o setting all that stuff on that page.

[00:30:24] <StephenLynx> or if you want to make sure everything goes fine, just get the value, try to use the incremented value, and keep increasing if it hits a duplicated index. when you are done, update with the incremented value.

[00:32:39] <StephenLynx> ah, I just read the page.

[00:32:47] <StephenLynx> is what I do, mostly

[00:33:03] <kba> so I'd just search to find the highest id and increment that?

[00:33:22] <kba> that seems a little messy

[00:33:28] <StephenLynx> I like to pre-aggregate it.

[00:33:48] <kba> what do you mean, StephenLynx?

[00:33:55] <StephenLynx> with forums, for example

[00:34:02] <StephenLynx> each forum has its own internal counting.

[00:34:13] <StephenLynx> for threads

[00:34:24] <StephenLynx> so I store the latest generated id on the forum.

[00:34:38] <StephenLynx> I just have to handle a single document to get the highest id.

[00:34:52] <StephenLynx> instead of get all threads, sort them, and get the first.

[00:34:54] <kba> right, but then I'd need a collection just for that

[00:35:11] <StephenLynx> if you wouldn't have a collection that would pivot your other collection

[00:35:22] <StephenLynx> you don't need this custom auto-incremented index.

[00:35:29] <StephenLynx> just use the _id.

[00:35:47] <kba> what do you mean pivot? I need the custom index because it's more readable

[00:35:55] <StephenLynx> why do you need to read it at all?

[00:36:04] <kba> because it'll be in the URL

[00:36:06] <StephenLynx> if the only unique index is the _id?

[00:36:26] <kba> I don't want my URLs to include 24-length hex string

[00:36:29] <StephenLynx> if it will be in the url, I assume it has some other unique field?

[00:36:42] <kba> No, why is that a given+

[00:37:07] <kba> if we look at your forum analogy, 2 threads can have the same contents or title or whatever

[00:37:08] <StephenLynx> so all these entries belong to nothing? they are in the top-level of your schema?

[00:37:17] <StephenLynx> that is why I have their ID

[00:37:21] <StephenLynx> that is pivoted by the forum.

[00:37:31] <StephenLynx> and the forum has an unique uri.

[00:37:33] <kba> they belong to users, but they don't belong to users in the scheme, they have the users referenced, though

[00:37:40] <kba> since I'll be pulling out this stuff as a list, regardless of the owner

[00:37:57] <StephenLynx> still, they are pivoted by the users, aren't they?

[00:38:05] <StephenLynx> each user will own his own set of entries.

[00:38:13] <kba> I don't know what you mean by pivot

[00:38:23] <kba> yes, a user own a set of entries

[00:38:36] <StephenLynx> so you can store the highest id of entry of the user in the user itself.

[00:38:57] <kba> in the database, the entries don't belong to the users, per se

[00:38:58] <StephenLynx> and use in the url the unique index of the user and the generated id of the entry

[00:39:01] <kba> they have their own scheme

[00:39:11] <kba> since that'd be faster when listing all entries

[00:39:15] <kba> (I assume)

[00:39:31] <StephenLynx> 2 things:

[00:39:32] <kba> the users are rarely of concern, people don't search for users, they search for the entries

[00:39:51] <StephenLynx> 1- having a 24 long string is an url is not an issue at all.

[00:40:01] <StephenLynx> I don't think your users will have to type these strings.

[00:40:01] <kba> I think that's a subjective matter

[00:40:10] <StephenLynx> no, its a practical matter.

[00:40:20] <StephenLynx> because it is causing you to model your schema after it.

[00:40:31] <StephenLynx> so is not subjective at all.

[00:40:31] <kba> only in the sense that I want a readable id for each entry

[00:40:37] <StephenLynx> why?

[00:40:45] <StephenLynx> will anyone have to manually enter it?

[00:40:47] <StephenLynx> to memorize it?

[00:40:52] <kba> they may

[00:40:55] <StephenLynx> they may?

[00:40:57] <StephenLynx> may?

[00:41:02] <StephenLynx> do you even know what you are developing?

[00:41:06] <StephenLynx> what are your requirements?

[00:41:09] <StephenLynx> what is your design?

[00:41:22] <kba> you're not going to be able to change my mind about whether I want readable ids :)

[00:41:41] <StephenLynx> so just model your database after your whims.

[00:41:49] <kba> a sequential or pretty url isn't an outrageous demand

[00:42:08] <StephenLynx> the problem is the compromise needed to achieve that.

[00:42:11] <kba> that is what I was attempting, StephenLynx, I was asking about advice on how to best do it

[00:42:17] <StephenLynx> and I told you.

[00:42:32] <StephenLynx> then you gave me a completely arbitrary reason to discard my advice.

[00:42:36] <kba> to create a collection just for that

[00:42:43] <kba> now you're misunderstanding, then

[00:42:45] <StephenLynx> no, because you already have your users.

[00:42:52] <StephenLynx> you can pivot the entries after them.

[00:43:06] <StephenLynx> I think its impossible your entries cannot be pivoted after anything.

[00:43:23] <kba> I can't immediately think of something

[00:43:35] <kba> in my database, they're a "top-level scheme" or what you'd like to call them

[00:43:47] <Razerglass> ok so i understand the document ID's are different, http://pastebin.com/XF6M2FFD but what am i doing wrong that isnt printing out my whole document?

[00:44:18] <Razerglass> im actually using findOne, sorry it isnt in the example i had changed it

[00:44:22] <kba> StephenLynx: but even if they were in the users' schemes, I wouldn't want the ids to be /<username>/<entryid>/

[00:45:44] <kba> the url, I mean.

[04:14:38] <grazfather> flaksdjf

[04:16:53] <grazfather> I am trying to do an aggregate where I combined objects together that were like id 1, field a value A, id 1, field b, value B so they combined to id 1, a:A, b:B, and I am matching the id using $in: [list]. I am getting "A pipeline stage specification object must contain exactly one field."

[06:06:36] <SNow> Hello

[06:07:20] <SNow> I did rs.remove arbiter and I have added it back again and the state is

[06:07:23] <SNow> "stateStr" : "REMOVED",

[06:17:30] <Boomtime> SNow: shutdown the arbiter, check the status while it is down - the config may not have updated on the arbiter

[06:17:56] <Boomtime> if so, you can remove the content of dbpath on the arbiter then start it up again

[06:19:18] <Boomtime> (note that arbiters do store a very small amount of information in dbpath for the local config)

[06:25:35] <grazfather> Hey guys, I am trying to do an aggregate where I combined objects together that were like id 1, field a value A, id 1, field b, value B so they combined to id 1, a:A, b:B, and I am matching the id using $in: [list]. I am getting "A pipeline stage specification object must contain exactly one field."

[07:18:16] <yeahiiiiii> hey, I am new to mongodb and struggle a bit with my schema. lets say I have a products collection and a category collection. common many to many relation. Products could either store a list of category ids and I do a second lookup or store category names so I can fetch all information in one query. In the latter case, when I update a categories name, how can I ensure that this update will be propagated to all products and their category list?

[07:58:06] <foofoobar> Hi. I have this aggregate to group my „pageviews“ and display two values: hits and uniques per day. This works as it should: http://hastebin.com/avayizuzad.js

[07:58:38] <foofoobar> As you can see I also do an $avg on the $onPage time, which works, but instead of the average I want to calculate the median. Because this is not possible with mongodb I want to do this in javascript.

[07:59:05] <foofoobar> To do this I want to strip the $avg and just output all onPage times like this: [5, 1, 3, 5, 1]

[07:59:40] <foofoobar> I tried it this way: http://hastebin.com/nomonudadu.js

[08:00:11] <foofoobar> But then average_time is an empty list. The first $group in my pipeline works as it should, all values are combines in a list

[08:00:17] <foofoobar> But the second does not. Where is my error?

[08:04:40] <Boomtime> foofoobar: have you tried running it in the shell?

[08:04:59] <foofoobar> Boomtime: Why should this change anything?

[08:05:07] <Boomtime> it might tell you more

[08:05:28] <Boomtime> it will also seperate the issue from driver to mongodb

[08:06:15] <Boomtime> btw this last bit is an interesting thing -> "average_time": { "$addToSet": "$_id.average_time" }

[08:06:29] <Boomtime> .average_time is itself an array yes?

[08:06:52] <foofoobar> Boomtime: Yes

[08:06:54] <Boomtime> do you know how big this can grow?

[08:07:23] <Boomtime> you have an array of arrays, whose worst case is one entry for every document that exists on that day

[08:07:38] <Boomtime> if you exceed 16MB, it halts

[08:07:47] <foofoobar> I dont want an array of arrays, a simple array with the values is what I was looking for

[08:08:13] <foofoobar> I want all average times in an array

[08:08:20] <Boomtime> your first stage creates an array: "average_time": { "$addToSet": "$onPage" }

[08:08:23] <foofoobar> *all average times of a page on a specific day

[08:08:29] <foofoobar> Boomtime: correct

[08:08:45] <Boomtime> your second stage embeds that array value into another array: "average_time": { "$addToSet": "$_id.average_time" }

[08:09:08] <Boomtime> i think you want a $each in there, or to unroll that first array some other way

[08:09:10] <foofoobar> Is there a way to merge them instead of creating an array of arrays?

[08:09:46] <Boomtime> check out $each, or check if $unwind stage inbetween does what you need, i'm not sure of the solution in this case

[08:09:58] <Boomtime> also, i am sorry, i have to go

[08:10:00] <Boomtime> good luck

[08:10:22] <foofoobar> thanks

[08:17:31] <foofoobar> Cant get this with $each to work

[08:17:39] <foofoobar> exception: invalid operator ‚$each'

[08:19:49] <foofoobar> Any hints?

[08:20:08] <koren> didnt get your query just logged in, can you paste it again?

[08:22:07] <foofoobar> koren: Yeah, give me a second

[08:22:42] <foofoobar> koren: http://hastebin.com/ucegacapah.js

[08:22:50] <foofoobar> You helped me with this query yesterday

[08:22:52] <foofoobar> :)

[08:23:50] <koren> I remember yes ^^

[08:24:38] <koren> well about your query, it seems that $each has to be in a field of $addToSet

[08:24:42] <koren> http://docs.mongodb.org/manual/reference/operator/update/each/

[08:25:20] <koren> "average_time": { "$addToSet": {"myField": {$each: "$_id.average_time" }}}

[08:26:19] <foofoobar> But this results in the same error

[08:26:34] <techie28> #mongodb

[08:26:51] <foofoobar> techie28: Yes, this is #mongodb.

[08:27:09] <techie28> yea thanks..

[08:29:53] <techie28> I am developing a small webservice which takes an array of JSON objects and in each of them there is an index called "ACTION" which will tell the service to add/edit the data depending on its value.

[08:30:23] <techie28> I will have to make one call for ADDING and one for updating.

[08:31:00] <techie28> I was wondering that I could use "Multi' and "Upsert" here to do that everything in one call only?

[08:33:19] <techie28> foofoobar: any ideas?

[08:40:33] <foofoobar> no, sorry

[08:43:49] <foofoobar> koren: any further hints?

[08:44:23] <koren> foofoobar: no sorry I'm realy not confident with aggregate yet but I think you can't use $each on $addToSet into it, not sure

[08:44:25] <valera> hello. my mapreduce functions fail with utf-8 errors (JS Error: TypeError: malformed UTF-8 character sequence...). is there a way to locate documents with malformed utf-8 ? exception did not give an ObjectId

[08:44:42] <koren> techie28: take a look at bulk upsert: http://docs.mongodb.org/manual/reference/method/db.collection.initializeUnorderedBulkOp/#db.collection.initializeUnorderedBulkOp prepare your queries and execute them in one call

[08:46:47] <foofoobar> koren: it should be possible: http://docs.mongodb.org/manual/reference/operator/update/each/

[08:47:50] <koren> they provides only exemples with update, and when I look at $addToSet in aggregation framework, they say it support only accumulator expressions: http://docs.mongodb.org/manual/reference/operator/aggregation/#aggregation-accumulator-operators

[08:48:06] <koren> http://docs.mongodb.org/manual/reference/operator/aggregation/addToSet/

[08:48:35] <foofoobar> Also why is there only an addToSet

[08:48:48] <foofoobar> I want something like addToArray, there should be duplicates allowed

[08:50:03] <koren> maybe with $push

[08:50:44] <foofoobar> ok

[08:55:07] <foofoobar> koren: http://hastebin.com/ocucolibem.js

[08:55:11] <techie28> koren: how about this db.getCollection('users').update({'email':'abc@mail.com'},{'firstName':'name1'},{"multi" : true,"upsert" : false});

[08:55:20] <foofoobar> I think I will use this and then flatten „by hand“ in javascript when I got the result

[08:55:52] <techie28> should this call not update the firstName if that email was found else add new?

[08:57:22] <techie28> db.getCollection('users').update({'email':'abc@mail.com'},{'firstName':'name1'},{"multi" : false,"upsert" : true});... im sorry this way I mean

[08:57:46] <techie28> setting upsert to true? is it not a good idea?

[08:57:59] <koren> it is

[08:58:02] <techie28> but I m getting "duplicate key error index"

[08:58:47] <koren> what is your id ?

[08:59:00] <koren> should be auto generated _id field

[08:59:17] <koren> and you should not have duplicate

[09:00:28] <koren> ho I see

[09:00:44] <koren> you update the whole document, wiping every field except firstName

[09:00:54] <koren> {$set: {'firstName':'name1'}}

[09:01:15] <techie28> yea that went wrong

[09:01:28] <techie28> ill have to use $set

[09:02:30] <koren> does this solve your issue?

[09:03:49] <techie28> im trying

[09:04:19] <techie28> this is my first time to working on mongo

[09:04:26] <techie28> or a noSQL database

[09:16:39] <techie28> koren: yes it did.. thanks a lot. :)

[09:17:45] <techie28> koren: I am updating one value only here.

[09:18:36] <techie28> koren: If that query criteria doesnt match.it creates a new doc with that name and the queried email..it is impossible to have all other fields there?

[09:22:11] <koren> it is possible yes, you want them empty ?

[09:22:54] <techie28> yes

[09:23:05] <techie28> just like MySql behaves

[09:23:27] <koren> then in your update, {$set: {"field": "value"}, $setOnInsert: {"fieldEmpty": ""}}

[09:23:30] <koren> etc.

[09:23:34] <koren> but it's not usefull with mongo

[09:23:58] <koren> mongodb is made in a way that if you dont have the field, just not set it and it will not break

[09:24:16] <koren> and if you want to query for field existing, just use $exists

[09:24:27] <koren> you don't have to set fields to null or "" like in sql

[09:25:33] <koren> it is possible but it's bad practice imo

[09:25:39] <techie28> like there are fileds in the table createdISO,updatedISO etc which have timestamps in them.. in case of upsert TRUE they are not present in the created doc

[09:25:55] <koren> that's why you have $setOnInsert

[09:26:06] <koren> for createdAt

[09:26:11] <koren> updatedAt just goes into $set

[09:26:32] <koren> {$set: {updatedAt: new Date()}, {$setOnInsert: {createdAt: new Date()}}

[09:26:50] <techie28> we have to manually insert the timestamps

[09:27:14] <koren> yes

[09:28:25] <techie28> okay

[09:28:32] <techie28> thanks again koren :)

[09:45:51] <ramnes> is mpobrien somewhere around here?

[09:49:23] <ramnes> or Kyle Erf?

[09:59:13] <joannac> ramnes: erm, why do you need specific people?

[10:05:40] <ramnes> joannac: to talk about mongo-tools code

[10:09:13] <makinen> db.copyDatabase({"copydb": 1, "fromhost": 'safekoe', "fromdb": 'glaucoma', "todb": 'glaucoma', "slaveOk": true})

[10:09:22] <makinen> what's wrong with that?

[10:10:02] <makinen> the beginning of the stacktrace:

[10:10:04] <makinen> Error: assert failed at Error (<anonymous>) at doassert (src/mongo/shell/assert.js:11:14)

[10:10:30] <ramnes> fromdb == todb?

[10:10:48] <ramnes> ah

[10:10:52] <ramnes> copying from a remote host

[10:11:12] <makinen> yes

[10:12:01] <makinen> should I give username, nonce or key too?

[10:13:02] <ramnes> if your remote has authentication enabled, yes

[10:13:04] <makinen> seem to be optional options and I can connect just to the db on the remote host if I connect there by ssh first

[10:13:15] <makinen> just fine*

[10:13:59] <makinen> can the authentication be enabled only for the remote connections?

[10:23:09] <hrusti> I am staring to develop web application which will have semi-big datasets and many relations and I cant decide between noSql and sql. I am thinking about mysql and mongo. I have read that nosql databases are not good with many relationship. What do you suggest?

[10:26:43] <vilcoyot> Hi, I would like to install mongodb 3.x on debian jessie. But the repo is only providing wheezy packages...

[10:26:54] <vilcoyot> Is there a plan to deliver them?

[10:27:00] <Derick> they will install on jessie though

[10:27:05] <Derick> (and run)

[10:27:28] <vilcoyot> Derick: Yes but... then why not having an alias for a jessie repo?

[10:27:36] <Derick> that is a good question :-)

[10:27:41] <vilcoyot> Haha

[10:27:44] <Derick> I'll add a ticket

[10:28:25] <Derick> oh, already there: https://jira.mongodb.org/browse/SERVER-18329

[10:28:49] <vilcoyot> I was looking for this ticket system..

[10:30:24] <vilcoyot> Derick: Thx a lot.

[10:30:38] <rasputnik> check my thinking here: devs. have a one off query they want to do and without an index they're going to take a while, so have asked me to create an index.

[10:30:43] <rasputnik> but this is a one-off query

[10:31:05] <rasputnik> so won't it be a waste of time? the work to create the index will be as much as the work to do the query, right?

[10:31:36] <rasputnik> plus we have the cost of the indexed query, which is unlikely to be 'free'. and the maintenance overhead of that index for all writes.

[10:32:20] <vilcoyot> how long is "a while"?

[10:32:52] <Derick> rasputnik: creating the index is as slow as doing *one* full table scan

[10:33:22] <Derick> rasputnik: so yeah, for a one-off query it makes no sense to create an index

[10:36:13] <joannac> ramnes: in what context? contributing? bug reports?

[10:40:22] <ramnes> joannac: https://github.com/mongodb/mongo-tools/pull/20

[10:40:30] <ramnes> see my code comment

[10:41:25] <ramnes> I would have prefered to talk before the pull request rather than the opposite

[10:42:13] <joannac> ramnes: put a comment on the ticket then?

[10:43:12] <ramnes> I didn't want to pollute the ticket with unrelated code stuff like this

[10:44:05] <joannac> ramnes: https://github.com/mongodb/mongo-tools/blob/master/CONTRIBUTING.md

[10:44:32] <joannac> begin a discussion on the MongoDB Developers Forum

[10:45:34] <makinen> how do i find out if the authentication's been enabled

[10:46:19] <joannac> makinen: check ps -ef output, or the config file

[10:46:45] <joannac> or try and connect in the mongo shell without authenticating and doing something and seeing if you get an error :)

[10:49:54] <makinen> i can start mongo shell and give a command 'use glaucoma'

[10:50:09] <makinen> so I guess the authentication's not enabled

[10:51:02] <makinen> maybe a firewall blocks copyDatabase()

[10:51:44] <makinen> btw is copyDatabase the best method to move the whole database into another server

[10:52:48] <joannac> use database doesn't do anything

[10:53:25] <koren> makinen: I feel retarded I used mongodump + mongorestore... thx

[10:53:36] <joannac> makinen: do a find() or something

[10:55:59] <makinen> find() returns fine

[10:56:23] <makinen> maybe I should use mongodump & mongorestore

[10:58:12] <koren> its slow, I think copyDatabase is faster

[11:01:49] <makinen> hm

[11:02:17] <makinen> now I switched to admin and got connection refused

[11:02:38] <makinen> > db.runCommand({copydb: 1, fromdb: "glaucoma", todb: "glaucoma", fromhost: "safekoe.pointsit.fi"})

[11:02:41] <makinen> 2015-05-28T13:59:24.883+0300 I NETWORK trying reconnect to 127.0.0.1:27017 (127.0.0.1) failed

[11:02:44] <makinen> 2015-05-28T13:59:24.883+0300 W NETWORK Failed to connect to 127.0.0.1:27017, reason: errno:111 Connection refused

[11:06:44] <makinen> but is it trying to connect to localhost

[11:06:47] <makinen> +why

[11:15:37] <makinen> okay it's not firewall related since I can connect with netcat to the mongodb at the remote host

[12:56:10] <makinen> is it a good idea to add a new mongo instance to a replica set, wait for it to synchronize and use it as a new master?

[12:58:11] <cheeser> hard to say. what problem are you trying to solve?

[13:17:54] <makinen> cheeser: I'm moving a mongodb database to a new server

[13:18:40] <makinen> cheeser: downtime must be minimized

[13:19:00] <cheeser> you can do the sync, then ask the current primary to step down. is there a heavy write load?

[13:19:13] <cheeser> i.e., any reason to worry about replication lag?

[13:23:16] <deathanchor> usually I just set the new machine at a higher priority, once it is caught up voting makes it primary.

[13:29:54] <cheeser> that's an option. though mucking with priorities can confuse things. and changing priority requires a node restart.

[13:36:07] <_ari> is it possible to filter by $regex and $group together?

[13:39:01] <_ari> for example, can we use $regex in an aggregate function?

[13:39:06] <_ari> or is it only for find()

[13:54:55] <deathanchor> _ari: $regex would be part of the $match part of your aggregation no?

[13:55:17] <pamp> hi

[13:55:36] <pamp> How can I drop a sharded collection

[13:56:07] <pamp> when I try I get this error: exception: collection's metadata is undergoing changes. Please try again.

[13:56:21] <pamp> I already stop the ballancer but get the same error

[13:56:37] <deathanchor> did you check that the balancer is not in a running state?

[13:57:07] <pamp> yes

[13:57:09] <pamp> is not

[13:59:16] <deathanchor> anything in the logs?

[14:00:11] <deathanchor> I think if the chunksplitter is running it won't let you do collection operations.

[14:01:52] <pamp> no

[14:02:01] <pamp> just unclean shutdhown

[14:02:05] <pamp> http://dpaste.com/0C815ZJ

[14:02:18] <_ari> right, thanks deathanchor

[14:04:46] <deathanchor> pamp: nothing in the logs about the drop attempt?

[14:07:01] <_ari> is it possible to make my aggregate function group objects by a certain element and display the rest of the elements?

[14:07:14] <_ari> within the object

[14:08:29] <pamp> 2015-05-28T14:05:42.498+0000 I COMMAND [conn3] DROP: newModel.vertexTest 2015-05-28T14:05:42.498+0000 I SHARDING [conn3] about to log metadata event: { _id: "CFN-DT-SG-NSQL1-2015-05-28T14:05:42-556720b6dd8039a2552f89c5", server: "CFN-DT-SG-NSQL1", clientAddr: "N/A", time: new Date(1432821942498), what: "dropCollection.start", ns: "newModel.vertexTest", details: {} }

[14:10:35] <pamp> after a few more attempts could eliminate collection

[14:10:37] <pamp> thaks

[14:25:20] <deathanchor> pamp: sounds like it was the chunksplitter probably running on that collection or DB.

[14:45:48] <cpennington> I've got a mongo instance (on compose.io) where reading a 1.6mb document takes ~3 seconds (using the commandline client) and similar from the python client

[14:46:33] <cpennington> I'm looking now into how to shrink the size of the document (remove data that we shouldn't be storing anyway, and minimizing document keys), but the time to load the data seems odd to me

[14:48:43] <cheeser> how much of that is network IO?

[14:50:17] <makinen> cheeser: probably not the server change will be done at night

[15:06:03] <cpennington> cheeser: not much, I don't think. is there a way to get that measurement in the js cli client?

[15:08:03] <cheeser> cpennington: i don't know

[15:14:24] <deathanchor> anyone have recommendation for a free mongo GUI client for non-tech folks to use for reads/simple lookups?

[15:18:59] <koren> web based ?

[15:19:53] <koren> I used it a lot for simple read: https://github.com/lovetheidea/MoaDB works pretty well and fast to setup

[15:20:21] <koren> but no more maintained

[15:20:32] <koren> otherwise https://www.humongous.io/ looks great but not tested

[15:20:53] <koren> I looked at the list from mongodb.org but every mongo gui looks like WIP :/

[15:21:11] <koren> the one used by compose.io is quite nice but you have to query like you would do in a mongo shell

[15:23:07] <kadosh> Hi

[16:02:50] <deathanchor> koren: thx for info

[16:19:08] <kadosh> Hi everyone

[16:19:10] <kadosh> I have a question

[16:19:19] <kadosh> I set up a replica set with 1 master 2 slaves

[16:19:34] <kadosh> I see the replication, but when my app (nodejs)

[16:19:52] <kadosh> reads or write just the master does the work

[16:20:12] <kadosh> is there something that can be set up to balance the load?

[16:21:40] <koren> you can set the readPreference but it is normal that only the master is working

[16:21:49] <koren> replica set is made to replicate, not balance

[16:22:15] <koren> in the connection string, set ?readPreference=secondaryPrefered if you want to balance and dont need up to date data

[16:22:27] <koren> because data in secondaries have a replication lag

[16:22:42] <koren> 2 or 3 sec so if you insert and then query the inserted data quickly, you get the old data

[16:22:45] <koren> use carefully

[16:23:44] <koren> you may be more interested in sharded clusters to balance load but it is a bit more complicated than replica set (even if mms makes it quite easy)

[16:26:49] <kadosh> I see thanks

[16:31:27] <svm_invictvs> Hello

[16:31:38] <svm_invictvs> I'm curious how do I renamne afile in GridFS?

[16:31:51] <svm_invictvs> Is it as simple as theFile.set("name", "newName") and saving it?

[19:42:19] <UnNaturalHigh> Is anyone here aware if its possible to use apache spark with mongodb without hadoop? I noticed the hadoop-connector but it isn't clear whether hadoop is also a requirement?

[19:45:14] <StephenLynx> you will have to check with the spark guys

[19:47:08] <cheeser> UnNaturalHigh: currently no but that's being investigated

[19:50:54] <saml> if you're using mongod and hadoop and spark.. you're really big data

[20:07:58] <f31n> hi i'm new to the nosql idea i'm used to work with relational databases. my question is is that a propper schema or is there a way to make it better: http://pastebin.com/twzRAz3C

[20:12:09] <StephenLynx> it seems to be some kind of mistake there

[20:12:28] <StephenLynx> at first I thought you nested all that, but then I noticed _id in the last depth

[20:13:19] <f31n> yes that would be a foreign id

[20:16:34] <StephenLynx> so rewrite without the confusing tabs.

[20:16:58] <f31n> okay i give my best

[20:19:01] <Streemo> I noticed that doing an update with a basic $inc and $set, sometimes the $set does not work. is that standard?

[20:19:35] <Streemo> db.collection.update({_id:someId},{$inc:{'score.up':rating.u,'score.total':rating.t}, $set:{'score.calculated':calculated}})

[20:20:22] <StephenLynx> sometimes?

[20:20:32] <StephenLynx> that would be extremely odd.

[20:20:33] <Streemo> at high volume, my data checking reveals that the score.calculated fields of some documents do not match generateScore(score.up, score.total)

[20:20:39] <Streemo> it really is

[20:20:55] <Streemo> is it a writeFail due to high volume?

[20:21:13] <StephenLynx> hm

[20:21:27] <StephenLynx> maybe another operation on it with a lower value went in first?

[20:21:34] <Streemo> hmmm

[20:21:41] <Streemo> so its a race condition

[20:21:43] <StephenLynx> after*

[20:21:48] <StephenLynx> might be.

[20:21:59] <Streemo> you are probably right! ill look into it. thanks

[20:22:26] <StephenLynx> I have a point in applications, generating ids where this might happen.

[20:22:47] <StephenLynx> I just check for the error, if its a duplicated index error, I add to the id and try again.

[20:22:58] <Streemo> i see

[20:23:09] <StephenLynx> but you wouldn't be able to do that.

[20:23:41] <StephenLynx> perhaps if you were to use the inc to add the delta.

[20:23:52] <Streemo> its a nonlinear function

[20:23:52] <StephenLynx> instead of using set to replace the value.

[20:23:59] <Streemo> so that wont work

[20:24:01] <StephenLynx> hm

[20:24:03] <StephenLynx> welp

[20:24:06] <Streemo> :/

[20:24:17] <StephenLynx> so a queue?

[20:24:28] <Streemo> yeah!

[20:24:34] <Streemo> i think that is the right solution

[20:24:48] <Streemo> thanks stephen

[20:24:50] <StephenLynx> np

[20:25:39] <UnNaturalHigh> cheeser: is there any sort of ticket or feature request started that you know of that I could subscribe to?

[20:26:20] <cheeser> not yet, no. it'll be a summer project for some of us here.

[20:27:28] <f31n> StephenLynx: http://pastebin.com/CSUJRtfe thats better now?

[20:27:54] <StephenLynx> wouldn't questions be an array?

[20:28:27] <f31n> yes it should, how to indicate that?

[20:28:34] <StephenLynx> []

[20:29:03] <StephenLynx> and you forgot a comma after the end of the answers object.

[20:29:31] <StephenLynx> in general it is a little bulky.

[20:29:47] <StephenLynx> but not too bad.

[20:30:16] <StephenLynx> one issue you might encounter is when querying the event, you would have to query for the users on a separate query.

[20:30:33] <StephenLynx> because you wouldn't be able to join the users collection based on the event's user.

[20:31:40] <StephenLynx> https://gitlab.com/mrseth/LynxChan/blob/master/doc/Model.txt

[20:31:43] <StephenLynx> this is how I model.

[20:33:22] <f31n> du kommst aus wien?

[20:33:40] <StephenLynx> :v

[20:34:02] <StephenLynx> applestrudel?

[20:34:35] <f31n> the meal or the cocktail?

[20:34:50] <StephenLynx> i dunno

[20:35:22] <StephenLynx> and why you thought I was from viena?

[20:35:45] <f31n> cause its posted from sergio augusto vienna

[20:35:49] <StephenLynx> vianna

[20:35:57] <StephenLynx> its a family name.

[20:35:59] <f31n> sorry my bad ...

[20:36:02] <StephenLynx> not original

[20:36:28] <StephenLynx> because my grand-grand father came from an italian family so poor they gave him for another family to raise, the viannas.

[20:37:02] <cheeser> well that had to suck

[20:37:10] <StephenLynx> meh

[20:37:18] <f31n> ah

[20:37:22] <StephenLynx> everything turned fine by the time my grand father had my mom.

[20:38:07] <StephenLynx> because he was really good with money, had a stable job and sent both daughters to college.

[20:38:21] <StephenLynx> my mom learned from him how to handle money properly, her older sister not so much.

[20:40:18] <f31n> and your family stayed in italy?

[20:40:22] <StephenLynx> oh no

[20:40:37] <StephenLynx> this italian family were immigrants.

[20:40:42] <StephenLynx> that is why they were so poor.

[20:40:48] <StephenLynx> they moved to brazil, I guess.

[20:41:14] <StephenLynx> or they were descendants from immigrants.

[20:41:24] <StephenLynx> that part of the family history is quite unknown.

[20:41:56] <StephenLynx> brazilians are not used to keep track of their family roots.

[20:42:15] <StephenLynx> all I know is that there were some natives in the grand-mother's family

[20:42:24] <StephenLynx> and my father comes from a noble portuguese family.

[20:42:29] <StephenLynx> and that thing about the viannas.

[20:44:38] <f31n> sounds like an impressive family history

[20:45:05] <StephenLynx> pff

[20:45:43] <StephenLynx> it isn't much different than canine genealogy.

[20:47:49] <f31n> true, but there is and will always be something you can tell about your family, for your children for example ... i loved hearing the family tales of my family names

[20:49:13] <StephenLynx> >children

[20:49:32] <StephenLynx> the only relative I talk is my mother once of twice per month.

[20:49:38] <StephenLynx> what I know will be buried with me.

[20:54:13] <f31n> xD ic ... got an update: http://pastebin.com/ZKw24sPY

[20:54:44] <StephenLynx> about to go home

[20:54:46] <f31n> thats the reason why user is outside of the whole other tree

Log file Viewer

Help | Karma | Search:

#mongodb logs for Thursday the 28th of May, 2015