pmxbot IRC Log Viewer

[02:17:32] <mastyd> Can someone help me with aggregation: https://gist.github.com/dmastylo/f8da4a3dead2323d5ed1. The firstPlace attribute isn't working, I want to count the number of "0" ranks a player has all together

[02:18:22] <mastyd> but I don't want only players who have gotten first place to be in the results, which is why the $match is not outside, but obviously $match doesn't work when inside of $group

[02:19:51] <joannac> mastyd: um, I don't actually understand what you're trying to do

[02:21:10] <mastyd> @joannac, I'm grouping players by their playerId (external reference to another DB), and I want the specific fields distance, prizeCents, gamesPlayed, coins, and firstPlace to be available in the result. firstPlace should be the number of times that group has a rank = 0

[02:21:27] <mastyd> Sorry grouping by userId.

[02:21:32] <mastyd> So imagine you have 10 users with userId 1

[02:22:07] <mastyd> 5 of those have a rank 0, and the other 5 are 2, 3, 10, 5, 8, etc. doesn't really matter. But I want to display that firstPlace = 5 because 5 of those in that group have a rank 0

[02:22:14] <mastyd> Does that make more sense? Hopefully

[02:24:25] <joannac> okay

[02:24:52] <joannac> have a project stage where you turn every document with rank=0 to isFirst=1

[02:25:03] <joannac> and every document with rank != 0 to isFirst = 0

[02:25:10] <joannac> and then sum up the isFirst values

[02:25:41] <mastyd> project stage before the group stage, okay let me try that and let you know if I run into issues. Thanks a ton!

[02:31:27] <mastyd> @joannac: having a hard time putting this together, this is honestly my first time doing an aggregation query in mongo

[02:33:15] <mastyd> @joannac: I have this: https://gist.github.com/dmastylo/f8da4a3dead2323d5ed1 but I'm not getting the expected results. I'm getting a null '_id' result and just one row

[02:36:13] <mastyd> @joannac: Nevermind I got it! Thanks a ton for the projection advice. So basically that lets you select which fields to return and manipulate them before the group, and the group lets you sum or avg or whatever on those fields

[02:36:17] <mastyd> I think I'm starting to get aggregation

[02:36:32] <joannac> mastyd: yup

[02:38:07] <mastyd> I'm so stoked now haha

[02:38:11] <joannac> mastyd: I find it helpful when writing an aggregation pipeline to do it one bit at a time, and check the documents look how I want

[02:39:07] <mastyd> @joannac: yeah that's what I was doing. Are aggregation queries fast or no? I mean I guess it depends what you're doing and if you have proper indexing and whatnot

[02:40:05] <joannac> exactly. depends what you're doing, how much data you're precessing, what else is running at the same time, etc

[02:41:09] <mastyd> how would you recommend to index fields in a query like i'm doing?

[02:41:24] <mastyd> I just updated it to my final version

[02:42:19] <stuntmachine> any good guides for fixing this on centos 6? http://pastebin.com/raw.php?i=bGZdsqaz

[02:42:42] <stuntmachine> this shows up as soon as I run mongo on a default install of mongo 3.0.5

[02:42:57] <joannac> stuntmachine: http://docs.mongodb.org/manual/tutorial/transparent-huge-pages/

[02:43:07] <joannac> first hit in google for "transparent hugepages mongodb"

[02:43:18] <stuntmachine> @joannac the problem is this doesn't solve the issue

[02:43:42] <joannac> How so?

[02:44:16] <stuntmachine> The second warning message (re: defrag) remains. I already tested on another box.

[02:45:51] <joannac> stuntmachine: there's no message around "defrag" in that pastebin

[02:45:57] <stuntmachine> i just thought maybe there were better details out there than the official ones around this particular issue on centos 6.x. i spent a few hours on this the other night and saw a few stackoverflow pages, etc, but none helped me get rid of that defrag warning. i'm trying to set things up as perfectly as i can from the getgo

[02:45:59] <joannac> do you mean rlimits?

[02:46:07] <joannac> oh

[02:46:07] <stuntmachine> 2015-07-28T21:38:11.781-0500 I CONTROL [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/defrag is 'always'.

[02:46:08] <joannac> sorry

[02:46:16] <joannac> I'm blind

[02:47:42] <stuntmachine> no worries

[02:47:53] <stuntmachine> pretty frustrating when the official docs don't help

[02:47:58] <joannac> How did you fix it?

[02:48:01] <stuntmachine> i didn't! :)

[02:48:12] <joannac> THe rc.local fix will fix it

[02:48:17] <stuntmachine> tried it :/

[02:48:19] <joannac> but not early enough to remove the warning

[02:48:41] <stuntmachine> ah maybe that's the issue

[02:48:43] <joannac> As in, the root problem will be fixed, but you will still see the warning in startup warnings

[02:48:55] <stuntmachine> well... doesn't that sort of negate the point of the warning?

[02:49:01] <joannac> That page is being fixed but I don't have the details

[02:49:10] <stuntmachine> i mean -- i know you're not the person who wrote the thing :P but nonetheless... unless you are :)

[02:49:22] <stuntmachine> oh okay cool

[02:49:32] <joannac> Yeah, I know

[02:50:14] <joannac> I think, if you restart the mongod process, the warning will go away?

[02:50:24] <stuntmachine> you mean after boot after editing rc.local right?

[02:50:36] <joannac> yes

[02:50:39] <stuntmachine> gotcha

[02:51:01] <joannac> currently the sequence is boot, rc.local, and somewhere in there mongod also starts

[02:51:38] <joannac> if you restart just mongod, it'll re-check, realise thp/defrag is fixed, and the warning will never be logged

[02:52:08] <joannac> (this is not my strong suit so apologies in advance if I'm not telling you the right thing)

[02:52:12] <stuntmachine> my perfectionism is biting me in the butt :)

[02:52:17] <stuntmachine> no worries. i appreciate the help.

[02:53:33] <stuntmachine> i just wanted to make sure i'm not crazy here :)

[02:54:57] <mastyd> @joannac: thanks again for the help!!

[05:20:57] <pulpbrittle> join #python

[05:48:09] <sabrehagen> is there a way to query against a field name/document key where the match is a regex. e.g. my documents have keys 'event', 'event2', 'event3'. i want to query all possible eventX keys in the document for a certain value.

[06:29:21] <Boomtime> sabrehagen: you can't - in order to construct a query you need to be able to specify the field on which you are querying - dynamic field names is poor schema design, usually you should be using an array or sub-document instead

[06:30:02] <Boomtime> in your case, it sounds like you should have a single 'event' field which is an array of subdocuments, or similar

[06:32:38] <sabrehagen> Boomtime: thanks for the advice

[07:02:11] <hemangpatel> Hello

[07:09:10] <joannac> hi

[07:15:08] <bentinata> I got error sizeStorer.wt: read checksum error, and sizeStorer.wt: encountered an illegal file format or internal value.

[07:15:28] <bentinata> Even after I delete mongod.lock and wiredTiger.lock.

[07:16:31] <bentinata> So, I just delete that sizeStorer.wt file, and WiredTiger.wt file.

[07:16:51] <bentinata> Without deleting index-* and collection-* file.

[07:17:04] <bentinata> Any idea how to recover my database?

[07:21:37] <Boomtime> bentinata: probably not now.. do you have a backup? and what version are you using?

[07:23:14] <joannac> and do you have a support contract?

[07:24:05] <bentinata> I don't have any support contract. Is this channel reserved for support contract only?

[07:25:00] <bentinata> Boomtime, I do have backup from a running MMAPv1 one. And I'm using MongoDB 3.0.5.

[07:25:06] <joannac> No, but it might be an involved process

[07:25:25] <joannac> If you had a support contract you could get more help

[07:25:35] <joannac> In here, you're basically relying on goodwill :)

[07:25:52] <joannac> (and people who have time to help)

[07:25:53] <bentinata> Yep. Thanks for your attention. :)

[07:26:39] <joannac> is it part of a replica set?

[07:27:39] <bentinata> No, it's just a simple wiredTiger engine.

[07:27:49] <bentinata> Single, no clustering, no sharding.

[07:27:56] <joannac> are you willing to lose the data snce the backup?

[07:29:26] <bentinata> Actually, I don't mind losing the data, since I have two running database. One on MMAPv1 and the other wiredTiger.

[07:30:01] <bentinata> But, restoring from a dump from MMAPv1 is taking a long time.

[07:30:43] <joannac> how large is the data?

[07:30:50] <bentinata> If this problem is common, I wouldn't use wiredTiger.

[07:31:16] <bentinata> About 2.5GeB.

[07:32:23] <joannac> the problem is not common

[07:32:53] <joannac> the last time I saw it, it was from moving / editing / something while the mongod was still running

[07:33:08] <joannac> your problem seems to be that you removed 2 of your data files

[07:33:45] <joannac> that's a very bad idea

[07:34:06] <joannac> are you seeing progress with your mongorestore?

[07:34:07] <bentinata> Yeah, previous error message is "mongod.lock is not empty".

[07:34:14] <joannac> right

[07:34:20] <bentinata> Mongorestore should be easy.

[07:34:22] <joannac> that indicates an unclean shutdown

[07:34:39] <joannac> also not a good sign

[07:34:52] <bentinata> Yep, and from the official doc, you should remove it. Then running mongod --repair.

[07:37:07] <bentinata> If there's no way to restore from just index-* and collection-* file, I'll just use mongorestore.

[07:37:25] <joannac> I thought you were already running mongorestore ?

[07:37:46] <bentinata> No problem. Just asking out of curiosity.

[07:37:54] <bentinata> Not yet, it's time consuming process.

[07:38:03] <bentinata> Thanks. :)

[07:38:27] <Boomtime> bentinata: such a fault is not known on 3.0.5, what compression algorithm do you use? (or say if not specified)

[07:38:47] <bentinata> Unspecified.

[07:39:46] <bentinata> Is this problem possibly happen because of power shutdown?

[07:43:03] <joannac> Hard to say, would need to see the files

[07:55:20] <bentinata> Done dumping MMAPv1 and restoring to wiredTiger.

[07:55:30] <bentinata> Took 10 mins haha.

[07:57:01] <bentinata> Thanks joannac, Boomtime, for sparing your time. :)

[08:35:20] <gemtastic> Any mongoDB and node devs in here?

[08:35:34] <gemtastic> In either case, Hi :)

[08:43:06] <morenoh149> yea

[08:50:28] <Derick> gemtastic: better to just ask your question!

[08:50:58] <gemtastic> Derick: Well, that WAS my question.

[08:51:50] <gemtastic> Most people in here is logically assumed to work with MongoDB in one way or another, but that's no guarantee that any one working with MongoDB through Node is here :P

[08:52:10] <gemtastic> I was just curious if they hang here

[09:10:26] <gemtastic> It's ok, noone has to out themselves with working with Node if they don't want to ;P

[09:43:21] <morenoh149> I work with node and mongodb

Log file Viewer

Help | Karma | Search:

#mongodb logs for Wednesday the 29th of July, 2015