pmxbot IRC Log Viewer

[00:00:18] <mwillbanks> if i need to query something based on a subdocument array how does one go about doing this?

[00:01:04] <mwillbanks> aka: db.something.find() returns { item: [ "1", "2" ] } and i need to do a db.item on that?

[00:01:50] <joannac> mwillbanks: not sure what you mean, sorry

[00:02:29] <joannac> db.something.find({item: 1}) ? db.something.find({"item.0" : 1}) ?

[00:03:14] <slikts> what could be the reason why calling .populate('bla') on a query would populate only one bla field and leave the rest as null?

[00:03:23] <slikts> all the fields have the same value

[00:03:34] <slikts> an ObjectId to a model

[00:03:57] <slikts> I'm calling it like this: req.model.find().populate('createdBy').exec(function(err, data) { next(err || data); });

[00:04:48] <slikts> if I leave out the .populate then I get the same ObjectId in all createdBy fields, but if I include it I get only one populated field and the rest are null

[00:05:31] <mwillbanks> joannac: basically i am attempting to iterate over a collection which contains an element masks that is an array of string MongoIds

[00:05:36] <mwillbanks> err ObjectIds

[00:06:10] <mwillbanks> then i need to take those arrays and find the corresponding record in a different collection

[00:08:18] <joannac> You can't do that in one step

[00:08:48] <slikts> mongoose docs: MyModel.update({ age: { $gt: 18 } }, { oldEnough: true }, fn);

[00:17:56] <TylerE> I'm running on 2.5.x...what do I have to do to enable external sorts?

[00:18:08] <TylerE> .e.g I'm getting exception: Exceeded memory limit for $group, but didn't allow external sort

[00:18:20] <TylerE> but I can't find anything in the docs as to how to actually turn it on

[00:22:39] <joannac> TylerE: https://github.com/mongodb/mongo/blob/master/src/mongo/db/pipeline/document_source_group.cpp#L361

[00:24:27] <TylerE> hrrm

[00:24:33] <TylerE> well, now I'm getting a different error

[00:24:34] <TylerE> "errmsg" : "exception: Sort exceeded memory limit of 104857600 bytes, but did not opt-in to external sorting. Aborting operation.",

[00:25:12] <TylerE> and I'm definatly calling db.coll.aggregate([pipeline],{allowDiskUse:true})

[00:30:21] <joannac> Looks like a bug, then. File a SERVER ticket

[00:34:14] <skot> TylerE, are you sure you are using a new shell?

[00:34:23] <LouisT> so i assume the mongodb-10gen package for ubuntu is still using the vulnerable openssl?

[00:34:29] <TylerE> skot, ooh, good catch, I might not be

[00:34:35] <TylerE> I updated to thje new server

[00:34:39] <skot> The old shell didn't have a second arg, so it would be ignored.

[00:34:40] <TylerE> but I didn't install the shell serpeatly

[00:35:03] <skot> what does version() say in the shell?

[00:36:02] <TylerE> 2.5.2

[00:36:07] <TylerE> so that might not be it afterall

[00:36:55] <skot> you need to upgrade, I don't think that feature was in until 2.5.5

[00:37:35] <TylerE> ahh, sigh

[00:37:43] <TylerE> that's actually the newest in the 10gen ubuntu repo ;(

[00:37:56] <TylerE> might as well just build 2.6 nightly from source I guess

[00:38:08] <skot> There should be 2.6.0-rc3

[00:38:11] <skot> I'd suggest using that.

[00:38:51] <TylerE> k, downloading

[00:38:54] <skot> I think the package names may have changed somewhere back around 2.5.2/3, but I'd have to go look.

[00:39:09] <skot> It should be in the release notes.

[00:39:25] <skot> rc3 is basically the 2.6 release.

[00:39:32] <TylerE> some of the 2.6 stuff is pretty exciting

[00:39:45] <TylerE> but the one that would really slay me was if $out could be made to work with upserts

[00:39:50] <TylerE> for incremental aggregation

[00:54:28] <proteneer_> how does mongodb deal with sharding when one of the shards permanently dies

[00:54:32] <proteneer_> due to some catastrophic failure

[00:57:49] <_sri> perhaps the bulk api many drivers are implementing is just an afterthought and was not actually planned

[00:58:37] <joannac> proteneer_: restore it from backup?

[00:58:46] <proteneer_> joannac: what if I don't have backups

[00:59:00] <proteneer_> or the backup is from like a month ago

[00:59:04] <proteneer_> and there's tons of data missing

[01:01:40] <joannac> proteneer_: the shard isn't a replset?

[01:01:56] <proteneer_> should it always be a replset?

[01:02:30] <cheeser> always

[01:02:32] <_sri> i too like to live dangerously

[01:02:59] <joannac> proteneer_: well... you're going to have a long recovery process

[01:04:08] <joannac> actually, it's probably not that bad

[01:09:06] <joannac> I think if you just restore, the ChunkManager will do the right thing. But you're going to lose any inserts to chunks on that shard, and also any chunks that were moved to that shard

[01:28:24] <hydrajump> hi does anyone here work for mongodb?

[01:34:18] <joannac> hydrajump: yes. what's up?

[01:38:00] <hydrajump> hi joannac. So I've never worked with databases before and I'm looking at possibly using mongodb for a web app. I saw that there is something called MongoDB University which offers free courses and I'm very interested in the one involving node.js (which I also don't know) and mongodb. I saw that the course has already started and lasts till May. However, I can't wait till May because I want to get started right away. Is the course material

[01:38:00] <hydrajump> available prior to the course ending?

[01:41:00] <joannac> hydrajump: can you make an account and just sign up for it?

[01:41:23] <joannac> I think that's what I just did, and I see the first 2 weeks

[01:41:53] <joannac> Oh wait, you want content in advance?

[01:42:01] <hydrajump> yeah if possible

[01:42:30] <hydrajump> I don't care about the assessment or certificate I just want to learn how to use mongodb the right way

[01:46:16] <joannac> I don't know for sure; why do you need to get started so fast anyway?

[01:48:07] <joannac> you could try your luck at education@mongodb.com I suppose...

[01:49:27] <hydrajump> Just eager to figure out how to get started with this web app.

[01:51:19] <joannac> You could try http://mongodbschool.io/ while you wait

[01:53:47] <hydrajump> Thank you I'll have a look.

[01:57:07] <hydrajump> I haven't been able to find a tutorials showing how to structure/ model data with mongodb. I don't mean a simple todo app, but something more complicated such as linkedin. How a user is provided with lots of filters and checkboxes to narrow done their search.

[01:57:45] <hydrajump> Do you know of any tutorials or OSS projects that do anything like that with mongodb?

[02:39:36] <paulkon> what's the recommended max size for a shard before another shard should be added?

[02:39:44] <paulkon> does that depend on the dataset?

[02:43:18] <joannac> wait, did they leave already?

[03:02:10] <garbagecollectio> db.nutrient_data.mapReduce(map, reduce, { query: {"Nutr_No" : 208}, finalize:finalize, out:{inline:1}}).results[0]

[03:02:39] <garbagecollectio> how i write this to automatically get for every unique nutrdat

[03:31:08] <niftylettuce_> hey guys, im trying to use livedb-mongo and not able to enter in my username/password

[03:32:22] <niftylettuce_> https://www.irccloud.com/pastebin/E0T7Gs34

[03:34:22] <niftylettuce_> i have feeling i have syntax err :)

[07:22:39] <IVBaker> hi all, I have a big issue: I'm using a sharded cluster and one of my configuration server died. I can't access the information that it had. I read the process to reset the configuration server here: http://docs.mongodb.org/manual/tutorial/migrate-config-servers-with-different-hostnames/ but I have an error when I relaunch with my new config server

[07:23:04] <IVBaker> ERROR: could not verify that config servers are in sync :: caused by :: config servers

[07:25:01] <joannac> config servers X and Y differ?

[07:25:28] <IVBaker> yes, the new config server is from scratch

[07:25:40] <IVBaker> I don't have the data it was hosting anymore

[07:26:19] <IVBaker> let's say I still have config1 config2 but not config3 and I'm creating a config4

[07:26:54] <joannac> IVBaker: yup, talk me through what you did

[07:28:13] <IVBaker> joannac: for now I'm only testing. I did the following: -create RS1 with 2 shards -connect the shards together -create rs2 with 2 shards -connect together

[07:29:08] <IVBaker> joannac: then create 3 config server, create one mongos with the configdb linking to the 3 config servers

[07:30:02] <IVBaker> joannac: one config server died and I haven't backed up any info on it. Now what I want is to relaunch a third config server to this configuration

[07:30:10] <joannac> okay

[07:30:47] <IVBaker> joannac: I didn disable the "balancer" would it help?

[07:31:19] <joannac> Um, yes. Like the docs say, you should disable the balancer.

[07:31:29] <joannac> But you still haven't told me what you actually did

[07:31:36] <IVBaker> joannac: let me try that

[07:32:55] <joannac> IVBaker: sure, but it won't fix your problem

[07:35:31] <IVBaker> joannac: still not working

[07:36:02] <joannac> IVBaker: are you going to tell me what you actually did, after config3 died?

[07:37:27] <IVBaker> joannac: stopped all the shard cluster, I stopped the 2 other config server, the mongos

[07:37:59] <IVBaker> joannac: restarted the 4 shards, the 2 config servers, created a new config server

[07:38:48] <asturel> (@joannac) asturel: looks like you fell off the oplog -> how can i fix it?:D

[07:39:00] <asturel> i tried delete the dbpath but didnt helped

[07:39:25] <joannac> IVBaker: um, is that the order you did it in? because that's not right

[07:39:42] <IVBaker> joannac: tried to restart mongos with the new third config server and I have the error message: ERROR: could not verify that config servers are in sync :: caused by :: config servers "config1" and "config4" differ

[07:39:56] <joannac> asturel: shut down the server you're getting the message in. remove everything in the dbpath. start the mongod again

[07:40:12] <asturel> i did that

[07:40:28] <joannac> asturel: paste your logs in a pastebin from that server

[07:40:35] <asturel> rs.config?

[07:41:02] <joannac> no, your mongod log

[07:41:13] <joannac> the value you pass --logpath

[07:42:02] <asturel> http://pastebin.com/JZmZkiEc

[07:42:10] <joannac> IVBaker: can you please answer my question: "is that the order you did it in?"

[07:42:22] <asturel> on the primary the last opt time is today

[07:42:45] <joannac> asturel: that's the logs before you cleared the dbpath

[07:43:42] <asturel> ah ok i do it again

[07:44:19] <IVBaker> joannac: not sure, I used a script, let me check/do it again

[07:45:30] <asturel> http://pastebin.com/3CALXBSR

[07:46:44] <asturel> i guess the replset the problem?

[07:46:50] <joannac> asturel: upgrade your mongodb installation

[07:47:12] <joannac> you're 2 major versions behind.

[07:47:54] <joannac> but yes, you're using --slave (which is not valid for a replset), and you're getting "non-matching repl set name"

[07:49:52] <joannac> also I'm not sure why this member thinks it has an oplog...

[07:50:11] <asturel> well i removed the slave thing but didnt helped

[07:50:40] <asturel> hmm where should i add --replset in debian?

[07:50:57] <joannac> how do you start mongodb?

[07:51:06] <joannac> actually

[07:51:07] <joannac> edit /etc/mongodb.conf

[07:51:44] <asturel> ba init.d

[07:51:46] <asturel> by*

[07:51:52] <asturel> i added rest=true and replset in that config

[07:54:13] <asturel> Tue Apr 8 09:48:29 [rsStart] replSet exception loading our local replset configuration object : 13132 nonmatching repl set name in _id field; check --replSet command line

[07:54:23] <asturel> Tue Apr 8 09:48:30 [rsMgr] replSet total number of votes is even - add arbiter or give one member an extra vote

[07:54:36] <asturel> Tue Apr 8 09:48:19 [rsStart] replSet info you may need to run replSetInitiate -- rs.initiate() in the shell -- if that is not already done

[07:54:57] <asturel> but if i tpye rs.initate it gives me error: "errmsg" : "already initialized" "info" : "try querying local.system.replset to see current configuration",

[07:55:14] <joannac> if you query it, what does it say?

[07:55:26] <asturel> how do i query ?

[07:55:36] <asturel> local.system.replset() didnt works :D

[07:55:45] <joannac> use local; db.system.replset.find()

[07:56:12] <asturel> { "_id" : "rs1", "version" : 1, "members" : [ { "_id" : 1, "host" : "78.24.185.47:27017" } ] }

[07:56:16] <asturel> bah.. it has to be rs0

[07:56:36] <joannac> why do you have a document in there? I thought you wiped the dbpath?

[07:56:50] <asturel> yes i did

[07:57:08] <joannac> tell me what you did

[07:57:28] <asturel> "dbpath" : "/var/lib/mongodb", ;; sudo /etc/init.d/mongodb stop && sudo rm -f "/var/lib/mongodb/*" && sudo /etc/init.d/mongodb restart

[07:57:43] <joannac> you need -R

[07:57:53] <joannac> to remove everything in the dbpath including directories

[07:59:06] <asturel> bah, works, thanks

[08:00:06] <asturel> i couldnt do queries on slave?

[08:00:10] <asturel> secondary*

[08:00:18] <asturel> Tue Apr 8 09:59:46 uncaught exception: error: { "$err" : "not master and slaveok=false", "code" : 13435 }

[08:02:19] <IVBaker> joannac: I did stopped all my instances in this order: mongodb, config server, mongos

[08:02:45] <IVBaker> joannac: (even if we can say the dying config server stopped first)

[08:23:03] <joannac> asturel: rs.slaveOk()

[08:23:54] <joannac> IVBaker: i meant, you shut down the server, copied the files, then brought them back up?

[08:24:14] <joannac> you didnt copy files while yhe process was running?

[08:27:45] <IVBaker> joannac: I don't have the file of my config server 3

[08:27:52] <IVBaker> the one that died

[08:28:24] <asturel> do i have to type it everytime?

[08:29:03] <asturel> also, isodate is in gmt+0 and DST isnt applied?

[08:29:08] <asturel> or i screwed something?

[08:32:06] <asturel> i do mongo::Date_t from unixtime in cpp driver

[08:35:04] <asturel> http://pastebin.com/0yNJc0X2

[08:39:11] <rspijker> asturel: you can give the offset as part of ISODate

[08:39:29] <rspijker> ISODate('2014-04-04T10:39:00.000+02:00Z')

[08:42:08] <asturel> ISODate("2014-04-08T07:29:14Z") now

[08:42:20] <asturel> but it should be 9:29

[08:43:19] <rspijker> the date formats and ways they are stored are always a bit of a pain

[08:43:25] <rspijker> sometimes there is a d and o element

[08:43:46] <rspijker> the d will hold the date as an isoDate and the o will hold an int offset

[08:43:46] <rspijker> it depends on the application how it's stored exactly, I think

[08:46:47] <asturel> but by 'default' its in gmt+0 w/o DST?

[08:48:47] <asturel> coz i dont see any other params @ mongo::Date_t

[08:52:15] <joannac> IVBaker: all 3 config servers are the same. copy the files from another server after you shut down mongod

[08:53:07] <IVBaker> joannac: I'll try

[09:02:15] <rspijker> asturel: think so, yeah

[09:02:42] <rspijker> asturel: but the output you get could be preformatted by the client you are using. That is, the mongo shell gives me a different response than, for instance, robomongo

[09:09:28] <IVBaker> joannac: I'm tried to copy the db from config server 2 to 4 bu it's still not working

[09:09:40] <IVBaker> exit

[09:28:51] <pontus_> anyone got any pointers on how to group coordinates? I would like to take thousands of ducuments and group by say the ten closest. I assume aggregation query is the way to go...

[09:29:12] <pontus_> closest would mean close to each other. not to me

Log file Viewer

Help | Karma | Search:

#mongodb logs for Tuesday the 8th of April, 2014