PMXBOT Log file Viewer

Help | Karma | Search:

#mongodb logs for Monday the 22nd of June, 2015

(Back to #mongodb overview) (Back to channel listing) (Animate logs)
[01:06:20] <AbuDhar> hey
[01:06:27] <AbuDhar> can mongo db be used for all kind of projects?
[01:18:56] <cheeser> AbuDhar: yes, it can within reason.
[01:20:41] <AbuDhar> and can I get schemas in mongodb?
[01:20:57] <joannac> what does "get schemas" mean?
[01:21:31] <AbuDhar> I mean have my collections(tables) with defined attributes like with MySQL? lol
[01:21:42] <joannac> no
[01:22:00] <cheeser> there is no enforced schema in mongodb
[01:22:43] <AbuDhar> but there are no libs that can extend mongodb that offers this?
[01:24:52] <cheeser> depends on the language. i think mongoose supports that if you use python. morphia does if you use java. c#'s driver will.
[01:25:23] <cheeser> with statically type langauges you kinda get that schema enforcement for free. most ODMs would give you some version of that.
[01:26:42] <AbuDhar> how about the node.js driver? :D
[01:27:15] <joannac> mongoose is nodejs I though?
[01:27:45] <cheeser> oh, it is actually.
[01:27:53] <cheeser> they all blur together for me.
[01:27:57] <joannac> AbuDhar: there are a steady stream of people through this channel who use mongoose without really understanding mongodb, and get themselves into trouble
[01:28:14] <joannac> if you're going to use mongoose, make sure you understand what it's abstracting
[01:28:39] <AbuDhar> oh ok.
[01:29:05] <AbuDhar> sounds good :)
[01:31:19] <cheeser> a general rule of thumb for any abstraction
[01:33:05] <AbuDhar> cheeser, we two know each other
[01:33:17] <cheeser> i know
[01:40:06] <preaction> ugh. leaky abstractions :(
[01:41:27] <AbuDhar> oh this guy is also here.. all my enemies gathered in one channel lol
[01:44:20] <dreamreal> You have enemies? Why?
[01:44:54] <AbuDhar> It was a joke.
[01:45:03] <AbuDhar> but former enemies maybe :)
[03:36:41] <Freman> What fs do ya'll recommend we put our mongo on?
[03:36:52] <Freman> it's huge, random reads, sequential rights
[03:36:57] <Freman> s/rights/writes/
[03:38:39] <cheeser> http://docs.mongodb.org/manual/administration/production-notes/#kernel-and-file-systems
[03:39:17] <Freman> so not btrfs (I kid I kid)
[03:54:31] <krebipeti> hi all mongo db ninjas
[03:54:50] <krebipeti> i have read about mongodb replication
[03:57:25] <krebipeti> i would like to create a system which is insert data on the primary server, and all read will be balanced to the secondary server. i'm thinking about spreading the load between read/write
[03:57:58] <krebipeti> it will be one write server and maybe 2 or 3 read server
[03:58:18] <cheeser> replication isn't for that. it's more about durability.
[03:58:32] <krebipeti> so to achieve that...what should i do?
[03:58:48] <cheeser> if your system is that busy, look in to sharding.
[03:59:41] <krebipeti> ahh...alright. i'm actually a bit confuse when to use replication and when to use sharding. so i can do that by sharding?
[04:00:28] <cheeser> sharding is for spreading read/write loads. replication is for durability and failover.
[04:00:33] <cheeser> to oversimplify
[04:00:47] <krebipeti> alright. thank you very mush @cheeser
[04:00:50] <cheeser> good luck with that. i'm off to bed.
[04:00:51] <cheeser> np
[04:00:52] <krebipeti> much
[04:01:01] <krebipeti> hehe...sweet dream
[11:01:33] <Sp4rKy> Hi, I have a replicaset with a primary and a secondary. I want to remove the secondary (so that I move to a standalone instance) and remove the db from the (old) secondary
[11:02:03] <Sp4rKy> I used rs.remove to remove the secondary from cluster, but then I get "not master" when trying to remove the db on the secondary
[11:02:19] <Sp4rKy> I guess I should get ride of the RS config on the secondary ?
[11:17:12] <coudenysj> Sp4rKy: are you issuing the remove command on the secondary or on the primary?
[11:17:15] <einyx> is upstart broken in the upstream version of mongo?
[11:17:18] <einyx> restart doesn't work.
[11:17:30] <Sp4rKy> coudenysj: on the primary
[11:17:51] <Sp4rKy> coudenysj: I achieved it by then restarting the (old) secondary with no replSet param
[11:18:00] <coudenysj> idd
[12:38:19] <djMax> I have a 3 member replica set that got "moved." All the hostnames are now different. I can't seem to reconfig them as it keeps telling me "LOADINGCONFIG" failed. Is it possible to revive them?
[12:41:17] <deathanchor> djMax: not recommended, but you could start them up standalone and modify the local db where the replica info is stored. (db.system.replset) then bring them all back online
[12:41:42] <djMax> ah. Makes sense.
[12:42:09] <djMax> and my attempts to reconfig() are meant to (and will forever) fail?
[12:42:32] <deathanchor> yeah I had that issue once, let me check my notes real quick
[12:44:23] <deathanchor> yeah, what I did was just trust one to be the primary. stated it in standalone, dropped the 'local' db and started it up in replset, and did an rs.initiate() on it, then added the other hosts one at a time.
[12:44:53] <deathanchor> but you might be able to "force" it by changing the local db directly
[12:45:05] <djMax> interesting. And blew away the db entirely on the others? Or just let replication take care of it?
[12:45:09] <deathanchor> again not a recommended action
[12:45:22] <djMax> If only there was a recommended option :)
[12:45:23] <deathanchor> replication will automatically drop things
[12:45:56] <deathanchor> but I don't think the others will allow it to sync if it has an old config in place
[12:46:18] <deathanchor> so you need to blow those away too (or config them manually in standalone mode)
[12:46:31] <cheeser> the recommendation, i think, is to add new replica set members on the new hosts and remove the old ones one sync is done
[12:49:41] <djMax> hmm, it's saying I'm not authorized to update the db.system.replset collection
[12:50:05] <cheeser> do you have auth enabled?
[12:50:13] <djMax> yes - but I'm siteRootAdmin
[12:50:39] <cheeser> i don't think that'd be sufficient in this case
[12:50:59] <djMax> what user is?
[12:51:31] <cheeser> the user matters less than the role but i'm not sure the exact role name.
[12:51:41] <cheeser> for my tests, i have a user with the 'root' role
[12:52:19] <deathanchor> since you are starting it in standalone to make the small mod, why not just turn off auth and just do a bind to the localhost to avoid anyone else using it while you mod the local db?
[12:52:31] <cheeser> that'd be my advice, too.
[12:52:36] <djMax> ok
[12:52:50] <cheeser> and if that fails, pour gasoline on it and light it up.
[12:52:58] <djMax> :)
[13:05:48] <djMax> seems like it's back up. One is still "RECOVERING" which seems odd because there shouldn't be much to recover
[13:07:47] <djMax> its optime date is not moving... I assume it should be.
[13:10:48] <djMax> docs seem to suggest I should remove the dbpath, but wouldn't that lose the replica set config?
[13:12:32] <djMax> yikes, and my local.1 file is 2GB. That seems odd.
[13:15:30] <joannac> *eyebrows*
[13:15:34] <joannac> what did you do?
[13:16:56] <joannac> djMax: what do the logs say?
[13:18:49] <djMax> logs seem quiet, rs.status() says everything is fine
[13:41:08] <nalum> hello all, I'm running an aggregate function to add 7 hours to a date, is there a way to restrict it to only affect documents that have a specific value set in a field?
[13:44:14] <deathanchor> nalum: $match
[13:45:29] <nalum> Okay, that'd go before the $project?
[13:52:22] <nalum> deathanchor: thanks, got it done
[13:52:44] <deathanchor> nalum: depends when you want the filtering done :)
[13:55:50] <vagelis> Hello, can u see my message? ChanServ says that I have to register my nickname even though i think I have done it.
[13:56:02] <StephenLynx> aye
[13:56:27] <vagelis> StephenLynx are you talking to me?
[13:56:31] <StephenLynx> yes
[13:56:32] <deathanchor> vagelis: he was
[13:56:49] <vagelis> Oh cool thanks guys. Ok i have a mongodb question, may I ask please?
[13:57:37] <deathanchor> vagelis: you don't need to ask permission to ask a question related to mongo (or it's various flavors)
[14:00:49] <vagelis> Well I had the impression that even though the room is called Mongodb its not supposed to be a place for questions :s I dont i just had this impression. Ok so, my question is, well I am trying to understand the explain(). So in winning plan the inner brackets say IXSCAN. Even though the query is find({ random_field { $gt: 5 } }).sort({ index_field: 1 })
[14:01:21] <vagelis> So even though the query doesnt use an index for find, it uses index for sorting.
[14:02:06] <vagelis> The answer to the question if query uses index to find and match document is NO why the inner brackets show IXSCAN?
[14:02:51] <vagelis> I know the answer but I cant understand it from the explain().
[14:04:21] <vagelis> random_field is just a random field. index_field is a created index. I tried to name them like that so you can understand.
[14:06:39] <deathanchor> vagelis: not sure of the question exactly, but if the query uses one index, then the sort will either use the same index or not at all.
[14:06:58] <deathanchor> well that's my experience with old old mongo 2.2
[14:07:00] <vagelis> no the query doesnt use index
[14:07:27] <deathanchor> vagelis: it has to use some index, even if it is the worst one ( _id ).
[14:07:49] <deathanchor> how about sharing a gist of the explain output?
[14:07:54] <vagelis> btw in the key_pattern says that use index_field: -1
[14:08:05] <vagelis> but i dont get it how it uses it
[14:08:48] <vagelis> Maybe if theres no index in the find(), it looks in the sort() if theres one?
[14:11:52] <deathanchor> how about sharing a gist of the explain output? you can sanitize any private info
[14:12:58] <vagelis> well i dont know what is supposed to be private info :S i am currently in the mongouniversity and I dont know what should/should not show
[14:13:20] <vagelis> let me create a similar by my self and show to u
[14:16:27] <vagelis> Ok here: https://bpaste.net/show/1675e4a972f6
[14:17:33] <vagelis> Also i dont get why it does it backwards. I asked that and I am waiting an answer but this is less important for me right now.
[14:21:50] <deathanchor> ok, so the query didn't use an index to filter because random_field wasn't indexed, but the sort was indexed, so I used the index for sorting only it appers.
[14:22:30] <deathanchor> I'll need to read up more on explain("executionStats"), that's new to me
[14:22:54] <deathanchor> I've always just used a cursor explain
[14:22:55] <vagelis> ok deathanchor thanks for ur effort
[14:23:13] <deathanchor> no wonder.. it's 3.0 :(
[14:23:25] <vagelis> yes
[14:24:14] <deathanchor> vagelis: I have no mongodb 3.0 exposure, (hence my stating I'm always on the old old fogy version of 2.2, 2.6)
[14:24:35] <vagelis> no problem guys thanks for ur responds
[14:28:25] <fxmulder> so I found that after setting up sharding and migrating data, the original shard still contains the old migrated data leading to duplicate documents between the shards, whats the best way to clean that up?
[14:29:25] <cheeser> maybe this http://docs.mongodb.org/manual/reference/command/cleanupOrphaned/
[14:33:05] <fxmulder> cheeser: that sounds promising, thanks
[14:34:24] <cheeser> np. good luck.
[14:52:13] <saml> is there a way to grep entire collection?
[14:52:27] <saml> i don't know structure of documents.. just want to grep for a string foobar
[14:53:03] <saml> db.docs.forEach(function(doc) { JSON.stringify(doc).indexOf('foobar') ?
[14:53:09] <deathanchor> saml: grep from all the various values?
[14:53:16] <saml> yah
[14:53:27] <saml> in key or value... doesn't really matter
[14:53:41] <deathanchor> saml: mongoexport | grep but you won't like how long it takes
[14:53:50] <saml> yah
[14:56:11] <deathanchor> saml: you can also try a strings on all the files and grep and hope you find something useful
[14:58:53] <fxmulder> should I have an admin database?
[14:59:44] <deathanchor> fxmulder: it's a special db
[15:00:25] <GothAlice> fxmulder: "admin" and "local" are used to control MongoDB behaviour and store internal state.
[15:01:08] <deathanchor> admin doesn't always show up when doing a show dbs.
[15:04:14] <GothAlice> It's automatically created the first time it's needed. I.e. when adding the first user after enabling authentication, when setting up a replica set or shard array, etc.
[16:39:15] <unseensoul> Hi
[16:39:45] <unseensoul> How do I search for a field which is both different than null and ""
[16:39:46] <unseensoul> ?
[16:52:17] <StephenLynx> $and
[16:52:30] <StephenLynx> unseensoul
[16:53:12] <StephenLynx> I don't know if $exists will return false in case of a null
[16:53:25] <StephenLynx> if it doesn't, you will have to check if field
[16:53:29] <StephenLynx> 1- exists
[16:53:33] <StephenLynx> 2- is not null
[16:53:42] <GothAlice> StephenLynx: He left.
[16:53:49] <StephenLynx> :v
[16:54:03] <GothAlice> Also, sub-optimal answer. {$nin: [null, '']} is somewhat more legible. ;)
[16:54:08] <StephenLynx> ah
[16:54:09] <StephenLynx> indeed.
[17:13:25] <shlant> anyone know how I can give truly "full access" in mongo short of assigning every role to a user? My app seems to not be able to access some data or create some db's and in attempting to troubleshoot if it's permissions, I tried using a "root" user, but that doesn't seem to give me the "full access" permissions I want.
[17:18:36] <tejasmanohar> does indexing a field make it necessarily forced to be unique?
[17:22:46] <tejasmanohar> or whats the very definition of an index
[17:24:45] <GothAlice> tejasmanohar: http://docs.mongodb.org/manual/core/indexes/
[17:25:44] <tejasmanohar> GothAlice: i read that unique is a property you can apply on index
[17:25:51] <GothAlice> Correct.
[17:25:54] <tejasmanohar> GothAlice: so unique is not mandatory for indexes?
[17:25:56] <tejasmanohar> seems right
[17:25:56] <tejasmanohar> yeah
[17:27:30] <cheeser> correct. requiring indexed fields to be unique would incredibly limiting.
[17:28:35] <tejasmanohar> gotcha
[18:25:32] <Argorok> Hi! How can I sum a field and group by dayOfWeek? I wanna build a query that retrieve for me the sum of the field grouped by day of week of another field
[18:26:38] <deathanchor> Argorok: Aggregation framework, useing $project, $dayOfWeek, and $sum
[18:26:55] <StephenLynx> that
[18:27:10] <StephenLynx> put the day of week in _id
[18:27:13] <deathanchor> and $group can't forget our friend $group
[18:27:14] <Argorok> deathanchor, Is the only way to do it? Ok, let me try. I saw it in the docs, but looked like a lot more than what I need
[18:27:25] <StephenLynx> afaik, yes.
[18:27:46] <StephenLynx> if you need to perform any operation that relates several documents to produce another one, you have to use $group
[18:28:03] <StephenLynx> and afaik you can only do that on aggregate, but I am not 100% sure of all that.
[18:28:08] <StephenLynx> just about 90% or so
[18:29:30] <deathanchor> Argorok: here is an example: db.collection.aggregate([
[18:29:30] <deathanchor> { $match : {
[18:29:30] <deathanchor> mykey : "searchkey",
[18:29:31] <deathanchor> time : {
[18:29:31] <deathanchor> $gt : ISODate("2015-01-01T00:00:00Z"),
[18:29:33] <deathanchor> $lt : ISODate("2015-02-09T00:00:00Z") }
[18:29:35] <deathanchor> }
[18:29:38] <deathanchor> },
[18:29:40] <deathanchor> { $project : {
[18:29:43] <deathanchor> dow : {
[18:29:45] <deathanchor> $dayOfWeek : "$time"
[18:29:48] <deathanchor> },
[18:29:50] <deathanchor> totalSent : 1
[18:29:53] <deathanchor> },
[18:29:55] <deathanchor> },
[18:29:58] <deathanchor> { $group : { _id : "$dow", totalSent : { $sum : "$totalSent" } } },
[18:30:00] <deathanchor> crap
[18:30:03] <deathanchor> sorry
[18:30:03] <StephenLynx> ol
[18:30:05] <deathanchor> meant to paste: https://gist.github.com/deathanchor/03a645b3c6cddaf95210
[18:30:08] <deathanchor> :(
[18:32:36] <Argorok> No problem :D Hmmmmm, project is like projection and not a project itself. I was reading the word project and translating in the wrong way when I first read
[18:33:33] <StephenLynx> lol
[18:38:29] <deathanchor> I read it as Pro-Ject (project onto a screen) not Pra-ject (like a working project)... NY accent here.
[18:39:37] <StephenLynx> I read both the same :v
[18:39:58] <StephenLynx> I just know mongo uses as a verb and not a noun
[18:40:12] <StephenLynx> because of "projection"
[18:48:41] <m3t4lukas> hey guys
[18:48:49] <Argorok> deathanchor, StephenLynx, english isnt my mother lang :P
[18:49:04] <StephenLynx> neither is mine.
[18:49:17] <m3t4lukas> is there a way to tell the mongodb c connector to stop spaming the console?
[18:49:45] <cheeser> i imagine there's configurable logging in the mix somewhere.
[18:50:05] <m3t4lukas> me, too, but I can't find where
[18:50:15] <m3t4lukas> I've not found it in the doc
[18:50:29] <cheeser> http://api.mongodb.org/c/current/logging.html ?
[18:50:43] <m3t4lukas> thing is when I use logstash it's difficult when there are multiple logging formats...
[18:53:39] <m3t4lukas> thanks cheeser
[18:53:45] <StephenLynx> I always got curious. What kind of scenario uses both C and mongo? What are you using mongo for?
[18:54:31] <m3t4lukas> I use Mongo in D and I have my own model access layer
[18:54:39] <cheeser> D++
[18:54:52] <m3t4lukas> I use it for ultra fast and flexible and feature webapps
[18:55:06] <m3t4lukas> *feature rich
[18:55:30] <StephenLynx> so you just use the C driver for D?
[18:55:36] <m3t4lukas> yep
[18:55:38] <StephenLynx> ah
[18:55:54] <m3t4lukas> as soon as I find the time I will write a driver for D
[18:55:58] <m3t4lukas> a native one
[18:56:16] <m3t4lukas> currently I'm wrapping the parts I need, which is most of it :P
[18:56:56] <m3t4lukas> a port would not be that difficult since D encapsulates everything C has
[18:57:09] <m3t4lukas> but, time time time :/
[18:57:18] <cheeser> is on my side?
[18:58:19] <StephenLynx> wat
[18:58:42] <cheeser> https://www.youtube.com/watch?v=wbMWdIjArg0
[18:58:51] <StephenLynx> ah
[18:58:52] <StephenLynx> :v
[18:59:30] <StephenLynx> lukas, you got a benchmark for web on D?
[20:45:27] <tejasmanohar> name: { first: String, last: String } vs firstName: String, lastName: String
[20:45:29] <tejasmanohar> modeling name data
[20:45:31] <tejasmanohar> need a separation
[20:45:43] <tejasmanohar> can't just split because there may be "Bob John APple" as first name
[21:46:28] <Siamaster> I tried to run this command from the admin
[21:46:30] <Siamaster> it didn't work
[21:46:34] <Siamaster> db.createUser({"user": "Siamaster", "pwd": "myPassword", "roles" : [{"role": "dbOwner", "db" : "MyAppDb"}]});
[21:46:37] <Siamaster> the admin database
[21:46:40] <Siamaster> but!
[21:46:56] <Siamaster> because it says Siamaster already exists in admin
[21:46:57] <Siamaster> but!
[21:47:09] <Siamaster> when I execute it on MyAppDb
[21:47:09] <Siamaster> it works
[21:47:32] <Siamaster> I'm already saying db is MyAppDb , I'm getting confused, what am I missing?
[21:54:19] <Siamaster> and show collections aren't working in the admin database when I'm logged in with a user with the userAdminAnyDatabase role
[21:54:25] <Siamaster> that is created in the admin database
[23:54:58] <shlant> anyone know why I would get "connected to: blah" with mongorestore and that's it? it doesn't restore anything and no output