PMXBOT Log file Viewer

Help | Karma | Search:

#mongodb logs for Saturday the 21st of February, 2015

(Back to #mongodb overview) (Back to channel listing) (Animate logs)
[00:20:23] <MacWinner> should tailable cursor and capped collections work when connected through mongos?
[00:23:14] <Boomtime> dunno, can't really see any problem, but be aware that you can't shard a capped collection
[00:24:22] <MacWinner> k
[00:38:49] <dev__> I have set up MongoDB 2.6.7 on an ubuntu server and have been trying to make it enforce authentication. I have set up a superuser and added auth=true to mongodb.conf. and restarted the service. what else is there to do? because it still just takes me directly to the test data without asking for a username and pw.
[00:39:13] <dev__> database*
[01:25:32] <dev__> I have set up MongoDB 2.6.7 on an ubuntu server and have been trying to make it enforce authentication. I have set up a superuser, added auth=true to mongodb.conf, and restarted the service. what else is there to do? because it still just takes me directly to the test data without asking for a username and pw. What step am I missing?
[01:27:12] <checkit> Hey guys... Mongo newb here... Trying to figure out the simple things... Start/Stop?
[01:28:07] <dev__> how to start and stop the service?
[01:28:17] <checkit> Yes
[01:28:27] <dev__> what are you running mongo on?
[01:28:34] <checkit> OSX
[01:31:06] <dev__> hmm idk about osx. i am new to it too but i am using ubuntu
[01:31:37] <dev__> for ubuntu it is sudo service mongod <operation>
[01:31:50] <dev__> where <operation> could be start, stop, restart
[01:32:39] <checkit> Right...
[01:32:45] <checkit> Don't have that on OSX.
[01:34:45] <dev__> well i can't seem to find anything about it for osx so far with my googling
[01:34:48] <dev__> i'll keep trying though
[01:38:18] <dev__> to start it i guess you could just type mongod
[01:38:26] <dev__> to end it you could look in to this
[01:38:27] <dev__> http://serverfault.com/questions/370545/shutting-down-mongodb-with-a-command-line-command-is-it-possible
[01:39:15] <dev__> but that is from 2012. idk what version mongodb that is for.. seems ridiculous that there isn't a clean way to start and stop the mongod process on osx
[01:41:27] <checkit> dev__:
[01:42:00] <checkit> I was able to find a few things.. Like entering into mogo and issueing "use admin" and "db.shutdownServer()"
[01:42:00] <dev__> yes?
[01:42:06] <dev__> ah
[01:42:09] <checkit> Or by simply issueing kill pid
[01:42:39] <checkit> dev__: I was thinking there might have been a more graceful approach.
[01:43:29] <dev__> on ubuntu there is but idk why there isn't on osx
[01:47:22] <dev__> checkit: i don't suppose you k now how to set up authentication on mongodb? I have set up MongoDB 2.6.7 on an ubuntu server and have been trying to make it enforce authentication. I have set up a superuser, added auth=true to mongodb.conf, and restarted the service. Any idea what step I am missing? because it still just takes me directly to the test database without asking for a username and pw.
[01:48:08] <checkit> dev__: No idea but...
[01:48:09] <checkit> Sorry
[01:48:22] <dev__> no worries
[02:02:05] <dev__> any of your guys(or gals?) that just game in up for helping me out setting up mongodb?
[02:02:11] <dev__> of you*
[02:31:26] <acidjazz> hey every1
[02:31:31] <dev__> hi
[02:32:40] <acidjazz> Fatal error: Uncaught exception 'MongoConnectionException' with message 'MongoClient::__construct(): php_network_getaddresses: getaddrinfo failed:
[02:32:56] <acidjazz> was going to bug/google this but i think its actually a DNS error on my side
[02:37:13] <dev__> well good luck getting that answered. I brand new to mongodb, been sitting in here for a few hours asking my question occasionally without any response
[02:37:21] <dev__> I am*
[02:42:14] <acidjazz> whats your question
[02:42:34] <acidjazz> i figured mine out it was DNS to the mogno server
[02:44:18] <dev__> acidjazz,
[02:44:19] <dev__> I have set up MongoDB 2.6.7 on an ubuntu server and have been trying to make it enforce authentication. I have set up a superuser, added auth=true to mongodb.conf, and restarted the service. Any idea what step I am missing? because it still just takes me directly to the test database without asking for a username and pw.
[02:47:06] <dev__> i am not sure if it isn't asking me for authentication because I am running mongo locally via root... so i've been trying to remote in to the mongodb server from my own machine and haven't been able to successfully connect
[02:53:42] <Boomtime> dev__: "it still just takes me directly to the test database without asking ..." -> have you tried doing anything?
[02:53:45] <Boomtime> like, run a query
[02:54:06] <Boomtime> just because the cursor says "test" doesn't mean the shell has actually attempted any operation
[02:55:27] <dev__> ok so it would allow someone without credentials to still open a connection and keep it open? even if it won't let them perform operations?
[02:55:42] <Boomtime> how would it stop someone from 'opening' a connection?
[02:56:12] <Boomtime> it's true it should drop it after a while, i don't know if there is a way to do that
[02:56:26] <dev__> by asking them for a username and password and if they don't have it turning them away
[02:56:42] <dev__> could I ask why you put opening in quotes?
[02:56:47] <Boomtime> sadly, that isn't how the wire protocol works, all commands are push
[02:57:07] <Boomtime> because opening a connection is at the TCP layer, the server has no say
[02:58:28] <Boomtime> if you want proper TCP level auth, you should use X.509, that way the authentication is done on establishment of the connection, and only access rights remain to be processed
[02:58:43] <dev__> ok
[02:58:49] <dev__> i'll look in to that
[02:59:05] <dev__> thank you
[02:59:27] <dev__> but yes, i just tired altering test and it still lets me do it
[02:59:41] <dev__> tried*
[03:00:04] <Boomtime> interesting, auth is apparently not actually enabled
[03:01:01] <dev__> ok nevermind sorry. i just exited the shell and reopened it.
[03:01:14] <dev__> i just remembered i had previously called db.auth
[03:01:28] <dev__> so i tried it again and it stopped me
[03:01:31] <dev__> thank you
[03:01:59] <Boomtime> excellent, good to hear
[03:19:59] <bros> mongo.count(mongo.models.account, { 'users.email': email }) === 0.
[03:20:04] <bros> Save collection = Error: MongoError: E11000 duplicate key error index: db.accounts.$users.email_1 dup key: { : null }
[03:22:59] <Boomtime> hi bros, do you have a question?
[03:23:07] <bros> Boomtime, I'm lost.
[03:23:23] <Boomtime> ok, apparently you have a unique index on users.email
[03:23:28] <bros> I do.
[03:23:50] <Boomtime> well, you have a document in there with a null value for that field
[03:23:57] <Boomtime> and you just tried inserting another one
[03:25:13] <Boomtime> if you need users.email to be permitted to be null but unique if it is set, you might consider a sparse index
[03:26:53] <bros> Boomtime, is there any chance
[03:26:56] <bros> that
[03:27:01] <bros> it's trying to recreate the collection?
[03:27:09] <bros> because some guy in #node.js lied to me about modules
[03:27:31] <Boomtime> 'recreate the collection' - what do you mean?
[03:27:46] <acidjazz> dev__: sorry man never set up mongo to require auth, always just setup my stuff to where only my web servers can connect
[03:27:59] <acidjazz> ah you figured it out cool
[03:28:04] <Boomtime> ok, so full disclaimer: i'm telling you what the error code means, not whatever crazy ODM or library you are using is doing
[03:28:05] <bros> Boomtime, I'm using some ODM that is making me rip my hair out called Mongoose.
[03:28:13] <bros> I think it like... creates the collection if it doesn't exist?
[03:28:22] <bros> I'm not trying to insert null anywhere.
[03:29:48] <Boomtime> ok, yes, you get that same error code when a unique index is created that aborts due to unique violation
[03:30:21] <Boomtime> your ODM might be trying to create the index
[03:30:37] <Boomtime> but if so, then data already exists which violates the key contraint
[03:31:02] <Boomtime> as always, i'd recommend you use the mongo shell to see what the reality is
[03:32:28] <acidjazz> ah mongoose
[03:33:18] <acidjazz> does mongoose have some option to show the raw mongo command? it shoudl at least dump that upon an error
[04:00:20] <GothAlice> I've got 99 problems, and Mongoose is most of them. :/
[05:36:23] <GothAlice> "[I] never set up mongo to require auth, always just setup my stuff to where only my web servers can connect" — ah. acidjazz: That's… one approach. Firewall rules?
[05:37:53] <GothAlice> 'Cause source address limiting might prevent data egress, but it won't stop someone from deleting everything. Or injecting data. Or spontaneously joining your server to a replicaset.
[05:37:54] <GothAlice> ¬_¬
[05:38:31] <GothAlice> (Won't replicate, of course. But it would certainly disrupt service.)
[09:51:23] <pedroabreu> is it ok if I make quite a noob question? .-.
[09:54:02] <pedroabreu> what is it the easiest way to share the database in a githubproject if I don't want to host it?
[10:00:21] <metathink> Hi
[10:02:04] <metathink> Is there a Bulk system which can handle insertions on many collections ?
[10:15:13] <metathink> I use mongodb to save logs, but because I need to frequently delete a large amount of categories of this logs, so I insert each category in a specific collection to be able to use db.collection.drop() without delay. I also put the received logs in a "buffer" and bulk insert them every X seconds. But if there was to many differents categories buffered before the X seconds passed, I have to do to a lot of bulk
[10:15:15] <metathink> insert in the same time. Any idea?
[10:43:04] <metathink> Well sorry, it seems to be efficient.
[11:14:35] <evangeline__> hi
[11:14:55] <evangeline__> how can I count the number of elements this query returns: db.train.find({sectname: {$exists : true }}).pretty()
[11:15:30] <evangeline__> ah .count() works ... sorry for the bother
[11:52:26] <mazzy> hi guys
[11:52:55] <mazzy> I'm trying to mount ebs volume on coreos by adding options flag to systemd
[11:52:56] <mazzy> Options=defaults,auto,noatime,noexec 0 0
[11:53:00] <mazzy> but I get an error
[11:53:18] <mazzy> someone of you have some experience of coreos/systemd?
[11:54:27] <metathink> :q
[11:54:34] <metathink> (sorry)
[14:25:49] <quakephil_> I'm doing a db.blah.insert() and then a db.blah.find() but the find is not finding the latest insert. But if I do the find a couple seconds later, it finds the inserted document. So it looks like a timing issue on my end. Is there a better way to do an insert-one-document-and-then-find-all-documents-including-the-inserted-one than the way I'm doing it?
[14:32:08] <quakephil_> I suppose the right approach to do the find first, add the to-be-inserted document to that local collection, and then do the insert separately?
[14:57:23] <mazzy> I'm trying running mongodb but I got this error SEVERE: Failed global initialization: FileNotOpen Failed to open "/log/mongodb.log"
[14:59:07] <mazzy> here is my config file http://pastebin.com/yz5m1dtm
[15:21:32] <jaitaiwan> quakephil_ you probably want to check the write concern to make sure your insert doesn't return till its been propogated to all servers and disks: http://docs.mongodb.org/manual/core/write-concern/
[15:22:16] <jaitaiwan> mazzy: does /log/ actually exist? if so, does it have the right file permissions?
[15:23:46] <mazzy> I have just figure out that maybe it doesn't have the right permission to execute
[15:23:59] <mazzy> I'm running mongo into official mongo docker image
[15:24:28] <mazzy> I need to fix the permission of the log file giving it mongod user and group
[15:28:54] <jaitaiwan> sounds right mazzy :)
[15:29:23] <jaitaiwan> probably also needs enough permissions to write to /log/ too because it may archive old logs
[15:29:30] <mazzy> I just need to figure out how to give right permission without rebuild again mongo docker image
[15:30:04] <lilgiant> hello, one question: i run a "db.COLLECTION.ensureIndex({FIELD:1},{unique: true, dropDups: true})" command on a COLLECTION. After some factorization I want to change the field, which should be indexed and unique. Is it save to just run "db.COLLECTION.ensureIndex({NEW_FIELD:1},{unique: true, dropDups: true})" ?
[15:32:50] <jaitaiwan> lilgiant: depends on what you mean by safe
[15:33:14] <jaitaiwan> Im pretty sure you have to delete the index either way to run ensure index a second time
[15:37:13] <lilgiant> yes, "safe" is a littel bit fuzzy: do i have to run a command to remove the index and the unique attribute for the field FIELD or is it done implicit by running the new ensueIndex command
[15:39:14] <jaitaiwan> lilgiant: If its the same field you'd have to delete the index first but in this case you're creating an index on a new field. You should be able to create the index without deleting the existing one
[15:41:55] <lilgiant> thanks :)
[15:44:22] <lilgiant> one more question: does "mongodump --coolection MYCOLLECTION ..." also backup the index of the collection or do i have to created it after restoring a backup?
[15:46:53] <jaitaiwan> lilgiant: can't be entirely certain but I don't think it does: http://docs.mongodb.org/manual/core/backups/#backup-with-file-copies
[15:49:03] <lilgiant> ok, thanks again
[16:09:21] <Fanthomas90> hi!
[16:10:12] <Fanthomas90> Is it possible to let the mongodb hadoop connector store to mongodb with sharding enabled?
[16:13:26] <mazzy> if the mongodb server is running on an aws instance the bind ip address which should be?
[16:13:30] <mazzy> 0.0.0.0?
[16:13:54] <jaitaiwan> mazzy: that would bind it to all interfaces on the device
[16:14:08] <mazzy> I have used 127.0.0.1
[16:14:10] <jaitaiwan> If you wanted to bind it to a specific networking interface, use its ip address instead
[16:14:34] <jaitaiwan> 127.0.0.1 will make it availible only to processes on the server itself
[16:14:34] <mazzy> ok then I should use the ip of that interface
[16:14:49] <jaitaiwan> mazzy: yes
[16:14:59] <mazzy> thank you jaitaiwan
[16:15:10] <jaitaiwan> no worries :)
[16:23:30] <lilgiant> is there a way to check, if the index was created on all docs of a collection? i created a new index and i couldn't see any cpu or io impact on the mongo server.
[16:24:08] <jaitaiwan> The index isn't stored on a collection so much as stored seperately as far as I remember
[16:24:42] <jaitaiwan> If you can see the index in the listindexes then you should be right
[16:25:31] <lilgiant> yes, the index is shown on db.system.indexes.find()
[16:25:41] <jaitaiwan> great. That should be fine then
[16:26:21] <jaitaiwan> I only saw a massive difference in cpu when I did a multiple text index on a 50million strong collection
[16:30:11] <lilgiant> ok, i have currently around 30k docs in the collection. creating indices on a similar mysqldb does affect the performance of the server much more, so i was just wondering :)
[16:30:30] <jaitaiwan> :)
[16:34:59] <mazzy> jaitaiwan
[16:35:18] <mazzy> by assigning the private ip of the interface
[16:35:31] <mazzy> I get ERROR: listen(): bind() failed errno:99 Cannot assign requested address for socket: 172.31.1.17:2701
[16:35:48] <mazzy> 172.31.1.17:27017
[16:36:03] <mazzy> I'm running mongo in a docker container
[16:36:10] <jaitaiwan> what's the config for the listen param
[16:36:12] <mazzy> port are exposes correctly
[16:36:21] <mazzy> and since it is running on aws
[16:36:28] <mazzy> port is open in SG
[16:36:29] <jaitaiwan> Should be just the ip, port should be defined separately
[16:36:44] <mazzy> net:
[16:36:44] <mazzy> bindIp: 172.31.1.17
[16:36:44] <mazzy> port: 27017
[16:38:34] <jaitaiwan> whats the output of ifconfig?
[16:39:41] <mazzy> eth0: 172.31.1.17
[16:39:59] <mazzy> and docker0 172.17.42.1
[16:41:07] <jaitaiwan> http://stackoverflow.com/questions/20466250/mongodb-bind-ip-error-bind-failed-errno99-cannot-assign-requested-address-fo
[16:41:40] <jaitaiwan> looks like you have to use 0.0.0.0 for now and use a firewall to restrict what ipaddress it will recieve from
[16:42:34] <mazzy> yes
[16:42:38] <mazzy> it seems so
[16:50:15] <blizzow_> I amd trying to create a user mor monitoring/maintenance on my mongos instance that will have access to look at local oplog on my sharded replica set members. What db on my mongos instance do I need to add the user to and what roles do I need to add? I tried to add a maintenance user to the admin database with clusterAdmin and I get access errors about connecting to local.oplog.
[17:52:16] <bros> What does this error say to you? Error: MongoError: E11000 duplicate key error index: db.accounts.$users.email_1 dup key: { : null }
[17:52:26] <bros> I understand everything except the { : null } aspect.
[18:01:47] <preaction> you're trying to save a new document that doesn't have a value for "email"
[18:17:12] <bros> preaction, I have a collection called "account" with an embedded array of objects called "users" with then unique index on the email key of users.
[18:17:34] <bros> How do I create an account, then create a user?
[18:17:55] <preaction> you can't
[18:21:27] <bros> Thank you very much. I was not aware. My half-ass SQL background was screwing me more than it was assisting me.
[18:25:13] <preaction> if you flattened that schema into a relational table, it'd be the same result. you could have an accounts collection and a users collection, then they wouldn't be so dependent on each other
[18:45:56] <bros> preaction, I was told to embed them.
[18:46:01] <bros> I don't see the benefit.
[18:46:19] <preaction> yeah, i'm frequently told to do things that i don't do because it doesn't work or is otherwise stupid
[18:46:54] <preaction> tell them they can either have data integrity, or they can have inner documents. that error is a _good thing_
[18:47:15] <preaction> and just because you _can_ have inner documents, doesn't mean you must
[18:47:25] <bros> preaction, when you write schema for mongo, do you *ever* use inner documents?
[18:47:30] <preaction> yes, all the time
[18:47:51] <preaction> my latest project involves a unique index of an entire inner document
[18:49:07] <bros> I thought you just referred to them as possibly superfluous
[18:49:19] <bros> Does that last statement mean you embedded the entire document?
[18:49:36] <preaction> you were told to embed users into the account. which you tried. which doesn't work because of the unique index
[18:49:53] <preaction> which means you need to push back and say "i won't do this, because X"
[18:50:05] <preaction> and that kind of situation is what i was referring to
[18:50:29] <bros> Or I could just create the account with the users array initialized opposed to pushing the entry later (directly after)
[18:50:40] <preaction> yes, that is another option
[18:50:52] <preaction> and in that respect: how could an account exist without a user?
[18:51:11] <preaction> that is probably what i would do in this situation
[18:52:06] <bros> So, embedding isn't Satan?
[18:52:17] <bros> What are the benefits? I've searched and have yet to been able to explain it to myself.
[18:52:23] <bros> Less queries/joins?
[18:52:49] <preaction> if you would otherwise _always_ join a table in order to have a meaningful query, then embedding a document is good
[18:53:24] <preaction> if the information is tightly-bound and is not really meaningful without the parent document, embedding it is probably good
[18:55:08] <bros> The same could be achieved with arrays of reference ObjectIds
[18:55:22] <preaction> if you were previously serializing xml or json and embedding that in a column, embedding it is probably good
[18:55:32] <preaction> sure, but then you need to go look up that data in another place
[18:55:39] <preaction> you _can_ do lots of things
[19:04:33] <GothAlice> I often link http://www.javaworld.com/article/2088406/enterprise-java/how-to-screw-up-your-mongodb-schema-design.html as a general overview of schema design considerations. There's also http://effectif.com/mongodb/mongodb-schema-design, and two presentations: https://www.youtube.com/watch?v=PIWVFUtBV1Q (O'Reilly Webcast) and http://www.mongodb.com/presentations/schema-design-example-2 (Schema Design by Example)
[19:05:05] <GothAlice> Not to mention many more search results and many hideously designed blogs one could persue.
[19:05:37] <GothAlice> (From the first link: "Identifying documents is key.")
[19:10:44] <bros> preaction, This is a bit off topic. Would you bother to check if a database result is null after the user has authenticated their credentials through encrypted tokens?
[19:11:08] <preaction> huh?
[19:11:32] <bros> Are you familiar with JSON Web Tokens?
[19:13:06] <bros> How do I make a key a unique index only relative to the document that contains it?
[19:16:00] <preaction> make it a combined key with the parent document's id, is my guess
[19:19:49] <bros> preaction, I don't understand. :/
[19:29:58] <mordonez> Hi guys, if I create an index after some data was already added, is the data already added included in the index? if no, whta is the right way to handle that situation?
[19:30:09] <GothAlice> mordonez: Yes, it is.
[19:31:01] <GothAlice> If you build the index normally when the operation completes your data is indexed. If you tell it to index in the background it may take a _very_ long time before the index becomes usable, however again, all your data will have been included. (The former method will block use of the collection being indexed until it's done, the latter will not.)
[19:35:44] <mordonez> excellent
[19:35:56] <mordonez> what about changing the index?
[19:35:58] <mordonez> same process?
[19:36:10] <mordonez> for example, adding a new field to the index
[19:36:19] <GothAlice> Well, you can't really "change" them. You can remove an old one, and add it back with the modification, AFIK.
[19:36:35] <mordonez> so the process will be the same
[19:36:39] <GothAlice> Yup.
[19:36:40] <mordonez> thank you GothAlice
[20:18:28] <giowong> hi is anyone here familiar with using heroku mongolab?
[22:48:02] <bros> How do I select just _id?
[23:35:39] <cheeser> bros: db.collection.find({}, {_id:1})