Log file Viewer
#mongodb logs for Monday the 20th of October, 2014
[04:19:59] <unpro> would findOne({'md5': '12345678901234567890123456789012'} where md5 is a text indexed field be reasonable, and always return a complete text match?
[04:20:38] <unpro> because ive got like 8million documents, and i'm expereincing some weird behavior
[04:23:27] <unpro> the weird behavior i'm experiencing is i'll try to find the same document using the same code, and it will either hit or miss
[04:27:37] <unpro> hm i don't even know that it is a text index, i did self.db.file.ensure_index('md5', unique=True, dropDups=True)
[04:37:34] <unpro> becuase i swear, i have assert() in place that verifies that the find_one for the field/value returns None, and then hits when I do it a 2nd time >_>
[05:12:33] <mehwork> when i try to create a sparse index, nothing seems to happen. Do you have to create the index manually first, then specify that it's sparse later?
[05:22:41] <mehwork> "Some options that you can specify to ensureIndex() options control the properties of the index, which are not index creation options. For example, the unique option affects the behavior of the index after creation."
[05:23:16] <mehwork> doesn't that mean i can't create the index with {unique: true} at the same time i create the index?
[05:25:47] <joannac> mehwork: http://api.mongodb.org/python/current/api/pymongo/collection.html#pymongo.collection.Collection.ensure_index
[08:17:24] <Soothsayer> I am creating a Collection which stores all URLs (for URL rewriting).. I was thinking of having the _id field in the Document schema as an md5 of the URL
[08:35:43] <glontu> is there q way to enforce order among subdocuments ? i would like them to stay sorted by 2 fileds
[08:57:50] <glontu> Soothsayer, i need to recheck the collection of subdocuments when i add or remove one of them. When i add one, if the new document is "better" than all others i need to make the previous best as not best anymore
[08:58:23] <glontu> when i remove a subdoc, i need to decide again if the "best" had changed and redo the subdocs properties
[09:00:07] <glontu> Soothsayer, i am in symfony so i figure that if i manage the subdocs in the owning docs add... and remove... methods it will get flushed anyway right after that
[09:01:32] <Soothsayer> glontu: you will not be able to achieve this if you do your adds / removes via the ODM
[09:01:57] <Soothsayer> to each of your sub-documents, add an ‘id’ field which is some kind of id/hash which needs to only be unique among the sub-documents of that document.
[09:02:33] <Soothsayer> next, when you query the Document.. you can loop through the sub-documents in symfony and check what’s the next best sub-document
[09:03:47] <Soothsayer> and then remove whatever you want to remove using the $pull command with the ‘id’ field of the sub-document
[09:04:09] <Soothsayer> lastly, i would suggest storing the ‘id’ to the isBest sub-document in the main Document somewhere
[09:04:33] <glontu> Soothsayer, but i don't want to remove them. i just want to update the previos "best" as not best anymore and add the new one as best.
[09:05:51] <Soothsayer> glontu: hint, use Symfony2’s Symfony\Component\Security\Core\Util\SecureRandom
[09:08:10] <glontu> one more thing. since i give them the id can the best document be mapped as an embedsone and reference the same subdocument from the collection ?
[09:09:20] <Soothsayer> glontu: there is no provision in MongoDB or Doctrine ODM to reference embedded sub-documents
[09:30:17] <af__> guys in oplog, the timestamp is saved as "ts" : Timestamp(1395663575, 1),.... I believe '1395663575' is in unix time.. but what is '1'... ? and also how do I access it in Pymongo?
[09:53:39] <_R_B_R> Hi all, I started looking at mongodb yesterday and it seems really good. Only trouble is, I want to be able to retrieve a single record from a sorted table in a fraction of a second. This is a (possibly terrible) code I've written to do this: http://pastebin.com/Rh8JMbPi
[09:56:41] <_R_B_R> The reason I'm doing this is because I want the user to be able to hover their mouse over a horizontal bar representing the whole dataset and for a tooltip to display what the value is at that position.
[09:57:32] <_R_B_R> At the beginning of the dataset queries are fast, but once I get past a few hundred thousand I begin to get some lag
[12:59:34] <skot> Yes, query using an indexed field. You should not use skip to get to the nth item. Do you know anything about the document you are looking for, like its _id value?
[13:00:57] <skot> You are trying to solve pagination by using skip/limit which will not perform well.
[13:03:17] <skot> using skip requires "skipping" all the previous value which is going to cause way more work than you need, hence the suggestion to use range based pagination, by using a query instead of skip to get where you want to go
[13:04:10] <skot> Go research how to do pagination without using skip/limit. There are lots of solutions/suggestions and they are for more than just mongodb.
[13:05:43] <skot> here is another mongodb SO example: http://stackoverflow.com/questions/20960815/range-query-for-mongodb-pagination
[13:43:55] <mskalick> could I have a question? - are test in mongodb 2.6.x functional and maintained?... thanks
[13:48:46] <cheeser> as in, does the server have tests that are maintained and run as part of the build?
[13:58:27] <mskalick> I've built MongoDB from source files and run "python buildscripts/smoke.py --mongod `which mongod` --mongo `which mongo` --dont-start-mongod --port 27017 --mode suite all" from that directory
[14:07:17] <dump> mskalick: just use the binary packages. You don't need to run tests, unless you are MongoDB core developer
[14:09:28] <mskalick> dump: I am packaging MongoDB and I would like to have "%check" section in rpm...
[14:19:01] <dump> mskalick: Have you checked the RPMs packages at http://docs.mongodb.org/manual/tutorial/install-mongodb-on-red-hat-centos-or-fedora-linux
[14:46:03] <mskalick> dimon222-cloud: because I have already running one mongod instance (but I've tried it also without it...)
[14:58:55] <dggb> I have a real time service that logs 40 items from a remote sensor node every minute. Could anyone give me some suggestions on document design>
[15:05:04] <kali> dggb: http://blog.mongodb.org/post/65517193370/schema-design-for-time-series-data-in-mongodb
[15:05:35] <dggb> kali: Thank you! I actually found that right after asking. How would I scale that document from 1 value per minute to 50
[17:06:01] <globaly> hello dears, how is it possible to return the number of distinct values in the command db.collection1.runCommand( { distinct: 'collection1', key: 'title'});
[17:27:37] <dimon222-cloud> is it possible to select just single parameter? not select whole document and then take parameter from it
[17:28:54] <dimon222-cloud> so find({"$and":[{"stype":stype}, {"snum":snum}]} and only receive something like sdetails back
[17:30:36] <dimon222-cloud> i want just get values without retrieving complete documents (dem, they're huge)
[17:31:06] <cheeser> dimon222-cloud: http://docs.mongodb.org/manual/reference/method/db.collection.find/
[18:01:58] <doug1> ceejh: Oh? Ever get "Database command 'listDatabases' failed: not authorized on admin to execute command { listDatabases: 1 }" ?
[18:02:19] <doug1> I don't know why I'm getting that. It's on a node before the router configures the admin user
[18:07:05] <doug1> Looking at cookbook code, I see it's trying to connect as the admin user. If I do a 'mongo -u admin' without a password at the prompt, it fails.. says it requires a password, but none has been set up yet so I assume the cookbook would fail for the same reason
[18:08:24] <ceejh> I just need an idea of your setup. Mongod, mongos, config sirs, or just a standalone node?
[18:09:05] <doug1> ceejh: full on setup. I've brought up 3 config servers, and this error happens on one of the data nodes. There's three of them. Haven't created the router yet
[18:09:31] <doug1> ceejh: Will eventually be 3 config servers, 2 shards of 3 nodes each and some number of routers
[18:09:57] <ceejh> so the config servers come up fine, then you start bringing up the mongod servers and it fails
[18:10:00] <doug1> The router, which creates the password for the admin user hasn't been created yet, it gets done last
[18:10:54] <ceejh> And I assume you're using the user management recipe on purpose because you want to initialize the admin
[18:11:01] <doug1> ceejh: from a month or two ago yes, but I had to fork it locally and make changes because there's bugs in the replica set naming. grrr
[18:11:57] <doug1> ceejh: for now... we have to put the router on a rightscale box (legacy) so there was scripts to create the admin user etc
[18:12:36] <ceejh> O_o your mongo instance must be running with auth=true to say it requires authentication
[18:14:41] <doug1> i'm confused tho... if I set the key file and the cookbook auto sets auth, what happens initially when the nodes are being created, before the admin user password has been set?
[18:15:16] <ceejh> Just to answer that generally, you won't be able to do any command except db.createUser
[18:15:40] <doug1> ceejh: no listdatabases? That's obviouslty what's being called by the cookbook (although a grep doesn't find it )
[18:18:54] <doug1> The docs have "node['mongodb']['admin']" The admin user with userAdmin privileges that allows user management .... but nothing about how to assign privs to it
[18:19:20] <ceejh> It sounds like you don't have that recipe, right, and trying to change things over to it might be more trouble than it's worth
[18:19:39] <ceejh> (Because if you pull straight from berkshelf, you get the last tagged version, you'd have to pull straight from git to get latest)
[18:20:19] <doug1> I just did a git pull a month or two ago. i had to fork because it straight up doesn't work with the way the replicaset names are done
[18:21:39] <doug1> very careful to remove old nodes from chef before each deploy, otherwise the chef search gets all outta wack
[18:25:00] <doug1> ceejh: last change to that file was 4 months ago. That's def earlier than my last pull where it still had it. wish I could remember what I changed
[18:27:01] <ceejh> anyways, here's a vagrant example of 3 cfg svr, 3 mongod, 1 mongos using chef-solo
[18:28:55] <ceejh> It's kind of hard to give you a tutorial of the user management recipe... atm it doesn't work for replica / sharded instances
[18:30:13] <ceejh> "Before using on a new database, ensure you're overwriting the node['mongodb']['admin']['username'] and node['mongodb']['admin']['password'] to something besides their default values."
[18:32:06] <ceejh> Haha, well it won't on master, but like I said, my branch works for me in chef-solo
[18:33:32] <ceejh> I wasn't able to get a cluster of mongo working at the time, so getting it working with multiple nodes came later
[18:34:21] <doug1> people expect me to get this to work, so I go to the cookbook and there's nothing there about all the issues. I start playing with it, it works, confidence builds and then for some reason it stops working.
[18:34:57] <doug1> which then makes me look like an idiot because people assume i either never had it working, or... i dunno, i'm just an idiot or something
[18:41:31] <arussel> I've got a collections with : {a: Date(), b: Int} is there a way to find all the document where a + b days > now ?
[18:42:14] <hannya> Whats the difference between: { type: ObjectId, ref: 'Comment' } and commentSchema?
[19:31:10] <Forest> Hello,i am getting issue Can't insert document: WSARecv tcp 127.0.0.1:28406: An existing connection was forcibly closed by th e remote host. What can be the problem?
[20:06:55] <Neal_> I have two collections: User and Subscription. I want each Subscription to "link" to a User somehow. I am very new to mongo and just want the best way of doing this.
[20:09:08] <Neal_> In the Subscription schema, can I have something like "user: { type: ObjectID }" and whenever I create a Subscription, I make that ObjectID a users' _id?
[20:23:31] <eucalyptus> well, im interested in configuring the underlying java/scala driver. i dont see a way to do that via the Hadoop configuration
[22:02:35] <DrakeL> Hi everyone! I've been using MongoDB for a while and it's great. The only problem I seem to have is... I have no clue how authentication works.
[22:03:06] <DrakeL> I'm trying to have an admin (superuser) access for myself to all databases, and one user to access one database read/write.
[22:05:59] <DrakeL> Boomtime: I tried that guide but the db.createUser for siteUserAdmin is not following some new schema for something?
[22:08:42] <DrakeL> Use Robomongo when server is in no auth mode and create a user. I can't auth with the details I make it with.
[22:09:45] <Boomtime> nowhere in any of the guides is robomongo mentioned, please follow the guide, you'll have better luck
[22:10:54] <DrakeL> I have, but when I reached the part of the guide that didn't work (not updated to latest schema), then I tried alternatives.
[22:14:33] <Boomtime> yeah, it sucks when you follow the docs and it works, leaves you having to blame the docs for being correct
[22:16:43] <doug1> I've been looking at how to create a cluster for an hour and I can't find it. when you can't create a cluster via a GUI, I'd call that a fail
[22:18:36] <doug1> Boomtime: Nothing. I was trying to deploy the cluster with chef. MMS is an act of desperation.
[22:20:06] <Boomtime> if you guys would actually describe the problem you might get some help, but right now you're just whining
[22:20:43] <doug1> Boomtime: Or, where are the docs that tell me how to create a cluster in the MMS GUI?
[22:21:41] <DrakeL> Sorry. Err. I'm able to create a superuser account with unrestricted access, but I cannot log into the account with the mongo command or in a visual thing like Robomongo.
[22:25:16] <DrakeL> I was running a semi-production mongodb instance with noauth and connections allowed from anywhere. I figured that I should probably get some authentication.
[22:28:44] <doug1> Boomtime: Any idea on a solution to my quantified problem description that doesn't qualify as whining?
[22:37:57] <Boomtime> also, this user is a user-admin which will let you create any type of user, but little else
[22:45:12] <Boomtime> while auth is off, helpers like db.dropUser might not work properly, you can just delete the document from system.users though
[22:54:07] <Boomtime> when auth is enabled you should use the helpers: http://docs.mongodb.org/manual/reference/method/js-user-management/
[22:56:17] <Boomtime> that sounds like robomongo trying to do something tricky, sorry not sure what is going on
[23:03:20] <Boomtime> a shard isn't a shard until you do sh.addShard - before that it's just a regular standalone instance
[23:03:38] <doug1> this is so frustrating, the chef cookbook tries to authenticate as the admin user but fails. This was working previoudly.
[23:05:17] <doug1> this cookbook however is still trying to create the replicaset. that's where I'm failing
[23:12:42] <doug1> eg: if the docs said the correct order to start instances in a cluster, and what instance to run the useradd command on, it would help in debugging the cookbook
[23:16:14] <Boomtime> doug1: this document tells you quite explicitly trhe order of servers: docs.mongodb.org/manual/tutorial/deploy-shard-cluster
[23:16:37] <Boomtime> this document tells you quite explicitly how to enable auth in a sharded cluster: http://docs.mongodb.org/manual/tutorial/enable-authentication-in-sharded-cluster/
[23:17:53] <DrakeL> Boomtime: I tried adding a user with this form: https://gist.github.com/ihatecsv/18908ec6fa0307fb9809 and it executes fine but when I'm in the Laxatus database and do db.system.users.find() it returns nothing
[23:22:02] <Boomtime> you should be using the helpers when auth is enabled, but (fyi) all users are stored in admin
[23:22:26] <DrakeL> But I'm trying to connect to the database in PHP to query it and it's saying I don't have permission to query
[23:28:37] <doug1> Boomtime: so, it seems the order is in fact cfg, mongos then replicasets. That means I have to configure the mongos twice. Darn, ok
[23:30:30] <Boomtime> not sure what is going on there, i would play around with roles and see what it takes to get php driver to be happy
[23:31:04] <Boomtime> for example, it might actually be asking for the list of collections on the database - not sure if that is covered by readWrite