Log file Viewer
#mongodb logs for Thursday the 25th of June, 2015
[00:02:14] <fxmulder> this is the first time this is being rebuilt on 3, prior rebuilds had been on 2.4
[01:28:16] <pyCasso> my understanding of mongo is limited so I need to know what are the gains from using mongodb oppose to an api call the returns json objects?
[01:29:23] <pyCasso> folks at work argue that it runs on a different server environment oppose to our native server environment
[01:50:24] <d-snp> cheeser: you just know that guy represents some fortune 500 company with a need for 50.000 mongodb instances right ;)
[03:29:39] <Pak> Hi channel, I am new to mongodb (and db'sin general) - I have a test project here http://quote.machinu.net
[03:31:01] <Pak> you can compare it with http://quote.machinu.net/1950 - you can see that this one is noticibly faster (only queries with .find({year: 1950}))
[09:11:56] <vassilux> I tried to repair but get error 12 , cannot allocate memory, any workaround ...
[12:08:19] <d-snp> 05:30 <@cheeser> "get random document" isn't something that mongodb is optimized for
[12:08:22] <d-snp> 05:30 <@cheeser> "get random document" isn't something that mongodb is optimized for
[12:09:02] <d-snp> isn't mongodb based on a key value store, so it'd be very fast at getting random documents?
[12:13:43] <talbott> [u'AutoReconnect: [Errno 110] Connection timed out', u' File "flask/app.py", line 1817, in wsgi_app', u' File "flask/app.py", line 1477, in full_dispatch_request', u' File "newrelic/hooks/framework_flask.py", line 98, in _nr_wrapper_Flask_handle_exception_', u' File "flask/app.py", line 1381, in handle_user_exception', u' File "flask/app.py", line 1475, in full_dispatch_request', u' File "flask/app.py", line 1461, in dispatch_request', u
[12:13:43] <talbott> ' File "newrelic/hooks/framework_flask.py", line 40, in _nr_wrapper_handler_', u' File "/location-service/app.py", line 49, in geocode', u' File "interactors/location_interactor.py", line 9, in geocode', u' File "services/location_caching_service.py", line 18, in get_location', u' File "mongoengine/queryset/base.py", line 309, in first', u' File "mongoengine/queryset/base.py", line 160, in __getitem__', u' File "pymongo/cursor.py", line
[12:13:44] <talbott> 595, in __getitem__', u' File "pymongo/cursor.py", line 1076, in next', u' File "pymongo/cursor.py", line 1020, in _refresh', u' File "pymongo/cursor.py", line 933, in __send_message', u' File "pymongo/mongo_client.py", line 1217, in _send_message_with_response']
[12:24:44] <deathanchor> in know in pymongo you can turn off the timeout, but you should ensure you close/exhaust your cusor.
[12:25:20] <deathanchor> talbott: doesn't matter, if you get a cursor, wait 10 minutes and try to read from it it will be dead.
[12:25:21] <talbott> so what is probably happening, is we are not being explicit about closing the cursor?
[12:26:12] <deathanchor> if you turn off the timeout, you need to ensure your program closes/exhausts the cursor before exiting/dying
[12:27:17] <deathanchor> perhaps you are reusing the same cursor? you can call something like cursor.exhaust() in pymongo to exhaust it. or cursor.close() to kill it off.
[12:28:04] <triplefoxxy> Hunting down an issue where all the RAM is allocated and system grinds to a halt. I see WARNING: /sys/kernel/mm/transparent_hugepage/enabled is 'always' could this be connected in anyway?
[13:13:16] <talbott> and, with mongoengine you dont really get low level access to cursor exhaust and cursor close
[13:34:39] <deathanchor> http://docs.mongoengine.org/en/latest/apireference.html#mongoengine.queryset.QuerySet.timeout
[14:01:14] <greyTEO> something similar to how it works for an insert. https://gist.github.com/dmregister/31bdfc3e903bbd7f3cfa
[14:02:04] <StephenLynx> some drivers, at leat the node.js one, has specific operations for finding one and finding multiple documents now.
[14:03:28] <StephenLynx> look at your driver documentation, I know I have used find and modify with upset at least once.
[14:05:44] <StephenLynx> from what I heard, the first developed driver was the java one. I wouldn't be surprised if its the most complete out there.
[14:06:35] <cheeser> it was. the c# driver i know is pretty complete and offers some features the java driver doesn't. (though i'm working on one of those gaps)
[14:07:34] <deathanchor> cheeser: yeah, I use those manually, but I was wondering if anyone wrote code for checking currentOp and killing it if it met certain conditions
[14:24:33] <fxmulder> so I had been running cleanupOrphaned on one of my replica sets, the server it was running on crashed and now when I try to run cleanupOrphaned again I'm getting "server is not part of a sharded cluster or the sharding metadata is not yet initialized."
[15:03:14] <hashpuppy> i need to take down one of my mongodb servers for maintenance. i have m1, m2, and arbiter. i'm taking down m1 (master). what's the best way to do this? just stop the instance?
[15:20:57] <synergy_> Are object ids unique over a whole database, or only in a document's collection?
[15:21:52] <cheeser> well, ObjectId is unique globally. but _id values are only required to be unique per collection
[15:23:34] <fxmulder> so what does "server is not part of a sharded cluster or the sharding metadata is not yet initialized." mean if this replica set has a sharded collection?
[15:31:04] <cheeser> sometimes those values are also unique globally but that's immaterial for a given collection's IDs.
[15:32:13] <deathanchor> synergy_: if you didn't explicitly set it, then you are safe that it is unique globally
[15:34:48] <deathanchor> synergy_: you can see what cheeser is talking about if you did a db.test.insert({ _id : "something" }) twice
[15:38:09] <fxmulder> so db.runCommand( {"shardingState": 1} ) on this replica set says that it is not a shard member while the other replica set still shows that it is a shard member
[15:38:36] <fxmulder> is it safe to enable sharding on the database again in the replica set that doesn't think it is a shard member?
[17:37:19] <leandroa> hi. what happens to existing data on a collection when I create a TTL index? I mean, with documents that match the expiration rule
[21:19:25] <akoustik> i'm trying to set up a replica set with members in multiple AWS regions. i'm trying to make sure all our traffic between regions is encrypted. i'm assuming it's necessary and sufficient to enable SSL and make sure each host has the keypair. is that about right?
[21:27:00] <GothAlice> akoustik: Within the datacenter I additionally use a private VLAN w/ IPsec. No actual firewall rules needed because literally only hosts on the authorized VLAN can communicate, and mongod only listens to that VLAN.
[21:28:53] <akoustik> GothAlice: cool. curious then, what benefit do you get from doing that in addition to just using restricted VPCs in AWS? just a little more confidence, or is it actually a stronger defense?
[21:29:30] <akoustik> (ie confidence in the case that amazon's security isn't as good as advertised, or something like that?)
[21:29:40] <GothAlice> I'm not technically on AWS, since the last time I was they ate my data despite cross-zone failures being against their SLA.
[21:30:25] <GothAlice> Notably their reliability. (It took me the 36 hours prior to getting my wisdom teeth extracted to reverse engineer the corrupted on-disk files… thanks EBS.)
[21:32:12] <akoustik> interesting. well, technically our data source is salesforce, so if/when AWS crashes/burns we at least don't actually lose data and have to worry about that while switching providers.
[21:32:53] <akoustik> main concern is just doing what we can to make transactions reasonably secure.
[21:35:16] <GothAlice> You say "transactions" in #mongodb, and I take a long quaff of my coffee. Using MongoDB for financial information?
[21:37:25] <akoustik> it doesn't automatically integrate with your mobile credit card reader? wtf? i thought this shit was In The Cloud
[21:45:24] <hahuang65> anyone know what safe: true in mongoid refers to? Is it write concern with majority? all? 1?"
[22:50:37] <rusty78> If I have a list of users, and I want to make sure no user can register a capitalization change like uSer1 instead of user1
[22:50:47] <rusty78> Anyone know a way to do this without actually grabbing all the users from my db?
[22:51:40] <GothAlice> rusty78: This is much harder than you might expect. Ref: https://en.wikipedia.org/wiki/Unicode_normalization and http://www.unicode.org/reports/tr15/
[22:52:08] <GothAlice> I.e. if you want to be case-insensitive, lowercase usernames anywhere they are entered and store them lower-case.
[22:53:13] <rusty78> I just do not want a user to be able to register the same name with only a capitalization change
[22:53:30] <GothAlice> Yes. That's called "case insensitivity". It is "insensitive" to capitolization.
[22:53:38] <GothAlice> If user1 and uSer1 are the "same", lower-case the input (uSer1 -> user1) and then test.
[22:53:58] <GothAlice> Anywhere a username is entered by a user, lower-case it before doing anything else to it. Problem solved.
[22:54:47] <GothAlice> Then you will need to store both, but the mixed-case one is for display purposes only. You would continue to query the lower-case version.
[22:54:49] <rusty78> if someone registers uSER, I want to be able to return uSER as their name - I also want to check if the name is unique
[22:57:05] <GothAlice> rusty78: Also, pleasepleaseplease restrict your usernames to the following regular expression: [a-z_][a-zA-Z0-9-_.]+
[22:58:32] <GothAlice> Depending on your normalization, that can either be the single letter "e with grave accent", or it can be two: "grave accent" followed by an ASCII "e".
[23:37:38] <Kharahki> How would you add default data to a collection when it is created? I'm using mongoose and I have a json file that I want to 'import' into the collection when its first created. Is there a good way of doing this?