Log file Viewer
#mongodb logs for Wednesday the 10th of June, 2015
[00:04:02] <joannac> background index builds take longer but are non blocking. How large is the database and index?
[00:04:22] <joannac> did you collect io statistics during the index build to see what your disks were doing?
[00:08:12] <Boomtime> do a db.stats() on the database you restored, this will provide the details of index size
[00:11:44] <Boomtime> it isn't mongodb's fault that the design generates a huge amount of indexes, those are choices made entirely by you (or your developers)
[00:13:34] <Boomtime> you could raise a bug report with nodebb and ask them to explain why they need an index ratio of 1:3 (which is absurd)
[00:14:12] <brotatochip> it's a forum, the collection that index was building for is one for searching posts
[00:36:32] <brotatochip> the only time it matters Boomtime is if there is a disaster and I have to restore using mongodump (which is less likely as I have a secondary)
[00:37:14] <brotatochip> a hidden slave for backups is still a good idea but restoration is going to never be fast as far as I see it
[00:38:03] <ianseyer> hi all. could somebody help me construct an aggregation pipeline? have been struggling to no avail
[00:38:53] <ianseyer> need to group all my documents by a field called 'code', and then find all those documents matching that code that have distinct titles
[00:44:05] <ianseyer> well, it's definitely worth mentioning this is my first serious exploration of mongo
[00:44:39] <ianseyer> but, here's my pipeline: [{'$group':{'_id':'$code'}}, {'$project':['title', 'code']}]
[00:44:41] <brotatochip> Boomtime: any idea as to how I can find out why my secondary keeps getting veto'd out of an initial sync?
[00:44:59] <ianseyer> grouping my code, then trying to get the title for each resulting document to see if any code groupings had multiple titles?
[00:45:28] <joannac> ianseyer: erm, get rid of the second bit (the $project) and just print out what you have
[00:48:26] <brotatochip> oh, nevermind, I just found out why in the log 2015-06-10T00:46:14.077+0000 [rsBackgroundSync] replSet error RS102 too stale to catch up
[00:49:42] <ianseyer> i figured it would be that, but would combine all those documents with matching codes into some sort of superdocument...
[00:50:31] <tejasmanohar> should i host mongodb on a separate ec2 instance than my node.js server for production usage?
[00:52:05] <tejasmanohar> cheeser: would it be wise to use an ACL in front of the EC2 instances then so only certain IPs can access?
[00:53:11] <brotatochip> tejasmanohar why would you ever not only permit IP access to a server unless it is explicitly needed?
[00:53:20] <tejasmanohar> is latency a big deal when im hosting them both on us east servers-- should be AWS so im guessing not?
[00:53:29] <tejasmanohar> brotatochip: i guess my qeustion is - is an AWS ACL the best way to do that?
[00:53:43] <brotatochip> latency is no issue between two servers in the same region tejasmanohar as far as I have seen
[00:56:58] <tejasmanohar> are there any other dedicated mongodb hosts in the AWS US East region that I should give strong consideration?
[00:57:25] <brotatochip> I don't know, I've been hosting it myself in AWS, but mongolab looks like it might be something I could really benefit from
[00:59:24] <ianseyer> joannac: oh hey! i think i got it. [{'$group':{'_id':'$code', 'titles':{'$addToSet': "$title"}}}]
[01:00:20] <tejasmanohar> if mongolab goes down i wont be the one diagnosing the issue in the server
[01:02:09] <tejasmanohar> Just because it was easy on Heroku, and it works for our internal URL shortener that doesnt need that much space, 500mb is >>>>> than enough
[01:04:13] <GothAlice> https://mongolab.com/plans/pricing/#dedicated-cluster-plans < high storage M6.
[01:05:27] <tejasmanohar> seems like MongoLab is the industry leader for managed mongo tho, right? GothAlice
[01:05:40] <GothAlice> Yeah. For me, a cluster of HS-M5s from mongolab would cost me… $76K/month, going with the less-RAM one. $106K/mo. for the more RAM one.
[01:07:17] <tejasmanohar> I've heard a lot of things about MongoDB going to be terrible if my app goes to scale and uses multiple nodes due to its pooor distributed consistency and especially because im charging users for things and then saving it in the db so if it doesnt save thats not good
[01:07:34] <tejasmanohar> Is this just people are not configuring the DB properly and out-of-the-box that's a common situation faced?
[01:08:38] <GothAlice> https://blog.serverdensity.com/does-everyone-hate-mongodb/ covers most of the naysayers.
[01:09:34] <GothAlice> The default settings used to be less… tolerant of ignorance… than they are today.
[01:10:24] <tejasmanohar> But w/ a hosted MongoDB platform like MongoLab, I should see less of that? GothAlice
[01:10:56] <tejasmanohar> Is there just a top notch book or somethign I can read about configuring, scaling, etc. MongoDB?
[01:11:00] <GothAlice> You'll still need to be aware of things like write concern and read preference.
[01:11:51] <tejasmanohar> So later, I can do Replica Acknowledged I guess if I have a muti-enode setup ? GothAlice
[01:12:36] <tejasmanohar> At the point that I put Replica Acknowledged, is there really the high speed of Mongo compared to other dbs?
[01:13:45] <GothAlice> tejasmanohar: https://www.mongodb.com/blog/post/high-performance-benchmarking-mongodb-and-nosql-systems
[01:14:57] <tejasmanohar> We then tested a configuration that prevents any possible data loss. In this configuration, MongoDB outperforms Cassandra and Couchbase by more than 25x, with latency that is more than 95% better than Cassandra, and more than 99.5% better than Couchbase.
[01:15:09] <tejasmanohar> not something i have to worry about right now much because i have no one using my app
[01:15:19] <tejasmanohar> but its just something i want to think about like do i need to be considering switching etc
[01:15:43] <GothAlice> Understanding how to construct your sharding keys for maximum benefit is something you'll need to worry about later, too.
[01:15:50] <tejasmanohar> GothAlice: for any financial application, would you recommend pushing up to a higher write concern than the default Acknowledged like Journaled?
[01:16:09] <tejasmanohar> With a journaled write concern, the MongoDB acknowledges the write operation only after committing the data to the journal. This write concern ensures that MongoDB can recover the data following a shutdown or power interruption.
[01:16:32] <GothAlice> For financial information I recommend using a transactional database (TokuMX, a fork of MongoDB, or any actual transactional DB), or figuring out two-phase commits and rollback strategies.
[01:18:11] <tejasmanohar> Shoot, is TokuMX the same interface tho? Like if i have a node mongoose express app can i still use it in the same way as mongo?
[01:20:18] <GothAlice> Any system where users can potentially exploit race conditions for direct or indirect financial gain.
[01:22:10] <GothAlice> You want to know that if the user hits "Stop" in their browser part-way through a deduction from one account and addition to another, that either both operations succeed or both operations fail, also.
[01:22:38] <GothAlice> If not, either money disappears from existence, or money can be spontaneously created.
[01:24:51] <cheeser> in practice, financial transactions are never as simple as a single transaction "take from one give it to another" action
[01:25:09] <GothAlice> In TokuMX you can perform a bulk operation set that will either all succeed, or all fail, regardless of the client connection failing or even the server being written to disappearing. On startup, the incomplete transaction will be rolled back safely.
[01:25:56] <tejasmanohar> So you're saying I can connect to TokuMX server the same way I can connect to mongo servers etc?
[01:27:34] <tejasmanohar> Alright, I'll have to give it a shot I think... so how do things just magically get better and more ACID compliant without changing the way i interact with the DB?
[01:28:47] <cheeser> toku's txns, e.g., only apply to a single shard. mongo's philosophy to date has been to do nothing that can't be applied equally well in a sharded sitation
[01:28:57] <GothAlice> TokuMX uses a very different behind-the-scenes structure that allows for point in time everything.
[01:30:03] <GothAlice> (Those needs being über performance, compression at the time was unique to TokuMX, and the aforementioned point-in-time everything that facilitates true ACID transactions.)
[01:30:14] <StephenLynx> toku is a very advanced tool that one shouldn't adopt without studying it a lot.
[01:31:41] <StephenLynx> they don't seem to be very concerned about honesty from what I felt from their website.
[01:32:54] <StephenLynx> yeah, they just put it like they did magic and its perfect and you should TOTALLY use it.
[01:33:52] <tejasmanohar> Does someone have a performance stats of Journaled vs Acknowledged write concern speeds?
[01:36:00] <tejasmanohar> Is Write Concern something set at the driver level when you write to the DB or in the DB's configuration itself? Looks like a driver level but checking since I don't see it written anywhere in the docs (prob just missing something)
[01:37:50] <StephenLynx> I wouldn't be surprised if they went with 0 at launch for marketing reasons.
[01:40:37] <tejasmanohar> https://whyjava.wordpress.com/2011/12/08/how-mongodb-different-write-concern-values-affect-performance-on-a-single-node/
[01:42:03] <cheeser> https://www.percona.com/blog/2015/04/14/tokutek-now-part-of-the-percona-family/
[01:42:37] <tejasmanohar> if i screw up and cause downtime that could potentially lose money too lol
[01:42:39] <GothAlice> Ah, tejasmanohar: http://edgystuff.tumblr.com/post/93523827905/how-to-implement-robust-and-scalable-transactions
[01:43:50] <GothAlice> Depending on how you structure your updates (i.e. using two-phase, synchronization, stacks, queues, etc.) they can be highly resistant to partial writes.
[01:44:19] <cheeser> tejasmanohar: eventual only applies to the secondaries. so long as you do primary reads, you're fine.
[01:44:53] <GothAlice> Only for queries where getting stale data is A-OK (i.e. user profile data like profile picture, etc.) do you direct them at secondaries.
[01:45:20] <cheeser> you need to be wary of rollbacks during primary elections but those are relatively rare and you can mitigate by using a higher write concern
[01:48:43] <GothAlice> For things like large-scale analytics, the laggiest data we've had back was 15 seconds old, and that's perfectly acceptable for fifteen-minute to one-hour granularity graph data. The data being closer to the people querying it was more important. :)
[01:49:02] <GothAlice> Hard to beat free. There's also a bunch of courses available on mongodb.com, I believe.
[01:50:26] <GothAlice> I also find the whitepapers and whatnot that are on mongodb.com (but not .org for some reason) to be quite useful in gauging capabilities and approaches to problems.
[01:51:20] <cheeser> so .org is more about code, contributions, and open source woowoo! where .com is more for business interests
[01:52:18] <cheeser> i'm not speaking philosophically. that's exactly how those two sites break down.
[02:09:00] <GothAlice> Not sure of the applicability of these results: http://techidiocy.com/write-concern-mongodb-performance-comparison/
[02:09:14] <cheeser> all writes are journaled. the journaled write concern waits for that to happen vs acknowledged is just the server ack'ing the write back to the driver
[02:11:48] <cheeser> "This is true because you can only really get an idea of performance when you’re testing your own queries on your own hardware. Raw figures can seem impressive but they’re not representative of how your own application is likely to perform."
[02:13:15] <tejasmanohar> "all writes are journaled. the journaled write concern waits for that to happen vs acknowledged is just the server ack'ing the write back to the driver"
[02:17:43] <Doyle> Hey. Does this look about right for a geo distributed replication-set with sharding? https://drive.google.com/file/d/0B5g2nsz5NekdSnB1RVoyeC1FM1k/view?usp=sharing
[03:47:04] <sabrehagen> given that objectids store a timestamp in them, is there a query to check if any documents in a collection are newer than a given document by comparing ids?
[03:53:25] <sabrehagen> arussel: thanks, was not immediately obvious when googling. will try this now.
[04:11:57] <tejasmanohar> 557668d932d0f724c8af263a Unhandled rejection Error: Invalid val: {"_bsontype"=>"ObjectID", "id"=>"UvjÒ¬\u0092\f B\u0082\u009D8"} must be a string under 500 characters
[09:58:05] <roelof> sorry: I mean this problem : https://groups.google.com/forum/#!topic/mongodb-user/FnxKJnFJFik
[10:07:19] <roelof> Can someone help me with this problem : https://groups.google.com/forum/#!topic/mongodb-user/FnxKJnFJFik
[10:24:43] <pagios> i am willing to run a db on my rpi, rpi can crash at anytime (by removing the power for example) is mongodb a good db to use in this case?
[11:23:58] <mehdy314> I have a python script that insert docs of a mongo collection into another db using pymongo. the mongo collection has 306099 docs after running the script without any error just 33426 of docs has been inserted. i use find method by the nway
[12:11:05] <Doyle> When running a distributed mongodb setup, is it recommended to place config servers and query routers at different geographical locations?
[12:14:07] <Doyle> Say you have US-EAST and US-WEST with a sharded replication set. You place the shards in EAST (Primary/Secondary/Arb), and another secondary for each shard (priority 0) in West.
[12:14:31] <Doyle> In this setup you have all the config servers and query routers in East. No problem.
[12:16:38] <Doyle> Can you place a new shard in West (Pri/Sec/Arb) and add it to the configuration so the config servers track all the meta data for it as well? Is there anything to be concerned with when spanning config servers and query routers across different sites?
[13:05:53] <leev> i have a cluster with a primary, secondary and arbiter. when I shutdown the secondary, the primary drops back to secondary. shouldn't it stay primary as it's also connected to the arbiter?
[13:09:20] <leev> Wed Jun 10 13:06:23.362 [rsMgr] can't see a majority of the set, relinquishing primary
[13:18:33] <leev> i'm guessing this has something to do with the upgrade i'm doing at the moment ...
[13:19:03] <leev> following the guide. i was on 2.4. all mongos are now 2.6, config servers are 2.6, now trying to do the mongod's.
[13:21:13] <leev> Wed Jun 10 13:06:26.426 [ReplicaSetMonitorWatcher] Socket say send() errno:9 Bad file descriptor 10.192.8.39:10002
[13:24:26] <leev> i also get "[LockPinger] Socket say send() errno:9 Bad file descriptor" to all config servers
[13:26:31] <nfo> Hi. About the reuse of disk space freed by deleted documents, does setting a TTL on documents is more efficient than removing the docs with the `delete` command ? http://docs.mongodb.org/manual/tutorial/expire-data/ As far as I know, with mmapv1, one has to compact() or repairDatabase() or resync the collection/DB to be able to reuse disk space freed by deleted documents, is it true ?
[13:27:21] <nfo> ho it's more or less documented here: http://docs.mongodb.org/manual/faq/storage/#faq-empty-records
[14:00:55] <rickardo1> I am new to mongodb and need it only to handle a large amounts of products ~1gb they are listed under root with key "Data" : [....] Is there any way I can import them directly from cmd line?
[14:02:18] <StephenLynx> I have never used mongoimport, but I think it might be able some common standards.
[14:02:48] <StephenLynx> if you have any change of automatically importing it, it will be with mongoimpor.
[14:12:21] <rickardo1> StephenLynx: mongoimport -d test -c Data data/products.json it goes to 100% but then a memory error.. have 8 gb on this machine.. :/
[14:16:31] <Doyle> Hey. When spanning multiple sites with a replication set, are the config and query servers distributed as well?
[14:16:39] <tubbo> any mongoid users here? wondering how i can solve a timeout issue with an aggregation i have. https://gist.github.com/tubbo/aecf639c8e683b43777c
[14:17:10] <saml> db.adminCommand('listDatabases').databases.map(function(x){return x.name;}).filter(function(x){return /^test/.exec(x);}).forEach(function(x){db.getSiblingDB(x).dropDatabase();})
[14:41:06] <StephenLynx> but then you would have to split your json in multiple parts to make that easier.
[15:19:32] <yauh> I have a collection with docs that contain highscores for a game. I want to show the top 10 plus the current player
[15:20:10] <yauh> I only want to find 10+1 documents, but the current player's position in the score list (you are on position #27153) should be listed as well
[15:22:14] <yauh> pre-aggregated seems like an inefficient approach because almost all documents must be changed if some of the first positions change
[15:25:46] <StephenLynx> a quick search revealed that mongo can't give you the position of a document.
[15:26:23] <StephenLynx> http://stackoverflow.com/questions/10813908/get-position-of-selected-document-in-collection-mongodb
[15:26:41] <StephenLynx> so you can get the amount of documents with a score higher than your player.
[15:53:57] <Lonesoldier728> is there something wrong with this query using mongoose http://pastebin.com/2cZ36mP1
[15:59:05] <diegoaguilar> "Failed: error scanning filesystem: error reading root dump folder: open dump: no such file or directory"
[15:59:40] <tubbo> StephenLynx: i suppose you could use it however you want, but it's definitely intended for use in a rails app. follows ActiveRecord's lead on a lot of things.
[16:00:15] <StephenLynx> that its a three out of three of stuff I wouldn't touch with a mile long pole.
[16:01:03] <StephenLynx> lone, that looks plain wrong from everything I know from the standard driver.
[16:07:11] <Lonesoldier728> well there is my code and question http://stackoverflow.com/questions/30761620/mongoose-populate-not-returning-results
[16:35:49] <jecran> Hello guys. I am using mongodb with node. https://gist.github.com/anonymous/d1920d655b0c1655e03f simple code here, I cannot figure out how to use the findOne() method instead of find(). Please help!
[16:37:04] <jecran> var dbs = db.collection('users').findOne(query); i tried this and similar and keep getting errors
[16:38:48] <jecran> https://gist.github.com/anonymous/d1920d655b0c1655e03f the sample code. Just want to apply findOne() instead of find() lol.... 'object is not a function' is the errors that I get
[16:40:02] <svm_invictvs> In Morphia, if I have an @Reference annotated Map<String, Foo> does that basically make it a mapping of strings to object ids?
[16:40:45] <svm_invictvs> And second question, does morphia cascade the put operation when persisting the object?
[16:41:26] <jecran> var dbs = db.collection('users').findOne(query, {}, function(err, doc){}) .... this produces null value without any errors
[16:43:45] <jecran> StephenLynx: query = {name:data.name}; the sample here works fine and displays the data correctly, but I want just the 1 piece of data, not all.
[16:47:12] <jecran> StephenLynx: any change at all in the code to try to make it findOne() results in an error, except for: var dbs = db.collection('users').findOne(query, {}, function(err, doc){}) Which results in null. If I exclude the middle param {}, I get actual errors
[16:50:10] <jecran> StephenLynx: https://gist.github.com/anonymous/b52d0d47e7ad1705d82b I put the error in a comment next to the line
[16:59:45] <jecran> StephenLynx: latest attempts..... no crashing, but still null. with or without using a cursor... https://gist.github.com/anonymous/79af1207cd4a19243355
[17:00:51] <StephenLynx> you just check if it doesn't exist and close the connection pool if it doesnt.
[17:01:35] <StephenLynx> you never do anything with the returned value or output any message based on its existence.
[17:04:09] <jecran> StephenLynx: Yes. its there, and displays when I search through the whole collection
[17:05:43] <jecran> StephenLynx: I don't know , but I just erased it all and started over, and it worked first try.... what the hey lol .... thanx for your input
[17:08:42] <StephenLynx> and when you assert the average skill of developers, you will want anything but the work of more people in your project.
[17:09:20] <cheeser> on the other hand, if your teammates are so crappy, having them write less code by using libraries is a net win.
[17:09:41] <jecran> Agreed. But some of the stuff I do i would be lost without some api's.... I have only been developing for 4months now lol
[17:12:24] <jecran> StephenLynx: turn's out that all the reading I have done in my spare time has paid off. Learning concepts, not code, like inheritance and encapsulation etc
[17:15:57] <jecran> agreed. And I still love it. And honestly, I learned regular java, and am more than happy to span off to all this: node, mongo, js, unity3d and c# , php.... And my code, less refactoring every week :P
[17:16:41] <svm_invictvs> http://docs.mongodb.org/manual/tutorial/create-an-auto-incrementing-field/
[17:16:59] <svm_invictvs> cheeser: Also looking at this example here: https://github.com/mongodb/morphia/blob/8f70190862c0094f02f3bf27713ac0f63d1f2cd2/morphia/src/test/java/org/mongodb/morphia/utils/LongIdEntity.java
[17:18:04] <GothAlice> Also, notably, you run into the Twitter problem when you start using "auto increment". I.e., it doesn't scale.
[17:18:36] <GothAlice> StephenLynx: How do you have two machines simultaneously insert new data without a collision?
[17:19:54] <GothAlice> StephenLynx: To the point they wrote https://github.com/twitter/snowflake/tree/snowflake-2010
[17:19:56] <StephenLynx> I just keep retrying and upping the id if the error its because of a collision :v
[17:21:27] <svm_invictvs> GothAlice: I understand that it's not relational, but that's not why I asked.
[17:21:43] <svm_invictvs> GothAlice: That wouldn't prevent soembody from trying to add something like that to Morphia
[17:22:17] <GothAlice> It is, but. No enforcement, no cascading rules, and ODMs that add such behaviour often trap users into thinking MongoDB does support it, and are surprised when the command-line tools don't do the same thing.
[17:22:38] <svm_invictvs> GothAlice: THre have been such things as poorly written tools, that's why I asked.
[17:24:07] <GothAlice> When db.collection.remove(somecriteria) in the shell and db.collection.find(somecriteria).remove() in my app do different things… there may be a problem.
[17:24:56] <jecran> StephenLynx: https://gist.github.com/anonymous/3cec295353b57cd62658 .... any recommendations on this? I get all expected results now..... coolbeans, although stupidly simple
[17:26:25] <ggoodman> It appears that in the 2.x series of node-mongodb-native, the semantics of `findAndModify` have been significantly changed.
[17:27:33] <ggoodman> What is now available to do a conditional update on many documents and return the modified documents resulting from the operation?
[17:27:34] <GothAlice> The edge cases are substantial: http://docs.mongodb.org/manual/reference/method/db.collection.findAndModify/#return-data
[17:28:04] <GothAlice> Like, way substantial. Not worth using level substantial. Just modify… then find.
[17:38:03] <shlant> anyone know why MMS automation agents take 5-10 minutes before even realizing that the servers they run on don't exist anymore???
[17:40:06] <GothAlice> shlant: MMS uses a once-per-minute ping, AFIK, and debouncing means that that once a minute thing gets stretched for a few minutes before the web interface really notices the problem.
[17:40:32] <GothAlice> (A single failure to ping might not indicate a problem. Two or more in a row missed = a problem.)
[17:42:26] <GothAlice> Debouncing is useful, but does add to the latency of alerts. Nobody wants their monitoring to flap, though. (It's down! It's back! It's down! It's back! …)
[17:43:05] <shlant> understood, just seemed like 5 minutes of losed connectivity is more than a "flap"
[17:43:26] <shlant> I still have servers showing on MMS that _had_ monitoring and backup agents on them
[17:45:41] <GothAlice> Where else are you seeing the hosts, then? Under the main Deployment section?
[17:48:19] <GothAlice> Hmm, also check Administration > Agents and try the "…" button, "Remove from MMS", if available in the server list against each host.
[17:48:43] <GothAlice> (If still enrolled in a replica set, you'll have to "Remove from Replica Set" first.)
[17:53:38] <shlant> like they are on the server and yet I don't have the option to uninstall them anymore as I already did
[18:05:02] <mbeacom> Hello everyone! Can someone please assist me? http://pastebin.com/YwLhKGMB In the provided JSON example, you'll find three example mongodb 2.6 results. I need to filter down the results to only return documents where the sub/embedded document in each document doesn't have more than one messages.author_id over all 'messages.$.author_id'
[18:21:16] <mbeacom> If there is a Bueller out there? If so, does Bueller want to help me resolve a simple aggregation pipeline issue? :D http://pastebin.com/YwLhKGMB
[19:18:40] <jacksnipe> Hey, I'm using mongo with python, flask, and the MongoEngine extension. Since I'm pretty sure MongoEngine is just a wrapper over the basic python mongo lib, do I have to worry about ints vs longs or other variable size stuff?
[19:33:02] <GothAlice> jacksnipe: As a note, there's also #mongoengine, and sometimes people even talk there. ;)
[19:41:51] <qrome> Let's say I wanted to start purging data where it would remove it from my mongodb after a certain time period and store it somewhere slow. Is this normal?
[19:46:47] <GothAlice> Typical is to stream the data into the "elsewhere" as it comes in, and let MongoDB clean itself up.
[19:47:04] <GothAlice> https://github.com/10gen-labs/mongo-connector to sync the data out to another service
[19:50:50] <deathanchor> WTF? rs_a:SECONDARY> rs.reconfig(cfg, { force : true }); "errmsg" : "exception: need most members up to reconfigure, not ok : host2:27018"
[19:52:46] <GothAlice> Thus no way to propagate the reconfig to other nodes reliably, and yeah, even if you force it, it'll say no to that.
[19:55:17] <GothAlice> If DNS names are in there, DNS becomes critically important to the reliability of your DB cluster, which is why I have /etc/hosts managed by the cluster to automatically include hard references to all other hosts in the cluster. No DNS problems. :)
[19:55:45] <deathanchor> yeah, I think someone changed the box from hostname, to hostname.fully.qualified
[19:57:55] <deathanchor> yeah, hence the testing, someone messed with it and didn't bounce the service
[19:59:49] <GothAlice> If you have fewer than 9 hosts, for the free tier, I can highly recommend https://mms.mongodb.com/ as a service to manage your cluster. Full disclosure: satisfied customer.
[20:00:12] <GothAlice> You'll still need to resolve the DNS problems yourself, but everything else is made easier by MMS. :)
[20:06:13] <jacksnipe> uh wtf is a 2dsphere -- as in a sphere with 2 dimensions to specify a point (i.e., lat/long)?
[20:07:36] <GothAlice> It also makes approximations about your height from sea level and things like that.
[20:08:54] <GothAlice> The old "2d" index, meant for backwards compatibility with 2.2 and earlier, assumes a flat two-dimensional euclidian plane.
[20:08:55] <cheeser> 2d sphere, as I understand it, is a more appropriate way to handle geo stuff than geo2d because distances and the like aren't as simplistic as a flat surface
[20:09:01] <jacksnipe> ah that's it, what's the speed difference between a Geo2D index and a 2dsphere?
[20:09:32] <GothAlice> There's a reason planes fly in a curve across the surface of the Earth instead of taking a "straight line" approach. Calculating distances around spheres can be mind-bending, and a curved path is often shorter.
[20:10:09] <GothAlice> Optimization without measurement is by definition premature. Try both, and measure. ;)
[20:10:53] <jacksnipe> yeah, thought so. ATM I'm just looking for users in a radius, and calculating the distortion from the rect mapping to the sphere mapping
[20:11:29] <GothAlice> 2dsphere would handle that for you. You can specify a search with a central point and circular radius.
[20:11:57] <GothAlice> http://docs.mongodb.org/manual/reference/operator/query/centerSphere/#op._S_centerSphere
[20:12:27] <jacksnipe> yeah I know what functionality I'd get I'm really _only_ concerned about the speed of the queries :P
[20:13:03] <GothAlice> I have peculiar requirements not covered by MongoDB's geographic capabilities, assuming operation on the surface of a spherical planetoid is highly restrictive for my needs. ;)
[20:14:01] <jacksnipe> yeah I need very low precision/accuracy (adding or dropping ~20 miles is nbd) so I was thinking that the Geo2D was good enough but I guess I'll take a look
[20:14:54] <GothAlice> And the note that geo2d is meant for legacy compatibility with 2.2 (and we're in 3.0…) should be a sign that its use is flatly deprecated.
[20:18:21] <GothAlice> I'm a fan of ringworlds. The benefits of a plane, and 1000x the surface area of a single sphere.
[20:20:36] <GothAlice> That just doubles the "width" of the strip. Still a flat plane for distance calculations.
[20:34:40] <jacksnipe> how do I direct mongo to autotimestamp something? Just have an empty field called "ts" on insert?
[20:45:44] <GothAlice> The more records you have, the more the field name overhead adds up. Renaming like this, though, really hurts when I choose to use non-Python tools like the mongo shell, though.
[20:47:34] <GothAlice> MongoEngine (and ODMs in general) provide a bit more functionality than just field re-naming. ;) MongoEngine is a full schema enforcement system, with triggers (signals), etc.
[20:48:47] <GothAlice> Also handles things like caching references (storing "remote" fields alongside the ObjectId of the reference) automatically, such that updating the referenced document updates all cached values, too.
[20:59:00] <saml> i have integration tests that inserts documents and removes them.. and executes count and other queries very fast. and sometimes tests fail
[21:08:26] <greyTEO> GothAlice, have you ever had an object override itself? e.g. lets say you have a web admin. Page 1 has an object and page 2 has the same object. page 2 updates. will page 1 updates override page 2? Similiar to a race condition with "stale" data.
[21:08:56] <greyTEO> this would be give the user has 2 tabs open at the same time...all hypothetical of course.
[21:09:46] <GothAlice> Github, on the wiki portion of projects, will alert a user in the process of editing something that someone else managed to hit save first, and leaves resolving the conflict to the user.
[21:10:33] <GothAlice> The other extreme from "whoever saves last wins" is to have every field fully live. Someone changes something in one place, all other places are immediately updated, regardless of manually refreshing.
[21:11:11] <greyTEO> That is what I thought. I was wondering if there was some magical method for merging docs...lol
[21:12:01] <GothAlice> Versioning would be a simple approach, and would leverage update-if-not-different queries.
[21:12:58] <GothAlice> I.e. user A and B load the form, each get version 1. User A presses save, db.foo.update({_id: …, version: 1}, …). User B presses save, the update can't find the record because the version is different, and user B is presented with the updated data, with their changes applied, and a message that someone else got to it first.
[21:14:36] <greyTEO> Yea that would be a fairly simple implementation. For my case, last to save wins works.
[22:42:37] <jecran> Hi guys. Using express , I am just trying to do a simple 'post' but I cant access my data. If I do a 'get' everything is there, but I need post lol. app.post('/login', function(req, res){}); This is the line that handles the post. I make it inside this function but can't retrieve my data. Any ideas on what im doing wrong?