PMXBOT Log file Viewer

Help | Karma | Search:

#mongodb logs for Friday the 2nd of September, 2016

(Back to #mongodb overview) (Back to channel listing) (Animate logs)
[03:41:18] <SimpleName> back up mongodb replica set
[03:42:06] <SimpleName> how to handle it, I know I can use mongodump, or snapshoot to restore a single server, but how to handle replica set
[03:53:32] <GothAlice> SimpleName: Add a replica member to maintain a near-realtime streaming backup.
[03:54:06] <SimpleName> GothAlice: but if you delete some data, the near-realtime server will delete it too
[03:54:26] <GothAlice> SimpleName: Ensure it can't win elections by setting a lower priority; this would protect against the failure of the other nodes on the set. You can even "delay" the replica used for this purpose (or have another that's delayed) to protect against user error, such as bad or mass deletions, bad updates, etc.
[03:54:37] <GothAlice> https://docs.mongodb.com/manual/core/replica-set-delayed-member/
[03:55:27] <GothAlice> We run a realtime member and a delayed member in the office, using chained replication to have the delayed one pull from the live in-house one to avoid excessive bandwidth use: https://docs.mongodb.com/manual/tutorial/manage-chained-replication/
[03:57:02] <SimpleName> GothAlice: if I use LAN ip config the replica set, it will be fail, primary become SECONDARY, and the SECONDARY can’t connect primary
[03:57:32] <SimpleName> GothAlice: but if I change the public IP, it works well
[03:57:34] <GothAlice> As mentioned, you use priority to control which members can become primary or not: https://docs.mongodb.com/manual/core/replica-set-priority-0-member/
[03:57:50] <GothAlice> I.e. our backup members can never become primaries.
[03:58:19] <SimpleName> GothAlice: there just another question, use LAN ip does not work, use public ip, works well
[03:58:31] <GothAlice> I'm not seeing the question in there.
[03:59:23] <GothAlice> Yes, all members should be able to "see" and communicate with each-other. It's highly, highly recommended to enable authentication and SSL in any scenario where your members are accessible outside a private VLAN in the datacenter.
[03:59:59] <GothAlice> Ref: https://docs.mongodb.com/manual/core/authentication/ and https://docs.mongodb.com/manual/tutorial/upgrade-cluster-to-ssl/
[04:00:22] <SimpleName> I use x509 internal auth for replica set
[04:00:55] <GothAlice> x.509 refers to authentication, not encryption of protocol data.
[04:01:08] <GothAlice> So SSL is still recommended, in addition to whichever form of authentication you use.
[04:01:43] <SimpleName> GothAlice: ok, I solve it, because I just make the firewall open for the specify public ip
[04:02:11] <SimpleName> but the document said, you better use x.509 with production
[04:02:57] <GothAlice> Note that your data being replicated over the public internet will be vulnerable to sniffing unless you use SSL. --sslMode and --clusterAuthMode are separate configuration options.
[04:03:28] <SimpleName> ok, let me look the document for more detail
[04:04:06] <SimpleName> have you used MongoDB Cloud Manager GothAlice the document always recommand it, but I just like free and opensource
[04:04:54] <GothAlice> Since version 3.0 the SSL options are available in the community edition (open source / free) version of the server. Prior to that it was available if you compiled the server yourself, but not available in the community edition binaries.
[04:05:08] <GothAlice> (I have used the Cloud Manager; I quite like it, in fact. :)
[04:06:08] <GothAlice> Because my servers use Gentoo, a compile-from-source Linux distribution, SSL has always been an option on that distribution. ;)
[04:07:18] <SimpleName> wow, you are amazing, I just use centos 7
[04:07:34] <SimpleName> how much money the cload manager one year
[04:07:45] <GothAlice> Also, apparently I'm tired—almost midnight here. ^_^; Indeed, if you're using X.509, you have SSL enabled as a requirement.
[04:09:09] <GothAlice> SimpleName: Please don't PM without asking first. Standard Cloud Manager is $39/server/month, or $468/server/year.
[04:09:30] <SimpleName> GothAlice: ok
[04:09:35] <GothAlice> There is, however, a free tier.
[04:09:45] <SimpleName> it’s cheap
[04:10:17] <GothAlice> https://www.mongodb.com/cloud/cloud-manager — see the pricing section on this page, which includes a feature matrix to see what the differences are between the versions.
[04:10:26] <GothAlice> Indeed, it's quite reasonable.
[04:11:51] <GothAlice> Cloud Manager assumes you have your own servers to use, though. MongoDB Atlas is a full hosted solution where servers are provided and managed for you: https://www.mongodb.com/cloud/atlas/pricing
[04:12:05] <SimpleName> ok, thank you
[04:12:48] <SimpleName> there is a version namded Cloud Manager Free??
[04:12:59] <GothAlice> Yes, given you have your own servers to use.
[04:13:45] <SimpleName> :D may be I can write a new manager for myself,
[04:14:01] <GothAlice> It does not include the more advanced automation (zero-downtime upgrades, database creation / modification, query optimization) that the paid versions have.
[04:14:03] <SimpleName> It ’s just a web server, and I am a pythoner
[04:14:45] <GothAlice> It's… a bit more than just a web server. It performs server automation (at the paid tiers) to perform controlled upgrades, manage user accounts, monitoring, and even analyze query performance to suggest indexes.
[04:16:10] <GothAlice> It also performs streaming backups.
[04:16:19] <SimpleName> about index, I think I can handle it well,
[04:17:03] <SimpleName> streaming backup, just like you say the “a single streaming server in replica set”?
[04:17:56] <GothAlice> ^_^ Certainly; there's nothing stopping people from writing their own automation platforms for MongoDB. Anything Cloud Manager can do, you can do, too. I appreciate the rock-solid reliability, analytics, support, and peace of mind using an official product can give. One must weigh the cost of using any existing service against the cost to develop the solution yourself.
[04:18:00] <SimpleName> mongodump —oplog
[04:19:22] <SimpleName> 8) yes, I will pay for it, because my company handle 1000,0000 data one day, just insert these new data
[04:19:33] <SimpleName> and analytics it one week, or one day
[04:20:23] <SimpleName> in fact, I want to know more about mongodb, before use the manager
[04:21:39] <GothAlice> It never hurts to learn more. :)
[04:28:33] <GothAlice> SimpleName: 预祝被翻译的手册编写的。:) I think that's… almost correct. ^_^;
[04:28:47] <SimpleName> - - 你居然也会说中文
[04:29:37] <GothAlice> 我只能说一点中国 ^_^
[04:31:23] <SimpleName> ;) but your chinese grammar is bad , just like My english is poor
[04:32:37] <GothAlice> 我不说话中文够...我不是非常好。Yeah. :P
[04:32:48] <GothAlice> I learned Japanese first…
[04:33:37] <SimpleName> :) Why learn Japanese first
[04:33:50] <GothAlice> It's the exotic language they offered at my school.
[04:34:57] <SimpleName> What College did you graduate from?
[04:35:42] <SimpleName> I wil go for dinner, see you later,
[04:35:43] <GothAlice> I actually learned Japanese in high school.
[04:35:46] <GothAlice> Have a great one!
[04:42:20] <GothAlice> Hmm. I do seem to recall the documentation was translated at one point… I just can't find it. The latest traffic on https://groups.google.com/forum/#!forum/mongodb-translators is from 2014 and https://mongodb-documentation.readthedocs.io/en/latest/meta/translation.html links (ancient) are dead. T_T
[05:31:07] <SimpleName> GothAlice: I can translate it from English to Chinese
[07:01:51] <SimpleName> when add a new secondary, it will be sync data from primary, read the primary’s oplog, right???
[08:27:00] <juodumas> Hello. I have this structure: `{items: [ {price: 12, worker_id: 1}, {price: 13, worker_id: 2} ], total: 25}`. I am running `find({items.$elemMatch: {worker_id: 2}})` to return documents with a particular worker_id. What is the best way to make mongo transform the items array so that it returns only those items that match the specified worker_id?
[08:58:41] <lohfu> how do i save RegExp to database using mongoose?
[08:58:55] <lohfu> mongoose refuses to accept `{ type: RegExp }`
[08:59:06] <lohfu> even though i can easily save RegExp to database using the native driver
[09:01:11] <lohfu> ok... i can just set the type to Object, but that seems silly as i then manually have to validate that it is in fact a regexp
[11:58:08] <SimpleName> GothAlice: keyfile vs x.509, which is more suit production
[11:58:33] <Zelest> wide open, no auth, no encryption :D
[11:58:44] <Zelest> over as many WAN links as possible :)
[11:58:45] <SimpleName> ;) are you kidding
[11:58:51] <Zelest> I am :D
[11:58:57] <SimpleName> - -I think so
[11:59:06] <SimpleName> so, what’s your recommand
[11:59:32] <Zelest> I actually use that.. but without the WAN links
[11:59:55] <SimpleName> use what??? no internal auth???
[11:59:56] <Zelest> we have the cluster firewalled and not available from the global net
[12:00:01] <Zelest> mhm
[12:00:23] <Zelest> {internet}----{web}---[ firewall ]---{ replicaset }
[12:00:29] <SimpleName> but if you enable the auth, the replica set won’t be work if you don’t set internal auth
[12:00:34] <SimpleName> I use firewall too
[12:01:07] <SimpleName> but I want to make more limit for my colleague
[12:02:00] <Zelest> ah
[12:02:13] <Zelest> i'm not sure what the best practice solution is so
[12:02:41] <SimpleName> ok , at least , I will use firewall + keyfile
[12:02:45] <SimpleName> 8)
[13:18:03] <nikitosiusis> can I disable wiretiger cache somehow?
[13:18:45] <nikitosiusis> I find that each time my cache get 95% my database freezes. but the performance is the same with 16gb, or with 128gb cache.
[13:21:29] <nikitosiusis> also, does it support anything except lru?
[13:41:38] <hemangpatel> Hello there, I'm getting this error while saving doc, WriteError({"code":14,"index":0,"errmsg":"Argument must be a string"
[13:42:55] <StephenLynx> well need the query.
[13:44:29] <hemangpatel> In local machine it is working but in ci while running test cases it' not working
[13:44:31] <cheeser> do you have a non-string key in your doc?
[13:48:05] <hemangpatel> let me send you doc
[13:49:19] <hemangpatel> cheeser, http://pastebin.com/3KJ1VA3c
[13:52:12] <cheeser> nothing wrong with the doc.
[14:08:34] <hemangpatel> cheeser, mongodb 2.4.9
[14:08:43] <cheeser> whoa.
[14:08:51] <cheeser> still, that doc inserts fine for me.
[14:09:06] <hemangpatel> is it a problem ?
[14:09:22] <GothAlice> hemangpatel: 2.4 is no longer supported, so it's concerning.
[14:10:07] <GothAlice> (3.2, 3.0, and 2.6 are supported; if you decide to upgrade at some point, remember to upgrade one version at a time until current instead of jumping straight to the latest.)
[14:10:44] <hemangpatel> Ok i know version is old but is it the reason for problem ?
[14:21:33] <cheeser> just missed her!
[14:25:04] <GothAlice> cheeser: Did you see the issue?
[14:27:14] <cheeser> i didn't
[14:27:34] <cheeser> but i didn't dig too deeply. fighting jenkins. :)
[15:07:50] <SimpleName> GothAlice: have you use mongoengin, it can create a base class for inheritance, but whatever create how many child class, it’s just create one collection, it’s not my wanted
[15:08:02] <SimpleName> GothAlice: do you have any idea
[15:19:29] <GothAlice> SimpleName: Avoid MongoEngine like the plague.
[15:19:43] <GothAlice> For all of these reasons and more: https://github.com/marrow/contentment/issues/12
[15:20:22] <GothAlice> To answer your question, however, you'll want to mark the base class as being "abstract" to prevent it anchoring the collection used at that level.
[15:20:31] <cheeser> 324 open issues!
[15:20:41] <GothAlice> About a third of which are regressions since 0.9.
[15:21:47] <GothAlice> #529 is particularly egregious, given the time frame. (Can't use $ positional projection operator properly.)
[15:21:53] <GothAlice> Open since 2013.
[15:22:02] <SimpleName> GothAlice so , is there any more suit way to define an model
[15:22:27] <GothAlice> SimpleName: Yes: in MongoDB. https://docs.mongodb.com/manual/core/document-validation/
[15:23:32] <SimpleName> ok, thanks
[15:23:45] <GothAlice> As a nice side effect, document validation is _way_ more powerful than an object schema. You can do anything you can do in a find operation, including $or and friends.
[15:26:16] <SimpleName> ok, I will change my code like this
[15:36:54] <SimpleName> GothAlice: good night, I will go to sleep :P
[15:37:01] <SimpleName> and thanks for your help
[15:37:27] <GothAlice> No worries. Have a great rest! :)
[23:13:07] <Skaag> Hi! Question: why is it so difficult to find the mongod.conf replica set configuration options for mongodb 3.x?
[23:13:17] <Skaag> I'm literally fishing for it via google searches
[23:28:42] <wgreenberg> hi! so I have a bunch of documents like { parent_ids: ['a', 'b', 'c'] } and I'd like to get stats on how many documents each parent id has, e.g. { 'a': 10, 'b': 30 } -- are Aggregates the right tool for this?
[23:28:55] <wgreenberg> I assume doing one .count() query for each parent_id is not a great idea..
[23:29:24] <wgreenberg> also, by "each parent id", I don't mean for each parent id in the entire database, just each parent id in some arbitrary subset
[23:56:29] <cheeser> yes. you'd $unwind the arrays then group/count