PMXBOT Log file Viewer

Help | Karma | Search:

#mongodb logs for Friday the 17th of June, 2016

(Back to #mongodb overview) (Back to channel listing) (Animate logs)
[04:41:11] <subir> hi all, would anyone be able to help me on this one? https://groups.google.com/forum/#!topic/mongodb-user/UsWCGCAK6MQ
[04:41:14] <subir> thanks
[04:42:29] <Boomtime> what does smoothly mean?
[04:43:18] <Boomtime> -> shut down the member, repoint the domain dns info, wait for that to propogate, start up new member at the new location
[04:43:52] <Boomtime> if you like you can copy the dbpath from the old member (after shutting it down) across to the new member before starting it up
[04:47:11] <jbu> hi all: I have a collection of posts which can be parents or children (1 level deep). I need to implement paging to grab posts such that it pulls in the most recently updated parent post first, then for each parent post it gets most recently updated children first within the children...I don't believe there's some kind of grouping / query that would make this feasible to work with paging...anyone have any idea
[04:56:16] <subir> by smoothly, I mean without causing outage
[05:00:13] <subir> Boomtime, by smoothly, I mean approach to not cause outage of the replicaset
[05:01:03] <subir> Boomtime, as it is in production environment
[05:15:08] <Boomtime> @subir: do you use majority writeconcern?
[05:15:39] <Boomtime> you have a PSA replica-set, a single copy redundancy - if you don't use majority writeconcern then you can afford to have one of these down for a short time
[05:16:02] <subir> Boomtime, no, we're not using majority writeconcern
[05:16:13] <Boomtime> however, i have a question; why do you need the new host to have the same DNS name?
[05:16:34] <Boomtime> why can't you temporarily bring up a new host (new name), then decommision the old one?
[05:19:24] <subir> Boomtime, we're using connection string something like: uri: mongodb://mongodb01<domain>:27017,mongodb02<domain>:27017/dbname in all the apps
[05:20:20] <subir> Boomtime, for connecting to the replicaset
[05:20:28] <Boomtime> that's ok - you can change that when you're able, it doesn't need to be at the same time - the connection string is only a seed list for discovery
[05:21:28] <Boomtime> for example: if you have a client connected with that connection string, then use rs.add to add a couple new members and then rs.remove two of the existing ones, that client will still discover the full set and track the primary correctly
[05:22:43] <Boomtime> to be clear; the connection string is only used for first contact - once even one of the members of the replica-set is contacted, the connection string seed list is thrown away and the replica-set configuration is used instead
[05:26:55] <subir> Boomtime, so let's say. to make it smoothly :), if I use rs.add to add mongodb03, and use rs.remove to remove mongodb02, then the above connection string with mongodb01 and mongodb02 should still be fine, right?
[05:28:15] <Boomtime> wait until mongodb03 is fully sync'ed so you know you have redundancy
[05:28:40] <Boomtime> but yes, the application will remain connected, and continue to be able to connect - however, you should update the connection string when you can
[05:29:33] <subir> Boomtime, but we have about 20+ apps which need to be built all over again, and deployed to production
[05:29:50] <Boomtime> you hard coded the connection string in? seriously?
[05:30:20] <Boomtime> how do you test it?
[05:30:25] <subir> Boomtime, and then after this step, I could use the same step as above to add a new mongodb02, and then remove mongodb03 to be where we wanted to
[05:30:48] <subir> Boomtime, yeah, the connection string was already there hardcoded from before
[05:30:49] <subir> :(
[05:32:34] <subir> Boomtime, there are different configs pointing to different environments
[05:33:30] <subir> Boomtime, we wanted to use consul for this purpose, but were not sure how to do that for replicaset
[05:36:10] <subir> Boomtime, so when adding mongodb03 to the replicaset, does it need to be hidden and without voting rights as it would create 1 primary, 2 secondary and 1 arbiter?
[05:36:33] <subir> Boomtime, or should it not make that much of a difference for that transition?
[06:23:57] <Lope> I need to have some kind of 2d grid object with 1's and zeros in each block. What data structure should I use to store it?
[06:46:17] <sumi> hello
[07:23:27] <dddh_> result of db.collection.find({..}).limit(5).count() should be <= 5 ?
[07:24:10] <Boomtime> wut?
[07:24:14] <Boomtime> no
[07:24:31] <Boomtime> limit applies to the cursor not to the count command, so limit won't even be seen
[07:25:09] <Boomtime> or to put it another way; count returns exactly 1 result always
[07:26:03] <dddh_> Boomtime: 1 result but I expected that this result should be between 0 and 5
[07:26:29] <Boomtime> limit is the number of results, count is a single result
[07:26:49] <dddh_> ok, I see
[07:27:22] <Boomtime> i.e count always returns a single result document where you interpret the value to be of significance in the form of a document count
[07:27:37] <dddh_> x = db.collection.find().limit(5).count()
[07:27:39] <dddh_> print(x)
[07:28:18] <dddh_> x <= 5 is true ?
[07:28:51] <Boomtime> no
[07:29:09] <Boomtime> i mean.. it _might_ be
[07:29:19] <dddh_> https://docs.mongodb.com/manual/reference/method/cursor.count/#limit-documents-in-count
[07:29:49] <Boomtime> holy shit, i'm wrong then apparently
[07:29:52] <Boomtime> it never used to
[07:30:10] <dddh_> yes, that is why I was wondering why it returns 8 in my case ;(
[07:30:18] <Boomtime> what version?
[07:30:38] <Boomtime> run db.version()
[07:30:38] <dddh_> 3.2.7
[07:31:59] <Boomtime> heh, you need the parameter to get fluent to see it
[07:32:06] <Boomtime> cont(true)
[07:32:10] <Boomtime> count(true)
[07:32:19] <Boomtime> that was a lucky typo that could have gone badly
[07:32:42] <dddh_> it counts where filter results true?
[07:33:09] <Boomtime> the count cursor function has this definition ->
[07:33:09] <Boomtime> function (applySkipLimit) {
[07:33:36] <dddh_> I see ;(
[07:33:36] <Boomtime> i.e it needs to be told - because, as i said, it never used to have this behavior, so presumably this was done to avoid having the behavior change between versions
[07:33:54] <Boomtime> til
[07:33:56] <Boomtime> TIL
[08:29:40] <jbu> all: Is there any guarantee about ordering after the $unwind operator is applied?
[08:30:45] <bratner> HI all, is this a good place to ask c++ mongodb driver related questions?
[12:31:42] <jayjo_> I'm using mongo compass, but in my queries I need to use some application programming (like getting the current date, or what date was two days ago, etc.) does compass build off of an api that I can use some code like this in my queries?
[14:48:01] <jayjo_> When I run mongoimport I'm getting an error "fatal error: bad stack size
[14:49:07] <jayjo_> this is my stacktrace: https://bpaste.net/show/d81abc4c0e78
[14:52:11] <jacoscaz> How do I disable the bson c++ extension in mongodb's nodejs 2.0.x driver?
[15:00:05] <jacoscaz> nvm, got it
[15:03:03] <jayjo_> I suspsect its openssl... how do I figure it out what version of openssl is being run with mongo on the server & my local machine? But I can connect just with the mongo shell just fine. Can I reinstall mongoimport only?
[15:08:34] <edrocks> jayjo_: run `mongodb --version`
[15:08:48] <edrocks> jayjo_: I mean `mongod --version`
[15:11:58] <jayjo_> edrocks: They are in fact different (even the ssl) will this cause the problem: https://bpaste.net/show/a28899de7627?
[15:13:25] <edrocks> jayjo_: Possibly. I don't know that much about openssl version schemes. I imagine it would have caused problems a long time ago and had been resolved to make them use the same one if it did cause issues
[15:36:41] <unix4linux> is it possible to have mongo prompt for password instead of passing it directly with the -p option?
[15:58:41] <jayjo_> If I have a db with ssl enabled, can I still connect over localhost? Now I need the hostname to match the certificate, and when I pass a hostname it tries to connect to this hostname. Does this take localhost off the table?
[16:12:34] <saml> https://github.com/mongodb-labs/mongo-connector do you use this?
[16:12:56] <saml> __path__ = extend_path(__path__, __name__) what is this?
[16:27:54] <pamp> hi
[16:28:43] <pamp> I have my HD full, I'm using wired tiger mongodb 3.2
[16:28:56] <pamp> mongodb doesn t start
[16:29:25] <pamp> what can i do to remove some data ? and set up my mongod instance?
[16:34:00] <saml> https://github.com/mongodb-labs/mongo-connector/wiki/Writing-Your-Own-DocManager#step-2-distribute-your-docmanager this thing doesn't eevn work
[16:58:50] <alexi5> i am starting to see documents everywhere
[17:59:47] <saml> it's bad
[18:20:08] <jayjo_> Is it possible to connect to a localhost instance with ssl enabled?
[18:22:33] <cheeser> yeah, should be.
[18:24:16] <jayjo_> But if I don't specify the host then there is a mismatch between the certificates hostname and the connection fails.
[18:24:29] <jayjo_> If I do specify it, then it will connect through the external IP
[18:25:33] <cheeser> and that's a problem how?
[18:36:08] <jayjo_> because I can't connect to the localhost with this servers public certificate
[18:36:31] <yopp> um
[18:36:33] <yopp> hey
[18:37:46] <yopp> I have a aggregation query with unwind. Before applying limit/skip I need to memoize the count of the documents and the step
[18:37:50] <yopp> is it possible?
[18:38:06] <yopp> (say I'm doing the pagination on aggregation result, and need to get number of pages)
[18:41:43] <kurushiyama> jayjo_ Simply connect to the external one? Since it is his own, that connection should stay on the host.
[18:44:06] <jayjo_> Ok so connect externally.... so I can't connect to the localhost with ssl enabled
[18:45:09] <kurushiyama> jayjo_ You do _not_ connect externally. You connect to the external IP of your host. That is _quite_ a difference.
[18:47:51] <kurushiyama> jayjo_ To understand the difference: Give your laptop a static IP config, fire up a service, plug out the network, and connect to said service.
[18:55:06] <schneider> how can I store encrypted string inside my collections for further decrypt? i junt dont want to store plain text
[18:57:39] <kurushiyama> schneider either get MongoDB enterprise (which supports transparent encryption of the data files) or do it application side.
[18:58:33] <kurushiyama> schneider I'd prefer the latter, anyways.
[19:06:48] <jayjo_> kurushiyama: thank you, it was working, but I didn't understand it. You cleared it up
[19:09:06] <jayjo_> If I have a collection of json documents that are 11.8 GB, and I put them in with mongoimport, should I expected the dataSize to be 11.8 GB?
[19:09:12] <jayjo_> of the collection
[19:10:09] <kurushiyama> jayjo_ Basically, think of your IP stack as your companies post office. Package comes in for the external IP address. "Hey, look, a new package! Where does it go to? Oh, that is our external address? Deliver it right away, we do not need the external post for it."
[19:10:31] <kurushiyama> jayjo_ I'd say less, depending on your setup.
[19:12:39] <kurushiyama> jayjo_ Compressed WT (default as of 3.2): You bet it is much smaller. MMAPv1: vice versa.
[19:15:01] <jayjo_> should it really be _much_ smaller? I have a collection now that I put this data into, and it's 10.7 GB, when the data itself in json is 11.8 GB. I just want to verify it all 'made it' in a sense
[19:16:30] <n1colas> Hello
[19:17:40] <kurushiyama> jayjo_ Uhm count the records?
[19:17:48] <kurushiyama> Hi n1colas
[19:21:27] <jayjo_> I can't very easy count the records.... the json themselves have variable amount of lines
[19:23:44] <jayjo_> I guess... there are ways
[19:24:39] <kurushiyama> jayjo_ You should know the number of objects in you json files. If not, even a regex should do the trick of counting them
[19:41:18] <saml> given docs like: {numbers: [1]} how can I query for docs whose numbers contain something other than certain number?
[19:41:50] <saml> a = {numbers:[1]}, b = {numbers:[1, 2]} given these, I want to get b only, not a
[19:42:10] <saml> db.docs.find({numbers: has other than 1})
[19:48:58] <kurushiyama> saml $not $in?
[19:49:46] <saml> ?
[19:49:55] <saml> I want b, which contains 1
[19:50:03] <saml> $not $in [1]
[19:52:39] <kurushiyama> saml dodgy.
[19:55:28] <saml> did it in a loop
[19:55:51] <saml> forEach(function(x) { if (pred(x.numbers))
[19:58:17] <saml> why does mongo-connector doesn't do anything?
[19:58:21] <saml> it just hangs. no log
[20:02:55] <kurushiyama> saml How is that interconnected?
[20:04:42] <saml> different issue
[20:04:59] <saml> working on two things. hehe finished that $not $in query
[20:05:08] <saml> now working on mongo-connector
[21:07:57] <MacWinner> Is it possible to create an index using db.collection.ensureIndex(..) but not block the collection?
[21:08:08] <MacWinner> i don't mind if it takes a bit longer
[21:08:47] <cheeser> set background : true
[21:09:08] <MacWinner> thanks!
[21:09:22] <MacWinner> is there anyway to get an estimate of how large an index will be before creating it?
[21:09:27] <cheeser> not really
[21:10:40] <MacWinner> what happens if you create and index and it exceeds your memory side?
[21:10:42] <MacWinner> size
[21:11:32] <cheeser> well, the indexes live on disk and are kept in memory if they can be.
[21:12:34] <MacWinner> will $in queries use indexes?
[21:12:46] <cheeser> yes
[21:13:21] <MacWinner> I currently have an index for "sender_1_objecttype_1_utcts_1_recipient_1" .. now i want to find all documents that have a sender in a set... will this compound index get used?
[21:14:13] <MacWinner> like { sender: {$in: ['foo@bar.com', 'foo2@bar.com']}}
[21:14:32] <cheeser> it should. .explain() can help you figure that out.
[21:18:15] <MacWinner> awesome, thanks again!
[21:18:27] <cheeser> np
[21:24:36] <MacWinner> cheeser, sorry, one last question.. with the index definition above, where _recipient_1 is at the end, will queries for recipient also be indexed?
[21:25:24] <MacWinner> i'm a little scared to do the query on my box since there is a ton of data
[21:26:19] <MacWinner> oh.. explain doesn't actually run the query
[21:26:51] <cheeser> no. a query by recipient won't use that index.