Log file Viewer
#mongodb logs for Wednesday the 14th of October, 2015
[00:03:20] <Bioblaze> I plan to use the _id of the Product + the Security Token as a Auth Key more or less :X
[00:19:24] <Bioblaze> I wanna store my Date information as ISO 8601 format. Is there anyway to do that? XD @.@
[00:44:08] <happyken> Hi steffan can you help me with this question: have mongo db with multiple databases having common prefix. I want to have common account credentials for dbs having common prefix. Is this use case possible by any chance ?
[00:53:10] <steffan> happyken: There is not, afaik, any way to do this automatically. You could write something to generate / synchronize credentials, perhaps, or possibly use external authentication
[01:19:54] <steffan> Well, there is this: https://www.mongodb.com/products/mongodb-enterprise-advanced
[01:20:14] <steffan> There are some features that only exist at this time in the commercial version. SSL used to be that way
[01:21:24] <steffan> Naturally, there is freedom for someone to independently develop, e.g., Kerberos support for the open-source version, but to date, I don't believe anyone has done so
[01:27:00] <StephenLynx> http://info-mongodb-com.s3.amazonaws.com/MongoDB_Enterprise_Advanced_Datasheet.pdf
[01:44:42] <steffan> Yes, using an external Kerberos / LDAP / Active Directory server is what I was referring to by external auth
[10:50:35] <braqoon> Hi, I have a replicaset that i need to change ID of it. I know this is not possible but ID have to be changed anyway. Will removing all nodes from replicaset and re-adding them under new replicaset without removing any data in DB is possible ?
[15:22:20] <dhanasekaran> Hi Guys, I am new to mongodb, Currently I have some with mongo with wiredTiger storageEngine, It's every thing keep memory, but not Reclaiming memory to OS please guide me.
[15:27:12] <StephenLynx> A: WT is leaking. try running with the default engine, WT will only be the default by 3.2
[15:28:12] <StephenLynx> nvm, you probably checked that, you asserted that the mongod is consuming the RAM.
[15:30:03] <dhanasekaran> I already stopped application. application running different machine, I need compression , that's why moved to WT
[15:33:28] <StephenLynx> I would make a test to see if it consumes as much RAM with the default engine, at least.
[15:50:33] <bdiu> I'm using the aggregation framework to sum 15 or so fields on ~ 2 million records... the performance here seems slow 30 seconds + per query. Anyone interested in providing some suggestions or advice for this type of reporting?
[15:51:38] <bdiu> I'm not sure if Mongo is the appropriate solution to our problem... I'm considering alternatives such as a Postgres (with or without the foreign data source connector) or elasticsearch
[16:00:55] <bdiu> @StephenLynx here's an example of one use: https://gist.github.com/bendalton/e7b5a7a781764cfc291b
[16:03:52] <bdiu> a byproduct of the code generating the query... not intended. good eye... also the text of that query changed the date format to a string... so not, in itself, a valid query
[16:10:47] <bdiu> yes.. the double match is unintended... should be combined into one.. testing the impact of that now
[16:13:30] <StephenLynx> it would add an additional update when you add a log, but since you got so many logs, it would be much more effective when reading.
[16:13:56] <StephenLynx> or you could just fetch this data from a single user at time based on user-demand.
[16:14:24] <bdiu> okay.. glad to know you've got the same instinct. the trick here is that the selection criteria for the logs is variable i.e., the $match portion here changes based on external, dynamic critera
[16:18:09] <bdiu> so, triplogs has properties mode and dateTime, we need to be able to basically say, I want user totals of triplogs between dates X and Y, where triplog.mode is $in:['mode1','mode2']...
[16:20:31] <bdiu> though not the full 2mm documents, frequently the resulting count from the match is 1 million + records...
[16:21:01] <dhanasekaran> StephenLynx: Have inserted data to MMAPv1 still holding memory, not releasing to OS, any idea please guide me
[16:24:19] <StephenLynx> it shouldn't be by no means using more RAM than enough to storage the whole dataset.
[16:24:46] <bdiu> StephenLynx: if you were faced with the requirement of looking at this data, dynamically, for each user on millions of records without a reasonable way to pre-aggregate, would you have an alternative technology choice?
[16:27:13] <ViolentR> thanks bdiu ;-) i have problem to extract data from primaryHardwareType hash i need only names but not values
[16:31:04] <bdiu> the date range, and selection criteria for the triplogs change frequently... it's easy to pre-aggregate some of the common cases, but not all by any stretch. We already have pre-aggregation in place, but can't use it for most of the queries needed for our reporting purposes.. there are a dozen potential values for the modes property and we need to support queries against every combination. so short of pre-aggregating
[16:31:04] <bdiu> every combination (12 factorial, right?) I'm kind of stuck with dynamically aggregating these individual documents
[16:37:58] <StephenLynx> the whole operation took too long, so if one could request enough times per second
[16:38:37] <bdiu> yeah, we pre-aggregate per user on day, month, year and choose the appropriate granularity based on the search, combining them together as best we can... e.g., searching from 1/1/2014-3/13/2015, we'd select the values for all of 2014 + the month values for jan, feb, and finally the 13 day values for march
[16:39:13] <bdiu> it works well where we can use the pre-aggregates but where we cant, we have to resort to scanning the collection, and grouping by user
[16:40:45] <bdiu> back to my former question though... if you HAD to provide the full selectability and 30+sec queries were not reasonable, is there an alternative technology you might turn to?
[16:41:34] <StephenLynx> I don't really know anything else that can both store this much data and provide a fast search through it.
[16:41:58] <StephenLynx> redis can be faster than mongo, unless your dataset is larger than your RAM.
[16:44:51] <bdiu> here's an example where we are using it: https://travelwisetracker.com/#/challenges/551ebeab8f8b81ac6d493473
[16:47:12] <bdiu> in this case, we heavily cache.. and this particular site has a much smaller overall data set...
[19:09:39] <dj3000> i have a query using the morphia java driver, and I want to sort the results on one of the fields. any ideas?
[19:11:09] <cheeser> http://mongodb.github.io/morphia/1.0/javadoc/org/mongodb/morphia/query/Query.html#order(java.lang.String)
[19:16:19] <dj3000> so lets say i sort (order) ascendingly on a numeric field foo. will limit(5) return me the smallest 5 results?
[19:29:16] <deathanchor> think of limit like eating hotdogs.... limit(5) means you can handle the first 5 hotdogs, but anymore and you'll puke.
[19:34:59] <StephenLynx> not really. is more about having the discipline to know you can't handle more than 5 hotdogs and not eating a sixth.
[19:37:14] <jr3> Is there a way to tell mongo to sort an array by some property them remove all elements over a certain index?
[20:01:56] <chrisOlsen> Hi everyone, is it possible to update a document with $set where the value is a further subdocument? e.g. { $set: {"My.Object.Chain": {"SubKey": "SubValue"}}}
[20:49:39] <jr3> cheeser: I have a document that has an array of integers, I want to sort this array then remove any elements in the array after index 10
[21:00:40] <shlant> does secondary for replica set readPreference mean that even if the secondary is down, it won't read from Primary?
[21:02:36] <shlant> it says "In most situations, operations read from http://docs.mongodb.org/manual/reference/glossary/#term-secondary members but if no http://docs.mongodb.org/manual/reference/glossary/#term-secondary members are available, operations read from the http://docs.mongodb.org/manual/reference/glossary/#term-primary."
[21:03:12] <shlant> does that mean that it ONLY reads primary when secondary is unavailable? what is "most situations"?
[21:58:07] <SAIF> There is nothing much in there.. http://pastebin.com/i9P2aBeb and this is the error http://pastebin.com/TsArQrTB
[23:05:07] <fxmulder> so I have 2 sharded replica sets going, I added the second replica set earlier this year and it migrated data to from the first replica set to the second replica set. The first replica set seemed to retain the original records it migrated so we are doing a cleanorphan on it to clean it out.
[23:05:49] <fxmulder> I would expect that the empty space left by the removed orphans would be reused by new entries into the collection but disk space keeps climbing