Log file Viewer

[09:31:46] <mongokid> hi everyone

[09:33:36] <mongokid> does anyone know why when I save these 'double' values in a document: 18.555555 & 0.123457

[09:34:01] <mongokid> they show up like this in the dump: 18.555554999999998245 & 0.1234569999999999973

[09:34:03] <mongokid> ?

[15:00:15] <VectorX> is there a discord channel or some other chat which is active ?

[15:36:55] <edrocks> VectorX I don't think so. Do you have any specific issues?

[15:55:02] <VectorX> edrocks, i want to make a db where i store domain names and changes in the ip addresses or cnames, then went on to all dns entries, the idea is to be able to query a domain name and return all domain names that share the same IP or CNAME (one or the other) based on the latest entry by date, having some issues figuring out the best schema for this since i am new to mongodb

[15:58:55] <edrocks> VectorX if you have some example documents and collections already that'd be great

[15:59:10] <edrocks> with mongodb though you really model your data based on your intended queries

[15:59:24] <edrocks> ie if you need to lookup X format the data as X

[15:59:32] <edrocks> (on insert)

[16:03:11] <VectorX> edrocks i havent created anything coz im trying to figure out if this would work

[16:03:29] <edrocks> it will work just depends how fast you need it to be

[16:06:56] <VectorX> edrocks there will be writes happening continuously aswell as reads though the reads wont be as frequent

[16:07:10] <edrocks> then you should be fine

[16:07:21] <edrocks> the issues are usually on read heavy stuff

[16:07:47] <edrocks> writers are likely just a cron job I imagine so who cares how long it takes not that it would take a long time

[16:10:04] <VectorX> the db is a scan of known domain names, and the IP/CNAME latest value added say as an array item if the previous value was different

[16:10:52] <VectorX> the idea is provided input of a domain, a query will find the latest entry for that domain and re-query the collection or collections to find all domains matching that IP/CNAME

[16:11:01] <VectorX> if dont know if that is efficient to do in mongo

[16:11:57] <edrocks> if you are ok with the total doc size being under 16MB you are fine to put previous ips in an array

[16:12:23] <edrocks> otherwise you'd duplicate the domain name and make a doc with {ip, domain}

[16:12:51] <edrocks> keep in mind duplicating data is very common for high perf systems. You trade storage for read/write speeds

[16:13:30] <VectorX> ok but how would that query look like then, say the input is 'google.com'

[16:14:04] <edrocks> db.stuff.find({domain:"google.com"})

[16:14:29] <edrocks> then you'd get a bunch of `[{domain:"google.com", ip:xyz},{..., ip: abc}]

[16:15:51] <VectorX> right but the result is supposed to be all the domain names matching the latest IP of google.com to the latest IP added for each of those

[17:27:49] <VectorX> edrocks not sure if you saw my last message about the desired query

[17:27:58] <VectorX> <VectorX> right but the result is supposed to be all the domain names matching the latest IP of google.com to the latest IP added for each of those

[17:28:07] <edrocks> VectorX No, I missed it went off to lunch sorry

[17:31:20] <edrocks> I think you need two queries then

[17:31:29] <edrocks> or maybe try with the aggregation framework

[17:32:10] <VectorX> right but if writes are going on, wouldnt that be inneficient

[17:32:45] <edrocks> I mean it

[17:32:54] <edrocks> it's either multiple queries or build that data when inserting

[17:33:17] <edrocks> which you can do it's just more work but you would get much better perf

[17:33:41] <edrocks> I'd just do the multiple queries or whatever is easiest to implement then improve perf if needed. Better to have something working then nothing

[17:33:52] <VectorX> ok

[17:34:02] <edrocks> especially when you are so new to mongodb. Not knowing how to implement the simple option before trying the more advanced ones is a recipe for failure

[17:34:19] <VectorX> that is not a good recipe

[17:34:32] <edrocks> You might not use that initial implementation but it will teach you way more than reading any docs or talking to someone will ever do

[17:35:19] <edrocks> but yea the perf aware option is to do fancy writes that probably write to an input/preprocessing collection which you'd use to build up your optimized collection that you'd read from

[17:35:34] <edrocks> that's the basic idea

[17:36:20] <edrocks> You do crazy formatting so that when you do a query the db has your data exactly as your client needs it. That's the end all be all for read perf

[17:36:25] <VectorX> so when you say 2 queries, that is first query returns the google.com doc to the application, then the app queries the db again for that ip and records with max added date sort of thing ?

[17:36:35] <edrocks> yep

[17:36:47] <VectorX> ok

[17:37:12] <edrocks> the optimized version would basically be caching/pre-generating that output into another collection which you'd then read from instead when processing queries from your client

[17:37:59] <VectorX> that would be a lot of storage which i dont think would be possible

[17:38:20] <edrocks> that's the tradeoff you make for better read perf

[17:38:39] <edrocks> but that's like way over optimized unless you have super high traffic

[17:39:05] <edrocks> basically you won't need unless you commonly have 100+ queries to run instead of 2

[21:29:39] <MaxLeiter> has anyone ever (using mongoose) had a populate() that returns an empty array/object?

[21:30:03] <MaxLeiter> im not doing anything differently than my programs 20 other findOne(...).populate().then()s

[21:30:19] <MaxLeiter> i can tell from database in compass that the array does have objectids in it

[23:21:48] <MaxLeiter> to add to the above mystery, populating a different field works fine /: