Log file Viewer
#mongodb logs for Wednesday the 18th of February, 2015
[00:25:01] <BadHorsi1> I'm wondering how to save an IPv4 address (not in string notation) in the mongo shell... So like saving a 4 bytes value...
[00:33:38] <BadHorsie> Like: perl -e 'use NetAddr::IP::Util qw(inet_aton);print inet_aton("192.168.1.1");'|od -c
[00:34:59] <Boomtime> this sounds like a data formatting problem, i think you need to decide how you want the data stored before trying to figure out how to do it
[00:56:25] <MacWinner> i have a sharded cluster with rs1 and rs2.. within rs1, I have a test_db.testcollection which I want to shard.. i've gone through the steps, but for some reason on mongos, if I do db.printShardingStatus(), it shows on rs2 as having chunks.. even though all the data is on rs1.. also, I don't see any entry for chunks on rs1
[01:11:13] <MacWinner> Boomtime, I'm connected via mongoshell to rs1 and I see the data there.. also I ran mongorestore against rs1.. (i had mongodumped from our production database)
[01:11:40] <Boomtime> so you pushed datya direct to a shard and expected the rest of the cluster to somehow know this?
[01:12:10] <Boomtime> if you have a sharded cluster you must do all operations through a mongos or you are on your own
[01:14:03] <MacWinner> Boomtime, but what if I'm converted from a single replica set environment into a sharded cluster?
[01:14:07] <Boomtime> never talk direct to a shard unless you are doing something that relates directly to that shard - replica-set config, for example
[01:14:48] <MacWinner> isn't it possible for the original replica set to have all the data without it being pushed via mongos? (because mongos was not setup initially)
[01:14:53] <Boomtime> in order to upgrade from a single replica-set to sharded cluster, the upgrade necessarily involves adding your existing replica-set as the first shard
[01:16:22] <MacWinner> i think I was skipping a step.. I setup a sharded cluster first, then imported all the data directly into a shard to try to simulate my initial state before setting up a shard key and such..
[01:18:28] <MacWinner> what's the expected behavior if you do the mongorestore through mongos before setting up a sharded collection or shard-key? will it by default put all the data into a single shard?
[01:20:21] <Boomtime> once you shard a collection in a database, it no longer uses the "primary" shard for anything special
[04:28:33] <Jonno_FTW> hi, I'm trying to copy from localhost to a remote host that requires auth, how do I specify the auth?
[04:34:13] <MacWinner> i have a bunch of advertisement images being uploaded.. there sized will usually be below 100kb.. not reaching the 16MB bson limit. Is it still a best practice to store the actual ad image binary data using GridFS?
[04:34:43] <Jonno_FTW> joannac: the sysadmin won't open the firewall on my machine for remote connections to mongod
[04:35:22] <joannac> if you can't connect to the remote mongod from your machine, how would it work in either direction?
[04:35:28] <Boomtime> MacWinner: gridfs is so you can exceed the 16MB limit, if you know you'll stay below that (with certainty) then you have no need of gridfs
[04:35:52] <Jonno_FTW> I can connect to the remote, but the remote can't access my machine because of the firewall
[04:49:19] <Boomtime> Jonno_FTW: let me see if I understand: you can login to the remote MongoDB with a mongo shell and you want to push a database from your local machine over to the remote MongoDB?
[04:56:24] <Boomtime> you are better of using dump, but alternatively, you can copy the db files that comprise the db you want to another directory and run restore against just that directory
[05:05:45] <Jonno_FTW> mongorestore.exe -d mack0242 --host remote.trafficdb --port 27017 --username mack0242 --password mypass --dbpath D:/dump
[05:14:06] <GothAlice> Quick question, since my Google fu is weak tonight: is there a way to $inc and limit to a maximum/minimum? I.e. I'm wanting to implement "trickling pool"-based rate limiting rules where you have a maximum pool size (say, 10), performing the limited action $dec's by one if possible, and every N minutes M elements are added back to the pool, not exceeding the maximum.
[05:17:37] <GothAlice> Want to abuse atomicity to rate limit API calls, signup attempts, even things like "is this username available" calls.
[05:25:02] <GothAlice> Hmm. I guess I could do it, with the overhead of potentially many extra documents, by exploiting upserts. (TTL indexes FTW.)
[05:25:27] <NoOutlet> My only other idea is to perform a second update after each element with $min: http://docs.mongodb.org/manual/reference/operator/update/min/#up._S_min
[05:30:24] <GothAlice> db.limits.update({_id: {event: "auth.available", slice: ISODate(…)}, pool: {$gt: 0}}, {$inc: {pool: -1}, $setOnInsert: {pool: 10}}, {upsert: true}) — the unique constraint will explode if one is found but the pool is empty. This handles a pool that completely refills after each division of time. Which is close.
[05:33:53] <giowong> how do i save my database so that when i push to git my teamamates cna access the collections i inserted?
[05:35:51] <GothAlice> NoOutlet: http://www.shorewall.net/ConnectionRate.html < This is the "algorithm" I'm attempting to implement. 2/min:5 — allow a burst of up to five attempts, adding two attempts back to the pool each minute. The use: if a single source IP attempts to register multiple accounts on my webapp too rapidly they'll be blocked for a short time. (With a friendly message saying "try back in X time".)
[05:36:35] <Boomtime> GothAlice: i feel you could create a rate limiter using an array of dates, updated by $push, with a $slice to limit entries to your rate quantity, the query predicate would include a date that is set back in time by the allowed frequency..
[05:36:37] <GothAlice> giowong: Yeah, don't do that. Source code management systems like Git are designed for text, not large binary blobs. Instead, consider having a central server somewhere hosting the MongoDB data for all to utilize.
[05:37:23] <giowong> i was wonder if there was a way to share the data i inserted so that they wouldn't have to do it on their own
[05:38:11] <Boomtime> i.e to update it must contain a date entry that is old enough to be replaced, the slice operator ensures that one of the old slots is removed and the new one is pushed in
[05:38:24] <GothAlice> giowong: There is. Dump/restore is one approach, another is "fixtures". Basically your code detects that it's running on a clean database and bulk-loads stock data from YAML, mongodump, or some other data source.
[05:39:21] <GothAlice> cheeser: Ah, but hopefully not mongodb on-disk stripes from a live mongod instance?
[05:40:53] <GothAlice> cheeser: :P On the other hand, I *do* perform filesystem snapshot syncs of my 26 TiB Exocortex dataset to offsite backup provider Backblaze. Took three months for the first backup…
[05:41:57] <GothAlice> giowong: An alternative, depending on how much data you have, is to sign up for a trial of something like https://mongolab.com/plans/ — their free tier should be more than sufficient for school work.
[05:42:56] <NoOutlet> You could also use one of 40,000 openly available mongodb servers that are unsecured on the internet.
[05:45:17] <GothAlice> Well, I wouldn't be so conspicuous as to promote random hosts on the 'net to a shard bonnet. Too much interference with normal operation. ;)
[05:48:30] <Boomtime> how about a fixed size array of dates, 5 dates indicating the last time they logged in (or whatever), you find on having a date in the array "old enough" (2 mins?), use the positional operator to update it to current date
[05:49:11] <MacWinner> anyone have a recommendation on a PHP odm? I prefer simple. I see a bunch.. mandango, php-mongo-odm, purekid, phalcon
[05:49:34] <Boomtime> that gets you your burst ability and slow recovery, though i doubt it's exactly the algorithm you linked to (i didn't check)
[05:50:06] <GothAlice> MacWinner: Python and MongoEngine. ;) Sorry, suggesting people switch to a real^H^H^H^H^H different language is a compulsion. :P
[05:50:55] <MacWinner> if I switch, it's going straight to nodejs.. i think end-to-end javascript is the future.
[05:51:31] <GothAlice> MacWinner: https://www.destroyallsoftware.com/talks/wat (Gary Bernhardt's famous lightning talk) and http://youtu.be/et8xNAc2ic8 (WTFJS) may be amusing, or possibly depressing, for you. :)
[05:51:52] <NoOutlet> It would be pretty cool if all these unsecured databases were the inspiration for a paradigm shift in open data and cloud computing. That is a system of intentionally opening up thousands of database servers to the public.
[05:51:52] <MacWinner> Boomtime, i've learned to love javascript.. i use to hate the thought of it.. but have come to appreciate it quite a bit after doing a bunch of angularjs development
[05:51:54] <GothAlice> I consider PHP to be a useful template engine, and JS to be an anti-language. :P
[05:53:50] <GothAlice> NoOutlet: The dubious legality of non-forceful intrusion into others' systems is the only thing stopping me from slurping all of the data from all of those unsecured hosts into my Exocortex for analysis. ¬_¬
[05:55:14] <GothAlice> Imagine the glorious infographic one could create from all that, even just using the _id ObjectIds… a calendar heat map of the time of creation of every record across 40,000 hosts…
[06:00:34] <GothAlice> Some neat things could be made without ever really needing to access confidential data. For example, one could calculate how "green" MongoDB is by getting the collection stats from all 40,000 of 'em. Compare actual data size vs. on-disk stripe size to determine "efficiency". ;)
[09:56:43] <robopuff> Hi guys, what is the proper way to get a calculated value (I've got a basic price, but also in calendar embedded document price can appear - need to combine them and filter through this) and then use it to filter collection, or maybe it's not possible
[11:09:13] <guest9999> hello. how can i figure out which index is being used with .explain() ? its a hash.. and getIndexes() doesnt give me hash for each index
[12:27:39] <esko> Hi guys, i need a rest api for my local mongodb so i can post some stuff with jquery to it. any suggestions?
[14:00:56] <Tug> Hey guys, I'm doing a test in the mongo shell and it seems I can't make the TTL feature work: http://pastebin.com/R9mK9Hee
[15:23:00] <Tug> cheeser, yes :) http://stackoverflow.com/questions/28586271/documents-not-expiring-using-ttl-in-mongodb-2-6-7
[15:26:43] <NoOutlet> I was thinking about your problem. Isn't there a way to pre-allocate a specific size for documents?
[15:27:10] <GothAlice> NoOutlet: Actually, part of my code-on-my-phone-in-bed time last night was spent figuring out his solution.
[15:27:32] <GothAlice> Depending on how MongoDB handles $push with a $limit and $pop on the same field in an update, it might work. I can have it self-clean.
[15:27:48] <cheeser> Tug: sadly it's just json. there's no real schema to it. just values we look for and use. :/
[15:28:47] <oznt> I'm lost with aggregartion ... how can you sum the elements of an array within a document?
[15:29:06] <GothAlice> NoOutlet: One minor point, Boomtime's solution will _require_ a periodic background task to repopulate the lists on a regular schedule.
[15:30:13] <d0x> NoOutlet: That was just one example. Anotherone would be to get the domain out of an (string) url. For that the String functions are not sufficient (yet)
[15:30:17] <GothAlice> http://docs.mongodb.org/v2.6/reference/operator/aggregation/unwind/#pipe._S_unwind and http://docs.mongodb.org/v2.6/reference/operator/aggregation/sum/#grp._S_sum specifically, oznt.
[15:31:04] <GothAlice> NoOutlet: Whereas my slightly less optimal solution self-populates by way of upsert, but can't drain and fill the pools at different speeds.
[15:31:51] <oznt> GothAlice, read those, but they are to complex. I am trying to understand how to sum a simple array inside one. I understand I need to use unwind, but not much further
[15:33:56] <GothAlice> document = {ages: [12, 24, 42]}; use test; db.foo.insert(document); db.foo.aggregate[{$unwind: '$ages'}, {$group: {_id: '$_id', average_age: {$sum: '$ages'}})
[15:34:17] <GothAlice> oznt: Have an example. This is first thing in the morning for me, so there may be errors in that due to a lack of caffeination.
[15:40:06] <NoOutlet> So a cron job for repopulating the lists or is there something more internally mongo that can do it, GothAlice?
[15:40:55] <GothAlice> NoOutlet: It'd have to be cron, or the equivalent. We use a distributed task execution system (basically the private version of marrow.task) that does scheduled one-off and periodic tasks, too.
[15:40:58] <oznt> GothAlice, that is very helpful. Can you concat the elements of an array if they are strings?
[15:41:42] <NoOutlet> d0x, so yes there are times when you can't completely process the data in a $project step. The general advise is to create the data in a schema that works for your application.
[15:41:54] <GothAlice> oznt: There is http://docs.mongodb.org/v2.6/reference/operator/aggregation/concat/, but I don't think it does quite what you're wanting.
[15:42:45] <NoOutlet> So if you need the domain of URLs a lot, store the domain when you create the document.
[15:42:52] <oznt> GothAlice, no it does not ... I am trying to to do the following [ "A", "B", "C"] should be in the end a string with "A,B,C"
[16:06:36] <izolate> how do you stop mongodb completely? i've done the "use admin; db.shutdownServer();", but if I type mongo, i can still connect to it ?
[16:12:27] <izolate> also, when I do "sudo service mongod start" I get a red "[fail]" - pretty vague. any way to get more information on that?
[16:14:49] <izolate> "exception in initAndListen: 10310 Unable to lock file: /data/db/mongod.lock. Is a mongod instance already running?, terminating"
[16:15:14] <izolate> seems to be the recurring problem. i have a persistent mongod process. can't kill it? thanks mongodb
[16:15:43] <GothAlice> It's pretty confident that there's a copy running. If you shut it down it's hardly persistent, eh?
[16:16:50] <izolate> and I'm not even convinced it's running since it doesn't exist in the ps aux list
[16:17:02] <GothAlice> izolate: Step one: enhance your calm. This can be worked out. Step two: cd /proc; grep mongo */cmdline — excluding self/cmdline, are there any matches?
[16:18:24] <GothAlice> Perfect, there's a copy of mongo (or mongod) running as PID 24856. The telnet thing will inform you if it's actually mongod and not something else with 'mongo' in the name.
[16:22:54] <GothAlice> As a note, if you have automation, don't ever expect a db.shutdownServer() to stick. ;)
[16:23:57] <GothAlice> (Also disclaimer: I roll my own cluster automation… and still manage to mix mms in there. ;)
[16:26:14] <izolate> i have another problem if you don't mind (mms related)? I get "Host is unreachable" in the mms dashboard. I thought I could fix it by putting "127.0.0.1 $HOSTNAME" into the hosts file, but no avail. off the top of your head, would you know how to fix this?
[16:28:12] <izolate> yeah definitely. but unsure why. incoming ports are limited, but all ports are exposed for outbound
[17:07:09] <preaction> i've got a collection that has 1GB of data in it, but has allocated 20GB of space. how do I prevent Mongo from doing that?
[17:08:14] <GothAlice> preaction: You can't, not really. It's allocating "stripes" on-disk for data to grow into. You can enable "smallFiles" on the server to reduce the rate at which it grows that space.
[17:09:08] <GothAlice> preaction: See: http://docs.mongodb.org/manual/reference/configuration-options/#storage.smallFiles
[17:10:02] <blizzy> GothAlice, I wasn't asking about that, I was asking if the comments were right about what is happening
[17:11:08] <GothAlice> blizzy: What I'm attempting to convey is that questions like that are most easily tested in an interactive Python shell. Copy and paste that script into one, then you can answer your question by typing "db" or "collection" (without quotes) and pressing enter.
[17:13:56] <GothAlice> Guest22055: Ask your question, don't ask to ask. What do you need help with? :)
[17:14:00] <blizzy> I just got confused between the database and collection. basically, the mongo client can store multiple databases
[17:14:24] <GothAlice> blizzy: *server, not client. The client is just asking the server for access to them.
[17:15:05] <GothAlice> blizzy: And yes. "Database" is analogous to use of that term in other database engines like MySQL. "Collection" is analogous to a table.
[17:16:39] <cheeser> Guest22055: http://docs.mongodb.org/manual/tutorial/install-mongodb-on-red-hat-centos-or-fedora-linux/
[17:17:17] <preaction> it looks like i'm already using --smallfiles, but it still allocates 512MB files (down from 1GB files). is a nightly repairDatabase something I should consider then?
[17:17:57] <GothAlice> blizzy: See: http://docs.mongodb.org/manual/reference/glossary/#term-database and http://docs.mongodb.org/manual/reference/glossary/#term-collection
[17:19:42] <cheeser> Guest22055: they're there: http://downloads-distro.mongodb.org/repo/redhat/os/x86_64/RPMS/
[17:19:47] <GothAlice> preaction: Depends on your data's inflation rate. I.e. do you frequently bulk-add data which is subsequently sparsely deleted? (That'd leave holes that might not be fillable later, requiring the stripes to grow.) Repair may be excessive, but compact might be an idea if that is the case. See: http://docs.mongodb.org/manual/reference/command/compact/
[17:20:01] <GothAlice> NoOutlet: I use the shell a lot. But it's mostly the Python shell, with all my MongoEngine goodies.
[17:20:26] <GothAlice> NoOutlet: This is important since I use MongoEngine's signal/event support to simulate triggers a lot.
[17:21:30] <preaction> GothAlice: the documents are long-lived, but have small bits added to them over time. I explicitly cap the number of bits I allow, so there's a limit to how big the documents get.
[17:22:13] <NoOutlet> I was wondering because I've been using mongo-hacker. It's got some niceties, but I think it's a bit too heavy, buggy, and version-specific.
[17:22:16] <GothAlice> preaction: Well, there's the 16MB hard limit. You may wish to investigate adjusting the document padding size to require moving records, and thus leaving holes, less frequently.
[17:24:38] <preaction> my avgObjSize is 113708, so you're saying if i adjust the padding, it'll leave more space for the updates so it doesn't have to move the document somewhere else, leaving a hole?
[17:25:12] <GothAlice> preaction: Aye. See also: http://blog.pythonisito.com/2012/09/mongodb-schema-design-at-scale.html ("Growing documents" section.)
[18:23:21] <abeard> I’m not sure the best way to ask this, but I’m trying to find out what can be done to optimize a less than query on a single indexed date field. I was expecting it to be a pretty quick query, but on two systems running mongo 2.6.7 it seems to be running suspiciously slowly
[18:32:52] <abeard> I was trying to to use a similar query to age out anything older than 3 weeks ago from the table, but it seems like trying to do a less than query on a single indexed date field is killing me
[18:41:55] <abeard> One think I’m noticing is that if I run the query with explain() indexOnly is set to false
[19:00:41] <medmr> i have found that indexing on a string version of the date instead of an actual date object gives better cardinality
[19:02:04] <medmr> looking up using $lt and $gt on a "YYYY-MM-DD HH:MM:SS" string has performed significantly better than doing it on an actual ISODate field
[19:18:33] <ubique> Hi! Is there any way to get know which member of replicaset produce change, while reading oplog ?
[19:28:58] <GothAlice> ubique: Primaries are the only hosts that can make changes in a replica set, and there can only be one.
[19:30:14] <ubique> GothAlice: ok, I will rephrase question - is there any way to get know which client made change through primary to a replica set ?
[19:30:59] <MacWinner> i wanted to convert my replica set into a sharded cluster using the steps here: http://docs.mongodb.org/manual/tutorial/convert-replica-set-to-replicated-shard-cluster/ I wanted to see if I could do it on my production system without any downtime.. assuming I'm writing data into my existing replica set every 3 seconds. my current data pumper is pointed directly at the replica set.. if I convert it to a shard, will any new data that is directed
[19:31:38] <MacWinner> i was doing some tests yesterday, and it seemed like if the data wasn't written via mongos, then it was not available to the cluster..
[19:32:39] <MacWinner> i'm planning on scheduling downtime.. but just more curious about if it's possible
[19:54:52] <edrocks> do I need to persist any data for an arbiter? ie if I run my arbiter in docker should I still use a volume for its data?
[21:45:43] <edrocks> daveops: I was testing it and It seems to be working fine on the same node even if i delete the container and rerun it without any volumes. I just have to pass in the name of replica set
[22:07:52] <MacWinner> i was reading this post about why fullcontact switched to cassandra: https://www.fullcontact.com/blog/mongo-to-cassandra-migration/ Is there reasoning FUD? I feel like they could have configured read operations to come from secondary nodes?
[22:17:46] <GothAlice> MacWinner: Without having read the article yet (it's processing…) there are a dozen FUD posts for every one with actually valid points.
[22:18:16] <GothAlice> https://blog.serverdensity.com/does-everyone-hate-mongodb/ is an excellent post for helping categorize the typical complaints.
[22:19:20] <MacWinner> i personally love mongo.. I only have a lingering concern in the back of my mind that things may slow down at scale.. we are not near scale yet. i also live in SF where it seems trendy to hate on mongo for no specific reason
[22:21:04] <GothAlice> "One day we realized our ‘data burn’ was on the order of 20GB a day and we had less than 200GB remaining on our SSDs." — Failure of monitoring and automated scaling. "We pushed Mongo onto the best database hardware AWS could provide…" — thinking just throwing hardware at a problem will solve a more fundamental architectural misunderstanding. "We could have done it in parallel, but…" — Admitting sharding was the right
[22:22:00] <fewknow> MacWinner: I have used mongo at scale and seen it actually speed up from scaling. The issue isn't mongo...it is the users of mongo. Not knowing how to design schema or access patterns
[22:22:22] <GothAlice> They also patched problems out instead of rewriting them out (law #50) resulting in duplicated code. FUD = Fear, Uncertainty, and Doubt
[22:23:39] <morenoh149> could someone explain to me. There's some limitation (or not allowed by mongodb) feature regarding one-to-many relationships?
[22:24:06] <fewknow> morenoh149: you shouldn't be using one to many relationships ... or simulating them
[22:24:34] <GothAlice> http://www.javaworld.com/article/2088406/enterprise-java/how-to-screw-up-your-mongodb-schema-design.html
[22:24:40] <salmaksim> If you wanted that you could create a scheme for making connections between things though, right?
[22:25:18] <fewknow> morenoh149: what are the relationships you have? why can't you use nested structure?
[22:25:53] <morenoh149> asking for a friend https://github.com/keystonejs/keystone/issues/1075#issuecomment-74867092
[22:26:27] <GothAlice> Problem: you have a user, and users are associated in some way with invoices. User A has placed 400 orders. You want to query them, so you get the user, pull the list of invoice IDs, then re-query the invoices collection passing in the most goddess-awful query ever imagined. (And $in with 400 elements.) User B has placed 60,000 orders, tries to place another, and the system explodes when it is unable to add the invoice ID to their
[22:26:57] <fewknow> morenoh149: you don't want to have relationships like that in mongo. It would be better to duplicate data than to have relationships
[22:27:21] <salmaksim> lol @ this risk "Risk writing a ranty hate blog that makes you look ignorant or worse"
[22:28:28] <salmaksim> i have to admit that switching from doing sql databases for a long time broke my brain when i started using mongo.
[22:29:21] <fewknow> salmaksim: at least you thought about it...developers just start wirting data because mongo will take the object model they are using in their code
[22:29:38] <fewknow> then months down the road they want to know why it is slow...or they can't do something.....
[22:29:41] <GothAlice> salmaksim: It isn't the easiest transition. That "how-to-screw-up" post covers many of the data modelling points, plus the many SQL vs. Mongo query comparison tables, esp. for aggregation. MongoDB really forces you to think about how you are going to *use* your data, rather than encouraging the fantasy of a "perfect schema".
[22:29:43] <MacWinner> i really fell in love with mongo after I was able to push json documents into it from AngularJS
[22:29:57] <fewknow> before the database would prevent you from messing up...now (with mongno) you can fuck yourself if you don't know what you are doing.
[22:30:41] <fewknow> MacWinner: you should have a data access layer written by a data engineer that you use to write to the database
[22:31:19] <MacWinner> it was a stopgap solution.. i've since come to love it for many other reasons
[22:31:26] <GothAlice> I go a step further. I have an AMVC separation, with API at the front. Controller speaks API and returns a view, API queries model, view consumes the data returned by the API.
[22:31:52] <salmaksim> GothAlice: i saw that it was about just using my data rather than dating it, getting beat up, and rejected by it
[22:32:55] <GothAlice> salmaksim: Indeed. Relational modelling is like an abusive spouse… typically before the invention of divorce due to lock-in. EAV (entity-attribute-value) is a testament to this.
[22:33:36] <GothAlice> Plus the many different models for inheritance. (Multi-table concrete, multi-table sparse, single-table, the dreaded EAV…)
[22:37:23] <fewknow> feeding it from HDFS with an avro schema ... converting to parquet adding timestamp and putting in impala
[22:38:47] <GothAlice> http://cl.ly/image/0t273e2e000f at least my dashboard is looking even more dashboard-y today. ¬_¬ Now to actually wire up those aggregate queries…
[22:40:40] <GothAlice> A breakout by type pie chart thingy will go in the large block of blank space.
[22:42:08] <NoOutlet> I was going to suggest making those three block divs bigger to fill the space, but a pie chart would be nicer.
[22:42:39] <NoOutlet> http://i.kinja-img.com/gawker-media/image/upload/s--rc5NiOhM--/18zqjbdo1l118jpg.jpg
[22:42:44] <GothAlice> It's important to see the breakout of the distribution of use of print vs. pay-per-click, etc.
[22:43:58] <GothAlice> (Also, all numbers from that screenshot are static and fake, FYI, except for the line/area and doughnut charts.)
[22:46:12] <NoOutlet> How can you know when it was print? Special URL on those materials? Or just by deduction (when it isn't from a search page or an ad)?
[22:47:11] <GothAlice> We do end-to-end tracking and link shortening for our job distributions. We can track an intention to apply for a job back to not just the job, but the individual order (in case a job is distributed multiple times).
[22:49:05] <morenoh149> the answer to my question was - multikey indexes can't be done on more than one array field http://docs.mongodb.org/manual/core/index-multikey/#limitations
[22:49:38] <GothAlice> morenoh149: Additionally, $elemMatch can only match within one list per query, AFIK.
[22:51:00] <GothAlice> However, except for certain situations with extremely small lists that don't grow or have bounded growth, storing one:many that way is The Wrong Solution™.
[22:54:37] <GothAlice> NoOutlet: https://gist.github.com/amcgregor/94427eb1a292ff94f35d is an example "record this click" upsert.
[22:57:15] <GothAlice> NoOutlet: And https://gist.github.com/amcgregor/1ca13e5a74b2ac318017 is an example aggregate query w/ complete sample record.