[14:41:10] <jessamynsmith> I have an idea for pip, and I'm wondering if anyone would like to talk it through with me?
[15:52:31] <carljm> jessamynsmith: I'm not actively working on pip these days, so my buy-in doesn't mean a lot, but I at least have "historical perspective" :-) What's your idea?
[15:53:05] <jessamynsmith> historical perspective is definitely helpful
[15:53:34] <jessamynsmith> I routinely work on apps that have test requirements I don't need or want in production
[15:53:45] <jessamynsmith> As far as I know, pip has nothing built in to support environments
[15:53:51] <jessamynsmith> (let me know if I'm wrong on this)
[15:54:26] <carljm> not exactly, no. though the ability to include one reqs file from another is useful in that situation
[15:54:31] <jessamynsmith> I actually built myself a little tool because I was tired of manually sorting new requirements into files
[15:54:38] <jessamynsmith> yeah, I use multiple files with includes
[15:54:50] <carljm> yes, that's how I handle that as well
[15:54:57] <jessamynsmith> the main case that isn't handled nicely is I add a new requirement, and I want to auto-add it
[15:55:18] <carljm> yeah, like `npm install --save` and `npm install --save-dev`
[15:55:21] <jessamynsmith> or,more often, I add some new lib that adds 4 other libs, and then I was manually cutting those out of the output of pip freeze
[15:56:44] <carljm> yeah, I definitely think there is room in the Python world for a tool (probably built on top of pip) that does some of what you're talking about
[15:58:19] <jessamynsmith> I hadn't even thought of going that direction
[15:59:11] <jessamynsmith> my current thinking is that cleaning up a virtualenv to match a set of requirements should live in virtualenvwrapper -- I'm going to see about getting a pr accepted
[15:59:19] <jessamynsmith> upgrading all is achievable with just pip
[15:59:25] <jessamynsmith> (I didn't realize at the time)
[15:59:37] <jessamynsmith> the big thing is splitting up and maintaining requirements files
[16:04:21] <jessamynsmith> it's so helpful to talk things over with someone else who gets the problem
[16:04:33] <carljm> sure. thanks for working on this
[16:04:40] <jessamynsmith> so far this all just came out of my own head, and I felt it was unnecessarily complex
[16:04:56] <carljm> you might also be interested in Mozilla's "peep" tool, which solves a very different problem, but is still in the category of "wrappers around pip"
[16:05:04] <jessamynsmith> I experiment with packages a lot
[16:05:15] <jessamynsmith> and I found virtualenv/requirements maintenance was taking too much time
[16:05:47] <jessamynsmith> I'm going to see about implementing this
[16:05:59] <carljm> yeah, for me it's always in that category of "I have ideas about what a better tool would look like, but the current tooling works just well enough that it never makes it to the top of the list"
[16:06:16] <jessamynsmith> maybe I install a lot of packages?
[16:06:31] <jessamynsmith> for me that line moved past the pain point
[16:07:14] <carljm> maybe. or maybe I've been doing this too long and my pain tolerance has gotten too high :)
[16:07:59] <jessamynsmith> I still think it would be great if pip eventually pushed people toward some standard naming
[16:08:14] <jessamynsmith> but for now, requirements files have so many names
[16:08:44] <carljm> funny story about npm install --save; the person on my team who is responsible for package.json wants the dependencies alphabetical, and npm doesn't do that, so we still have to go in and manually edit package.json anyway :-)
[16:08:52] <jessamynsmith> how do you organize your projects? requirements/env.txt or do you have them all top-level
[16:08:58] <carljm> there's always that one more step that the tooling doesn't quite handle automatically...
[16:09:13] <jessamynsmith> yeah, so, when pipreq divides them it makes it alphabetical
[16:12:08] <carljm> the one remaining problem is that gunicorn doesn't handle pdb gracefully, because of the multi-process setup and all
[16:12:10] <jessamynsmith> so yeah, that's a typical one
[16:12:30] <carljm> usually I don't need to pdb on an actual request, just in a test. but every now and then I fire up runserver so I can stick a pdb in a live request
[16:12:31] <jessamynsmith> I use pycharm which "just knows" how to run the django server, so I use that for debugging
[16:12:40] <jessamynsmith> oh, I debug the live server all the time
[16:13:40] <doismellburning> I've basically never used pdb
[16:13:56] <jessamynsmith> carljm: so you just export local vars to make things work like heroku?
[16:14:08] <doismellburning> I do need to work out how to get LiveServerTestCase using gunicorn though
[16:14:14] <doismellburning> jessamynsmith: fwiw, I use honcho
[16:14:16] <carljm> jessamynsmith: yeah, I've adapted the 12-factor "config via env vars" everywhere
[16:14:22] <carljm> and I also use honcho and a .env file
[16:14:33] <jessamynsmith> carljm: I still haven't read the 12-factor thing, but I do tend to do that
[16:14:36] <jessamynsmith> I ahve never heard of honcho
[16:14:40] <carljm> (though usually I aim to have my "default" settings - i.e. if there are no env vars - as close as possible to my desired dev config)
[16:14:56] <carljm> so generally on my projects you can fire them up for local dev with no env vars at all and it'll be reasonable
[16:15:09] <jessamynsmith> yeah, I learned that the hard way...
[16:15:25] <jessamynsmith> I have my django secret key in an env var, but then people can't just clone my repo and run without setting an env var
[16:15:28] <jessamynsmith> so now it defaults in dev
[16:15:45] <doismellburning> jessamynsmith: well hello ;)
[16:16:26] <carljm> what I do to avoid mistakes in prod is I have a MODE setting
[16:16:26] <jessamynsmith> I will look up 12factor and honcho
[16:16:29] <doismellburning> I still might change that behaviour in v2
[16:16:35] <carljm> which can be either 'dev' or 'prod', defaults to 'dev'
[16:16:43] <jessamynsmith> when I get back to django
[16:16:57] <carljm> but if it's set to 'prod' (via an env var of course), then a bunch of other settings (including SECRET_KEY) error out if not set explicitly via env var
[16:18:15] <jessamynsmith> doismellburning: that'd be amazing
[16:18:50] <carljm> doismellburning: I avoid using DEBUG as a proxy for dev vs prod, because DEBUG changes Django's behavior, and sometimes I want to test DEBUG=False locally without having to go into "full prod" mode on all my settings
[16:22:13] <carljm> doismellburning: nope, not your fault. but it does appear that either you were watching the discussion all along, or you have an IRC highlight set up for the word "12-factor" :-)
[16:22:29] <carljm> mention 12-factor and doismellburning appears, like a genie from a bottle.
[16:22:46] <jessamynsmith> like nedbat and coverage
[16:22:59] <carljm> jessamynsmith: now you've highlighted him twice!