Log file Viewer
#pypa-dev logs for Tuesday the 9th of June, 2015
[07:20:53] <tomprince> lifeless: Nothing very interesting. We use a version of versioneer that doesn't support PEP440 and we have tooling that parses the version.
[07:22:07] <tomprince> The non-matching version comes from new setuptools/pip using === for non-pep440 versions and/org old setuptools mangling versions that don't match its idea of what a version can be.
[07:53:09] <lifeless> tomprince: so I've put a patch up; it works in tests but be glad to know it works in practice :)
[08:21:34] <anthony25> for my project, I'm using a little program coded in C to interact with iptables where I do a setuid(0) (because I don't want it to run as root), so I have to add the +s flag to the executable
[08:22:14] <anthony25> or if you think about another solution about how could I do that outside the setup.py
[08:51:29] <lifeless> or you could run as root and drop privileges immediately, forking off a process to do stuff you need to do as root
[17:06:09] <maxking> dstufft: I am on debian sid, following this guide http://docs.docker.com/installation/debian/
[17:09:03] <dstufft> I think just docker-compose build && docker-compose up in the root of the Warehouse repo will give you warehouse running then
[17:09:26] <maxking> dstufft: I have a slow connection, It will take some time to get everything on my machine running.
[17:10:27] <dstufft> https://github.com/pypa/warehouse/issues?q=is%3Aopen+is%3Aissue+label%3Aeasy there's a few things I tagged easy
[17:12:30] <dstufft> https://github.com/pypa/warehouse/milestones/Become%20PyPI is everything we need to deploy to pypi.python.org fwiw
[17:14:04] <maxking> dstufft: Are there any other people actively developing warehouse apart from you?
[17:14:53] <dstufft> I am the main contributor, you can see who has commits landed here https://github.com/pypa/warehouse/graphs/contributors (or use the drop down to use any other metric that suits you)
[17:36:05] <hargup> Hell dstufft, maxking I'm an intern at Continuum Analytics and I'm working on automatically creating conda packages from PyPI
[18:29:27] <lifeless> dstufft: hai; I'd like a) a gist with what you want to make pbr compatible with pip's needs - e.g. 'implementation non-specific make X happen and I'll seriously consider using pbr'
[18:30:01] <lifeless> dstufft: secondly, I'd like to make a supported interface for parsing requirements files
[18:30:45] <lifeless> dstufft: I think the current one is stable-enough (req.req_file.parse_requirements) - what do you think? [we parse requirements files in some openstack meta-tooling, and do it badly]
[18:32:31] <sigmavirus24> lifeless: IMO, if we need something to parse requirements it should be pulled out into a separate library
[18:33:07] <lifeless> sigmavirus24: pip defined and owns this, and we're fine with that - we're not trying to define it as a new external thing
[18:33:38] <lifeless> sigmavirus24: dstufft has previously said he's fine defining stable bits in the library when folk ask for it
[18:35:05] <lifeless> I think putting InstallRequirement into a separate library would be high friction for low benefit
[18:35:42] <sigmavirus24> I think there's a lot of benefit: Namely a backwards compatibility guarantee
[18:36:12] <sigmavirus24> I know you're asking for that here, but I feel weird about pip being forced to export stable APIs when it may need to change them
[18:36:21] <lifeless> how would that add such a guarantee? it would be still subject to making changes, and it would be vendored into pip so it could easily do api breaks without affecting pip
[18:37:16] <lifeless> and - pip does major releases too, its not like I'm asking pip to never break it, just to be aware that there are users for that api and please be a litle bit gentle around its evolution
[18:38:02] <dstufft> I don't think we can make the parser a stable public API as it is without also making PipSession a stable API
[18:39:21] <sigmavirus24> lifeless: except the reaction to pip's major releases while overwhelmingly positive have had some very loud people (including people in OpenStack) complaining even though they are major releases
[18:39:54] <sigmavirus24> So there are some (artificial) expectations around pip that would probably be even stricter if pip started exporting APIs
[18:41:48] <lifeless> I'd like to call parse_requirements('global-requirements.txt') rather than what we currently do that is broken.
[18:42:54] <lifeless> I'd rather tweak high level code than be in the business of chasing a parser with no formal definition in a separate codebase
[18:43:13] <lifeless> I'd like the pip code to put in mild backwards-compat effort on changes to the signature of parse_requirements
[18:44:00] <lifeless> ^- bundle that up with whatever labels-for-the-outside; it seems a reasonable request to me. If its not, thats ok.
[18:47:49] <dstufft> lifeless: I don't have a problem with the concept of creating a stable API for parsing requirements files, but the problem I see is that the API as it is right now is that exposing parse_requirements as a stable API requires exposing a bunch of other things too because it has a bunch of layering violations (as does most of pip)
[18:48:16] <dstufft> parse_requirements takes a finder as an input, a session, an options dictionary, a wheel cache thing
[18:53:24] <dstufft> it "works" in that it doesn't raise an exception, but if you don't pass in a finder it just flat out ignores a bunch of stuff from the requirements files (--find-links, --index-url, --allow-unverified, --allow-external, --binary, maybe others), if you don't pass in the others it ignores other things
[18:54:43] <lifeless> http://logs.openstack.org/50/189250/1/check/check-requirements-integration-dsvm/e53b7f4/logs/devstacklog.txt.gz#_2015-06-08_10_37_25_988
[18:58:08] <dstufft> lifeless: I think the best thing to do is open an issue about stablizing that API. I know tox was looking to use the same API
[19:01:46] <dstufft> I don't want to expose an API that you need to use other, non exposed parts of pip to do useful things with that people want to do. So it's likely we'll need to refactor the API to remove the layering violations (likely make it so it takes a filename (or a string) and returns the parsed information and have the caller responsible for mutating finders and sessions and such instead of having the parser do it
[19:03:52] <dstufft> the other question is whether we want to expose parts of pip as an API at all or not. So far we've avoided that question (more or less) by splitting parts out of pip into their own libraries (via packaging). It'll effect how we do vendoring (there's problems with our current solution where it fails in certain situations, but we don't run into those situations very often because we don't support being called as a library)
[19:04:16] <dstufft> requests stole our vendoring shenagins for a version or two, and had to ultimately rip it out because it just broke too much
[19:06:06] <dstufft> the goal of our old shenagins was to remove the need (or reduce it) to have random downstreams patch our sources
[19:20:08] <tomprince> lifeless: I think a good long-term result would be for pip to become a shallow wrapper around other libraries.
[19:22:47] <tomprince> In that scenario, pip probably wouldn't ever grow stable APIs, but all the APIs people care about would be come stable apis of the underlying libraries.
[19:25:51] <lifeless> so I can see if I can come up with a design for them that doesn't squick me :)
[19:29:28] <dstufft> lifeless: I'll write something up later today that has what I want in a system and what the things pbr does or doesn't do that holds me back from using it
[19:29:46] <dstufft> maxking: we don't have sign up working yet, you can login with dstufft // password though
[19:37:50] <maxking> Documentation says we can get warehouse shell access using 'warehouse shell', but that probably needs to be done inside the container?
[19:38:17] <dstufft> maxking: you can't get shell access to the container that's already running, but you can spin up a second copy of the web container running bash instead of warehouse by doing docker-compose run web bash
[19:40:45] <dstufft> the praticial effects of this, is that you can't change files around in the running container, (but the current directory is mounted as /app/, so you can make changes to warehouse outside of the container and have thems how up inside)
[19:43:29] <dstufft> I don't know docker super well, I was just trying out docker-compose as an easy way for people to run warehouse
[19:52:37] <maxking> dstufft: Okay, its permission error. Probably I ran something with sudo and thus permissions have changed.
[19:54:27] <maxking> dstufft: Sorry for all the bugging, and thanks a lot for all your patient answers.
[22:48:48] <lifeless> dstufft: http://docs.travis-ci.com/user/languages/python/#Nightly-build-support
[23:03:52] <lifeless> dstufft: ^ [only nagging on this one cause I think it will mitigate the 'oh noes we have to use 6.1.1' hysteria
[23:28:34] <lifeless> dstufft: does https://github.com/pypa/pip/issues/2867 close automatically, or need manual poking ?
[23:29:30] <dstufft> it needs manual closing, it'll only close automatically if you include something like ``Closes #2867`` in the PR body before it gets merged