Log file Viewer
#pypa-dev logs for Friday the 3rd of July, 2015
[00:54:44] <tchaypo> i do love simple bugs with precise reproduction instructions and a clear description of the problem
[01:00:31] <lifeless> tchaypo: I think its important in the ecosystem. OpenStack isn't directly affected.
[03:00:45] <tchaypo> if I’m reading it correctly there’s two ways we could tackle this (without a major rewrite); one is right at the top of the code that does the marker evaluation, the other is down in the guts of the code that does the comparisons
[03:02:45] <tchaypo> if we do the latter we need two implementations (one in markerlib, one in pkg_resources) and both implementations currently don’t care what kind of marker they’re comparing since it’s all just a string comparison - we’d need to add special handling for anything where the name of the thing being compared is _version
[03:04:05] <tchaypo> the former sounds easier for simple cases where we just have one python_version in the marker; but is more complicated when we have to handle “python_version … or python_version …”, especially since it’s possible for the marker to have parenthetical groupings and stuff as well.
[03:33:56] <dstufft> if we split out the markers into their own PEP, we can just add a whole marker implementation into packaging
[03:40:56] <lifeless> Phrasing it differently, why does pkg_resources have a separate implementation at all at the moment ?
[03:42:35] <dstufft> given a PEP though, I think it makes sense to put the marker impl into packaging, since you'll need packaging anyways to get PEP 440 versions
[03:56:31] <dstufft> lifeless: I'd rather not put dependencies in packaging if possible since it's designed to be bundled. Also markerlib is PEP 345 (aka before wheels and such) and hasn't had a commit or a release in > 2 years. The code itself is just over a 100 lines and afaik setuptools is the only thing that really uses it (and that bundles it). I don't think there's a lot of value to keep the lib seperate here
[03:59:36] <dstufft> lifeless: re: heroku, last I heard was me giving $$ details and the email address. Nothing since then other than an auto reply from one of the names that they are on vacation till monday, so I figure there's a good chance the other person is on vacation too.
[04:01:19] <dstufft> lifeless: yea, you were CC'd in my last email on 6/29, the auto response was from uh.. Matt I think.
[04:02:07] <dstufft> will do, with it being all holiday and stuff I figure I won't start wondering till end of next week or so
[04:09:03] <tchaypo> lifeless: one implementation “uses the ‘parser’ module which is not implemented on Jython and has been superseded by the 'ast' module in Python 2.6 and later."
[04:14:24] <tchaypo> that’s the non-markerlib version. If I’m reading https://bitbucket.org/pypa/setuptools/src/d0d95997b068d46699d00adca960008e2488f3d6/pkg_resources/__init__.py?at=default#cl-1541 correctly, the markerlib version is *only* used if ‘parser’ is not in globals()
[04:27:03] <tchaypo> also, I think anything we do is going to be using pkg_resources.parse_version() which aiui already implements PEP440; it seems weird to have pkg_resources call markerlib call pkg_resources
[04:42:30] <tchaypo> so in summary, it sounds like we want to pull out https://www.python.org/dev/peps/pep-0426/#environment-markers and use it as the basis for a new PEP; which will also contain an overview of how we’re going to replace the two current implementations (in pkg_resources and markerlib) with a single implementation (that doesn’t use parser, if we want to
[04:42:31] <tchaypo> support 2.6 and jython; or that doesn’t support 2.6 or jython if we want to use parser, since it seems to be the default and most-up-to-date implementation at present)
[04:49:33] <dstufft> tbh the marker language is probably simpler enough to parse it with regex, but idc if it uses ast or whatever
[04:51:44] <tchaypo> *thinks* would we be talking about a complete replacement for markerlib, or would we only need to implement a subset of markerlib?
[04:53:44] <dstufft> it'd be a replacement for markerlib inside of packaging, but markerlib is only like 3 functions anyways
[04:57:14] <tchaypo> okay, markerlib implements the env marker spec from pep 345; what we’re talking about now is implementing the spec from pep440 (possibly pulled out to its own pep); so it seems like we need to at least heavily update markerlib
[04:59:37] <dstufft> PEP 426, but yea. Ideally what we'd do is split that out into it's own PEP that just defines the marker syntax without defning what it looks like in a whole requirement specifier (similarly to how PEP 440 defines >= without telling you how to specify _what_ project you want to install), then packaging would get a PR that implements markers as a Proof of Concept that will get worked on and merged when the PEP gets accepted.
[05:00:02] <dstufft> there's no backwards compatability requirements in that PR since it's a new lib, it's free to make the best choices
[05:02:35] <dstufft> I'm not sure if it can use AST actually, since if we're using PEP 440 specifiers we have ~= and === too
[05:07:27] <dstufft> I can add you to it if you want to work on one, or you can do it yourself in your own locations
[05:10:27] <dstufft> API wise, I think something closer to RequirementSet or Version would be nice, a class that offers introspection of the marker to see individual pieces of it and such (much how RequirementSpecifier gives you access to the version part and the operator part) with the ablity to evaluate that class against the current environment, or override parts of the current environment
[05:36:16] <lifeless> dstufft: we have a request that RequirementSet __str__ show in the order a human might write it
[05:37:44] <dstufft> the only implications is that RequirementSet(str(RequirementSet(...)) should be equivilant
[05:56:41] <dstufft> hg.python.org/peps/ is the official location, most of the packaging ones live in interopatbility-peps while we're working on them and then we periodically copy them into the hg.p.o repo
[05:56:58] <dstufft> but you can work on it wherever you want and then submit them to hg.p.o periodically
[06:06:03] <dstufft> tchaypo: works for me, do you want access to the repo or just want to submit PRs