Log file Viewer

[00:35:13] <tchaypo> morninge

[00:35:26] <tchaypo> did I miss a conversation with dstufft overnight? I see you talking to him but I’m not sure what you were responding to

[00:54:34] <lifeless> tchaypo: discussing the thing we were discussing

[00:55:04] <tchaypo> restart irccloud and I see it now

[00:59:59] <tchaypo> I will add “look at wheels” to my list of things to do

[01:00:07] <tchaypo> working with Nakato on other things this morning

[01:04:14] <lifeless> kk

[04:55:35] <tchaypo> okay

[04:56:18] <tchaypo> > python setup.py bdist_wheel

[04:56:18] <tchaypo> error in setup command: 'install_requires' must be a string or list of strings containing valid project/version requirement specifiers; Expected ',' or end-of-list in setuptools>=17; python_version == '2.4' at ; python_version == '2.4'

[05:43:24] <tchaypo> so dstufft said that “wheels already support environment markers"

[05:43:44] <tchaypo> but I can’t build a wheel that has a marker in install_requires

[05:47:07] <tchaypo> pip wheel -r will read env markers in requirements.txt

[05:48:34] <lifeless> tchaypo: you can't? how have you been trying ?

[05:48:47] <lifeless> tchaypo: I think you've forgotten the extras indirection shim.

[05:49:06] <lifeless> tchaypo: clone 'mock' from github and run 'python setup.py bdist_wheel', that will get you a wheel with markers.

[05:51:25] <tchaypo> that’d be https://github.com/pytest-dev/pytest-mock/ ?

[05:51:36] <tchaypo> nope

[05:53:22] <tchaypo> yep, that works - because it uses pbr, which reads a requirements.txt and creates extras (ie, the shim you mentioned)

[05:55:58] <tchaypo> and I see the extras in the metadata

[05:55:58] <tchaypo> Requires-Dist: jinja2 (<2.7); (python_version<"3.3" and python_version>="3") and extra == 'docs'

[06:07:03] <tchaypo> but still - bdist_wheel works for mock because mock has pbr has a setup_requires on pbr; if you put an env marker directly in the install_requires it can’t parse. installing the wheel works because the env markers are in a completely different format

[06:09:17] <lifeless> I think you're conflating user input and system output

[06:09:59] <lifeless> the user input we want to be able to do is install_requires=thing-with-markers

[06:10:21] <lifeless> the system output you pasted seems reasonable

[06:10:33] <lifeless> but your test is testing a changed input, not the behaviour of wheel

[06:10:41] <lifeless> / wheels

[06:16:02] <tchaypo> as dstufft said, wheels seem to be fine; if you can get the markers into the wheel, you can install the wheel fine on today’s setuptools

[06:16:31] <tchaypo> so I don’t think I’m interested in testing the behaviour of wheel?

[06:19:39] <tchaypo> I think https://github.com/jamezpolley/setuptools-380-testpackage/blob/install-requires/setup.py is an example of what we’re trying to enable?

[06:23:04] <lifeless> sure

[06:23:40] <lifeless> the point of wheels was to check that we don't break wheels vs their state today based on /how/ we enable that sort of setup.py

[06:45:54] <tchaypo> The metadata in the wheel is nice and clean and abstracted from our problem. Anything that can install a wheel today can already deal with markers, in install_requires at least.

[06:47:01] <lifeless> as long as the output stays the same :)

[06:47:26] <tchaypo> Building a wheel without using the extras shim will need a newer version of setup tools but the resulting wheel will work with older versions of setup tools

[06:57:37] <lifeless> tchaypo: assuming we don't change the output format. We should obviously try not to do that

[07:01:19] <lifeless> but

[07:03:40] <lifeless> dstufft: does anything generate pep-453 requires-external: entries today?

[12:06:17] <dstufft> lifeless: distutils2 which is still available on PyPI to be used

[12:06:24] <dstufft> dunno about the shim projects

[18:39:35] <buck1_> for your consideration https://github.com/asottile/virtualenv-hax/blob/master/virtualenv_hax.py

[19:23:51] <lifeless> dstufft: that only generates it when folk use it... so the question is does anyone use that feature.

[19:24:27] <dstufft> lifeless: to be clear, I'm just saying the generator exists. I don't think hardly anyone, if anyone, actually uses it

[19:24:42] <dstufft> if I pulled up the DB I could tell you for a fact, but I'm not able to do that right this second

[19:26:15] <lifeless> kk

[19:26:21] <lifeless> this is planning for pep 497

[19:31:27] <lifeless> dstufft: commented on https://github.com/pypa/pip/issues/3025

[19:33:08] <dstufft> lifeless: I did not notice it had a universal

[19:33:49] <dstufft> I think the problem still stands though, not for mako specifically, but like... if you have something like if python_2_6: install_requires += ["foo

[19:34:01] <dstufft> by default wheel is going to cache that for all py2

[19:35:01] <lifeless> its worse though

[19:35:13] <lifeless> some packages change install_requires based on what setup_requires sees

[19:35:57] <lifeless> https://bitbucket.org/runeh/anyjson/src/0026a68c035696bdc8db8628e364139eba9a8ba8/setup.py?at=default#setup.py-57

[19:36:19] <dstufft> yea

[19:36:28] <dstufft> there's some stuff we can't protect against at all

[19:36:28] <lifeless> any wheel built of anyjson in a hermetic environment will depend on simplejson

[19:36:49] <lifeless> so, a change to bdist_wheel so that the default is python minor version specific

[19:37:01] <lifeless> would be how I'd address this

[19:37:06] <lifeless> its the right layer

[19:38:57] <dstufft> it used to be that way, and it switched to py2 or py3 because for most projects it's the right default :/ Another problem is that it lets minor versions be used for any latter versions, like a py26 wheel can satisify a py27 wheel. It might be better to just pass --python-tag cp26 (etc for each version)

[19:39:22] <dstufft> cp26 doesn't have the same behavior as py26 where Python 2.7 will accept it

[19:39:28] <lifeless> isn't that something wheel could still default to ?

[19:40:30] <lifeless> so my concern here is user surprise

[19:40:43] <lifeless> 'pip built a wheel differently to doing setup.py bdist_wheel'

[19:41:03] <lifeless> -> how do I reproduce what pip does?

[19:41:07] <lifeless> -> pip is whats buggy

[19:41:08] <lifeless> etc

[19:42:10] <dstufft> well, the flipside is people are already thinking pip is being buggy, I was talking to the reporter in another IRC channel and he was really confused why mako was trying to install something on Python 3.2 that it shouldn't be. I agree that mako's --universal flag is wrong, but the same probably would have happened without if if mako had been installed on Python 3.4 first

[19:42:32] <dstufft> problem, not probably

[19:42:40] <lifeless> we could publish a blacklist on pypi

[19:42:46] <lifeless> of known packages with quirks

[19:43:37] <lifeless> first report of problem with X -> update the blacklist. When fixed (if fixed!), put an upper bound on the blacklist

[19:46:48] <lifeless> OTOH the problem is in the wild now with this version of pip and will be forever :)

[19:52:29] <dstufft> lifeless: I think producing a wheel that is more specific than needed is a better solution than a blacklist on PyPI (though I discovered a bug in pip's handling of cp26 and such). It's true that this version of pip will never cange, but people tend to upgrade pretty quickly. Importantly I don't think there is much of a downside to passing ``--python-tag cp27`` and such when building a cache wheel, only thing I can think of is that ``pip

[19:52:29] <dstufft> wheel`` will produce a different wheel... which I don't think is that big of a deal, and in some cases we'll have 2+ wheels when we could have gotten away with one.

[19:52:46] <dstufft> anyways, gotta go pick up some food :D

[21:44:31] <tchaypo> okay

[21:45:06] <tchaypo> lifeless: my schedule for today has a block of time this morning to deal with feedback on pep-496 from last weekend, and then write up what we’ve been talking about re install_requires

[21:45:29] <tchaypo> but first I need to finish breafast etc. back in 30 minutes or so

[21:46:34] <lifeless> cool