Log file Viewer

[01:58:26] <lifeless> dstufft: and ping?

[01:58:32] <lifeless> StevenK: url me up

[01:59:37] <StevenK> lifeless: https://github.com/pypa/packaging/pull/49 for just marker support or https://github.com/pypa/packaging/pull/50 for markers + requirements parsing

[01:59:52] <lifeless> StevenK: thanks

[19:50:11] <lifeless> dstufft: ping

[20:09:55] <ErikRose> What's everybody's take on conditional install_requires entries, i.e. `if sys.version_info<(2, 7, 9): install_requires.append('ndg-httpsclient')`?

[20:10:10] <ErikRose> We're having some trouble in Let's Encrypt with those + wheels.

[20:10:31] <ErikRose> Wheels of course don't get the opportunity to run such setup.py logic.

[20:11:00] <ErikRose> Nor do they have sufficient granularity to offer separate wheels for 2.7.8 and 2.7.9.

[20:11:23] <ErikRose> What's the best approach? Unconditional dependencies + runtime import twiddling?

[20:12:01] <ErikRose> Environment markers are another possibility,

[20:12:48] <ErikRose> but they got range operators only in setuptools 17.1, released 6 months ago.

[20:14:07] <ErikRose> I suppose, for the < case, we could just use multiple == conditions: 2.6.0, 2.6.1, 2.6.2, ..., 2.7.8.

[20:14:16] <ErikRose> That's supported as far back as setuptools 0.7,

[20:15:21] <ErikRose> Hmm, why is setuptools 0.7.x not on PyPI?

[20:15:32] <ErikRose> It goes right from 0.6 to 0.9.8.

[20:15:46] <ErikRose> nm, 0.7.2.

[20:17:08] <ErikRose> Looks like mid-2013 for setuptools 0.7.

[20:20:43] <dstufft> lifeless: hi, sorry. Been back and forth in email and such trying to arrange some stuff and it's just sucked up a bunch of time :(

[20:21:25] <dstufft> ErikRose: I Think environment markers support < don't they?

[20:21:40] <ErikRose> dstufft: Only fairly recently

[20:21:56] <ErikRose> You know how Linux distros are. :-)

[20:22:41] <ErikRose> Actually, you know what? We can go ahead and use < once we upgrade to our new installer, which pins everything.

[20:22:58] <dstufft> ErikRose: well, for wheels you care about the pip version, not the setuptools version

[20:24:00] <ErikRose> Oh, does pip do all wheel metadata handling itself?

[20:24:14] <dstufft> yea, you can install wheels without ever having setuptools installed

[20:24:19] <ErikRose> I guess it makes sense for setuptools to be confined to setup().

[20:24:32] <ErikRose> That's refreshing.

[20:25:04] <dstufft> there are two things, there's the wheel metadata which pip reads, and then setuptools also supports env markers inside of setup.py (which will be translated to inside the wheel when building)

[20:25:21] <dstufft> sadly, pip < 6 doesn't work with the setup.py env markers, though setuptools 0.7 does

[20:25:21] <ErikRose> So far so good.

[20:25:40] <dstufft> but, you can do the programatic thing inside of a setup.py, and explicitly override the dependency metadata during wheel generation

[20:25:52] <dstufft> depending on what versions of what you want to support

[20:26:20] <ErikRose> We can be sure that we build the wheels under a modern setuptools.

[20:26:44] <dstufft> ErikRose: can you be sure that nobody will be attempting to install the sdist with an old setuptools or an old pip?

[20:26:45] <ErikRose> Then people would just need pip >=6 to install properly, right?

[20:26:59] <dstufft> if it's wheels they are installing, it's pip 1.5+

[20:27:18] <ErikRose> Good point.

[20:27:29] <dstufft> if it's sdist, it's pip 6+ if you're using env markers inside of the setup.py, or pip reallyold+ if you do the programatic hack

[20:27:38] <dstufft> (twine is an example of the programmatic hack)

[20:27:45] <ErikRose> Thanks; I was about to ask. :-)

[20:28:04] <dstufft> https://github.com/pypa/twine/blob/master/setup.cfg#L9-L15 + https://github.com/pypa/twine/blob/master/setup.py#L28-L31

[20:28:28] <dstufft> the trick is that when the metadata section is inside of setup.cfg, bdist_wheel will completely ignore install_requires when building the wheel, and will use that instead

[20:28:38] <dstufft> (and setuptools and such doesn't otherwise use it)

[20:29:10] <ErikRose> That is super scarypants. :-) We'll never remember why we did that.

[20:29:35] <ErikRose> You're swinging me back to my original leaning, which was to come up with an unconditional set of dependencies. Thank you. :-)

[20:30:54] <dstufft> ErikRose: comedy option: Whenever you come back to the code in the future and you're confused why a thing exists, ping me because my head is a a database of all the sad things in packaging :D

[20:31:22] <ErikRose> Thanks! I knew that! That's why asked in here today. :-)

[20:31:36] <ErikRose> So glad you're getting paid for it now

[20:31:45] <ErikRose> dstufft, Packaging Oracle

[20:32:03] <dstufft> hope you like symlinking header files

[20:36:27] <ErikRose> Love it.

[20:36:43] <ErikRose> My favorite is to do it at random and watch people try to debug it.

[20:46:27] <ErikRose> dstufft: I support overriding the wheel metadata within setup.cfg would make everything alright as long as you're using a new enough pip that it always builds wheels, rather than installing straight from the sdist. Sanity check?

[20:48:08] <dstufft> I'm not sure exactly what you're asking. If you're asking if it's OK For sdist to be "broken" if built wheels are OK I'd say probably not, but there's other levels of "well sdists just don't do that" which are probably ok

[20:48:46] <ErikRose> I meant to ask under what conditions setup.cfg gets the opportunity to override install_requires.

[20:48:55] <dstufft> oh

[20:49:00] <dstufft> only during bdist_wheel

[20:49:00] <ErikRose> Since modern pips always build a wheel and then install that, I figured…

[20:49:14] <dstufft> which the pip wheel building uses bdist_wheel

[20:49:45] <ErikRose> (When did pip start always building wheels?)

[20:50:07] <dstufft> pip 7

[20:50:14] <ErikRose> Ah, pretty recently.

[20:50:40] <dstufft> to be specific, it doesn't always do it, if you don't have wheel installed it won't, and if bdist_wheel fails it will fall back to setup.py install, and people can disable it with --no-binary

[20:50:56] <ErikRose> hehe, never mind then. Thanks.

[20:54:01] <dstufft> the three (or four) reasonable options are generally A) Use environment markers in setuptools and eschew support for pip <6 and setuptools < 0.7 (or < newer if you need a newer feature) B) Do the old style conditional logic in setup.py and use the setup.cfg trick to get conditional in the wheel C) Hope that your conditions fall along a wheel platform line and don't share (e.g. py2 vs py3) D) Don't conditionally depend

[20:54:34] <dstufft> there's also the E) Don't support wheels at all, and make bdist_wheel explode. but that's not a very good one

[20:55:07] <ErikRose> Awesome. That breakdown is tremendously helpful.

[20:55:24] <ErikRose> I'll link you this comment when I'm done writing it; you may be amused.

[21:39:22] <ErikRose> dstufft: For your amusement: https://github.com/letsencrypt/letsencrypt/pull/2182#issuecomment-172099220

[22:00:26] <lifeless> dstufft: ouch :(

[22:00:34] <lifeless> dstufft: so - please review StevenK's packaging patches

[22:00:46] <lifeless> dstufft: they are needed to land before setuptools and pip can get pep 508 support

[22:01:26] <lifeless> dstufft: wrapping that up is more time sensitive than the abstract build system stuff,

[22:02:11] <lifeless> ErikRose: pbr has support for automatically Just Working too

[22:02:21] <lifeless> ErikRose: as of the latest commits in master; we should have that released next week

[22:02:41] <lifeless> ErikRose: (it uses markers in wheels and local conditional logic in non-wheel installs with pip)

[22:04:04] <ErikRose> lifeless: Hmm, pbr. First I've heard of it. Looks interesting.

[22:35:21] <dstufft> lifeless: okay, will review the packaging patches, just got back from getting alyssa

[23:07:28] <lifeless> dstufft: thanks :)

[23:11:25] <xafer> dstufft: if you can take a look at https://github.com/pypa/packaging/pull/47/ also ;)

[23:11:48] <dstufft> okay

[23:15:08] <dstufft> gotta go eat dinner, then I'll be back to reread the relevant parts of PEP 508 and review that PR

[23:30:26] <xafer> I had also this one: https://github.com/pypa/packaging/pull/51 but I could not figure the fail case :p

[23:32:22] <lifeless> dstufft: you can add cores anytime you want:)

[23:55:14] <dstufft> xafer: pep8 failures might be because of updated pep8 releases

[23:55:29] <dstufft> new releases sometimes add new rules

[23:55:47] <xafer> yup, it looks like it

[23:56:13] <xafer> But by fail case I meant a combination of version, spec concerned by this change

[23:56:28] <dstufft> lifeless: yea, it hadn't been a big deal since up until just recently nobody really touched it in any major capacity except me