Log file Viewer

[00:18:54] <berker> anyone have time to review https://github.com/pypa/pip/pull/2986 ? (I can also help with reviewing some PRs - I reviewed https://github.com/pypa/pip/pull/2980 a while ago, for example)

[03:10:10] <lifeless> o/

[19:16:00] <lifeless> dstufft: I wonder if we could get a little high bandwidth time from you today

[19:16:09] <lifeless> dstufft: we're working on packaging peps at the pyconau sprints

[19:48:03] <dstufft> lifeless: yes-ish

[19:48:35] <dstufft> lifeless: I've slept 2 hours in the past 48 hours, but I can devote whatever time you need right now, but the quality of the time may not be as great as I'd like :)

[19:52:50] <lifeless> dstufft: meep, everything ok ?

[19:53:19] <dstufft> lifeless: yea, I was just sick last week and got my sleep schedule inverted, then Alyssa had skate camp starting today so I couldn't sleep all day

[19:53:33] <dstufft> since I had to drop her off/pick her up

[19:53:52] <lifeless> ucky combo of things

[19:54:03] <lifeless> ok, so quick list of things

[19:55:36] <lifeless> 1) assertion: pip shouldn't grow to installing non-python things. e.g. general shared libraries. Agree or disagree ?

[19:56:15] <lifeless> 2) assertion: pip should grow to understanding the presence/absence of non-python things needed to run python things. e.g. that a given header file, executable, or library file is present

[19:56:59] <lifeless> 3) design discussion: should the things referred to in 2) be in the same language as existing dependencies (e.g. lib!libblas1.so could be placed in an install_requires line)

[19:57:23] <lifeless> or should they be partitioned off, which will drive having separate lists all over the place

[19:57:44] <dstufft> 1) Agree-ish. If someone shoves openssl into a wheel I think it's reasonable to install it, but we're unlikely to add any particular features or abilities specifically for installing non-python thingsd

[19:57:56] <dstufft> if that makes sense

[19:58:01] <lifeless> yeah

[19:58:12] <lifeless> like, we shouldn't have a wheel on pypi for libblas1.so

[19:58:27] <lifeless> thats too far into 'being a platform'

[19:58:48] <lifeless> (shouldn't, not 'can't)

[19:59:06] <dstufft> yea, generally

[19:59:23] <dstufft> if you want that, go use conda or something

[19:59:26] <dstufft> is the tl;dr

[19:59:35] <dstufft> for 2 uh

[20:00:12] <dstufft> there is a "requires external" thing, but it's mostly just opaque strings that are more advisory

[20:00:13] <dstufft> like pip can't do anything with them, but people can read them and do things with them

[20:00:31] <lifeless> blink

[20:00:39] <lifeless> ok so we've been designing the thing that exists :/

[20:01:16] <dstufft> requires external is from metadata 1.2

[20:01:22] <dstufft> e.g. it's never really been implemented

[20:01:36] <dstufft> I mean, PyPI understands it and will display it I think

[20:01:36] <lifeless> ah

[20:01:42] <lifeless> ok, I can read that

[20:01:57] <dstufft> but it was implemented in distutils2 and nothing else (other than PyPI)

[20:02:13] <lifeless> yeah

[20:02:18] <lifeless> so I think this would supercede that

[20:02:43] <lifeless> the specific cases we've been talking through are actual library files on windows/fedora/sue/bsd and how to deal with both source and binary deps

[20:02:47] <lifeless> without becoming conda

[20:03:11] <dstufft> yea

[20:03:32] <dstufft> I think you'd probably want it to be in a different list than install_requires

[20:03:54] <lifeless> so, thats one of the debate points

[20:04:05] <dstufft> I guess one of the questions is, what's the end goal here

[20:04:15] <dstufft> what UI/UX are you looking to accomplish

[20:04:20] <lifeless> right

[20:04:32] <lifeless> I'll make sure that we make that clear in the pep draft

[20:04:33] <lifeless> but

[20:04:51] <lifeless> fast errors rather than slow-after-compile-fails errors

[20:05:00] <lifeless> and clearer errors rather than compiler spew

[20:06:01] <lifeless> the proposal is essentially two things: a schema for external requirements and a modular platform translation layer (that e.g. conda would provide) to translate 'libblas1.so' [which is a specific known quantity] into 'blas' or whatever the vendors package name is

[20:06:17] <lifeless> the question about what list they go into

[20:06:57] <lifeless> I think it would be useful for deployments to be able to list those requirements in requirements.txt files, so that they get the same benefits for existing already uploaded packages

[20:07:01] <lifeless> [por packages with bad metadata]

[20:07:17] <lifeless> s/por/or/

[20:07:45] <lifeless> also, UI wise, I think having two lists of things that are visually identical but semantically totally different is confusing

[20:08:00] <dstufft> so you're hoping this translation layer would automatically install libblas1.so via whatever the platform tooling is

[20:08:02] <lifeless> setup_requires and install_requires have the same semantics for their content; they are used in different places

[20:08:21] <lifeless> setup_requires and external_requires looking the same but one being ... etc

[20:08:56] <lifeless> uhm, so - It seems to me that starting with 'please translate this missing dependency for me' is a big step up

[20:09:12] <lifeless> could that do installation as well? I think perhaps, but would not want to mandate that it had to.

[20:09:44] <lifeless> e.g. : no layer present, we output the raw string we had as a dependency.

[20:10:30] <lifeless> layer present, we ask the layer to translate, and it's range would be Either(new_message, retry)

[20:11:41] <dstufft> ok

[20:12:08] <dstufft> This sounds similar to some rough thoughts I had thought about before, but that I hadn't bothered to really move beyond the "random idea" stage

[20:12:44] <lifeless> there are plenty of questions like 'should it specify a filename (lib!libblas1.so) or should it specify a more abstract thing (lib!blas==1.0)

[20:12:53] <lifeless> which we'll need to iron out

[20:13:09] <lifeless> with digging into library semantics on mac/windows/linux

[20:14:05] <dstufft> yea

[20:14:27] <lifeless> but ok so you don't hate it :) that enough for me for now

[20:16:23] <lifeless> -> breakfast here

[20:16:31] <lifeless> thank you!

[20:16:32] <dstufft> I think I should probably warn that it feels like something that might be pretty solidly metadata 2.0 territory, but I wouldn't bet the farm that it can't be done in a way that can be integrated into the current metadata

[20:16:43] <lifeless> well

[20:16:49] <lifeless> I think a versioned dep on setuptools

[20:16:56] <lifeless> and integrated into the same lists

[20:16:58] <lifeless> and tada

[20:17:31] <lifeless> setup_requires=['setuptools>=23', 'pip>=10'] :)

[20:17:52] <lifeless> but even that really isn't needed

[20:18:07] <lifeless> since the things simply wouldn't be satisfiable by earlier libraries

[20:18:10] <lifeless> it would fail closed

[20:18:13] <lifeless> anyhoo - food

[20:18:20] <dstufft> ok

[20:18:24] <dstufft> have a good breakfast :D

[20:38:06] <lifeless> the fried eggs were nice

[20:49:33] <xafer> By the way, https://github.com/pypa/pip/pull/2980 is quite simple

[20:52:45] <xafer> thanks :)

[20:53:30] <dstufft> xafer: np!

[21:55:01] <lifeless> dstufft: now to see if https://rbtcollins.wordpress.com/2015/08/04/the-merits-of-careful-impatience/ freaks anyone out :)