Log file Viewer

[00:08:26] <diminoten> hey you guys should accept https://github.com/pypa/warehouse/pull/354

[00:08:33] <diminoten> so travis builds at least pass

[00:08:35] <diminoten> :3

[00:09:09] <diminoten> and I'll feel better about pulling the code that's from master rather than the year-old v14.2.1 tag

[01:05:03] <dstufft> lkasjdfsklfjsfjsfdsf

[01:05:04] <dstufft> fucking Ubuntu

[01:07:33] <lifeless> oh?

[01:08:16] <dstufft> they switched the default of pip so that it does ``pip --user`` outside of a virtual environment

[01:08:24] <dstufft> and they added a custom flag that only exists on ubuntu to switch it back

[01:14:00] <lifeless> ahahahahahahahaha

[01:15:10] <dstufft> A non trivial amount of my time is spent baby sitting debian and ubuntu

[01:15:20] <dstufft> to try and keep them from breaking pip, virtualenv, ensurepip, etc

[01:15:47] <lifeless> well

[01:15:50] <lifeless> stockholm syndrome

[01:16:01] <r1chardj0n3s_afk> Omfg ubuntu

[01:16:39] <dstufft> https://bugs.launchpad.net/ubuntu/+source/python-pip/+bug/1419695

[01:18:41] <dstufft> https://bugs.launchpad.net/ubuntu/+source/python-pip/+bug/1419695

[01:18:42] <dstufft> er

[01:18:43] <dstufft> wrong channel

[01:58:55] <abadger1999> Glad I talked the fedora maintainer out of making that same change unilaterally.

[01:59:57] <dstufft> abadger1999: I only recently started engaging downstream, but Fedora for me so far has been *awesome*

[02:01:40] <dstufft> I talk to slavek fairly regularly and I feel like working with Fedora is a lot more like trying to find the right common ground instead of trying to fight a battle

[02:01:53] <abadger1999> Cool :-) I'm very happy we have a good relationship with you :-)

[16:24:12] <msabramo> has the idea ever come up of vendoring "wheel" into pip? Seems like that would be convenient.

[16:24:31] <msabramo> pip is already vendoring so many other things; what's one more? :-)

[16:27:18] <xafer> I guess it will be the case when pip will wheel-cache its install

[16:30:28] <xafer> but you also need setuptools to build a wheel :-/

[16:31:52] <sigmavirus24> msabramo: I thought pip already did vendor wheel

[16:31:53] <sigmavirus24> Weird

[16:33:17] <msabramo> yeah it seems like it should but it doesn't

[16:34:20] <msabramo> https://github.com/pypa/pip/blob/develop/pip/commands/wheel.py#L110

[16:44:39] <dstufft> we don't vendor wheel because it's a build dependency of sdists

[16:44:45] <dstufft> we don't vendor build dependencies

[16:48:16] <msabramo> $ pip wheel

[16:48:17] <msabramo> ERROR: 'pip wheel' requires the 'wheel' package. To fix this, run: pip install wheel

[16:48:44] <dstufft> sure, and you'll get a similar error if you do ``pip install <thing whichi only has a sdist>`` except s/wheel/setuptools/

[17:06:35] <xafer> dstufft, so how do you envision the wheel-caching scenario without vendored wheel ? Check if the wheel package is available and if it isnt, let pip perform like it currently does ?

[17:08:09] <dstufft> xafer: treat setuptools and wheel as build requirements when we need to fallback to sdist, install them intoa temporary location, build the wheel, then throw away the temporary location and install the wheel

[17:08:36] <msabramo> so is vendoring wheel a bad idea?

[17:09:09] <dstufft> I think so

[17:11:28] <tomprince> Well, vendoring anything is bad, and the benifits of vendoring wheel don't out way that, unlike the rest of the things bundled with pip.

[17:15:47] <dstufft> yea that's a good way of putting it :)

[17:16:26] <dstufft> We vendor what we must in order to make it possible to install from wheels, if we can do that then we should be able to JIT install things from wheels that we need in order to _build_ things

[17:25:43] <msabramo> not sure I follow — pip vendors things like colorama and progress — those are niceties that could be optional

[17:30:17] <tomprince> Well, setuptools and wheel are things that are imported in a seperate process, by a packages setup.py, so the versions of them used don't matter as much to pip.

[17:31:37] <tomprince> Certainly, colorama and progress could be optional, but then pip would need to be able to tell if they are of a compatible version before importing them.

[17:32:11] <tomprince> And either they need to be installed automatically, polluting a virtualenv, or the default experience would be sub-optimal.

[17:32:35] <msabramo> `pip wheel` requires `wheel`; not just required by packages' setup.py

[17:34:17] <msabramo> not a big deal though; installing wheel to get `pip wheel` to work is easy enough, since the error tells you to do that

[17:35:44] <tomprince> https://github.com/pypa/pip/blob/01eb41cfba062d4a7802ad9dcfe6eee4f11194b6/pip/commands/wheel.py#L106-114 looks like the only place wheel is imported (and only to see if wheel is installed)

[17:36:14] <msabramo> oh interesting

[17:36:17] <tomprince> So, 'pip wheel' uses it, but only via 'python setup.py bdist_wheel'

[17:36:29] <msabramo> yeah

[17:36:36] <msabramo> I missed that point

[17:36:57] <tomprince> In fact, it seems like it would be tricky to get a vendored copy to support that usage.

[17:37:05] <xafer> the sad thing is that if you have wheel but not setuptools, it will also ask you to install wheel :p

[17:37:55] <msabramo> I guess a vendored wheel wouldn't help if it's needed by other packages

[17:38:12] <msabramo> well maybe with some monkeypatching nastiness or something

[17:38:19] <dstufft> long term I think the real answer is to remove the errors like that (we have one for wheel and for setuptools) and to just autoinstall build dependencies

[17:38:27] <dstufft> that generalizes a lot better, but it requires more work

[17:38:34] <msabramo> I think I see why you guys are against it

[17:41:03] <dstufft> "Required in order to install Wheels" isn't exactly correct as to the dividing line between what is a good candidate for bundling or not (as you pointed out, colorama and progress could both be made optional since they aren't strictly required). Build Dependency of the thing we're building vs dependency of pip itself is probably a better way to say what the dividing line is

[17:41:17] <dstufft> long term I don't want setuptools (and by extension) to be special, I want it to be just another option for a build system

[17:41:26] <dstufft> er, "and by extension, wheel"

[17:42:04] <dstufft> since we're not going to bundle every build system that people write, we'll want generic support for being able to automatically install wahtever dependencies any particular project needs in order to take an sdist and turn it into a wheel

[17:42:42] <dstufft> and if we have generic support for that, there's no reason not to use it to install setuptools + wheel for the "legacy sdist" case

[17:46:00] <xafer> btw I was looking at what distlib was doing for windows scripts and found https://bitbucket.org/pypa/distlib/src/712bebbecb6f8f29d16f7b10f6ac05b8b04c7dda/distlib/scripts.py?at=default#cl-293 with several launchers t32.exe, t64.exe, w32.exe, w64.exe inside of distlib... I guess they are the launchers presented in PEP397 but are they document ? and available from somewhere else than distlib ?

[17:48:15] <dstufft> I think it's https://bitbucket.org/vinay.sajip/simple_launcher

[17:50:01] <msabramo> cool, thanks dstuft

[17:50:08] <msabramo> cool, thanks dstufft :)

[17:51:16] <dstufft> at some point I should probably write down my high level ideas for what all this should look like

[17:51:39] <dstufft> so people can either argue with it or use it as a roadmap of sorts

[17:54:58] <xafer> shouldnt the pep397 launchers be installed and available through the windows installation ?

[17:55:49] <ionelmc> xafer: the py.exe?

[17:57:43] <xafer> well distlib (and pip if I'm not mistaken) seems to use t32/t64/w32/w64.exe files

[17:58:18] <msabramo> roadmap could be useful

[17:59:44] <ionelmc> xafer: those are script shims

[18:00:18] <ionelmc> totally different than pep397

[18:01:16] <xafer> mmh, ok. Any pointers I could look at ?

[18:04:29] <xafer> (I'm not sure what you mean by script shims)

[18:05:16] <ionelmc> they just launch whatever is specified in the shebang

[18:05:36] <ionelmc> so if t64.exe is copied to pip.exe

[18:05:43] <ionelmc> it will look for a file pip-script.py

[18:05:56] <ionelmc> read the first line (the shebang)

[18:06:10] <ionelmc> and execute it with pip-script.py as argument

[18:06:51] <ionelmc> there are variants for all platforms cause those binaries can't be changed (they have signatures)

[18:07:31] <ionelmc> 4 platforms: 32/64 and wide/narrow (i think)

[18:07:34] <xafer> https://github.com/pypa/pip/blob/develop/pip/_vendor/distlib/scripts.py#L162-L170 => ok so, the goal is to have a windows executable expecting a shebang and (apparently) a zipped __main__.py file to launch a script ?

[18:07:55] <ionelmc> sorry, not wide/narrow

[18:08:01] <ionelmc> w is for pythonw.exe

[18:08:04] <xafer> yup

[18:08:08] <ionelmc> t is python.exe (terminal)

[18:08:29] <ionelmc> i think they must be different cause process has to be created differently

[18:08:38] <ionelmc> or binary ?!

[18:08:45] <ionelmc> you have to read the code to be sure

[18:08:50] <xafer> but shouldnt the python installation provide those shims ?

[18:09:20] <ionelmc> xafer: python (since 3.x) provides the `py` lanuncher

[18:09:41] <ionelmc> wich does something entirely different, it finds the right python to lanuch in the windoze registry

[18:09:53] <ionelmc> py -2 lanuches python2

[18:10:03] <ionelmc> py -3 launches ptthon3

[18:10:10] <xafer> oh and with the py launcher, you could just put the shebang and the script, just like it is explained in pep397 ?

[18:10:18] <ionelmc> and i think it supports py -3.4 or -3.3 too

[18:10:52] <ionelmc> you can use the py launcher with no version spec and will figure the right python from the shebang (i think)

[18:11:10] <Wooble> yes.

[18:12:16] <xafer> setuptools seems to have similar cli-32,cli-64,gui-32,gui-64.exe files :-/

[18:29:17] <xafer> yup it seems to be simple_launcher, from its readme: "Note: More recently, the launchers have been updated to find their script in an archive appended to the executable, rather than a separate file. (This variant is enabled when APPENDED_ARCHIVE is #defined). This allows the launchers to be used to e.g. run .pyz archives as native Windows executables."

[18:29:37] <xafer> thanks for the pointers :)

[18:32:33] <ionelmc> xafer: there was some issues with appending stuff to them (if you want to sign executables it's bad)

[18:32:51] <ionelmc> i'm not sure what the exact details are