PMXBOT Log file Viewer

Help | Karma | Search:

#pypa-dev logs for Monday the 24th of March, 2014

(Back to #pypa-dev overview) (Back to channel listing) (Animate logs)
[09:01:30] <jezdez> Ivo: no worries
[09:01:34] <jezdez> IRC noobness all around :)
[09:01:37] <jezdez> glad we have dstufft
[09:06:12] <pmxbot> jaraco pushed 3 commits to setuptools (https://bitbucket.org/pypa/setuptools/) :
[09:06:12] <pmxbot> Force sdist zip format (preferred for _build_egg)
[09:06:12] <pmxbot> Remove test_build_egg. The test doesn't work and it's difficult to test _build_egg given the current interface.
[09:06:12] <pmxbot> Remove test_do_download, which also no longer works due to complexities around secure downloaders.
[10:54:59] <pmxbot> jaraco pushed 1 commit to setuptools (https://bitbucket.org/pypa/setuptools/) :
[10:54:59] <pmxbot> Update test_install to actually patch the hidden _python_cmd.
[11:00:33] <pmxbot> jaraco pushed 2 commits to setuptools (https://bitbucket.org/pypa/setuptools/) :
[11:00:33] <pmxbot> Remove test for use_setuptools, as it fails when running under pytest because the installed version of setuptools is already present.
[11:00:33] <pmxbot> Just remove test_ez_setup, as the last remaining test, test_install, only barely touches the functionality as the actually installation command is mocked. The test run in the travis config is much more comprehensive.
[11:35:49] <pmxbot> jaraco pushed 1 commit to setuptools (https://bitbucket.org/pypa/setuptools/) :
[11:35:49] <pmxbot> Update test to use local timestamps, so the tests pass in any timezone.
[11:48:53] <pmxbot> jaraco pushed 1 commit to setuptools (https://bitbucket.org/pypa/setuptools/) :
[11:48:53] <pmxbot> Use mktime for Python 3.2 compatibility
[11:52:44] <pmxbot> jaraco pushed 1 commit to setuptools (https://bitbucket.org/pypa/setuptools/) :
[11:52:44] <pmxbot> Remove unused UTC implementation
[18:59:21] <Ivo> dstufft: what do you mean by broken here https://github.com/pypa/pip/issues/1668#issuecomment-38460127
[18:59:46] <dstufft> Ivo: 3.4 rm -rf's ensurepip
[19:00:37] <Ivo> as in, debian have said 'fuck that shit' essentially
[19:01:10] <Ivo> but your comment mentioned venv?
[19:03:18] <qwcode> venv/pyvenv gets the pip it installs from ensurepip I think
[19:03:46] <Ivo> did i miss that feature in 3.4 O_O
[19:04:32] <dstufft> Ivo: yea, venv automatically installs using ensurepip into a 3.4 venv
[19:04:42] <dstufft> Debian didn't permanently break it though, they are looking to re-add it back in
[19:04:46] <dstufft> it was a temporary thing
[19:05:30] <dstufft> I think it's currently broken on Fedora too
[19:05:37] <dstufft> but they are doing some signifcant work to make it work actually
[19:05:44] <Ivo> hmm, archlinux 3.4 removed ensurepip as well
[19:06:15] <Ivo> although updated packages with arch aren't as much of a problem
[19:07:07] <dstufft> do you use Arch?
[19:07:11] <Ivo> y
[19:07:20] <dstufft> does it all blow up if you use python3.4 -m venv test
[19:07:29] <Ivo> 3.4 is in testing atm
[19:07:37] <dstufft> ah
[19:07:39] <Ivo> https://projects.archlinux.org/svntogit/packages.git/tree/trunk/PKGBUILD?h=packages/python
[19:08:06] <dstufft> oh they just do --without-ensurepip
[19:08:10] <dstufft> it looks like?
[19:08:23] <dstufft> that is fine to do
[19:08:37] <Ivo> does debian not just do that?
[19:08:39] <dstufft> no
[19:08:50] <dstufft> they do rm -rf /usr/lib/../ensurepip
[19:08:58] <dstufft> as in they delete the module entirely
[19:11:05] <dstufft> because of the policy against bundled software
[19:11:18] <dstufft> which is dumb because they allow virtualenv to do the same thing already
[19:11:23] <Ivo> why is --without-ensurepip not enough
[19:11:53] <dstufft> I think they just took a scorched earth policy while they figure out what they are going to do
[19:12:06] <Ivo> my naive assumption is surely you'll get a cleaner outcome by using the configure option the maintainer of the upstream nicely provided for you
[19:12:27] <dstufft> the outcome is different
[19:12:44] <dstufft> --without-ensurepip only prevents pip being installed into the global site-packages
[19:12:56] <dstufft> The ensurepip module itself still exists
[19:13:05] <dstufft> and the venv module uses it to install pip into a newly created venv
[19:13:28] <dstufft> what debian did was use --without-ensurepip to prevent it from being installed into the global site-packages, but then also removed the module from the stdlib all together
[19:13:33] <dstufft> which means venv breaks by default
[19:13:40] <Ivo> clearly i have a bad idea about what a --without-ensurepip option should do :S
[19:14:26] <Ivo> I always thought that venv was going to provide basic plumbing, and let something like virtualenv implement niceties over the top of it, like installing an pip
[19:14:54] <dstufft> with 3.4 the built in venv module is basically a complete replacement of virtualenv
[19:15:11] <dstufft> well except I don't think it handles -p
[19:15:33] <Ivo> I don't really like this 'well since we have a bundled pip dagging around, may as well chuck it in with the venv module as well'
[19:15:47] <dstufft> Why
[19:16:01] <dstufft> that was part of PEP453
[19:16:14] <dstufft> otherwise the venv module is pratically useless
[19:16:17] <Ivo> trying to get too much right, and the pip module will be static with the python version
[19:16:50] <dstufft> if it's good enough for the global site-packages it should be good enough for a venv
[19:16:58] <dstufft> and pip gets upgraded in patch releases
[19:17:05] <dstufft> so 3.4.1 will get a new version of pip bundled
[19:17:30] <dstufft> and something like virtualenv can still provide a shim over the venv module and provide newer pip if someone wants that
[19:18:18] <dstufft> It's kind of asymetric for Python to have ``pip`` available by default in the global site-packages, but not have it available in a venv
[19:20:48] <Ivo> i think it'd be asymmetric if python just bundled pip straight out, instead of a bootstrap module
[19:21:07] <Ivo> s/bundled/incorporated
[19:21:19] <dstufft> That's designed to be somewhat invisible to users though
[19:21:26] <dstufft> the Make file calls ensurepip by default
[19:21:29] <dstufft> the installers do as well
[19:42:11] <Ivo> dstufft: I always thought virtualenv would transition over to using venv instead of its own incantations when it found it was running on python >=3.3
[19:42:39] <dstufft> that's certainly a possiblity
[19:42:51] <dstufft> but there's very little reason to keep ensurepip in the stdlib at all if it's just an install time thing
[19:42:56] <Ivo> and by that means venv would certainly not be 'useless'
[19:44:14] <dstufft> useless by itself
[19:44:23] <dstufft> which is kind of a wierd state for a stdlib package to be in
[19:44:42] <Ivo> i sometimes use it as a cleanroom for testing pip itself...
[19:45:07] <Ivo> and get-pip.py
[19:45:46] <Ivo> also nice that it has an programmatic interface
[19:45:49] <Ivo> much moreso than virtualenv
[19:46:31] <dstufft> python -m venv --without-pip
[19:46:31] <Ivo> unfortunately that also suffers from noone making use of new features in python because who runs a newish python in production rite!?
[19:47:12] <agronholm> I use 3.3 in production
[19:47:24] <Ivo> agronholm: you're clearly no-one
[19:47:28] <agronholm> :)
[19:47:37] <dstufft> python.org runs 3.3 I think
[19:47:38] <dstufft> or 3.2
[19:48:23] <agronholm> the only really interesting thing about 3.4 was that it comes with pip
[19:48:31] <agronholm> at least the windows version does
[19:48:45] <Ivo> os x as well?
[19:48:49] <dstufft> yes
[19:48:51] <agronholm> haven't tried that
[19:49:02] <Ivo> asyncio yo
[19:49:07] <dstufft> ensurepip is in the python stdlib, the python.org provided binaries call it by defualt
[19:49:14] <dstufft> and so does the makefile
[19:49:18] <agronholm> Ivo: already backported
[19:49:29] <dstufft> SipHash is $$
[19:52:35] <Ivo> soooo, thinking in irc messages... python is now trying to provide a package manager, i.e make pip its official package manager, like ruby does with gem and node does with npm... but since it once was a '3rd party' package manager, distros don't know how to react?
[19:52:55] <dstufft> well
[19:52:58] <dstufft> it's less about that
[19:53:02] <dstufft> more about the way we did it
[19:53:41] <dstufft> if we just committed it straight to the stdlib then they would have just allowed it
[19:54:05] <dstufft> but instead we put an installer in the stdlib and bundled a package inside it and install it
[19:54:14] <dstufft> and distros reallly don't like bundled software
[19:54:22] <dstufft> it makes it harder for them to handle patches and stuff
[19:54:38] <Ivo> but not only that, we also make use of that to enable the shared nothing architecture that is a venv
[19:55:30] <Ivo> because you use that wheel to install a separate pip into a venv, which is now a feature of python (if you don't go removing stuff)
[19:55:51] <dstufft> for instance: If they want to patch pip to use the system provided certificates instead of the bundled ones, they have to patch the one provided by python-pip, and they have to unpack the Wheel, patch that pip, and repack the Wheel, and then stick it back inside
[19:56:28] <Ivo> so it would be less of a problem if we were more like node, in that the npm that handles a local node env is the same one that does global installs (you just add -g in your invocation)
[19:57:35] <dstufft> well it'd be less of a problem if we just shoved pip in the stdlib
[19:57:47] <Ivo> how do install it to a venv then tho
[19:58:09] <dstufft> you wouldn't have to install it, it'd be part of the stdlib
[19:58:10] <Ivo> or would you forgo that feature in that case
[19:58:19] <dstufft> so you'd get it just like everything else you get in the stdlib
[19:59:13] <dstufft> The reason we didn't do that is we wanted to be able to independetly upgrade pip in an rleady installed environment
[19:59:58] <dstufft> e.g they way we did it, even if you'er stuck on 3.4.0 and pip 1.5.4, but you want the latest pip (1.9 let's say), you're only a pip install --upgrade pip away from that
[20:00:49] <Ivo> so what was stopping the installer from putting a 1.5.4 pip in lib/site-packages
[20:01:08] <Ivo> and pip install --upgrade pip would also work
[20:02:32] <dstufft> well that's what it does more or less, it just does the unpacking of a Wheel at install time rather than commiting it to the repo
[20:02:46] <dstufft> I have to go and pick up my daughter
[20:02:49] <dstufft> will be back in a bit
[20:02:57] <Ivo> but the latter would have been easier for distros
[20:03:06] <Ivo> cus they no longer have to worry about a wheel
[20:23:57] <dstufft> Ivo: there were concerns about that which I don't recall exactly, and venv doesn't include the system site-pacakges by default either
[20:24:51] <Ivo> that was why i was asking about whether you'd roll back the venv-includes-a-copy-of-pip feature
[20:26:13] <Ivo> is your daughter having a good day?
[20:30:30] <Ivo> https://www.youtube.com/watch?v=xbHPTPUpQ1I&t=4m
[20:30:33] <Ivo> nice thing to listen to
[20:31:03] <dstufft> rolling back that would require deprecating and removing features from CPython
[20:31:11] <dstufft> and i'm not convinced it'd be worth it tbh
[20:31:43] <dstufft> the distros will sort things out, they are just kind of trying to do that now
[20:32:22] <Ivo> well the example you gave, ca certificates, seems a bit hypocritical that they made no fuss about virtualenv vendoring zips
[20:32:42] <Ivo> but when python proper does it, its a no go
[20:35:04] <dstufft> Well python is a bigger deal than virtualenv, and I wouldn't exactly call it making a big fuss or anything
[20:35:18] <dstufft> they are rying to figure out what their policies are for python now
[20:35:34] <dstufft> for instance: one thing PEP453 does is recommend installing python-pip with Python
[20:37:15] <Ivo> imho the deliniation should be made on whether something is system-y or developer-y. ensurepip's copy of pip is developer-y, so system maintainer shouldn't be trying to mess with it
[20:37:30] <dstufft> Fedora is planning on introducing a circular dependency for that, Debian doesn't want to do that because a circular dependency is kind of icky
[20:37:44] <dstufft> also pip is a developer facing tool in their eyes, but Debian isn't just a developer facing OS
[20:38:37] <Ivo> well the problem is vanilla pip by default is modifying the system python and system path, so I'd say it has to be considered system-y
[20:39:04] <dstufft> https://lists.debian.org/debian-python/2014/03/msg00045.html
[20:39:45] <Ivo> if pip went --user by default then it would turn to be much more developer-y
[20:42:24] <dstufft> well
[20:42:37] <dstufft> the proposals for --user by default don't affect what it'd do if you install it as root
[20:42:54] <dstufft> it essentially turns a permissions error into installing it into --user
[20:43:55] <Ivo> s/install/run/ ?
[20:48:18] <dstufft> yea
[20:50:03] <Ivo> but isnt the point that once --user is default, people won't be inclined to be running pip with root anymore
[20:51:29] <dstufft> ideally yes
[20:52:35] <Ivo> which is also what some debianers are complaining about in these MLs - 'pip, get off ma /usr'
[20:52:54] <dstufft> yea
[20:53:12] <dstufft> well Linux distros and language package maangers have a long history too :]
[20:53:43] <Ivo> i think pip is in many ways behind many other LPMs tho
[20:53:56] <Ivo> at least, nowadays
[20:53:57] <tomprince> I found the comment about installing to /usr/local/lib/.../dist-packages crazy.
[20:54:20] <tomprince> I thought the point of having dist-packages was so that /usr/local/lib/..../site-packages was for admin installed stuff.
[20:55:19] <dstufft> debian is kind of crazy
[20:55:29] <dstufft> I don't know why they switched from site-packages to dist-packages
[20:55:58] <Ivo> so debian's pip installs to a dist-packages?
[20:56:10] <Ivo> or debian python packages install under a dist-packages?
[20:56:25] <dstufft> python packages install under dist-packages
[20:56:34] <dstufft> pip doesn't really decide where to install stuff too
[20:56:41] <dstufft> it just asks distutils where it should go
[20:56:52] <dstufft> Debian patches their python to make it /usr/local/.../dist-packages
[20:57:14] <Ivo> so is debian's site-packages for anything anymore?
[20:57:25] <jezdez> ah, is debian fucking shit up again?
[20:57:28] <jezdez> hi all!
[20:57:35] <dstufft> Ivo: don't think anything uses it not
[20:57:37] <dstufft> no*
[20:57:48] <Ivo> so its effectively purely a rename
[20:57:54] <jezdez> I'm surprised it's not ubuntu this time
[20:57:55] <dstufft> jezdez: eh, Ubuntu/Debian's current Python 3.4 rm -rf's ensurepip
[20:58:00] <jezdez> teehee
[20:58:10] <dstufft> but they are talking about how they want to handle it long term
[20:58:21] <Ivo> i was thinking debian's python debs installed to one dir and direct distutils to install to another, but sort both in sys.path
[20:58:23] <dstufft> it's a temporary thing while they figure out how the policies apply and what they want to do exactly
[20:58:25] <Ivo> s/sort/source
[20:58:26] <dstufft> Ivo: they do
[20:58:38] <dstufft> python debs go to /usr/.../dist-packages
[20:58:42] <jezdez> gotcha, but they still want to package 3.4 already?
[20:58:50] <dstufft> python packages go to /usr/local/.../dist-packages
[20:59:01] <Ivo> ...oh
[20:59:04] <tomprince> Ivo: I think that was the intention of introducting dist-packages, but I don't think that is the reality.
[20:59:05] <jezdez> dstufft: obviously
[20:59:07] <jezdez> W(
[20:59:11] <dstufft> jezdez: yea, the next Ubuntu LTS uses 3.4 by default
[20:59:18] <jezdez> *gasp*
[20:59:34] <jezdez> dstufft: like `python` will be 3.4?
[20:59:39] <dstufft> jezdez: no
[20:59:42] <jezdez> ah.
[20:59:43] <jezdez> phew
[21:00:04] <dstufft> ``python`` will probably never be Python3 on any sensible distro
[21:00:14] <Ivo> ARCH IS SENSIBLE
[21:00:16] <dstufft> even when python2 is gone it'll probably be ``python3`` still
[21:00:48] <dstufft> jezdez: but afaik all of Ubuntu's internal stuff uses 3.4 in the latest LTS, and python 3.4 is preinstalled but python2 requires an apt-get
[21:00:59] <dstufft> for a minimal install at least
[21:02:10] <dstufft> https://lists.debian.org/debian-python/2014/03/msg00045.html
[21:02:11] <tomprince> Oh. Reading https://bugs.launchpad.net/ubuntu/+source/python2.6/+bug/362570 suggests that dist-packages in /usr/local is to distinguish from a manually installed python in /usr/local
[21:02:28] <dstufft> tomprince: ah that makes sense
[21:02:35] <Ivo> arch is a happy reverse mirror land of joy
[21:02:46] <Ivo> if you want to install a python2 version of something you need a 2 in the name
[21:03:03] <dstufft> Ivo: I would maybe consider using arch for a desktop if I ever left the land of Apple
[21:04:01] <dstufft> but yea, right now the various linux distros support (or don't) PEP453 to varying degrees while they figure out how mad they are at PEP453/me
[21:04:33] <Ivo> dstufft: you get either the latest version of a thing after a week or two, or at its .1 release if its big and probably needs a bug release
[21:04:49] <dstufft> but afaict most of them are probably going to enable it at least for the venv module
[21:05:10] <dstufft> I doubt many of them will use ensurepip to install pip into the system python
[21:05:17] <dstufft> but I never expected them to anyways
[21:05:39] <dstufft> Fedora is got some thing which will dynamically generate a Wheel from an already intalled pip
[21:05:48] <Ivo> i essentially think they'd be a lot less scared if pip already had '--user by default'ish behaviour
[21:08:09] <dstufft> probably so
[21:08:18] <dstufft> I posted to their respective mailing lists about it
[21:08:24] <Ivo> where does --user currently put scripts?
[21:08:24] <dstufft> and i'm in #debian-python and #fedora-python
[21:08:29] <dstufft> ~/.local/bin
[21:08:41] <dstufft> which is on $PATH by default in Fedora, dunno about other distros
[21:09:42] <Ivo> pip could always detect whether that wasn't currently on path and inform you of such
[21:10:02] <Ivo> s/path/$PATH
[21:10:08] <dstufft> sure
[21:10:27] <dstufft> well
[21:10:45] <dstufft> actually that'd break on anything but the most simple thing
[21:11:04] <Ivo> what's the that you're referring to
[21:12:19] <Ivo> a 'is its directory on $PATH' detection?
[21:15:47] <dstufft> yea
[21:16:12] <Ivo> what's a case you're worried it would break for?
[21:16:42] <dstufft> symlinks primarily
[21:17:08] <dstufft> either of the bin dir itself (ln -s ~/.local/bin ~/bin) or of stuff inside it
[21:18:13] <dstufft> that doesn't itself mean that we can't do it, just that it's something to think of
[21:43:11] <Ivo> apparently a shipping container falls off a boat somewhere in the world about once an hour
[21:43:52] <dstufft> TIL
[21:46:44] <EWDurbin> see nasdaq:OMEX