Log file Viewer
#pypa logs for Wednesday the 19th of February, 2020
[14:37:38] <toad_polo> To all who are interested, I have posted the official announcement of the PyCon US 2020 Packaging Summit on discourse: https://discuss.python.org/t/pycon-us-2020-packaging-summit-registration-and-topic-proposal/3341
[17:37:52] <pyusr> I think I've found a major bug in the pip, virtualenv, setuptools eco-system that basically borks tons of systems
[17:50:39] <pyusr> anyhow, this python 2 EOL thingy just manged to bork system python + virtualenv on ubuntu 16.04
[17:50:57] <pyusr> i'm trying to figure out where to complaint about it, and who should be fixing it
[17:52:08] <pyusr> (i have a detailed bug report with an easy docker reproducable, but i dont know where to put it)
[17:52:27] <nicksloan> I think you need to give more details. I'd say that sounds like an Ubuntu issue based on what you've said so far
[17:53:45] <pyusr> basically if your workflow on ubuntu 16.04 invololves apt-get install python-virtualenv, then creating a virtualenv using that package, it will use the bootstraped ubutnu version of pip 8. virtualenv installes setuptools, because pip 8, it will get the python 3 only version, and from them on your virtualenv is borked
[17:55:22] <nicksloan> also, I'm really not an expert here, so I won't likely be able to give you a solid answer on anything, but I think sharing the whole report will give others in here what they need to try to help you
[18:30:13] <pradyunsg> pyusr: thanks! It's midnight where I'm at right now, so I'mma head to bed. Chances are someone else would take a look before I would. :)
[18:30:33] <ngoldbaum> pip 8 isn’t new enough to know about the necessary metadata that would prevent the latest version of setuptools from being installed
[18:36:27] <pradyunsg> techalchemy: ^ look at the kind of pain and confusion all of the distro stuff causes. :}
[18:36:55] <pyusr> can pypi see the pip version in the USEr-agent of the request, and on older pip, not show py3 only packages ?
[18:37:25] <pyusr> since ubutnu 16.04 is LTS it's still supported, and maybe it's their issue, but I dont know
[18:39:19] <pyusr> if the user-agent can solve the issue, maybe it's best to put a stopgap on the pypi server code ?
[18:39:40] <pyusr> since then it's not depended on people making sure they have an updated python-virtualenv package
[18:40:16] <pyusr> altough then you might be in another world of pain, if your workflow is "virtualenv --no-download" :) ::) :)
[19:49:10] <techalchemy> hm pip 8 installs virtualenv 20.0.4 on python 2.7.17 in a fresh virtualenv for me
[19:52:13] <pyusr> you need to read virtualenv.py source code to figure how it uses pip to bootstrap
[19:52:52] <pyusr> basically when creating an venv, it does PYTHONPATH=/usr/share/python-wheels/pip.whl python -m pip install pip,septutools,wheel
[19:53:29] <pyusr> PYTHONPATH=file:///usr/share/python-wheels/pip.whl python -m pip install pip septutools wheel
[19:53:41] <techalchemy> the distributed version of virtualenv in 16.04 is broken is what i'm saying
[20:15:37] <toad_polo> This is one of the known problems with LTS distros. They do their own weird stuff to their packages and the support burden ends up falling on upstream maintainers who can't do much if anything about it.
[20:17:15] <toad_polo> If your LTS distro's version of pip is not able to work with today's version of PyPI, the support needs to come from the LTS distro, not the people who fixed this years ago.
[20:19:46] <techalchemy> pyusr, you are on an old version of the platform using old software where I'm sure few resources are being allocated. I dont know the people who packaged virtualenv; I did mention the problem
[20:20:22] <pyusr> techalchemy: yeah, but it's a supported one, which has a basic expentcy of CI / CD builds not failing for example
[20:20:30] <toad_polo> In any case, upgrading pip in your virtual env as part of the first command you run seems like a very simple solution and if it works it's remarkably unlikely anyone on the python packaging side will do anything further about it.
[20:21:04] <techalchemy> it's not a new problem though, which is why I know you can use --no-download when creating the virtualenv to avoid the automated install attempt when making a virtualenv (which thus avoids the failure)
[20:21:58] <toad_polo> Plus they can't do anything about it because the lever they mostly control is what code goes in new versions of these packages, and since those versions are all frozen there's nothing much to do.
[20:22:10] <sangy> I'm curious though, so pip calls pypi and says "give me setuptools" and pypi answers with "yeah, you should get 45.0.2" and nowhere there's any discrimination about the client running on py2?
[20:22:57] <techalchemy> pip then filters the list of packages to what it determines is ok to install
[20:23:28] <sangy> right, so shouldn't it 1) download the simple indeces (which iirc have a bunch of versions of the package) and 2) filter until it finds something that's satisfiable?
[20:23:47] <ngoldbaum> and sufficiently old versions of pip don't know about the extra metadata that's in the package that says what versions of python the package supports
[20:24:23] <techalchemy> it satisifies all the dependency constraints of "setuptools >= 18" or whatever
[20:32:31] <techalchemy> if you have enterprise ci/cd support with canonical i'm sure they will find a way to help you
[20:34:29] <techalchemy> well if you want to move the issue forward you'll need to address it with canonical, everyone in this channel is a volunteer who works on python packaging in their spare time for free
[20:35:05] <ngoldbaum> could also consider upgrading to a newer LTS distro, doesn't 16.04 go EOL in april?
[20:35:14] <techalchemy> canonical LTS does have enterprise support though I suspect they won't invest much effort fixing something that is basically dead
[20:35:39] <techalchemy> (unless, again, you were a customer, in which case from my experience they would do basically whatever you need)
[20:39:42] <sangy> pyusr: I've personally found that updating to more recent toolchains to be beneficial in *many* ways. E.g., a project that I work on was a hell to make work on certain ubuntu versions because they were still shipping gnupg <2.0 (which was ancient)
[20:39:57] <techalchemy> so from 2021-2024 you can (through 'ubuntu advantage') pay for 'extended security maintenance' (or as an individual you can enroll 3 machines in ubuntu advantage for free I think)
[20:41:07] <pyusr> sangy: there are production systems which run ubunt u14.04 with docker and kubernetes from 5 years ago :)
[20:42:24] <ngoldbaum> coming into upstream's IRC channel and complaining is not going to be terribly productive usually, FWIW
[20:42:48] <sangy> don't assume that because we contribute to open source we also don't work in industry
[20:43:07] <techalchemy> yeah like i said before most people in here are volunteers when it comes to open source
[20:43:24] <techalchemy> so the standard answer for your question about our thing we do for free in our spare time is: upgrade to the latest version