Log file Viewer
#pypa logs for Thursday the 6th of March, 2014
[10:46:35] <Arfrever> jaraco: Is there a plan to have 2.x-maintenance branch and more 2.* releases of Setuptools?
[10:47:01] <jaraco> Arfrever: I wouldn't be opposed if that's necessary. I'm hoping it's not necessary.
[10:48:10] <jaraco> But if there are high-priority fixes that need to go out to address defects in 2.x, that can be done. I'll add the bookmark just for clarity.
[11:05:28] <jaraco> My expectation is this - since the deprecation has been around for months, no one using those packages is keeping up with late setuptools releases.
[11:06:03] <jaraco> I put out the beta with several weeks' notice to give individuals and companies time to adopt and test the beta and identify shortcomings.
[11:06:23] <jaraco> But nothing short of actually releasing the deprecation is going to actually spur action.
[11:07:31] <jaraco> I expect then that in a month's time, most packages will have new versions released to support setuptools 3, which will be sufficient for most use cases.
[12:06:19] <ronny> anyone aware of a way to block out setuptools 3.0? it did just hit our deployments, and breaks some packages
[12:15:45] <dw> jaraco: if you haven't already released 3.0.2, there's a trivial syntax error in ez_setup.py tip
[12:28:18] <dw> oh. i see you made a release in the same minute you merged that request, so i guess this wasn't tested at all
[12:39:37] <ronny> im still trying to figure what causes the deployment ot install setuptools 3.0, when i run the same commands on a terminal, i get a 0.6c11
[12:54:24] <dstufft> jaraco: my guess is that people generally use whichever setuptools comes with their venv
[13:01:39] <ronny> dstufft: the target machine installs everything wit --no-index and --find-links, for some reason virtualenv itself puts in a old egg and a new egg
[13:04:10] <dstufft> back then virtualenv tried to download the latest version of pip/setuptools from PyPI
[13:04:35] <dstufft> You can use the bundled versions only by passing --never-download to the virtualenv creation
[13:04:51] <dstufft> or upgrade to a more recent virtualenv which no longer downloads and only ever uses the bundled versions
[13:33:42] <dw> anyone with commit setuptools commit privs fancy making the 1 byte trivial fix that'll allow many, many CI systems to resume functioning?
[13:34:15] <dw> sed -ir s/Zipfile/ZipFie/ ez_setup.py # https://bitbucket.org/pypa/setuptools/issue/158/
[13:37:52] <dw> i do py-lmdb maintainance one day every month, and for the past 4/5 months on that day i've discovered CI has broken in the meantime. i've removed every pypa dependency except setuptools, and looks like that too is about to be replaced with distutils. its really hard to put in words how unbelievable that we can't have such critical, basic tools work reliably on all versions of python>=2.5 for more than 30 days at a time
[13:41:34] <dw> apologies for pugnaciousness, its just wearing a bit thin.. will mail the list with proposal for a set of black box tests. clearly whatever tests exist just aren't enough
[13:50:26] <dstufft> Well most (all?) of the pypa tools require 2.6+ at this point, there's been some breakage in pip and virtualenv but most of those were planned breakages which went through a deprecation cycle. There were some bugs but I think the nature of the big ball of string we have now is that it's incredibly hard to change anything and not break something for someone :/
[13:52:57] <dw> i appreciate your pain, though i don't think the lack of testing is helping much. im happy to volunteer time sort out a TravisCI config that simply tries to run the supported versions in their target environment, its a really basic script to write
[13:53:56] <dstufft> dw: well I don't really work on setuptools and i'm not sure what testing is in place there
[13:54:27] <dstufft> for pip/virtualenv we do run tests via Travis CI (although virtualenv's test suite is hilarously bad)
[13:55:00] <dw> unit tests have low cost/benefit for this kind of thing. literally a 1 line shell script is vastly more useful
[13:55:35] <dstufft> something like 75% of pip's test suite involves invoking pip and asserting things about the state of the world afterwords
[13:56:34] <dstufft> Our test suite isn't perfect (there is more than one area where there's little or no coverage) but we don't have zero coverage :)
[13:57:10] <dw> this example on setuptools master right now, its similar to the 3.1 SSL bug from a few months back. you dont need to explicitly test it, just run the code under an environment where it should work
[13:59:18] <dstufft> Truthfully, outside of the breakages that were done on purpose, the biggest problem comes from the combinatorial of different environments, especially since we have a broken system (distutils) that is being monkeypatched by another system (setuptools) and on top of that we have pip wrapping around all of that to apply different set of expections.
[14:00:04] <dstufft> Especially when those systems have been more or less staginant for a long time so people worked around them and applied their own hacks on top of all of them :/
[14:00:22] <dstufft> But yea, some bugs are embarassing and just point to a need for better test infra :/
[14:01:31] <dstufft> But these set of programs are also somewhat fundamentally hard to test in their current state :/
[14:05:29] <dstufft> dw: FWIW one such example of a problem like that, which broke things for a lot of people was (and I can't recall the exact details or pip version now) that when you upgraded from distribute to setuptools via the empty distribute package which only depended on setuptools things would break because it would uninstall distribute, then try to install setuptools.
[14:06:22] <dstufft> But! Because of some fiddly bits this only ever occured if you were upgrading distribute to setuptools in addition to other packages and! those other packages came after setuptools/distribute in the internal list of packages
[14:07:10] <dw> check out https://github.com/dw/py-lmdb/blob/master/.runtests-travisci.sh and https://github.com/dw/py-lmdb/blob/master/.runtests-ubuntu-12-04.sh .. these scripts are incredibly ugly, but a reasonable example of how trivial it is to hit a good set of meaningful combinations. some bikeshed of these scripts is what i'm going to propose
[14:08:40] <dstufft> Yea, i'm not sure what setuptools testing infra is like so I can't say much about it :/
[14:09:50] <dstufft> for the projects i'm involved with it's basically 2.6, 2.7, PyPy, 3.2, 3.3, and 3.4 (well techincally 3.4.1 since I was pulling from a checkout for this) on Ubuntu Linux 12.04 via travis, and it all uses virtualenvs (which means we've missed bugs that only occur on the system installation and not inside of a venv)
[14:11:54] <dw> tox+venv work wonderfully for me on osx, but for i gave up on both under travis (reasons elude me now)
[14:13:45] <dstufft> One of the recent OSX bugs was that it when you were inside a virtualenv, pip would install scripts to the non virtualenv bin path
[14:15:05] <dstufft> We have OSX turned on for us in Travis, just no ones had the time to make it work because the OSX builders don't do Python out of the box :/
[14:23:12] <ronny> dstufft: does pip check if the shared objects in a wheel are importable by the target python?
[14:28:57] <dw> should be quite easy to get a python install sorted out on a base osx install, or somesuch
[14:29:41] <dstufft> actually anothe rproject i'm involved with has it more or less setup already, I just haven't had the time to port it over to pip
[14:34:52] <dw> jaraco: i guess im partly to blame for having CI point at master, but this is partially out of desparation after spending 48h trying to get a working CI env :)
[14:40:37] <dstufft> jaraco: I don't think using DeprecationWarning is going to be very useful for setuptools :/
[14:40:46] <jaraco> dstufft: https://bitbucket.org/cffi/cffi/issue/138/remove-use-of-features-removed-in
[14:41:17] <dstufft> Since 2.7 Python disables DeprecationWarning by default, and I'm guessing that not very many people turn them back on when installing packages
[14:41:59] <dstufft> you'd have to use the environment variable to turn them on for setuptools when isntalling from pip at all, and I'm not sure if that would work either
[14:55:09] <ronny> dstufft: can you report that? i cant do it at work due to lack of my github password
[15:04:30] <jaraco> ronny: https://bitbucket.org/pypa/setuptools/issue/58/feature-require_features-with-require#comment-5456923
[15:20:12] <Arfrever> dstufft: Re "you'd have to use the environment variable to turn them on for setuptools": A call to warnings.filterwarnings("always", "Features are deprecated ...", DeprecationWarning) before warnings.warn() would enable this warning.
[15:21:09] <dstufft> Arfrever: right, setuptools could force them on (and probably should if it's going to use deprecationwarning to signal deprecations)
[15:21:24] <dstufft> my statement was about what me as and enduser would need to do to see the warning as it exists today
[15:39:54] <jaraco> Arfrever: good idea - I've created https://bitbucket.org/pypa/setuptools/issue/159/enforce-visibility-of-deprecation-warnings
[16:03:11] <ronny> dstufft: any idea how to make wheel build binaries that dont include symbol versions?
[18:36:51] <dstufft> it's an unfortinate thing that often times the people who need to see our deprecation warnings aren't the people currently using that software
[18:37:14] <dstufft> e.g. even if a package author was still using 0.6, this change would affect them
[18:40:42] <jaraco> indeed - and my hope was that by adding the deprecation warning, it would signal the users of recent setuptools versions to signal the package maintainers of the upcoming issue.
[19:06:30] <Alex_Gaynor> Sorry for being so harsh btw, it's just that setuptools is sort of the gatekeeper to most people's first experience with 3rd party packates, and I want that to be a good one
[19:53:50] <dstufft> jaraco: Alex_Gaynor yes, I think important going forward is to figure out what the best way is to communicate this to people
[19:54:37] <dstufft> I can only guess they'll ignore the warnings for setuptools removing Feature again
[19:59:42] <qwcode> dstufft, well color, and actually having deprecation messages at all. don't think pip had ever done either before (or needed to). people DID notice the bundle messages. they were pretty loud
[20:00:56] <dstufft> qwcode: people didn't notice the warning about --allow-external and --allow-(insecure|unverified)
[20:04:10] <qwcode> dstufft, it seems a lot of the complaints with those options has just been "I hate the effort involved", not so much, "you didn't tell me"
[20:13:06] <qwcode> dstufft, the dependency links removal will be the first big pip change, that will force pypi packages to re-package to be compatible, right? that's what was so dramatic about the setuptools change.
[20:13:46] <dstufft> well they don't actually have to repackage for dependency links either in the general sense
[20:14:19] <dstufft> if the only reason they are using them is because something isn't uploaded to PyPI, you can just upload it to PyPI
[20:14:42] <dstufft> but if the reason you're using them is because you forked it and now you want to distribue that fork, then you have to modify the package to point to your forked name
[20:16:34] <qwcode> dstufft, I guess we have no idea how many of the 4005 were pointing to vcs patches with same name as pre-existing pypi packages
[21:52:12] <qwcode> jaraco, to be clear, I missed some of the IRC earlier, you're going to add Feature back real quick?
[21:53:36] <qwcode> jaraco, btw, see discussion on distutils. it's unclear on how extra could ever replace Feature
[21:54:30] <jaraco> I think the usage of Feature in the wild is different than the usage for which it was intended.
[21:55:15] <jaraco> Ivo: in general, that wouldn't work... because ez_setup.py gets bumped to a _future_ version.
[21:55:27] <qwcode> jaraco, maybe the new deprecation could point to a docs page that explains the rationale of how it's reallly ok to get rid of it
[21:56:19] <qwcode> jaraco, the sqlalchemy case where he wants to have --without-cextension is good one to explain
[21:58:01] <Ivo> I think one of the major issues with python packaging in particular is that it hasn't had any communication lines open to the 'general python public' for so long
[21:58:29] <Ivo> so now when it's trying to get better, and obviously there are tonnes of growing pains, it has no way to share these with anyone
[21:58:47] <Ivo> or alternately, they never know to listen in the first place before something breaks for them
[21:59:53] <Ivo> qwcode: but it might still be a good idea, the more ppl being aware of things the better
[22:01:50] <qwcode> Ivo, I want to write up the deprecation policy section I posted an issue about. that could be a good thing to post to major lists. "See here, we have a policy... we're trying..."
[22:32:11] <pmxbot> !8ball (8) !acronym (ac) !anchorman !annoy (a, bother) !bender (bend) !bitchingisuseless (qbiu) !blame !bless !boo !bottom10 (bottom) !calc !chain !cheer (c) !c
[22:32:11] <pmxbot> ompliment (surreal) !config !ctlaltdel (controlaltdelete, controlaltdelete, cad, restart, quit) !curse !dance (d) !danke (dankeschoen, ds, gm, germanmotivate) !
[22:32:12] <pmxbot> deal !define (def) !demotivate (dm) !disembowel (dis, eviscerate) !duck (ducky) !embowel (reembowel) !emergencycompliment (ec, emercomp) !esperantomotivate (em)
[22:32:12] <pmxbot> !excuse (e) !featurecreep (fc) !fight !flip !fm (frenchmotivate) !fml !gettowork (gtw) !golfclap (clap) !google (g) !grail !haiku !hal (2001) !hangover !help (
[22:32:12] <pmxbot> h) !hire !imotivate (im, ironicmotivate) !insult !jm (japanesemotivate) !job (card) !karma (k) !keelhaul (kh) !klingon (klingonism) !lm (latinmotivate) !log !lo
[22:32:13] <pmxbot> go !logs !lunch (lunchpick, lunchpicker) !meaculpa (apology, apologize) !motivate (m, appreciate, thanks, thank, gracias) !murphy (law) !nailedit (nail, n) !nas
[22:32:13] <pmxbot> tygram (nerf, passive, bcc) !norris () !notify !oregontrail (otrail) !panic (pc) !password (pw, passwd) !paste !pick (p, p:, pick:) !pm (piratemotivate) !progre
[22:32:13] <pmxbot> ss !quote (q) !r (r) !resolv !roll !rubberstamp (approve) !saysomething !send_text !simpsons (simp) !stab (shank, shiv) !storytime (story) !strategy !strike !tg
[22:32:14] <pmxbot> if !therethere (poor, comfort) !ticker (t) !time !tinytear (tt, tear, cry) !top10 (top) !troutslap (slap, ts) !urbandict (urb, ud, urbandictionary, urbandefine,
[22:32:14] <pmxbot> urbandef, urbdef) !version (v, e, r) !weather (w) !where (last, seen, lastseen) !wolframalpha (wa) !zinger (zing) !zoidberg (zoid)