Log file Viewer
#pypa-dev logs for Friday the 4th of April, 2014
[04:52:57] <Ivo> dstufft: would appreciate to know if I got anything wrong with https://github.com/pypa/pip/issues/1707#issuecomment-39531726
[04:55:28] <dstufft> I'd have to dig further into it, but my bet is that anaconda can solve it by fixing their Python to point to those dirs by default
[10:20:41] <pf_moore> Ivo: I added a comment - sounds to me like anaconda aren't compatible with standard builds so maybe they should host a stack-specific wheel archive if they want to support pip
[10:21:18] <pf_moore> But yes, as dstufft says, they sound like they are behaving much like a separate distro in terms of compatibility
[10:52:06] <Ivo> pf_moore: when they have a custom binary build to do they usually host their own conda recipe, not a wheel
[11:20:54] <dstufft> if kung fu was terrible and made you remember that everything is terrible, yes possibly
[11:23:05] <ghickman> oh yea I stumbled onto that trying to work out how to get globally entry points to work in a virtualenv
[11:29:40] <dstufft> I'm learning things that I never wanted to know about how the Python interpreter starts up
[12:32:07] <pf_moore> Have you found the bit where the interpreter sacrifices a goat to the great god shub-niggurath yet?
[13:32:01] <Ivo> pf_moore: my simplified, non-PC interpretation of that pyOpenSSL issue is that Anaconda hasn't supplied a conda recipe for cryptography yet, and the user has come wining to pip asking why we don't just automatically have the same level of convenience and just use what anaconda has
[14:06:30] <pf_moore> Ivo: yeah, probably. When a curated stack like conda works, it's great, but when they don't have the package you want, everything else looks clumsy
[14:07:13] <pf_moore> You have to do all the work the distro people normally do for you, and it's unpleasant :-)
[14:08:03] <Ivo> dstufft's suggestion that maybe they could modify their python's distutils to point to their own header / libarary folders is a good one though
[14:09:41] <pf_moore> Not sure how good having "conda python" "distro python" and "original python" is though...
[14:12:11] <pf_moore> Ivo: If you look at the code for virtualenv it makes your eyes bleed, using core stuff lets you keep your sight
[14:12:24] <Ivo> there have been a lot of PRs building up on virtualenv that haven't been merged or much reviewed or anything :/
[14:12:47] <dstufft> I haven't done any verification that this all works except looking at sys.path FWIW
[14:13:06] <dstufft> so I need to still figure out what else we need to do to make this function correctly
[14:13:08] <Ivo> I assume its just because there's noone around with deep arcane knowledge of how virtualenv should work across everything to actually say yes to things
[14:13:49] <pf_moore> Basically yes, I do what I can but daren't say something for <whatever Unix platform> works OK
[14:14:15] <Ivo> I assume https://github.com/pypa/virtualenv/pull/516 not being in might start pisssing off ppl using embedded arms for dev
[14:14:18] <dstufft> I think it's a combination of virtualenv code being bad, not having hardly any tests, virtualenv being a crap ton of hacks for stuff, and likewise
[14:14:44] <pf_moore> It's a nasty collection of bugfixes for obscure systems that 2 1/2 people in the universe use
[14:16:13] <pf_moore> Well, there's activate.fish. WTF is the fish shell and how many people use it?
[14:17:36] <dstufft> pf_moore: some parts of this I suspect we'll still do ;) We might still handle the activation scripts, I know we'll handle installing pip/setuptools
[14:18:12] <pf_moore> Actually, one of the things I think is *good* about virtualenv is that it caters for this sort of stuff
[14:18:15] <dstufft> if we push the isolation into the core interpreter for 3.3+, then for 3.3+ suddenly all the horrible hacks in virtualenv no longer matter
[14:18:59] <pf_moore> Yeah, the key thing here is to use the core for the isolation (that's where it should be supported) and let virtualenv do the user experience stuff
[14:19:55] <pf_moore> With a reasonable amount of luck, the isolation thing is actually pretty stable, and it's pretty much all win
[14:20:40] <tomprince> If you add a .cfg file, do you even need feature detection? Which is checked first?
[14:20:41] <Ivo> https://gist.github.com/datagrok/2199506#simplifying-activate would be nice for 2.0
[14:21:15] <dstufft> A good half of what i'm doing at the moment is reverse engineering virtualenv and CPython to figure out why various hacks exist and figure out a) if we still need them (virtualenv started as like a 2.4? project I think, a lot has changed since that and 2.6) and b) is there a simpler way to do it
[14:21:50] <dstufft> tomprince: we need feature detection to determine in virtualenv if when we're creating a virtual environment if we need to use the venv method or the hack method
[14:23:24] <dstufft> it's replacing a hardcoded in virtualenv site.py script that as far as I can tell was first added in the first iterations of virtualenv and has been frankensteined to a different version
[14:24:08] <tomprince> dstufft: If the hack uses pyvenv.cfg and if python >=3.3 looks for pyvenv.cfg first (and then ignores lib/pythonX.Y if it find it), it seems like you don't need feature detection.
[14:24:46] <tomprince> You could still use it to figure out if you could skip adding the os.py/site.py, but that wouldn't strictly be necessary.
[14:26:22] <pf_moore> They discussed using bin but decided too many tools hard-code the check and would go crazy if it changed...
[14:28:46] <pf_moore> Ivo: The stuff about an activate that starts a new shell is nice but sometimes you want that, sometimes you want to set PATH in your shell
[14:31:03] <pf_moore> (impossible to do binary redirection of stdin/stdout because everything's aon object or some such crap
[14:31:58] <dstufft> pf_moore: on the other hand, at least Windows is Windows and you don't have like 20 different variations of Windows ;)
[14:33:25] <dstufft> I don't actually hate Windows, it's just not what I use and it's different enough from what I do use that I have a hard time predicting what will work there :]
[14:33:40] <pf_moore> The main problem with Win vs Unix is that the kernels are *utterly* different and most good userland tools come from Unix but don't deal with that fact well (and why should they?)
[14:34:26] <dstufft> I use OSX because it's the only *nix with a UI that doesn't make me want to kill things
[14:36:18] <pf_moore> Quite a lot of MS is pretty open source friendly - they are open sourcing all sorts of things ATM (C# and VB compilers most recently)
[15:25:37] <Ivo> pf_moore: some dude coincidentally told me the actual guts of the compiler wasn't actually OS'ed, although maybe he's wrong
[15:31:33] <pf_moore> Ivo: I'd heard something about bootstrapping but maybe there's something in that, but whatever - getting a company as big as MS to do *anything* like that is pretty impressive
[15:55:58] <techtonik> dstufft: I named "hacks" warts and made a personal public list - wish somebody else prepared one, so I won't have to do this
[15:59:56] <techtonik> pf_moore: They've changed executive to a newer, morer updated version, which knows about open source.
[16:01:16] <techtonik> Python is a good OS on its own as long as there are not many injections from OS stuff.
[19:08:23] <barry> and yep, it was in some debian-specific patches to distutils and sysconfig. it sure would be nice if we could upstream some of that
[19:54:39] <barry> well, close. i've got it installing pip to the right place, but sys.path isn't right yet
[20:32:35] <barry> dstufft: ['', '/home/barry/env/python', '/usr/lib/python3.4', '/usr/lib/python3.4/plat-x86_64-linux-gnu', '/usr/lib/python3.4/lib-dynload']
[20:36:29] <dstufft> barry: I'm totally working to backport 3.3's venv to 2.6 through runtime monkeypatching via AST manipulations
[20:37:28] <dstufft> barry: my goal is to get something that will use the built in isolation on 3.3+ and something that will function as a dynamically patched Python on < 3.3
[20:38:35] <dstufft> https://gist.github.com/dstufft/a7514cf479ee97c7289b < that works, but I need to add more files than site.py to do that too
[20:38:56] <barry> dstufft: what i *really* want is a blessed, common api to answer that question, i.e. "am i in a virtual env". i think there are now at least 3 ways to guess that depending on whether you're using pyvenv or virtualenv. i want one single documented way (ideally sys.i_am_in_a_virtual_env or some such)
[20:39:41] <dstufft> if getattr(sys, "base_prefix", sys.prefix) != sys.prefix: will work for this if I'm successful
[20:41:27] <dstufft> if we want to wrap that up in a sys function I'm totally OK with that, though you'd have to do something like if getattr(sys, "am_i_in_virtualenv", lambda: False)()
[20:41:51] <dstufft> because virtualenv can't add stuff to sys if you're inside of virtualenv on older pythons
[20:42:46] <dstufft> if you only care about python 3.3+ you can just do sys.base_prefix != sys.prefix too
[20:45:33] <dstufft> but yea, that's my goal. To make the hacks virtualenv uses to work match pyvenv more, and to use pyvenv itself on 3.3+ so that everything can just standardize around the stdlib way (and maybe someday venv can drop all it's hacks and just be a UI thing on top of venv that's easier to update to a newer version of pip and setuptools)
[20:48:56] <dstufft> kinda wish we could just roll a 2.8 for PEP453, the TLS stuff, and maybe a few other things :V
[20:49:44] <Ivo> dstufft: do you want python 3 to go into disuse for either another 5 years, or forever?
[20:50:38] <dstufft> I don't think adding virtualenv and pip to the core is going to make anyone stay on 2.x that wasn't going to stay there anyways
[20:51:37] <dstufft> I do think that not having a 2.8 was a mistake, though I don't know that releasing a 2.8 *now* is fixing that mistake, or just adding another mistake
[20:52:04] <Ivo> the thing stopping people going to 3, imho, is the long tail of not yet ported packages, which is still most definitely present, but is actually receding at a pleasant pace also imho
[20:52:13] <barry> if we had released a 2.8, people would just be whinging about 2.9 and it wouldn't end
[20:53:39] <barry> well, a lot of people will never port. it's a sunk cost for projects that are already working fine on py2. what does it benefit them to pay for a port? (remember, they don't care about all the new features). what i think we need to concentrate on is making it a no-brainer for *new* projects to start in py3 and let the old projects bitrot themselves to their inevitable death
[20:54:18] <barry> but you're right, the mix of py3 compatible packages still isn't quite there. mm3 f.e. can't be py3 because of two remaining libraries (restish and storm)
[20:54:40] <dstufft> barry: well no I think the problem is, it wasn't until 3.3 that it was even reasonable to use Python 3 the language for a for serious project (excluding the ecosystem, just talking about core language stuff), so it's been roughly 2 years since a Python 3 tha someone could actually reasonably use was released
[20:55:00] <barry> and we've very nearly eliminated py3 from various ubuntu images, and almost switched debian over to py3 by default (which != /usr/bin/python changing ever)
[20:55:13] <Ivo> a year ago, you have some django application with 15 odd dependencies and 50% of them or more are python 2 only. Now, maybe 70% are python 3 as well, 15% can be replaced with other stuff, 5% you could rewrite. There's still a 10% (for some projects thats 20% for others thats 0%) but its getting there quite nicely tbh and I have no idea why a sudden spate of "OH NO PY3 IS DYING IF WERE NOT ALL THERE BY NEXT MONTH!" posts came out at the start of this year
[20:56:07] <dstufft> so you have a bunch of people angry because it wasn't into relatively recently that Python 3 was even a reasonable choice for a _new_ project, meanwhile 2.x wasn't getting anything new
[20:56:08] <barry> yeah, 3.2 was usable. i certainly used it for stuff. but 3.3 was better, 3.4 is better still, and 3.5 will be mo' better. but yes, you could very definitely build new projects in 3.3 and 3.4 is a fantastic base to start projects on
[20:57:22] <barry> i do think the web domain is a little trickier (e.g. restish is web-related and difficult to port), but i'm hoping for that to be a focus of 3.5
[20:57:24] <dstufft> e.g. I think it would have been better if instead of "supports" being "we'll fix bugs and stuff" for 2.x until 2015 but 2.7 is going to be a LTS sort of thing it would have been better to just stick to the normal release schedule for 2.x and squeeze another one in the middle of the 2.7 LTS cycle
[21:15:01] <agronholm> the trouble is that the current "no new language features" policy has resulted in the 3.4 release being less exciting than it could have been
[21:26:30] <agronholm> I'm pretty sure there are people dying to add $FAVORITE_THINGAMAJIG$ to python
[21:28:21] <Arfrever> "This PEP proposes a temporary moratorium (suspension) of all changes to the Python language syntax, semantics, and built-ins for a period of at least two years from the release of Python 3.1. In particular, the moratorium would include Python 3.2 (to be released 18-24 months after 3.1) but allow Python 3.3 (assuming it is not released prematurely) to once again include language changes."
[21:32:48] <techtonik> to stop people from dying, python needs to evolve into ownPython, so that people can add their own $FAVORITE_THINGAMAJIG$ to their own version
[21:35:08] <techtonik> i don't know why after 15 years there is no language with pluggable semantics
[22:04:20] <barry> there's no language moratorium for 3.4 and there are tons of new stuff, but i'm not sure that language syntax alone is a very good indicator of anything. by necessity, syntax changes have a very high bar to hurdle
[22:46:45] <agronholm> barry: the problem is that since so many of the new features can be backported, there is no pressure for the costly migration to 3.x
[22:48:08] <tomprince> agronholm: Why is that a problem? If there isn't a reason to upgrade, there isn't a reason to upgrade.
[22:49:14] <agronholm> because it means getting rid of 2.x will be harder and as most people continue to use 2.x, library authors won't feel pressure to port their codebases to 3.x
[22:49:46] <tomprince> And good news for those of use that don't feel the benefit of 3.x outweights the cost of porting.
[22:50:42] <agronholm> for me personally, just the clear distinction between bytes and unicode is worth it
[22:50:54] <ecoutu> hey guys, i fixed a bug in the pip python library and i'm trying to write a test for it. i can't get running individual tests working as per this documentation: http://www.pip-installer.org/en/1.2.1/contributing.html#running-tests-directly-with-nose - from the pip directory, i'm running - nosetests tests/functional/test_install.py:test_install_from_local_directory and i'm getting: TypeError: test_install_from_local_directory() takes exactly 2
[22:50:54] <ecoutu> arguments (0 given) - I'm following the documentation but it looks like i'm running the tests wrong.
[22:50:59] <barry> not really. most library authors know that they have to port to python 3 by now if they want to stay relevant. the ones that are lagging are lagging due to resources (i.e. round tuits) or because they are effectively abandoned, not because they don't understand the need.
[23:38:53] <meltingwax> did someone turn off travis on the pip github repo? i can't seem to get it to kick off for updates to my PR. been this way since yesterday