pmxbot IRC Log Viewer

[01:19:26] <r1chardj0n3s> it would be nice to have a better solution for users who pop up like this occasionally https://sourceforge.net/p/pypi/support-requests/448/

[01:19:31] <pmxbot> jaraco pushed 12 commits to setuptools (https://bitbucket.org/pypa/setuptools/) :

[01:19:31] <pmxbot> Reuse list2cmdline for argument quoting.

[01:19:31] <pmxbot> Extract method for getting script args

[01:19:31] <pmxbot> No need to pass the writer - just invoke it directly.

[01:19:31] <pmxbot> Move trailing comment

[01:19:31] <pmxbot> Move decision logic about windows/header generation closer to install_scripts, as it doesn't appear to be used elsewhere.

[01:19:31] <pmxbot> Move get_script_header into ScriptWriter

[01:19:31] <pmxbot> Remove unused imports

[01:19:31] <pmxbot> Moved get_script_header into ScriptWriter class

[01:19:31] <pmxbot> Deprecate and remove usage of easy_install.get_script_header.

[01:19:31] <pmxbot> Rename _gen_args to get_args (for consistency).

[01:19:31] <pmxbot> Extract method for parsing options.

[01:19:31] <pmxbot> Extract method for handling non-ascii exe. Strip out excess whitespace from option handling.

[01:19:49] <r1chardj0n3s> dstufft .. my last message (I got pmxbot'ed before typing this)

[01:20:51] <dstufft> r1chardj0n3s: should file a ticket agaisnt warehouse

[01:21:46] <dstufft> I don't think it's unreasonable to have a "soft limit" which is enforced by Warehouse with the ability to relax that limit for particular projects and a "hard limit" enforced by Fastly and/or nginx

[01:23:19] <r1chardj0n3s> thing is, I really only get requests like this once every few years

[01:23:22] <r1chardj0n3s> it's so rare

[01:23:34] <r1chardj0n3s> so it

[01:23:42] <r1chardj0n3s> hm

[01:24:37] <dstufft> r1chardj0n3s: well I don't think there's a better solution other than tell them to go away or add support to whitelist em

[01:25:18] <r1chardj0n3s> dstufft: yah

[01:25:51] <dstufft> almost got 100% test coverage for virtualenv-rewrite

[01:25:59] <r1chardj0n3s> we're still getting reports of the cookie-related 503 errors

[01:26:07] <r1chardj0n3s> \o/ virtualenv-rewrite

[01:26:09] <dstufft> r1chardj0n3s: yea it's on my stack of things to figure out

[01:26:25] <dstufft> it appears timeout related

[01:26:39] <dstufft> but i'm not sure what cookie would be causing pypi to spin there and do nothing until the timeout happens

[01:26:47] <r1chardj0n3s> fastly-related

[01:27:18] <dstufft> well the 503 error is a Fatly generated error that says "hey we hit the timeout waiting for a request from the backend server"

[01:27:29] <r1chardj0n3s> hm

[01:27:31] <dstufft> s/request/response/

[01:27:34] <r1chardj0n3s> yeah

[01:27:55] <dstufft> https://docs.fastly.com/guides/backend-servers/503-error-explanations

[01:33:01] <r1chardj0n3s> what *was* the nginx file size limit?

[01:34:25] <r1chardj0n3s> (and what is it now?)

[01:34:33] <r1chardj0n3s> the pypi limit is 60M

[01:34:59] <r1chardj0n3s> (sorry, dstufft ^^)

[01:35:12] <dstufft> um

[01:35:15] <dstufft> sec

[01:36:03] <dstufft> https://github.com/python/pypi-salt/blob/master/provisioning/salt/roots/salt/pypi/config/pypi.nginx.app.conf.jinja#L18

[01:36:13] <r1chardj0n3s> I would *never* find that ;)

[01:36:18] <r1chardj0n3s> thanks!

[01:37:15] <r1chardj0n3s> OMFG PyPI support backlog cleared

[01:37:33] <r1chardj0n3s> (for now, pending responses including to one dispute over an unwanted fork)

[01:38:15] <dstufft> r1chardj0n3s: heh, yea things are a little harder to find now, but that's the price we pay for being able to spin up new web nodes super quick

[01:38:21] <r1chardj0n3s> yep

[01:40:25] <dstufft> lol

[01:40:28] <dstufft> legacy-virtualenv ftl

[01:40:40] <dstufft> python -c "import site; print(site.getsitepackages())" works outside of a virtualenv, fails inside of a legacy-virtualenv

[01:41:11] <r1chardj0n3s> nice

[01:49:31] <dstufft> (note it works inside of a virtualenv-rewrite both with the venv builder and the legacy builder)

[01:53:21] <r1chardj0n3s> \o/

[01:54:37] <dstufft> ionelmc: sorry, I rebased my branch without thinking about it

[01:57:09] <ionelmc> just a moment, bad rebase

[01:58:09] <ionelmc> dstufft: "functional" is meant for the integration tests right?

[01:58:16] <dstufft> ionelmc: yea

[02:29:31] <dstufft> damn

[02:29:35] <dstufft> appveyor is sloooow

[02:29:49] <sigmavirus24> But of course! it's windows =P

[02:30:29] <dstufft> yea but I mean it sits there queued forever

[02:30:40] <Alex_Gaynor> dstufft: feeling pretty good about travis now ? :_)

[02:31:39] <dstufft> Alex_Gaynor: feeling better about it

[02:31:44] <dstufft> too bad it doesn't do windows

[02:32:28] <Alex_Gaynor> maybe some day

[02:33:21] <dstufft> https://ci.appveyor.com/project/dstufft/virtualenv/build/3 vs https://travis-ci.org/pypa/virtualenv/builds/45901551

[02:33:54] <Alex_Gaynor> Not wild about the blue background for the terminal

[02:34:49] <sigmavirus24> Alex_Gaynor: me either

[02:36:46] <dstufft> http://d.stufft.io/image/3X1Y2I0D1E3L is nice though

[02:38:33] <Alex_Gaynor> dstufft: yeah, that's pretty nice, hope jenkins gets support for that soon so we can use it on pyca/cryptographyt

[02:41:02] <dstufft> Alex_Gaynor: I'm supposed to hae a video chat with joshk sometime to talk about travis enterprise and seeing if it makes sense for PyPA and PyCA to run our own instance

[02:41:21] <Alex_Gaynor> dstufft: cheers

[02:42:04] <dstufft> (and if that would eliminate the need to run Jenkins, though Windows is likely a problem with that I guess since probably Travis doesn't support winrm or w/e the hell)

[02:44:05] <Alex_Gaynor> dstufft: I'd honestly rather find ways for us to give them money for more computers or whatever than to run our own stuff

[02:44:34] <dstufft> Alex_Gaynor: more compute power isn't going to give us the ability to run FreeBSD or whatever the fuck

[02:45:14] <dstufft> as far as I know their plan for custom images is going to revolve around Docker

[02:45:22] <dstufft> which means nothing non-linux for that

[02:45:23] <Alex_Gaynor> dstufft: mm, you're saying if we ran an enterprise we could have our images be FreeBSD, wacky debian, etc.

[02:45:35] <dstufft> Alex_Gaynor: that's one of the things I'm planning on asking him yes

[02:45:46] <Alex_Gaynor> dstufft: I think there's a plan for Docker to support jails? (should we move this to a different channel?)

[02:46:15] <dstufft> Alex_Gaynor: eh, it's just as relevant for PyPA, because we need something to cover those OSs, we're just deficient in that

[02:46:59] <dstufft> Alex_Gaynor: they might cover jails later on yea maybe, I don't know if OSX will work or not on Travis enterprise. I mean down the road if travis-ci.org ends up being able to customize things enough that we no longer need to run our own then I don't see a problem dropping it

[02:47:14] <dstufft> but right now PyCA in particular is already running test infra

[02:47:19] <dstufft> it's not like it's going from not running to running

[02:47:31] <Alex_Gaynor> true

[02:48:16] <dstufft> I have no idea how the talk is going to turn out, but I've been wanting customized travis for awhile for PyPA and If igured i'd ask about both

[02:49:26] <dstufft> especially if it's possible to get OSX and Windows shoe horned into travis enterprise

[02:49:34] <dstufft> that could take the CI systems down to 1

[02:51:15] <dstufft> appveyor is so bad

[02:51:19] <dstufft> 30 minutes to start a test job

[02:52:05] <Alex_Gaynor> jesus

[03:13:42] <pmxbot> jaraco pushed 6 commits to setuptools (https://bitbucket.org/pypa/setuptools/) :

[03:13:42] <pmxbot> Added new class CommandSpec, which will be used for abstracting the command handling for script headers.

[03:13:42] <pmxbot> Update install_scripts to use CommandSpec for generating script headers.

[03:13:42] <pmxbot> Use CommandSpec in get_script_header

[03:13:42] <pmxbot> Use CommandSpec in ScriptWriter, removing now unused methods.

[03:13:42] <pmxbot> Remove redundant line

[03:13:42] <pmxbot> Add test demonstrating how a custom launch command spec that could be passed as the script executable.

[03:50:03] <pmxbot> jaraco pushed 5 commits to setuptools (https://bitbucket.org/pypa/setuptools/) :

[03:50:03] <pmxbot> Allow CommandSpec to be constructed simply from a list.

[03:50:03] <pmxbot> Add test capturing expectation around sys.executable having spaces in the name.

[03:50:03] <pmxbot> Change the way string values are interpreted from build.executable - now they must be quoted or otherwise escaped suitable for parsing by shlex.split.

[03:50:03] <pmxbot> Fix failing test (now that expectation is different).

[03:50:03] <pmxbot> Update note in changelog

[04:16:07] <r1chardj0n3s> dstufft: if you're around, I can help diagnose ^^

[04:16:15] <r1chardj0n3s> or any other pip dev I guess

[04:16:50] <dstufft> r1chardj0n3s: what does ``which pip3`` return

[04:17:44] <r1chardj0n3s> added info to the bug report

[04:17:48] <r1chardj0n3s> at the top

[04:20:17] <dstufft> hmm

[04:20:19] <dstufft> that's strange

[04:20:45] <r1chardj0n3s> yeah

[04:20:50] <dstufft> I'm not sure what would cause that, it looks like maybe something in distlib, but I don't have have the mental capacity yo really deug it right this minute

[04:20:51] <r1chardj0n3s> so

[04:21:05] <r1chardj0n3s> the reason I'm trying to do this is because I can't see to make stevedore work ;)

[04:21:17] <r1chardj0n3s> I accidentally nuked it, and now virtualenv won't work

[04:21:24] <r1chardj0n3s> but pip installing it doesn't actually install it

[04:21:40] <r1chardj0n3s> so I tried forcing the issue, and hit the problem with pip reinstall

[04:21:54] <r1chardj0n3s> (because stevedore -> pbr -> PIP WTF ARE YOU DOING PBR)

[04:21:58] <dstufft> sounds like maybe you messed up your env somehow? do the shebangs match

[04:22:26] <r1chardj0n3s> yeah, pip shebang is correct

[04:22:41] <r1chardj0n3s> hm

[04:24:34] <r1chardj0n3s> I'm confused as to why it's not installing stevedore actually

[04:24:38] <r1chardj0n3s> when I ask it to

[04:27:31] <r1chardj0n3s> agh, and there's a bug in the self-check if the user cache dir doesn't exist

[04:27:34] <r1chardj0n3s> goddammit

[04:27:42] <r1chardj0n3s> I'll fix that

[04:29:21] <r1chardj0n3s> wait, no, the dir ecists

[04:29:25] <r1chardj0n3s> wha

[04:29:32] <r1chardj0n3s> my computer is funky

[04:30:01] <r1chardj0n3s> oh, no

[04:30:03] <r1chardj0n3s> omfg

[04:46:45] <sigmavirus24> r1chardj0n3s: how you doing?

[04:47:04] <r1chardj0n3s> sigmavirus24: oh, just dealing with the computers, they laugh at me

[04:47:33] <sigmavirus24> me too

[04:48:10] <r1chardj0n3s> at the moment, I cannot install stevedore

[04:48:12] <r1chardj0n3s> \o/

[04:48:31] <sigmavirus24> i saw

[04:48:39] <sigmavirus24> GOOD JORB PBR

[04:48:52] <r1chardj0n3s> I'm *this* close to nuking 3.4 and reinstalling from scratch

[04:49:02] <sigmavirus24> NOOOO

[04:49:44] <r1chardj0n3s> clearly something is quite fubar in my 3.4 install

[04:50:30] <r1chardj0n3s> I've just fixed a bug in the pip self-check but can't run the goddamn test suite because virtualenv is broken because stevedore because pip becaUSE PBFUCKGINR

[04:51:50] <sigmavirus24> r1chardj0n3s: https://github.com/kennethreitz/python-guide/issues/494 vent your frustrations there

[04:51:53] <sigmavirus24> :D

[04:52:01] <sigmavirus24> also is this virtualenvwrapper's fault?

[04:52:08] <sigmavirus24> what if you uninstall that?

[04:52:12] <r1chardj0n3s> hm

[04:52:13] <sigmavirus24> can you uninstall pbr?

[04:52:22] <r1chardj0n3s> lemme look

[04:52:54] <r1chardj0n3s> oh crap

[04:53:07] <r1chardj0n3s> I had dags of a python3 brew install lying around

[04:53:23] <r1chardj0n3s> *shouldn't* have affected anything tho

[04:54:53] <r1chardj0n3s> nup, virtualenvwrapper did not affect the pip thing

[04:55:06] <r1chardj0n3s> (which pbr just pokes, the pip install issue is the ultimate problem

[04:57:26] <sigmavirus24> curl -s https://bootstrap.pypa.io/get-pip.py | python3.4

[04:57:34] <sigmavirus24> & done

[04:57:36] <r1chardj0n3s> too late ;)

[04:57:41] <sigmavirus24> oh?

[04:57:49] <r1chardj0n3s> 3.4 is dead, jim

[04:57:58] <sigmavirus24> beam me up scotty

[04:59:12] <r1chardj0n3s> ./configure && make install

[04:59:16] <r1chardj0n3s> been a while between those

[04:59:40] <r1chardj0n3s> oh, if you're a Race for the Galaxy fan, http://keldon.net/rftg/

[05:01:59] <r1chardj0n3s> ok, back to pip

[05:03:33] <r1chardj0n3s> dstufft ... Python 3.4 install from source, all seems to work. "pip --version" tells me it's the python 2 version, all ok. but there's no pip3. "python -m pip" works.

[05:03:41] <r1chardj0n3s> er

[05:03:46] <r1chardj0n3s> "python3 -m pip" that is

[05:04:00] <dstufft> r1chardj0n3s: what does ``python3 -m pip --version say, is it the right place?

[05:04:22] <dstufft> r1chardj0n3s: and how did you install pip? did you rely on ensurepip?

[05:04:23] <r1chardj0n3s> yup all good there "pip 6.0.3 from /usr/local/lib/python3.4/site-packages (python 3.4)"

[05:04:33] <r1chardj0n3s> I did nothing after the "make install"

[05:04:37] <r1chardj0n3s> so ensurepip I suppose

[05:04:51] <dstufft> r1chardj0n3s: you did a regular install not an altinstall?

[05:04:58] <dstufft> acutally

[05:05:13] <dstufft> r1chardj0n3s: did you compile 3.4 from tarball or from the 3.4 branch?

[05:05:18] <r1chardj0n3s> tarball

[05:05:28] <dstufft> then you had pip left over in /usr/local/lib/python3.4

[05:05:41] <dstufft> because no released version of 3.4 comes with pip 6

[05:05:49] <dstufft> so it saw you already had it and didn't do anything

[05:06:04] <dstufft> https://github.com/pypa/virtualenv/pull/697 look at all dem issues

[05:06:13] <r1chardj0n3s> argh, you're correct, I'm looking at my shell history and I did forget to nuke the lib dir :/

[05:06:15] <r1chardj0n3s> sorry!

[05:07:36] <r1chardj0n3s> *alright* now we're cooking

[05:07:52] <r1chardj0n3s> installing from source tarball breaks :)

[05:07:57] <r1chardj0n3s> PermissionError: [Errno 13] Permission denied: '/bin/easy_install-3.4'

[05:07:59] <r1chardj0n3s> lulz

[05:08:16] <r1chardj0n3s> I have no idea how or why my system is broken, but it clearly is very

[05:08:45] <dstufft> r1chardj0n3s: it looks like you installed 3.4 to / and then installed it again to /usr/local maybe?

[05:08:55] <dstufft> and the metadata is confused between the two of them

[05:09:37] <r1chardj0n3s> there is no python3 in /bin

[05:09:45] <r1chardj0n3s> woah

[05:09:52] <r1chardj0n3s> but there *is* a /lib/python3.4

[05:09:57] <r1chardj0n3s> well done old boy!

[05:10:34] <dstufft> r1chardj0n3s: this is why I use pyenv to install all my Pythons

[05:10:53] <dstufft> you get unpatched (except to make it work) Python and ti handles all of that crap

[05:11:15] <r1chardj0n3s> pyenv is new to me

[05:12:17] <r1chardj0n3s> (assume you don't mean pyvenv)

[05:14:45] <r1chardj0n3s> lol, install pyenv with homebrew, of course ;)

[05:14:47] <r1chardj0n3s> ok

[05:14:53] <r1chardj0n3s> will give it a try

[05:28:03] <dstufft> r1chardj0n3s: yea

[05:28:21] <dstufft> r1chardj0n3s: I have some env varsI set in my zsh thing to make compilation match linux more

[05:28:23] <dstufft> dunno if you care

[05:28:47] <dstufft> https://github.com/dstufft/dotfiles/blob/master/zsh/.zprofile#L10-L16

[05:29:27] <dstufft> that makes the unicode settings give you that max unicodes across all the pythons (2.x and above) and makes the other crap match what ubuntu does

[05:35:13] <r1chardj0n3s> dstufft: ok, so I got 3.4.2 installed using pyenv, but installing 2.7.8 using it failed :/

[05:35:32] <dstufft> r1chardj0n3s: how did it fail?

[05:35:39] <dstufft> I have 2.7.9 installed via it, and I had 2.7.8

[05:35:54] <r1chardj0n3s> TEST FAILED: /lib/python2.7/site-packages/ does NOT support .pth files

[05:35:56] <r1chardj0n3s> :/

[05:36:23] <r1chardj0n3s> more context

[05:36:24] <r1chardj0n3s> Checking .pth file support in /lib/python2.7/site-packages/

[05:36:24] <r1chardj0n3s> /Users/richard/.pyenv/versions/2.7.8/bin/python2.7 -E -c pass

[05:36:24] <r1chardj0n3s> TEST FAILED: /lib/python2.7/site-packages/ does NOT support .pth files

[05:36:59] <r1chardj0n3s> I don't know why there's a /lib/python2.7/site-packages :/

[05:37:32] <r1chardj0n3s> wtf there's a /lib/python3.4 as well now

[05:37:37] <r1chardj0n3s> I nuked that

[05:38:45] <r1chardj0n3s> pyenv created it

[05:38:59] <tomprince> dstufft: Looking at the description of https://github.com/pypa/virtualenv/pull/697, I notice the relocatable isn't supported.

[05:39:07] <r1chardj0n3s> table flip time

[05:39:10] <r1chardj0n3s> rage quit

[05:39:27] <dstufft> r1chardj0n3s: o.O

[05:39:30] <dstufft> r1chardj0n3s: I don't get that

[05:39:37] <r1chardj0n3s> I know, right?

[05:39:53] <tomprince> I wonder if it would be possible to support creating a virtualenv in one location with the intention of relocating to a *fixed* *known-in-advance* location.

[05:40:07] <tomprince> (Bascially DISTDIR equivalent.

[05:40:08] <r1chardj0n3s> timestamp is very specific. pyenv created the /lib/python3.4 and tried to create /lib/python2.7 which broke its install

[05:40:30] <r1chardj0n3s> hm

[05:40:44] <dstufft> tomprince: yea, it never really worked right and the primary motivation behind it seems to be to prevent needing to recompile things.

[05:40:50] <dstufft> tomprince: hmm, something like DESTDIR might be doable

[05:40:52] <r1chardj0n3s> oh, I think I know what the culprit *might* be

[05:40:57] <r1chardj0n3s> $ cat ~/.pydistutils.cfg

[05:40:57] <r1chardj0n3s> [install]

[05:40:57] <r1chardj0n3s> prefix=

[05:41:02] <dstufft> r1chardj0n3s: ha

[05:41:10] <r1chardj0n3s> the "fix" for the stupid OS X error I had last week

[05:41:16] <r1chardj0n3s> FFFFFFFUUUUU COMPUTER

[05:41:28] <r1chardj0n3s> FFFUUUUUUUU

[05:41:33] <r1chardj0n3s> rm computer

[05:41:47] <dstufft> tomprince: it might require support in pip to make it doable though, can you comment on the PR?

[05:43:32] <r1chardj0n3s> nuked, trying again. last attempt tho, since Abbey is now playing Ni No Kuni on the TV right next to my work desk and that's far more appealing right now :/

[05:45:02] <r1chardj0n3s> also the ballonicorn has migrated to the living room (my work space ;)

[05:45:14] <tomprince> dstufft: I wrote a wall of text.

[05:45:56] <dstufft> tomprince: thanks!

[11:10:59] <straycat> Is there a reason pkg_resources.parse_requirements() replaces _ in the project_name with - seems strange since there are projects on pypi with _ in the project name?

[11:11:21] <straycat> I'm looking for some sort of project name spec but I don't see one

[11:33:28] <straycat> Ahh xafer just pointed me to http://legacy.python.org/dev/peps/pep-0426/#name in #pypa, which states: "All comparisons of distribution names MUST be case insensitive, and MUST consider hyphens and underscores to be equivalent." So I guess this is why parse_requirements makes that replacement.

[17:45:34] <dstufft> jaraco: ping

[17:45:45] <jaraco> pong dstufft

[17:46:18] <dstufft> jaraco: the require=False deprecating thing is making openstack go boom

[17:46:21] <dstufft> https://github.com/pypa/pip/issues/2326

[17:46:30] <dstufft> dunno what the right answer is

[17:49:25] <sigmavirus24> dstufft: they can use env vars to disable that, no?

[17:49:28] <jaraco> Well, it sounds as if the level should be set so at least the caller will be known in the warning.

[17:49:53] <sigmavirus24> So warnings can also be set to fire only once

[17:50:13] <dstufft> I don't think the once thing is actually going to work

[17:50:17] <sigmavirus24> I think matching is done on the message so if it is static that's cool. If they're running pip lots of times though, that can be a problem since it'll still hit the console logs

[17:50:27] <sigmavirus24> dstufft: you can also have all warnings go to the pip log

[17:50:28] <jaraco> I would have expected that to be the default for DeprecationWarnings. Also, I thought DeprecationWarnings were suppressed by default except in test suites.

[17:50:36] <sigmavirus24> jaraco: not on 2.6

[17:50:36] <dstufft> jaraco: not in python 2.6

[17:50:45] <dstufft> the default for 2.6 is that it's once per unique location

[17:50:46] <sigmavirus24> o/ dstufft

[17:50:56] <dstufft> so probably they are running something a bunch of times

[17:54:13] <sigmavirus24> dstufft: thoughts on configuring warnings to go to pip.log?

[17:54:21] <sigmavirus24> or debug.log, whatever it's real name is? :P

[17:58:09] <jaraco> Any idea what stacklevel is appropriate for that call?

[17:59:19] <jaraco> 2 I think

[18:02:43] <pmxbot> jaraco pushed 3 commits to setuptools (https://bitbucket.org/pypa/setuptools/) :

[18:02:43] <pmxbot> Bump stacklevel to 2 to inform the caller of the issue.

[18:02:43] <pmxbot> Merge deprecation warning fix

[18:02:43] <pmxbot> Update changelog

[18:04:03] <jaraco> dstufft, Any idea if an 11.2 release can be invoked in that environment?

[18:04:15] <jaraco> I'm just waiting for tests to pass to release.

[18:04:44] <dstufft> jaraco: well their problem is coming from the bundled copy of pkg_resoruces in pip

[18:04:53] <dstufft> but they'll update it pretty quickly

[18:05:11] <dstufft> I can do a 6.0.7 in a few minutes

[18:05:47] <pmxbot> jaraco pushed 2 commits to setuptools (https://bitbucket.org/pypa/setuptools/) :

[18:05:47] <pmxbot> Added tag 11.2 for changeset feb5971e7827

[18:05:47] <pmxbot> Bumped to 11.3 in preparation for next release.

[18:09:58] <pmxbot> jaraco pushed 1 commit to setuptools (https://bitbucket.org/pypa/setuptools/) :

[18:09:58] <pmxbot> Add support for linkifying pip issues.

[18:14:07] <pmxbot> jaraco pushed 1 commit to setuptools (https://bitbucket.org/pypa/setuptools/) :

[18:14:07] <pmxbot> Update changelog.

[18:20:58] <dstufft> jaraco: is there a ticket or anything about why Entrypoint.load(require=False) was deprecated?

[18:21:25] <jaraco> Not really, but I updated the changelog to give a little more detail.

[18:21:50] <jaraco> Basically, having a boolean parameter to a function which just bypasses a portion of that function is an antipattern.

[18:22:58] <dstufft> is _load() a public API even though it's got an underscore?

[18:23:29] <jaraco> Yes. I wasn't sure if it needed to be public, but now it's clear that it should be.

[18:24:19] <jaraco> I'm thinking it should have a clearly public name.

[18:24:22] <dstufft> might make sense to add an alias for _load that's not got an underscore

[18:24:45] <dstufft> compat code can do getattr(Entrypoint, "new_new", EntryPoint._load)()

[18:24:50] <dstufft> new_name*

[18:25:57] <dhellmann> jaraco: I'm not sure why a boolean flag to disable a behavior is an anti-pattern, but as long as there's a clearly public API that does the same thing I don't care too much.

[18:27:34] <dstufft> jaraco: next question, are you planning on doing the clearly public name today and if so should I hope off on a pip/virtualenv release?

[18:28:20] <jaraco> http://blog.iannelson.systems/back-to-basics-on-the-use-and-abuse-of-the-humble-boolean/

[18:28:28] <jaraco> dstufft: sure

[18:30:23] <dstufft> jaraco: eh, I don't know that method(some_flag=True) is quite the same as method(True), the blog seems to be against them because it's hard to know what it does, which I don't think is the case with keyword args

[18:30:56] <dstufft> I don't think that method(SomeFlagEnable) and method(SomeFlagDisable) is better than method(some_flag=True) or method(some_flag=False)

[18:31:20] <dhellmann> jaraco: I don't know if I buy it, but as I said it doesn't really matter as long as the feature is still available.

[18:31:22] <jaraco> Agreed.

[18:32:06] <jaraco> But "ep.require(env, installer); ep.load()" is much clearer than "ep.load(require=True, env, installer)"

[18:32:42] <jaraco> I'm thinking of calling the new method "load_" or "load_ex"

[18:32:51] <jaraco> I don't love either option, so I'm open to other suggestions.

[18:33:06] <jaraco> I considered "just_load" but that seems ugly.

[18:33:20] <dstufft> jaraco: yea that's clearer for sure, it's too bad that load() was already taken by the original api ;)

[18:33:38] <dhellmann> jaraco: simple_load()?

[18:33:58] <dhellmann> not especially descriptive

[18:34:19] <dhellmann> load_without_version_check()

[18:34:26] <jaraco> I'm leaning toward load_ and deprecating load entirely, to eventually have load replace load_.

[18:34:54] <dstufft> alternatively, if the functionality needs to stay (which I think it does and hope it does), and a suitable name can't be picked it may not be worth the code churn ? I dunno

[18:34:58] <dhellmann> jaraco: how would that work?

[18:36:27] <jaraco> dhellmann, it would require that no callers are using EntryPoint.load, then it could become an alias for load_.

[18:36:37] <jaraco> But that would likely be well into the future.

[18:38:00] <dstufft> jaraco: having Entrypoint.load() and Entrypoint.load_() (and tbh Entrypoint._load()) with slightly different behaviors feels a lot worse to me than the original problem :/

[18:38:03] <jaraco> I share dstufft's sentiment about code churn.

[18:38:15] <dhellmann> jaraco: I have to make stevedore work with whatever version of setuptools happens to be installed. It sounds like this is going to be a fair amount of code churn for little real benefit.

[18:39:23] <jaraco> I'm pretty sure _load can be removed. It's only been out a short time.

[18:40:20] <dhellmann> I'm not using it in stevedore yet, but I've been waiting to understand the full plan before I made any changes

[18:41:52] <dstufft> jaraco: well even Entrypoint.load() and Entrypoint.load_() with different behaviors feels worse to me than a named boolean argument. like I'm totally on board with cleaning it up if we can come up with a good name for it, but just appending a _ to the end makes me feel like maybe the remedy is worse than the disease in this case unless a different, but descriptive name that shows the differnce between the two can be thought up?

[18:42:32] <jaraco> dstufft, but if .load() is deprecated, then it can be disregarded as legacy, right?

[18:42:59] <dstufft> jaraco: theortically, but the two names look so much alike that it'd be hard to visually notice the two functions are different

[18:43:35] <jaraco> maybe .get or .fetch?

[18:43:42] <jaraco> or .prep

[18:45:19] <dstufft> jaraco: .import() ?

[18:45:31] <jaraco> I think that's reserved.

[18:45:36] <dstufft> .import_()

[18:45:45] <dstufft> or something

[18:46:06] <dstufft> it's essentially just a wrapper around __import__ right? /me looks at the code

[18:47:52] <jaraco> I think I got a clean fix.

[18:48:22] <dstufft> (isn't working on core libs so fun, everyone has an opinion on what to name your thing ;))

[18:48:33] <sigmavirus24> hehe

[18:48:39] <jaraco> I can call it .load.

[18:48:51] <sigmavirus24> it's like the 201 new flags people want requests to add to functions

[18:49:16] <dstufft> jaraco: going to drop the require=True default or something?

[18:49:29] <jaraco> http://paste.jaraco.com/q0hW9

[18:50:09] <jaraco> "call .require and .load (without arguments) separately"

[18:50:19] <dstufft> jaraco: +1

[18:50:52] <dstufft> jaraco: oh

[18:50:54] <dstufft> one downside

[18:51:14] <dstufft> how should people detect if they should call .load() or .load(require=False)

[18:52:19] <jaraco> Yes, i think I've missed something.

[18:53:01] <jaraco> Yeah, that doesn't work.

[18:53:11] <sigmavirus24> you can remove *args

[18:53:14] <sigmavirus24> just do **kwargs

[18:54:10] <jaraco> The current behavior of ep.load() is to invoke require implicitly.

[18:54:19] <dstufft> .fetch and .require, .import_ and .require, .get and .require

[18:54:29] <jaraco> I want a .load() that doesn't invoke require.

[18:54:44] <dstufft> I like the idea of ditching .load() and switching to two functions

[18:54:48] <dstufft> that's easy to detect

[18:55:30] <dstufft> and I think .require() is an obvious function name for the one

[18:55:53] <jaraco> And load is the obvious function for the other.

[18:56:02] <jaraco> Maybe resolve.

[18:56:04] <dstufft> the only problem is the "just do the import" side, which "import" is an obvious name, but reserved so requires munging, and .load() has the old behavior

[18:56:08] <dstufft> resolve is decent I think

[18:57:10] <dstufft> getattr(ep, "get", functools.partial(ep.load, require=False))()

[18:57:13] <dstufft> er

[18:57:17] <dstufft> getattr(ep, "resolve", functools.partial(ep.load, require=False))()

[18:57:27] <dstufft> there's the compat shim ^

[18:58:43] <dstufft> or getattr(ep, "resolve", lambda: ep.load(require=False))()

[19:04:48] <jaraco> dstufft, should I deprecate EntryPoint.load or just the require parameter? Given that ep.load() is a very common usage, I'm inclined not to deprecate it.

[19:05:45] <dstufft> jaraco: you might just document it as deprecated but in a we're not going to actually remove it way?

[19:05:52] <dstufft> or something

[19:05:59] <dstufft> Python did that with logging.warn vs logging.warning

[19:06:04] <dstufft> and some other things

[19:06:48] <jaraco> But should I recommend the usage to be: ep.require(); ep.resolve() ?

[19:06:55] <jaraco> in place of ep.load()

[19:07:27] <sigmavirus24> jaraco: that seens elegant

[19:07:43] <jaraco> sigmavirus24, which?

[19:07:52] <sigmavirus24> recommending using ep.require(); ep.resolve()

[19:08:08] <sigmavirus24> or rather, that replacement is clear and explicit and I like it

[19:09:30] <dstufft> jaraco: yea I think so

[19:09:52] <dstufft> basically I agree with everything sigmavirus24 just said

[19:10:51] <jaraco> hmm. I'm okay with deprecating the parameters to .load(), but I'm more inclined to allow it as a shortcut for exactly that most common usage (when no parameters are passed).

[19:10:55] <sigmavirus24> dstufft: so when do we become one mind with unfortunate amounts of knowledge about awful things like packaging and http?

[19:11:24] <dstufft> sigmavirus24: I'm not sure one mind could sustain that much awful

[19:11:41] <sigmavirus24> dstufft: you're right. it's better we just agree on lots of things and operate separately

[19:12:37] <dstufft> sigmavirus24: having that much awful in one might might be like trying to hold an infinity stone

[19:12:43] <dstufft> jaraco: that sounds ok to me too

[19:14:22] <jaraco> I'm going to start with the softer deprecation for now. It'll be straightforward to deprecate the method entirely in the future.

[19:15:43] <dstufft> Ok

[19:15:44] <dstufft> so then

[19:15:55] <dstufft> or getattr(ep, "resolve", lambda: ep.load(require=False))()

[19:16:01] <dstufft> should give the old behavior always

[19:16:24] <dstufft> except in the small window before resolve was added and load(require=False) was deprecated

[19:16:30] <dstufft> dhellmann: ^^

[19:20:14] <jaraco> dstufft: thanks for that. Which is better for the changelog? using lambda or functools?

[19:20:59] <dstufft> jaraco: lambda is shorter and doesn't require an additional import, i'd probably use that (though you should make sure it actually works since I just wrote it in irc)

[19:25:59] <pmxbot> jaraco pushed 3 commits to setuptools (https://bitbucket.org/pypa/setuptools/) :

[19:25:59] <pmxbot> Add EntryPoint.resolve and deprecate most usage of EntryPoint.load. Removed EntryPoint._load.

[19:25:59] <pmxbot> Added tag 11.3 for changeset a1a6a1ac9113

[19:25:59] <pmxbot> Bumped to 11.4 in preparation for next release.

[19:27:57] <dstufft> jaraco: so 11.3 has the new method?

[19:28:19] <jaraco> Yep. https://pythonhosted.org/setuptools/history.html

[19:29:43] <dstufft> cool thanks

[19:29:48] <dstufft> i'll do some releases shortly

[19:30:04] <dstufft> so we can updated the vendors so we can updated the vendored vendors

[19:30:04] <dstufft> etc

[19:33:51] <dhellmann> jaraco, dstufft: ok, 2 new functions works for me. I can make stevedore check for them and use them, or call load() as it does now.

[19:34:49] <jaraco> !thanks dhellman

[19:34:49] <pmxbot> you're doing good work, dhellman!

[19:34:53] <jaraco> !schneier dstufft

[19:34:54] <pmxbot> dstufft is the seed for your random number generator.

[19:48:44] <msabramo> wait so we're supposed to use EntryPoint.resolve now instead of EntryPoint.load?

[19:48:50] <msabramo> what's the difference?

[19:49:25] <dstufft> msabramo: EntryPoint.load() does pkg_resources magic to ensure that the correct version is available

[19:49:36] <dstufft> (this is now EntryPoint.require())

[19:49:47] <dstufft> then it imports the entry point and returns it

[19:49:55] <dstufft> (This is now EntryPoint.resolve())

[19:50:18] <dstufft> previouslu you couldn't call _just_ require, but you could call _just_ resolve by doing EntryPoint.load(require=False)

[20:01:05] <msabramo> so require=True or EntryPoint.require() is the thing that causes the error saying that you have the wrong version installed?

[20:03:48] <dstufft> msabramo: yes

[20:03:55] <dstufft> require=True is the default for EntryPoint.load()

[20:04:05] <msabramo> cool

[20:04:36] <msabramo> I've wondered sometimes what does that but never looked

[20:05:36] <dstufft> ionelmc: ping

[20:15:27] <dhellmann> jaraco, dstufft : does https://review.openstack.org/145042 look right to you?

[20:16:02] <straycat> awesome, does this mean we can add the entry point for the write_setup_requirements back in?

[20:19:22] <dstufft> dhellmann: yea I think so

[20:19:37] <dhellmann> dstufft: thanks

[20:19:50] <dstufft> you could probably make it shorter if you wanted too

[20:20:47] <dstufft> https://bpaste.net/show/abbcf813f685

[20:20:49] <dhellmann> dstufft: shorter?

[20:21:03] <dhellmann> ah

[20:21:28] <dstufft> can pass require=True to the first oen if you want to make sure the default doesn't change in the future

[20:21:31] <dhellmann> yeah, my way looks a little less tricky to me

[20:21:37] <dstufft> but either way is fine with me

[20:21:37] <dstufft> :D

[20:22:05] <dstufft> I tend to try to reduce branches because my projects tend to require 100% branch coverage ;)

[20:22:42] <dhellmann> you have the same number of cases to test either way, right? :-)

[20:24:49] <sigmavirus24> dstufft: probably a question better asked to elarson but do you know how pip's cache handles if there's more than one instance of pip running against it?

[20:25:49] <dstufft> dhellmann: yea probably

[20:25:58] <dstufft> sigmavirus24: it uses a lockfile to prevent concurrent access

[20:26:07] <sigmavirus24> ah okay

[20:26:33] <dstufft> https://github.com/ionrock/cachecontrol/blob/master/cachecontrol/caches/file_cache.py

[20:26:56] <dstufft> pip extends that class to catch all the file related errors and return None instead of erroring

[20:28:24] <sigmavirus24> dstufft: I like def _fn

[20:28:44] <dstufft> lol

[20:28:58] <dstufft> I like _secure_open_write

[20:29:02] <dstufft> writing that was a pain in the ass

[20:29:26] <dstufft> python makes it way too hard to open a file securely

[20:43:17] <ionelmc> dstufft: pong

[20:45:07] <dstufft> ionelmc: I was looking at your virtualenv-in-virtualenv change, and an idea came to me and I wonder if it wouldn't work and do it simpler - https://bpaste.net/show/7ce722a9029f

[20:45:57] <ionelmc> dstufft: i've already tried that

[20:46:05] <dstufft> it didn't work?

[20:46:14] <ionelmc> syconfig basically use sys.prefix and sys.exec_prefix

[20:46:19] <ionelmc> aaah

[20:46:25] <ionelmc> it also uses sys.executable

[20:46:53] <ionelmc> i'm a bit wary of patching sys.executable just to make sysconfig work

[20:47:33] <dstufft> it doesn't patch sys.executable

[20:47:40] <ionelmc> basically if you're in a virtualenv you can't figure the real sys.executable

[20:47:47] <ionelmc> because it's run with a different bin

[20:48:03] <ionelmc> thus sys.executable is different (will point to the bin in the venv)

[20:48:05] <dstufft> it asks sysconfig to read a value from the makefile

[20:48:10] <dstufft> BINDIR

[20:48:36] <ionelmc> dstufft: yes, but sysconfig _uses_ sys.executable to figure out where that makefile is

[20:50:08] <ionelmc> dstufft: it's all downhill from here https://hg.python.org/cpython/file/518f40815684/Lib/sysconfig.py#l103

[20:51:25] <ionelmc> if there's a different way, do tell :-)

[20:52:37] <dstufft> ionelmc: yea but that doesn't matter, because the Makefile has hardcoded values and we symlink (or copy probably if that isn't available) into the virtual env

[20:54:18] <dstufft> at least on OSX it ends up looking in $VENV/lib/python3.4/config-3.4m/Makefile and $VENV?lib/python3.4/config-3.4m/ is a symlink to the config-3.4m dir from the real python

[20:54:39] <ionelmc> dstufft: so which is least worst - figuring out the path to the real bin or figuring out the path to the global libs?

[20:55:07] <ionelmc> ok lemme make a quick pastebin

[20:57:00] <dstufft> ionelmc: well I think that it's more likely to work without needing to maintain a list of all the differences for each intepreter, Operating system, and isolation type (and include any relevant patches that the OS might make)

[20:57:42] <ionelmc> dstufft:

[20:57:44] <ionelmc> https://www.irccloud.com/pastebin/aLkxvfbu

[20:58:18] <dstufft> hmm

[20:58:26] <dstufft> that is not what I get on OSX

[20:58:44] <ionelmc> dstufft: i mean look at it all:

[20:58:46] <ionelmc> https://www.irccloud.com/pastebin/vS5mYxmu

[20:58:58] <ionelmc> they are all in the virtualenv

[20:59:02] <ionelmc> drives me mad

[20:59:39] <dstufft> >>> sysconfig.get_config_var("BINDIR")

[20:59:39] <dstufft> '/Users/dstufft/.pyenv/versions/3.4.2/bin'

[20:59:44] <dstufft> inside of the venv

[21:00:33] <dstufft> ah ha

[21:00:39] <dstufft> Windows doesn't have the Makefile apparently

[21:01:10] <dstufft> sysconfig intiializes those values to something based off sys.executable, and then on posix systems it pulls the real values from the Makefile

[21:01:39] <ionelmc> dstufft: yeah https://hg.python.org/cpython/file/518f40815684/Lib/sysconfig.py#l359

[21:01:42] <ionelmc> sad

[21:02:03] <dstufft> time to just drop support for windows!

[21:02:12] <sigmavirus24> \o/

[21:02:14] <sigmavirus24> party time!

[21:02:31] <dstufft> hm

[21:02:44] <dstufft> ionelmc: so I wonder

[21:03:07] <dstufft> on Windows we can probably more or less safely assume that the Python is using the layout of the PYthon.org installers

[21:03:31] <ionelmc> i mean we could pull the bin out of the windows registry

[21:03:49] <dstufft> ionelmc: what is sys.real_prefix on Windows

[21:03:51] <ionelmc> but that might have undesirable consequence

[21:04:13] <ionelmc> dstufft: 'C:\\Python27'

[21:04:27] <dstufft> ionelmc: and python.exe is at c:\\Python2.7\\python.exe yea?

[21:04:37] <ionelmc> for cpython yes

[21:04:50] <dstufft> is it somewhere different for pypy

[21:04:57] <ionelmc> pypy.exe

[21:05:02] <sigmavirus24> dstufft: does anaconda follow the layout of Python.org installers?

[21:05:14] <dstufft> sigmavirus24: dunno!

[21:05:29] <ionelmc> who needs anaconda

[21:05:42] <ionelmc> it doesn't even provide the `py` launcher :p

[21:05:53] <dstufft> I think it might make sense to try to do if WINDOWS: bindir = sys.real_prefix ELSE sys.config.get_config_var("BINDIR")

[21:05:57] <dstufft> probably using the flavors

[21:06:02] <sigmavirus24> ionelmc: ಠ_ಠ

[21:06:16] <sigmavirus24> ^^ my unamused face

[21:06:25] <sigmavirus24> I'm not an anaconda user, but I recommend it

[21:07:42] <ionelmc> better than nothing i guess, but if they made it for people that can't be arsed to install the compiler then they should at least make .py file associations, setup the PATH and provide the damn `py` launcher

[21:08:07] <dstufft> in general I'd much rather do things that will limit our need to rely on the python behaving a particular way, especially in *nix where downstream apply all sorts of wierdo patches

[21:08:44] <dstufft> gotta go pick up my daughter

[21:33:42] <ionelmc> dstufft: i'll give that bindir idea another shot, but later

[21:47:57] <ionelmc> also, can anyone explain why there's need for that "local/lib" - when is "local/lib" needed and when is "lib" ? what makes the distinction?

[21:49:41] <DanielHolth> on Linux FHS /usr/local/ is for stuff that is not package managed

[21:56:52] <tomprince> Maybe something to do with debian having /usr/lib/pythonX.Y/dist-packages and /usr/local/lib/pythonX.Y/dist-packages on sys.path

Log file Viewer

Help | Karma | Search:

#pypa-dev logs for Monday the 5th of January, 2015