Log file Viewer

[19:17:22] <Diaoul> hi

[19:19:30] <Diaoul> what is the best way to package a flask application to make it available on pypi? I'm having issues with the alembic directory and manage.py script which are outside the application folder and pip install ignores them

[20:53:28] <peke_> Hi, I work with clients with pretty typical corporate environment with proxies and Windows. IIRC, pip used to pick up proxy settings automatically earlier, but nowadays it requires setting https_proxy environment variable.

[20:54:36] <peke_> This is a small nuisance, especially with somewhat non-technicla people and especially when they don't even know what proxy they use.

[21:02:48] <peke_> My guess is that needing to set https_proxy was caused by taking request into use. urllib2 that was used earlier supports reading proxy config from Windows registry and OSX system config in addition to environment variables, but apparently request does not.

[21:04:10] <linovia> doesn't pip use requests ?

[21:04:59] <linovia> I got a corp. client and gave up to go through the proxy with the stdlib

[21:05:10] <peke_> Yes, pip uses requests nowadays. And apparently requests only supports environment variables for proxy config.

[21:05:42] <peke_> I mean, that's all it supports out-of-the-box.

[21:06:48] <peke_> It seems requests.get accepts proxy config in format that urllib.getproxies returns, though, and urllib.getproxies does support Windows/OSX system config: https://docs.python.org/2/library/urllib.html#urllib.getproxies

[21:09:27] <peke_> My question is, could pip get proxy config using urllib.getproxies and pass it to requests?

[21:11:56] <peke_> If that isn't considered a stupid idea, I could submit an issue about it and take a look at providing a pull-request.

[21:12:10] <peke_> Or is this better to discuss on #pypa-dev?

[21:33:49] <dstufft> peke_: I don't think that would be a terrible idea

[21:34:13] <dstufft> note myknowledge of how proxies are setup is pretty limited

[21:34:16] <dstufft> I don't really use them

[21:38:50] <peke_> dstufft: With "don't" and "terrible" in same sentence, I need to make sure: Did you mean that you think it is an OK idea?

[21:39:07] <dstufft> peke_: yea it sounds reasonable

[21:39:17] <dstufft> with whatever limitedknowledge I have of proxies ;)

[21:39:31] <dstufft> if there's more places to discover proxy settings, looking in those places sounds like a reasonable thing

[21:40:39] <peke_> dstufft: Cool! My proxy knowledge isn't huge either. But I think pip used to work behind my clients proxies earlier and the reason is most likely request only reading proxy from environment variables.

[21:41:06] <dstufft> peke_: note: I think when we switched to requests we stopped supprting non-CONNECT proxies

[21:41:44] <peke_> I have no idea what non-CONNECT proxies are....

[21:41:52] <dstufft> https://github.com/kennethreitz/requests/issues/1622

[21:41:57] <dstufft> we're using requests 2.x

[21:42:33] <dstufft> peke_: there are two ways proxies work with HTTPS sites

[21:42:58] <peke_> I don't think that matters in this case, though. My clients are able to use pip (and install it with get-pip.py) if they set https_proxy environment variable.

[21:43:13] <dstufft> ok

[21:43:17] <dstufft> then yea that should work for them

[21:43:31] <dstufft> CONNECT vs non-CONNECT is whether the proxy itself can see the HTTP request or not

[21:43:50] <dstufft> for CONNECT proxies, the proxy just tunnels the raw TLS protected TCP packets to the destination

[21:44:10] <dstufft> for non-CONNECT proxies the proxy itself "unwraps" the TLS from the destination (and might re-wrap it with it's own TLS)

[21:44:43] <dstufft> but if https_proxy variable works then your proxy will work, you just need a better way to get the config in :D

[21:44:54] <peke_> Ok. I'm fairly certain that's what my clients use. At least they don't use authenticating proxies, NTLM, or anything stupid like that.

[21:46:58] <peke_> Yeah, it seems to me that just getting correct proxy config ought to be enough. I alrady looked that urllib2 uses urllib.getproxies and according to this SO answer it works also with requests: http://stackoverflow.com/a/16311657/38083

[21:47:58] <peke_> I need to test this when I'm at my client office next time.

[21:48:35] <dstufft> peke_: note if you do it in the near future it'll get into the next version of pip

[21:48:54] <peke_> Interesting! What's the timeline?

[21:49:27] <dstufft> peke_: something something when it's ready, but more or less when we finish https://github.com/pypa/pip/milestones/6.0

[21:49:27] <peke_> I would be especially interested to get this into version bundled with Python 2.7 when ensurepip backport lands.

[21:49:33] <dstufft> well

[21:49:38] <dstufft> that's too late for that

[21:49:42] <dstufft> the ensurepip backport already landed

[21:49:46] <dstufft> and 2.7.9rc1 is already cut

[21:49:51] <peke_> Bummer =/

[21:50:01] <dstufft> pip 6.0 (next version) will be in 2.7.10 though

[21:50:25] <dstufft> the ensurepip PEPs makes the pip version exempt from backwards compat concerns

[21:52:18] <peke_> Ok. ensurepip alone makes my life a lot easier. Btw, do you know what will be the recommended way to run (ensure)pip on Python 2.7 on Windows?

[21:52:33] <dstufft> peke_: the 2.7.9 installer will install pip by default

[21:53:05] <dstufft> so unless people uncheck that option you'll get pip installed by default so you don't need to worry too much about ensurepip

[21:53:12] <peke_> Yes, but will it also set PATH so that I (and my clients) can just run `pip install ...`?

[21:53:21] <dstufft> (but if someone does, they can install pip just using python -m ensurepip)

[21:53:30] <dstufft> peke_: Hrm, I'm not sure if 2.7 includes the PATH changes or not

[21:53:58] <dstufft> it's probably saner to use python -m pip on Windows instead of ``pip``, especially since you can't do ``pip install --upgrade pip`` on windows

[21:54:07] <dstufft> but you can do ``python -m pip install --upgrade pip``

[21:54:21] <dstufft> peke_: there's a Script that'll do the path modifications though if the installer doesn't

[21:54:36] <dstufft> pf_moore is a better resource for best practices on Windows, I try not to touch it

[21:55:11] <peke_> The problem is, you cannot even run `python ...` on Windows command prompt without adding installation directory to PATH.

[21:55:56] <peke_> Or at least earlier 2.7 installers didn't set PATH. I know 3.x versions do.

[21:56:49] <dstufft> peke_: I'm not sure what if any changes have been made to how the PATH is modified during the installation of 2.7.9

[21:57:27] <dstufft> https://www.python.org/downloads/release/python-279rc1/ has windows installers if you want to test it out :D

[21:58:30] <peke_> Thanks, I'll check them! Assuming PATH is not set, would python-dev be the best place to ask about it?

[21:58:54] <dstufft> peke_: yea probably

[21:59:30] <peke_> If the idea behinkd ensurepip on 2.7 is to make pip easily available for Windows users, it would be convenient if they didn't need to touch PATH.

[21:59:50] <peke_> That is surprisingly complicated to some people...

[22:02:41] <peke_> dstufft: Thanks for you time/help! I'll check proxy config enhancements as soon as I can, and will also take a look at (ensure)pip on 2.7rc1.

[22:14:20] <peke_> dstufft: One more thing: Would this be the place where to set proxies read from system: # Handle configured proxies

[22:14:44] <peke_> Ooops, this one: https://github.com/pypa/pip/blob/develop/pip/basecommand.py#L89

[22:15:06] <dstufft> yea that'd probably be a reasonable place

[22:15:13] <dstufft> options.proxy should take precedence I think

[22:15:46] <peke_> Yeah, I was thinking about `else: session.proxies = urllib.getproxies()`

[22:59:45] <nanonyme> Doesn't requests handle automatic proxies pretty much out of the box?

[23:03:17] <peke_> nanonyme: My experience is that it doesn't. It seems to only read proxy config from environment variables.

[23:04:56] <peke_> On Windows that info is typically in registry and on OSX it can also be in some system config. Apparently urllib.getproxies() knows how to read it also in those cases.

[23:19:07] <tomprince> peke_: That seems like something that it would be sensible to add to requests, though.

[23:19:28] <tomprince> Rather than only in pip.

[23:20:35] <peke_> tomprince: Perhaps. Although it would then probably belong to urllib3 that requests uses.

[23:22:08] <peke_> I'm not sure are they interested to add that support when it is so easy to pass that proxy config to requests when you want, though.

[23:27:20] <tomprince> Hmm ... it looks like requests already looks at urllib.getproxies.

[23:30:05] <peke_> dstufft: Just tested that Python 2.7.9rc1 Windows installer has an option to "Add python.exe to Path". That's disabled by default, which probably is an OK default. Although it only mentions python.exe, it also adds Scripts directory so both "python -m pip" and "pip" work on terminal. Woot!

[23:30:38] <peke_> tomprince: Oh, interesting. Can you point where? Is that a new addition?

[23:32:14] <tomprince> https://github.com/kennethreitz/requests/blob/master/requests/utils.py#L522 and https://github.com/kennethreitz/requests/blob/master/requests/sessions.py#L239 and https://github.com/kennethreitz/requests/blob/master/requests/sessions.py#L617

[23:43:16] <peke_> tomprince: It definitely seems that requests can read proxy config using getproxies at least in some cases. Not sure why that doesn't happen in my client's environment. I definitely need to investigate that further. Thanks for pointers!