Log file Viewer

[01:30:23] <tdsmith> apollo13: high-five for the cool uris reference

[16:56:44] <indygreg> i pin hashes in my pip requirements files and i've been getting a number of hash verification errors from PyPI recently, possibly due to PyPI/pip using different source archive types (e.g. fetching a .tar.gz instead of a .zip). did something change?

[16:57:49] <dstufft> indygreg: yea, the URL format changed (and thus, the order the URLs appear on /simple/)

[16:58:33] <indygreg> bleh. so much for determinism over time :/

[16:58:47] <indygreg> thanks for the info

[16:59:11] <indygreg> (yes, i know i should run my own package server if i want determinism over time)

[16:59:27] <dstufft> indygreg: you can also just pun all the archive types

[16:59:30] <dstufft> pin

[16:59:38] <indygreg> oh yeah?

[17:00:19] <dstufft> yea, you can add multiple --hash

[17:00:29] <dstufft> https://github.com/pypa/warehouse/blob/master/requirements/main.txt

[17:00:32] <indygreg> oh, yeah, that's what i'm doing

[17:00:33] <dstufft> ^ example

[17:00:49] <dstufft> so pin both .tar.gz and .zip and now you're insulated!

[17:00:53] <indygreg> i read that as "pin the archive type" as if you could tell the requirements file which archive type to download

[17:01:53] <indygreg> it's too bad the server doesn't advertise the hashes of available files so the client can choose to download one that it has pinned

[17:02:20] <indygreg> that would insulate a bit more from future changes. e.g. if a package maintainer uploads a wheel, clients pinning the .tar.gz could still download it

[17:02:41] <dstufft> indygreg: yea, I plan to make the repo better :]

[17:02:51] <indygreg> awesome!

[18:25:51] <gthank> I want to use an implicit namespace package. I would *like* to use `find_packages`. Is this currently possible? I read a lot of discussion on the issue tracker, but didn't see a clear, final statement either way

[18:26:57] <dstufft> gthank: I don't think find_packages currently supports PEP 420 namespace packages

[18:29:07] <gthank> dstufft: thanks. So I should pass `namespace_packages` and also manually set the values for the `packages` and `package_dir` arguments?

[18:29:19] <dstufft> yea I think so

[18:29:52] <gthank> excellent

[18:39:18] <gthank> dstufft: the tests have an interesting thing in them; find_420_packages = setuptools.PEP420PackageFinder.find

[20:30:33] <simon_weber_> hey all, we've been seeing some odd flakiness from pip on our build servers over the past few months

[20:30:49] <simon_weber_> it got annoying enough that we switched to an internal devpi instance to cut out pypi

[20:31:13] <simon_weber_> but when testing it now, that doesn't seem to have solved our problem, so i'm wondering if anyone here has leads

[20:31:51] <simon_weber_> the main symptom is "SysCallError: (104, 'Connection reset by peer')" when pip is looking up an index for a vcs package subrequirement

[20:32:45] <simon_weber_> when we saw this with devpi hooked up we didn't see any index requests on its end, leading me to believe that pip may not be using the devpi index in this particular case

[20:33:27] <simon_weber_> wait, I lied. this isn't a subrequirement, it's just the next requirement in the requirements.txt.

[20:34:12] <simon_weber_> here's the full traceback: https://dpaste.de/KD6X

[20:36:33] <elarson> simon_weber_: are you sure your build servers can reach the upstream git repo?

[20:40:01] <simon_weber> yup, the clone step completed without problems. I think it was a red herring, since it doesn't seem like pip would go to the git repo for the index request?

[20:41:16] <simon_weber> what I'm really wondering: is it possible to verify that pip was trying to connect to pypi instead of devpi for this index request?

[20:41:40] <simon_weber> i'm working on recreating the problem now so I can try upgrading and whatnot, but it might take a bit to recreate

[20:47:39] <njs> it's possible that pip just needs some patch to make it retry dropped connections in some case where it isn't

[21:18:47] <simon_weber> hm, i wasn't able to recreate the problem with 30 minutes of continuous installs, either with devpi or without

[21:18:52] <simon_weber> annoying

[21:30:46] <[Tritium]> in pip, can --extra-index-url be specified multiple times?

[21:31:30] <[Tritium]> ...specified? passed, provided... poor word choice, Tritium...

[21:47:33] <simon_weber> njs: I suspect this is a problem down the requests stack. The pip code here https://github.com/pypa/pip/blob/fc04667164dd503fc3cbe5d4e16fab6018d7790b/pip/index.py#L777 expects requests errors, but we're seeing the SysCallError instead. I think it's something like https://github.com/kennethreitz/requests/issues/2543, so maybe upgrading requests will help

[21:51:10] <[Tritium]> is there a way to make pip not clobber other executables on install? (there is already a script named foo, a package wants to install a script named foo, is there a way to install the package without clobbering names)

[21:56:24] <njs> simon_weber: looking at line 188 of pip/_vendor/requests/packages/urllib3/contrib/pyopenssl.py, it seems that that recv is already wrapped in a try/except that tries to catch that SysCallError

[21:56:55] <simon_weber> I suspect it's not in 2.6.0

[21:57:05] <simon_weber> let me check

[21:57:21] <simon_weber> er

[21:57:29] <simon_weber> I see, you're talking about pip

[21:57:32] <njs> the relevant thing is your pip

[21:57:33] <simon_weber> I was on...6.0.0, I think

[21:58:45] <njs> yeah, dunno about pip versions, I'm looking at the copy of urllib3 in a git clone of pip :-)

[21:58:58] <simon_weber> yeah let me check it out

[22:03:45] <simon_weber> yeah, the bundled requests is only 2.5.0 in that version

[22:03:55] <simon_weber> so hopefully this will fix itself with an upgrade

[22:53:43] <Noldorin> I'm trying to install hgview, but getting an error to do with XML validation for xmlto

[22:53:43] <Noldorin> xmlto: Fix document syntax or use --skip-validation option

[22:53:48] <Noldorin> how can I tell pip to pass that?