Log file Viewer

[04:19:25] <Tcab> Just listened to talk python to me podcast re pypi and about how new website uses markdown or supports it

[04:20:00] <Tcab> Checked it out and https://pypi.org/project/requests/ does not render properly

[04:24:55] <ngoldbaum> i think they need to update their setup.py?

[04:24:58] <ngoldbaum> https://github.com/requests/requests/blob/v2.18.4/setup.py

[04:25:26] <ngoldbaum> long_description_content_type='text/markdown'

[04:25:44] <ngoldbaum> Tcab: hmm, easy PR I guess

[04:31:45] <ngoldbaum> Tcab: https://github.com/requests/requests/pull/4623

[08:10:15] <pyvkd> hi everyone today while doing pip and pip3 installs i have started to get these errors

[08:10:17] <pyvkd> ProtocolError('Connection aborted.', error(104, 'Connection reset by peer'))

[08:13:40] <pyvkd> i have purged the system python3-pip and installed via curl https://bootstrap.pypa.io/get-pip.py | sudo python3

[08:13:59] <pyvkd> any ideas

[08:18:18] <pyvkd> doesn't help

[09:56:51] <rindolf> Hi all! Would you guys appreciate a pypi equivalent to https://qa.perl.org/cpan-testers/ ?

[12:39:22] <ssbarnea> is here the right place to discuss pypi CDN issues? i got logs of flaky "No matching distribution found " errros on CI since the migration.

[12:41:52] <mgedmin> it's a good place to discuss those, I think

[12:46:07] <ssbarnea> mgedmin: we get more and more of these, on normal / random packages. my impression is that some CDN servers are reporting 404 for some folders and making pip believe packages do not exist. It doesn't retry, it just fails right away.

[12:47:59] <ssbarnea> mgedmin: here is one example: https://gist.github.com/ssbarnea/62d2374d9f101d5f596f27c136015d39

[12:48:30] <ssbarnea> and the problem is not specific to today, happened since the upgrade, seems random in nature.

[12:49:09] <mgedmin> how hard is it to reproduce this at will?

[12:51:44] <ssbarnea> mgedmin: i expect >100 occurences in last two weeks, maybe like 2-3% of builds. this is "impression" but I should be able to query the mongodb to give exact numbers.

[12:52:11] <ssbarnea> lucky for me I got a pattern check for this kind of error, so i only need to perform a query, which is not as easy as SQL.

[12:53:57] <ssbarnea> but if not really needed i would prefer to avoid the query task

[12:54:11] <mgedmin> if you had to get the output of pip install -v <something> failing like that, how hard would that be

[12:54:27] <mgedmin> note: I'm just a curious bystander, with no access to pypi infrastructure

[12:54:48] <mgedmin> I can't be sure what will be useful to actual sysadmins once they notice this conversation

[13:41:42] <ssbarnea> ok, in this case I will activate -v for a while. can I do it via PIP_ vars?

[13:46:37] <mgedmin> I think so

[13:47:18] <mgedmin> https://pip.pypa.io/en/stable/user_guide/#environment-variables

[13:47:24] <mgedmin> -v is --verbose, so PIP_VERBOSE=1 should work

[15:04:21] <di_codes> ssbarnea: we’ve had similar reports with jenkins, could you leave a comment on <https://github.com/pypa/warehouse/issues/3826> if you see this again?

[16:55:08] <carlwgeorge> The packaging tutorial says "If you are using setuptools, you don’t need to list README.rst in MANIFEST.in." Is that also true if you are using a README.md file?

[16:55:59] <nedbat> carlwgeorge: if in doubt, just list it

[16:56:58] <Wooble> carlwgeorge: you don't need it for README.{rst,txt,md} https://github.com/pypa/setuptools/blob/89ef5bb7e4812193d6f593d531bad36191517981/setuptools/command/sdist.py#L40

[16:57:33] <carlwgeorge> Awesome, thanks Wooble!

[18:12:20] <sumanah> I could use some help from folks here -- I'm working on making https://github.com/econ-ark/HARK pip-installable (and then I'll use conda-skeleton to make it conda-installable). My work-in-progress branch is https://github.com/brainwane/HARK/tree/installability

[18:12:23] <sumanah> As of this moment, on my installability branch, in a Python 2.7.13 virtualenv using the system setuptools (version 39.1.0), if I run python setup.py sdist, it dutifully creates dist/HARK-0.8.0.tar.gz which is about 25M in size (a lot of that is cAndCwithStickyE/Results). pip will dutifully install HARK and its dependencies but then you can't actually import any of it yet, what with the __init__s not working and so on.

[18:12:38] <sumanah> TODO: I need to get install_requires working nicely with requirements.txt, I need to specify version numbers and (I believe) add some further package requirements that the developers so far haven't had to worry about because Anaconda always bundles them, and there's a bunch of other polish I need to do.

[18:13:18] <sumanah> so my main question right now is: how do I avoid duplication between requirements.txt and the install_requires argument in setup.py? https://packaging.python.org/discussions/install-requires-vs-requirements/ is confusing me a little

[18:13:53] <sumanah> (I'm trying to do 1 thing at a time here, so, porting to Python 3 is on our roadmap but *not just yet*)

[18:16:56] <ngoldbaum> requirements is for a specific deployment of your app

[18:17:08] <ngoldbaum> it sets the *exact* dependency versions you want to use for that deployment

[18:17:11] <sumanah> (side note: Ben Folds is my soundtrack here, in case anyone wants to really understand my situation in full stereo)

[18:17:29] <ngoldbaum> install_requires specifies the minimal dependencies needed

[18:17:47] <ngoldbaum> if you need to specify version constraits (i.e. for a minimum version number or to pin something)

[18:17:51] <ngoldbaum> you can do that in install_requires

[18:17:55] <ngoldbaum> but only if you really need ot

[18:17:59] <ngoldbaum> need to

[18:18:32] <ngoldbaum> IMO projects shouldn't ship a requirements file unless they're building a webapp or something like that that needs *very* specific versions of dependencies

[18:18:57] <sumanah> So I think I don't need requirements.txt .... is it like, often, requirements.txt is for libraries that need to be super specific about dependencies, and install_requires is more appropriate for applications that do not need to be THAAAAT specific?

[18:19:06] <sumanah> thank you for your help ngoldbaum

[18:19:11] <ngoldbaum> :)

[18:28:05] <sumanah> ngoldbaum: just to be clear, was my understanding there (from a few min ago) correct?

[18:29:54] <ngoldbaum> yup

[18:29:58] <ngoldbaum> as i understand it

[18:29:59] <sumanah> Thank you!

[18:30:00] <ngoldbaum> others might disagree

[18:30:04] <sumanah> Right! Of course

[19:35:19] <ssbarnea> di_codes: I got new pypi failures with extra logging, check last comment on https://github.com/pypa/warehouse/issues/3826

[20:44:33] <sumanah> a cautionary tale https://blog.sory.biz/posts/matplotlib-gah/

[20:47:26] <ngoldbaum> sumanah: you only had that issue with pyplot becase pylot uses the interactive backends to plot things

[20:47:38] <ngoldbaum> just "import matplotlib" doesn't trigger the hooks to any GUI toolkits

[20:47:44] <sumanah> ngoldbaum: me?

[20:47:47] <ngoldbaum> (in your case the native MacOS GUI toolkits)

[20:47:49] <sumanah> ngoldbaum: I did not write that

[20:47:54] <ngoldbaum> not your blog?

[20:47:56] <sumanah> nope

[20:47:58] <ngoldbaum> ah

[20:48:06] <sumanah> my blog is https://www.harihareswara.net/ces.shtml

[20:48:11] <sumanah> accept no imitations ;-)

[20:48:25] <ngoldbaum> :)

[20:48:41] <ngoldbaum> tbh the MacOS backend is a huge mess

[20:48:42] <sumanah> yours?

[20:48:49] <ngoldbaum> it would be better if it didn't default to that

[20:48:57] <ngoldbaum> i don't have one

[20:58:55] <sumanah> np