Log file Viewer

[20:58:23] <StevenK> dstufft: Can haz https://github.com/pypa/packaging/pull/50 today?

[20:58:35] <dstufft> StevenK: yea

[20:58:43] <StevenK> Yay!

[22:48:59] <sigmavirus24> dstufft: is there a JSON API for pypi?

[22:49:14] <dstufft> sigmavirus24: /pypi/<name>/json

[22:49:16] <sigmavirus24> ah

[22:49:20] <sigmavirus24> thanks dstufft

[22:55:21] <dstufft> sigmavirus24: no problem

[22:55:40] <dstufft> sigmavirus24: it probably makes sense to do the retry logic on all 5xx errors fwiw

[22:55:45] <sigmavirus24> dstufft: sure

[22:55:51] <sigmavirus24> I don't want to arbitrarily retry

[22:55:59] <sigmavirus24> and requests' retries are not sophisticated enough here

[22:56:40] <sigmavirus24> dstufft: is there no common URI that I could use to do something like `https://pypi.python.org/pypi/{safe_name}/{filename} and

[22:56:48] <sigmavirus24> *`HEAD {uri}`

[22:56:53] <sigmavirus24> to see if a file has been uploaded

[22:57:34] <dstufft> you probably want to check the json

[22:57:45] <dstufft> the other problem is, you might want to actually download the file and compute a hash of it

[22:57:51] <dstufft> (maybe)

[22:58:02] <dstufft> one of the problems we run into with 500 errors is incomplete uploads :(

[22:58:06] <dstufft> (legacy pypi is real bad)

[22:59:40] <sigmavirus24> So if the upload is incomplete, how do I deal with that?

[23:00:00] <sigmavirus24> dstufft: is the md5 in the json just what I tell PyPI when I upload something?

[23:00:25] <sigmavirus24> or is it what pypi calculates?

[23:01:06] <dstufft> sigmavirus24: If the upload is incomplete the only thing you can do is error-- Doing a hash check like that might be as eperate thing like twine upload --verify or something I dunno

[23:01:46] <sigmavirus24> dstufft: so I can't just use the md5 returned in the json to do a hash check?

[23:02:43] <dstufft> sigmavirus24: re hash, it doesn't matter, PyPI buffers the whole file locally and compares the hash that is sent by twine to a hash it computes of the file before doing anything else. The incomplete upload thing happens when something interupts PyPI's writing the file to persistent storage

[23:03:19] <dstufft> it's basically PyPI being really bad at transaction management

[23:03:26] <dstufft> and warehouse doesn't have the same problem

[23:04:03] <dstufft> sigmavirus24: does that description make sense? I think it sounds confusing but idk

[23:06:02] <ronny> good evening

[23:11:51] <sigmavirus24> sigmavirus24: makes sense to me

[23:11:51] <sigmavirus24> thans

[23:16:29] <sigmavirus24> dstufft: so the 500s can be caused by failing to write to persistent storage. in that case, is there any recourse to fix that?

[23:20:27] <dstufft> sigmavirus24: bump a new verson and re-release

[23:20:34] <dstufft> -> goes to daughters school for a bit

[23:20:52] <sigmavirus24> thanks dstufft