Log file Viewer
#pypa logs for Saturday the 14th of May, 2016
[11:49:16] <apollo13> dstufft: btw I am mostly doing infra stuff recently and if you want I'd happily be a second/third hand for the ops team at pypi -- not that I have aeons of time, but whatever I can do to free you to do other things…
[11:51:45] <apollo13> dstufft: also, I'd like to help on warehouse if you can give me tasks and a little bit of a hand
[11:53:23] <dstufft> apollo13: mind posting an offer to infrastructure@python.org ? We don't have a clear way to authorize a new user for like, sudo access on PyPI or what not but if you make a psot to that ML offering to help (with any relevant background info) I can make sure nobody has any objections.
[11:57:07] <dstufft> apollo13: [Tritium] The tasks that I think are most important for getting Warehouse launched are tagged https://github.com/pypa/warehouse/milestones/Launch other items we have a few issues tagged easy but I don't think we're super great at actively categorizing issues as easy vs not
[11:58:47] <dstufft> If you're interested though, you don't have to limit yourself to just the Launch tasks, if a non Launch task looks interesting to you feel free to tackle it (although a few of them will be harder to do until after the old code base dies)
[11:59:14] <apollo13> ERROR: Couldn't connect to Docker daemon at http+docker://localunixsocket - is it running? -- but I am also getting issues with docker ps -- gotta fix some perms
[11:59:58] <dstufft> I think the only real thing we end up depending on having already setup on the host system is docker and docker-compose (and well make-- but even that's optional if someone felt like copy/pasting commands)
[12:02:31] <apollo13> hurray, it is installing :D time to do something else in the meantime, mail will come later too
[12:04:28] <[Tritium]> dstufft: ok, who has docker installed on windows and doesnt have visual studio?
[12:04:54] <dstufft> [Tritium]: No idea :] I haven't used Windows as a dev platform in like 8 years :)
[12:06:54] <[Tritium]> warehouse MIGHT be dev-able on windows IF you dont have to test it against gunicorn
[12:11:08] <dstufft> basically everything in Warehouse runs in a docker container with the host FS mounted inside it
[12:12:31] <dstufft> Docker is used so that people starting up don't have to get PostgreSQL, RabbitMQ, Elasticsearch, and redis installed
[12:13:28] <dstufft> Well operationally wise the PSF is fine having it run on Heroku, it's easy to move from Heroku to not Heroku if it ever becomes a problem since we purposely don't depend on any Heroku APIs (except during build)
[12:14:24] <dstufft> the problem is I don't know if Heroku is fine with us running there, (they donated credits which is great, but their AUP also states that no app can use more than 2TB/month of bandwidth... and legacy PyPI is currently taking up 10TB/month to the Rackspace backend servers)
[12:14:57] <dstufft> I talked to them like a year ago about it or so, but back then we were hovering right around 2TB/month and they said "well we don't really enforce that limit unless it becomes a problem"
[12:15:14] <dstufft> but sort of hovering around the limit and blowing past it 5x is a bit different I think
[12:18:37] <dstufft> (That's not _just_ for PyPI, it's for all of our stuff on Fastly which is largely PyPI, the second largest thing is www.python.org which I think is 5-10% of the bandwidth used by PyPI)
[12:19:37] <dstufft> I mean, the servers that Rakcspace donates and the other companies etc are all super important too
[12:19:58] <dstufft> but Fastly really takes the brunt of the load, which allows us to run with a skeleton ops team
[12:21:20] <dstufft> We had a mirror network before we used Fastly, the problem was nobody ever ran mirrors regularly. People would set one up, it'd run for awhile, then it'd get stuck and nobody would notice for months and months and we'd eventually delist them
[12:21:58] <dstufft> I think when we finally killed the last bit of automatic mirror support in PyPI/pip, like 80% of the "mirrors" were actually just aliases for PyPI itself at that time
[12:22:43] <dstufft> (due to the way the mirrors were structured we couldn't delete a mirror, we had to keep the hostname pointing somewhere so instead of deleting we just added another hostname to PyPI itself)
[12:22:48] <[Tritium]> I mean, was pypi ever setup to support a mirror network? I think CPAN uses some bulletproof rsync magic
[12:23:19] <dstufft> there was a PEP and mirroring clients. It wasn't rsync easy though (and still isn't)
[12:24:19] <dstufft> nobody used mirrors because they were unreliable, and nobody bothered to keep the mirrors reliable because nobody used them
[12:26:50] <apollo13> dstufft: ok, got warehouse running -- all images (ie http://localhost/static/images/logo-large.87e8a3ef.svg ) are black though, any idea :D
[12:27:33] <dstufft> apollo13: they're _black_? That is odd... I wonder if the image optimization is going haywire for you
[12:29:48] <dstufft> Commenting out https://github.com/pypa/warehouse/blob/master/Gulpfile.babel.js#L146 would remove the image optmization from the asset pipeline, though because of some terrible behavour with npm you have to rebuild the docker containers to get that to be reflected
[12:30:07] <dstufft> I wonder if that's a problem with the image optimization and firefox in general
[12:31:33] <dstufft> So, whatever is in the master branch is live on pypi.io modulo something that just got merged and hasn't been automatically deployed yet, _except_ the logos don't currently go through the image optimization because for IP reasons the logo is in a seperate repo
[12:33:50] <dstufft> That repo mostly just holds the new PyPI logo, and the logos of the sponsor companies across the bottom
[12:34:07] <[Tritium]> keeping it in the warehouse repo would license the image the same as the code...
[12:36:25] <dstufft> Okay, the black placeholder logo happens even with the original image commited to the repo
[12:36:40] <dstufft> so it's not any part of the build process, it's just the placeholder .svg not being compatible with FF I guess
[12:37:31] <dstufft> [Tritium]: Yea, the PSF doesn't want the logo licensed like that for trademark reasons or something
[12:38:27] <apollo13> if I change code, everything should restart just fine, or do I need to do something else?
[12:40:35] <[Tritium]> I need to start sucking at my job so that when someone calls in sick on a weekend... i am not the first person called
[12:44:30] <apollo13> dstufft: so I added child-src to csp (which is the new thing instead of deprecated frame-src) but it will not show up, it properly reloaded but it seems as if config.add_settings({'csp': … }) does not pick up any changes -- is there another config somewhere?
[12:45:53] <apollo13> oh and firefox properly shows the images in the network tab of the dev tools :D
[12:46:31] <apollo13> dstufft: oh nevermind, it is there now, was looking at the wrong cached request
[12:47:33] <apollo13> Note: This directive is deprecated. Use child-src instead, unless you are supporting browsers that use CSP 1.0 only (e.g. Safari 9).
[12:47:57] <dstufft> I could be remembering wrong, I think I looked at child-src and it inherited from default-src whereas frame-src did not
[12:49:37] <dstufft> https://github.com/pypa/warehouse/issues/1052 (more info on the decision for what browsers we decided to support if that sort of thing interests you)
[21:25:14] <rubenwardy> Why isn't my package installable? https://pypi.python.org/pypi?name=phpbb-parser&version=0.1.0&:action=display
[23:38:37] <[Tritium]> ionelmc: I think TOML has been settled on.... i think partly because there are six TOML libraries in the wild that could easily be vendored