Log file Viewer
#pypa-dev logs for Monday the 30th of April, 2018
[15:33:08] <cooperlees> sumanah: I've tried to reach out to people about the secure the internet grant
[15:36:00] <sumanah> And thanks for the kind words cooperlees about turning off Legacy. EWDurbin had a tweet https://twitter.com/EWDurbin/status/990966616803422208
[15:42:34] <xhae> Hi, I am looking at some scripts installed by pip, and instead of a long (and invalid) hashbang it's using bash and some glorious syntax hack on the second line: #!/bin/sh\n'''exec' /...
[15:44:00] <xhae> Now half of me is happy because the hashbang length limit was such a pain, but the other half is sad because a tool, dh-virtualenv, which rewrites hashbangs is broken. Can anyone tell me when this arrived, or if it's even pip doing it?
[16:16:41] <thea> @xhae can you tell us how you generated the scripts? like which commands you used?
[16:26:22] <xhae> @thea these are the setuptools entry_points of packages like `flask` and `fabric`, installed in a virtualenv with `pip install`
[16:34:58] <techalchemy> di_codes: do you know if any of the pip 9 series uses xmlrpc for anything besides searching?]
[16:37:58] <xhae> I found the change, anyway: https://github.com/pypa/pip/blame/87d2735487fe2e8296d777c43ecb9920a2201bd6/src/pip/_vendor/distlib/scripts.py#L139-L168
[16:38:01] <techalchemy> pipenv is all migrated to use the new url then, but we do have pip 9 vendored until piptools takes my pr
[17:19:09] <sumanah> lghampton: but maybe you could start an Etherpad or Google Doc or Dropbox Paper about problems we saw people having getting set up on Thursday night?
[17:20:27] <sumanah> lghampton: https://github.com/brainwane/warehouse/tree/make-clean-issue and https://github.com/pypa/warehouse/issues/2943 might be relevant lghampton
[17:21:34] <sumanah> lghampton: and I remember at least a few people basically saying "what's the architecture?" so we need to perhaps make that easier to find -- perhaps building on https://github.com/pypa/warehouse/issues/2794
[17:22:00] <sumanah> lghampton: and I remember https://github.com/pypa/warehouse/issues/2673 sort of being an issue -- multiple people were just waiting for some signal that `make serve` had completed and was ready
[18:27:06] <lghampton> sumanah: The NYC Python organizers got really good feedback from the sprint, many of our attendees said they really enjoyed the event
[18:27:23] <lghampton> sumanah: also, I have this etherpad of issues: https://public.etherpad-mozilla.org/p/issues-from-sprint
[18:34:24] <sumanah> lghampton: is there anything in your list that you didn't transfer over to https://public.etherpad-mozilla.org/p/issues-from-sprint ?
[18:35:42] <sumanah> lghampton: ok, so, next, I think we come up with new issues or reopen/add to existing issues for all these
[18:36:20] <sumanah> toad_polo: I think it went okay. We didn't have quite enough helpers to circulate, IMO
[18:37:14] <sumanah> lghampton: please file an issue for " One user was confused about whether to run `make serve` and `make initdb` from Getting Started docs in different windows simultaneously or sequentially" and let me know when you've done that
[18:37:27] <toad_polo> sumanah: Yeah, that was partially the problem with the dateutil sprint, too. Though honestly we had enough helpers it was more of a proper division of labor / lack of prep type deal.
[18:37:48] <toad_polo> I shouldn't have been helping people get set up when I was also the bottleneck for getting PRs merged and actual decisions about changes to the interface.
[18:38:40] <toad_polo> That was also another problem, evening-only sprints are really hard to actually pull off.
[18:39:32] <toad_polo> sumanah: Doing good. My son is 3 months old now. Paternity leave is great for me and the wife. :D
[18:45:34] <lghampton> toad_polo: Sumana had the great idea of making a list of activities related to the project that people could do at the sprint that weren't "set up a dev environment, close tickets"
[18:46:31] <toad_polo> Yeah. I think that's a good generalization of my idea of having some code examples to work on to give feedback on the documentation.
[18:47:48] <sumanah> di_codes: out of the issues people ran into during Warehouse dev setup, I think https://github.com/pypa/warehouse/issues/2673 came up most often.
[18:48:18] <sumanah> lghampton: what's your assessment? out of https://public.etherpad-mozilla.org/p/issues-from-sprint what was most frequent?
[18:49:10] <lghampton> sumanah: In my experience, it was the people on Ubuntu who had trouble with the Docker containers not networking
[18:49:36] <lghampton> sumanah: but it was 2 people, and it was a fairly time-intensive thing to diagnose/debug
[18:49:59] <sumanah> lghampton: as you may have guessed, I'm trying to use this data to fix stuff before the PyCon sprint :)
[18:50:33] <sumanah> di_codes: lghampton noted: "2 people on Ubuntu were having trouble with Docker containers not networking -- throwing "Port in use" errors from the containers"
[18:51:34] <di_codes> ok, so maybe we need to add a section to the guide which helps people figure out if any of the ports we need are in use
[18:51:49] <lghampton> di_codes: It looked like the issue was that the database container wasn't connecting to the other ones
[18:56:51] <sumanah> lghampton: you mentioned: Docker doesn't play nicely with Linux subsystem for Windows (Probably unavoidable, only 1 person had that issue)
[18:58:03] <lghampton> sumanah: I'm not sure if it's something we should fix before the sprints. I think running Docker on Linux subsystem for Windows is an "at your own risk" kind of thing
[19:02:05] <lghampton> sumanah: di_codes: this might be helpful: https://en.wikipedia.org/wiki/Windows_Subsystem_for_Linux
[19:05:11] <sumanah> lghampton: would you mind trying to figure out some other places in the developer docs that ought to link to the architecture overview?
[19:34:35] <lghampton> sumanah: How do you feel about adding a link at the beginning of "Submitting Patches" ? https://warehouse.readthedocs.io/development/submitting-patches/
[20:08:13] <sumanah> EWDurbin: di_codes: http://www.osc.dial.community/grants.html I don't PyPI/Warehouse counts as "free & open source software projects working in the humanitarian response & international development sectors" but I thought you might want to know of it in case you know relevant orgs/efforts
[20:34:19] <sumanah> EWDurbin: di_codes: I know Dustin is working on some of the Docker stuff to make new developers' lives easier for the PyCon sprint. I also want to suggest that we flesh out some other things newcomers can do at the sprint
[20:35:00] <sumanah> EWDurbin: di_codes: we had a pretty good experience having newcomers go through the PyPUG package distribution tutorial -- thanks thea for your work on that.
[20:35:57] <sumanah> and then if they have a moment of confusion, we hear about it and we can fix it
[20:36:19] <sumanah> https://docs.google.com/presentation/d/1JHTvs58DaHJzha8aN8oz_IYaiyZlupgioAB3d5MnnFw/edit#slide=id.g38f2482d74_0_0 is the slide we had up on the screen.
[20:36:46] <sumanah> EWDurbin: di_codes: lghampton: I think a useful thing we could add would be the equivalent of https://zulip.readthedocs.io/en/latest/testing/manual-testing.html
[20:37:18] <sumanah> cooperlees: in a conversation with dstufft today I heard that he's working on the beginnings of an API design document for the new Warehouse API
[20:37:37] <sumanah> cooperlees: and that dstufft will be at Day 1 of the PyCon sprint and leaving Monday night
[20:37:57] <cooperlees> I'll try get up to speed with the dev env and maybe churn out a n00b first PR to understand stuff and be useful
[20:38:08] <sumanah> cooperlees: dstufft will also be at the main talk days in case you want to find a spare table and work with him, talk with him before the sprint
[20:38:41] <cooperlees> Another hack / benchmark I'd like to do is see if asyncio + uvloop out performs twisted
[20:39:36] <sumanah> di_codes: EWDurbin -- remind me, were there Warehouse volunteer contributors at North Bay Python/PyBay that cooperlees should meet?
[20:40:15] <EWDurbin> sumanah: i'm actually not sure of any PyPAers that are out in that neck of hte woods
[20:42:32] <sumanah> EWDurbin, sure :D if we were going to expand our list of some things that it's useful to manually test, for Warehouse and related packaging tools, what might be on there?
[20:43:54] <EWDurbin> hmmm, the most core and criticial parts (uploads, json, simple, etc) are unittested.
[20:45:01] <sumanah> and btw when I say "expand" I'm thinking about stuff like https://warehouse.pypa.io/development/reviewing-patches/?#testing-with-twine which is the closest we have to something like https://zulip.readthedocs.io/en/latest/testing/manual-testing.html
[20:47:15] <sumanah> di_codes: EWDurbin: I know Yeray Diaz Diaz is working on the whole frontend test setup in https://github.com/pypa/warehouse/pull/3450 so even if we suggest frontend tests ("make sure this looks ok") it'll be obviated eventually
[20:48:43] <sumanah> cooperlees: You are absolutely welcome to call yourself a member of the PyPA -- there is no intake process and no specific time commitment :)
[20:52:25] <sumanah> lghampton: you know that I pointed you to http://thebraidytester.com/downloads/YouAreNotDoneYet.pdf a bunch of times :)
[20:54:55] <sumanah> lghampton: right..... like, if we were to make a manual testing guide to give useful tasks to very new people at the sprint, frontend testing and testing weird filenames, dates, and graphics might be a thing new folks could do
[20:55:35] <sumanah> (but I guarantee you there are people who want to print PyPI project pages to show other people, or print to PDF to share in, like, a tenure review packet)
[21:02:35] <techalchemy> sumanah: I haven't actually booked my flight or hotel yet but I will be attempting to be around on monday at least for part of the day
[21:25:54] <sumanah> thea: how is your time these days? how much time is Google giving you to work on this stuff?
[21:29:51] <thea> I took a bit of a break from actively working on pypug to help with warehouse and urllib3. But it's about time to come back to it.
[21:30:48] <sumanah> thea: if you want to develop a bit of a TODO/test script for people *testing* PyPUG I think that would be a great way for newcomers at the PyCon sprints to contribute
[21:33:11] <sumanah> thea: so https://docs.google.com/presentation/d/1JHTvs58DaHJzha8aN8oz_IYaiyZlupgioAB3d5MnnFw/ is the slide Laura made to guide participants at a one-evening sprint last week
[21:38:09] <sumanah> thea: I'm thinking people could work through the distro tutorial, or the "specifying your requirements" guide, and test it with varying inputs
[21:38:37] <sumanah> weird files & filenames.... figure out the current minimum versions of tools needed.... stuff like that
[21:38:47] <thea> We have something similar we call "friction logging" internally. I can certainly put something like that for the PyCon sprints.
[21:38:50] <sumanah> thea: I'm totally just brainstorming this. Could be wrong and it might not be useful
[21:42:02] <thea> I mean even though I plan on wholesale replacing that tutorial, having specific feedback on issues with the current one helps a lot to make sure the new one covers those cases.
[21:45:18] <cooperlees> di_codes: Am I dumb - how do I get the multi value keys out of the email I'm parsing?
[21:45:20] <sumanah> di_codes: I'm imagining you setting up IRC notifications specifically for the string "PEP 566"
[21:48:39] <di_codes> cooperlees: <https://docs.python.org/3/library/email.message.html#email.message.EmailMessage.get_all>
[21:50:29] <di_codes> i have a WIP branch to add a canonical way to do this to the `packaging` module, but the pypi launch got in the way
[21:51:04] <cooperlees> Is there a github issue? Maybe I can do that @ the sprints since I'm basically writing it now :)
[21:52:49] <di_codes> cooperlees: there isn’t, it was part of my effort to add the ability to validate metadata as well. it’s one of the things on my list for the sprints
[21:53:01] <di_codes> cooperlees: here’s the branch, you might find some parts helpful: <https://github.com/di/packaging/tree/metadata-validation>
[21:56:15] <di_codes> sumanah: not unless you have the ability to peer into my head (which even I lack sometimes)
[21:57:02] <di_codes> lol at all the people asking ewdurbin to move. c’mon, this guy loves Cleveland so much he made pycon move there!
[21:58:02] <di_codes> cooperlees: it’s on my list of things to do before sprints happen which is… also in my head
[21:59:55] <techalchemy> for those who haven't listened, go listen to 'Talk Python To Me' to hear about the pypi / warehouse rollout etc
[22:10:56] <sumanah> cooperlees: I looked at http://kafka.dcpython.org/day/pypa-dev/2018-04-30 but because of https://github.com/yougov/pmxbot/issues/75 I think I missed something you said :)
[22:11:37] <cooperlees> "cracks whip - Got them down for the nice organizer project manager to make sprints way more awesomer"
[22:17:48] <techalchemy> sumanah: My best friend lived there for a few years doing big data stuff for the cleveland clinic, it really was actually quite nice and there were a lot of good plays and such
[22:18:50] <sumanah> techalchemy: you said "I will be attempting to be around on monday at least for part of the day" re: the PyCon sprint -- I'm glad. Working on pipenv stuff?
[22:24:49] <techalchemy> no idea haven't actually managed to get more than 1 maintainer in the same conversation at once :p
[22:32:16] <sigmavirus24> Still adjusting to the new allergy normal of spring. sumanah based off the fact that you were sick and are now off-balance. Did you get the horrible flu-like infection that followed with a sinus infection and ear infection? I had that and was off-balance for a week and a half at least
[22:32:43] <sumanah> sigmavirus24: I got that in January! I was off-balance in April primarily because of sudden familycare needs and then catching up.
[22:34:15] <sumanah> Also sigmavirus24 I can't recall whether I've asked before: should I add you to the list of PyCon NA sprint participants? https://wiki.python.org/psf/PackagingSprints
[22:35:14] <sumanah> btw cooperlees: please feel free to file issues as you get a Warehouse development environment up, if you choose to try doing that -- https://github.com/pypa/warehouse/labels/developer%20experience we want to know about DX issues
[22:35:15] <sigmavirus24> I'm leaving on the 13th to come home and take care of my brother's kids, so I won't be at sprints :(
[22:36:49] <sumanah> sigmavirus24: to combat allergies: I hear people say eating local honey is useful, and I hear people say nasal irrigation is useful. I personally think my allergies are worse when I am underhydrated, so I think one benefit of various herbal teas is that they cause me to hydrate more
[22:38:14] <sigmavirus24> I've also heard the local honey thing too. There aren't many local apiculturists in the closer vicinity so I think it hasn't helped because most of the honey I can get that's local is ~50 miles away (at closest) :(
[22:38:27] <sigmavirus24> Also nasal irrigation is something I keep meaning to try but haven't, so I'll push that up the list
[22:38:51] <sumanah> sigmavirus24: as a person of Indian descent I feel like I could promulgate arbitrary herbal things here and say "uh, yeah, it's Ayurvedic! trust me!" but I don't ACTUALLY know. But I imagine stirring some turmeric, ginger, black pepper, ground coriander, and fenugreek into some boiling water and drinking it is unlikely to *hurt*
[22:39:40] <cooperlees> sumanah: Will do. I've also been locked inside Facebook so long that I've never used Docker. haha
[22:39:58] <sigmavirus24> That said, I am known to use large quantities of black pepper and tobasco in soup for the purposes of clearing out sinuses
[22:40:19] <sumanah> sigmavirus24: and how much air filtration do you currently do in your home? some folks do a lot of stuff with, like, HEPA filters
[22:42:00] <techalchemy> sumanah: have you ever actually tried the honey thing? i wonder if it works
[22:42:53] <sumanah> techalchemy: my allergies are usually not that bad.... the one place I get bad allergies is when I'm in Karnataka, where my folks are from, and I never have the energy to chase down local honey while I'm there. Maybe I could try to get my family to get some ahead of time before the next time I visit.
[22:43:43] <sumanah> sigmavirus24: finally, and this may be overkill, but I know at least one person who often had sinus problems, and then visited an ear-nose-and-throat doctor and got diagnosed with something, then had a small procedure to fix something, and has way fewer problems now. I don't remember the details, sorry
[22:43:58] <techalchemy> I get migraines super frequently and I only suspect that allergies are involved, but if eating honey is a solution I would definitely pursue it
[22:45:57] <sumanah> di_codes: whoa! is that towards https://github.com/pypa/warehouse/issues/1536 / https://github.com/pypa/warehouse/issues/2943 sorta?
[22:45:59] <sigmavirus24> sumanah: I've already had many things fixed in one surgery a while back but it didn't help with allergies :(
[22:46:47] <sumanah> sigmavirus24: so you'll be at PyCon during the talks days -- I hope to meet up a bit and talk twine
[22:47:55] <sumanah> hey I may as well take this opportunity to ask what accessibility concerns y'all (who get migraines) have with websites.... like, are there particular color ranges we shouldn't use?
[22:48:03] <techalchemy> yeah the relentless ones are the worst because you run out of things you can do for them
[22:48:20] <cooperlees> Mine are all muscular related from shoulder reconstruction ... so I just get a lot of massages and chiro work to avoid them
[22:48:43] <techalchemy> sumanah: uh saturated reds/yellows/whites, high contrast like white on red in the brighter ranges
[22:49:07] <sumanah> techalchemy: do you happen to know of any automated tools to check for this sort of stuff? is it already in the ARIA/WCAG guidelines?
[22:49:46] <dstufft> (go go sititng in the dark, with my monitor one step away from being turned off b/c the brightness is turned down so low)
[22:50:07] <techalchemy> I don't know of any tools, but personally I have a color modifier installed which temperature shifts everything and a filter that filters blue light and such on my glasses
[22:51:45] <techalchemy> honestly for me it's the same set of concerns as for epilepsy, any bright/flashing lights would be a potential trigger, even at baseline
[22:52:49] <di_codes> sumanah: it’s just cleaning up some cruft, we aren’t actually using twisted anymore
[22:54:50] <sigmavirus24> sumanah: I'd love to talk twine. I started a GitHub discussion, but that was in the hopes of not bugging you while you were sick. The short of my question is: Do you have a sense of a good way of writing up design documentation/proposals for projects? I have an API design for twine to make lots of people happy and want to put it somewhere. I've just had terrible luck with doing that via issues
[22:57:43] <sumanah> sigmavirus24: the short answer is: a wiki that other people can actually edit, or a static proposal on a blog post or something like that plus an email thread to talk about it
[23:00:06] <sumanah> sigmavirus24: For things that are more fluid, Dropbox Paper/Google Docs or similar can work. If we had a https://github.com/substance/texture/ installation, maybe we could use that? I find Etherpad kind of crashy and it isn't as formattable -- but for quick consultation or something a little more private to work out ideas before a public thing, we could use https://public.etherpad-mozilla.org/
[23:03:19] <sigmavirus24> yeah, i was more hoping to solicit feedback prior to spending lots of time working on it. I think I'll start a PR with the documentation so we can point to the rendered docs and then solicit feedback on the issues where people have requested this?
[23:03:58] <sumanah> sigmavirus24: I just reread your note from a few weeks ago, sorry, I misremembered your question/plan
[23:04:38] <sumanah> sigmavirus24: I would be happy to corral some of the relevant parties during the sprints and ask them to look at the PR :)
[23:05:48] <sumanah> sigmavirus24: and in a sense we're asking: how do you run community consultations so that the stakeholders actually show up and talk about their needs before there's a lot of work done that would need to be re-done?
[23:08:25] <sumanah> sigmavirus24: my take on that: asking the right questions, personal outreach to individuals, putting word out in their spaces (their issue trackers/lists/channels), holding live synchronous chats when possible, giving clear timelines for when people need to respond
[23:13:27] <sigmavirus24> I think my fear is that these folks have largely been of the opinion that *ANY* api would be good so I almost feel like they're not going to react very well to personal outreach, but I'll definitely try that
[23:22:32] <sumanah> sigmavirus24: "asking the right questions" to me also includes saying "here are two to four approaches we could use" so they have a somewhat more templated choice
[23:22:51] <sumanah> sigmavirus24: I added the Twine API topic to https://wiki.python.org/psf/PackagingSprints
[23:25:54] <cooperlees> I'm getting some timeouts to files.pythonhosted.com from my prod bandersnatch - But let me debug more before blaming something Warehouse is doing.
[23:25:55] <cooperlees> https://bitbucket.org/pypa/bandersnatch/issues/107/warehouse-issues-filespythonhostedorg
[23:35:00] <sigmavirus24> sumanah: I'll start working on that documentation tonight after my shift :)
[23:41:04] <sigmavirus24> I don't have a bunch of options though to give people. It's taken me over a year just to arrive at this API. I think it would vastly simplify twine's current internals though :)