PMXBOT Log file Viewer

Help | Karma | Search:

#pypa-dev logs for Monday the 2nd of April, 2018

(Back to #pypa-dev overview) (Back to channel listing) (Animate logs)
[12:47:07] <sumanah> hi dstufft and hope you're doing well
[13:07:39] <pf_moore> sumanah: Hi
[13:07:45] <sumanah> Hi pf_moore
[13:08:00] <sumanah> Happy Monday
[13:08:15] <pf_moore> sumanah: You too :-)
[13:08:28] <pf_moore> I've just pushed pip 10.0.0b2 to PyPI and I'm about to send out a quick release announcement
[13:08:38] <pf_moore> Am I OK to send it to pypi-announce again?
[13:09:21] <sumanah> pf_moore: let's talk about that
[13:09:32] <sumanah> pf_moore: how many betas do you think there will be for pip 10.0?
[13:09:44] <sumanah> 10.0.0 I should say
[13:09:57] <pf_moore> sumanah: Ideally no more. This was an emergency fox for a crashing issue on MacOS.
[13:10:28] <pf_moore> I'm happy to not send it to pypi-announce - people will get the latest beta whatever
[13:10:40] <pf_moore> and the other email channels should get everyone anyway
[13:12:02] <pf_moore> I probably won't bother adding a new notice to the Python blog, FWIW
[13:12:32] <sumanah> if people run `python -m pip install --upgrade --pre pip` right now they'll get the second beta. And if they ran that on Saturday or Sunday or earlier today, experience crashing on the Mac, and report it, we can easily tell them to upgrade again
[13:13:04] <sumanah> pf_moore: so let's not send it to pypi-announce. Where will you be announcing the second beta -- the Insider blog and distutils-sig/pypa-dev, where else?
[13:14:48] <sumanah> pf_moore: I actually think 9.0.3 needs more signal-boosting than it's gotten, because of *its* fix for Mac TLS stuff
[13:14:49] <pf_moore> sumanah: OK, that's fine with me. distutils-sig, python-announce and python-list. I'm not sure it's worth putting it on the Insider blog as I said.
[13:15:09] <sumanah> oh I just saw that you said that, sorry, my eye oddly skipped that
[13:15:29] <pf_moore> sumanah: Yes, I agree re 9.0.3, but traditionally we've kept releases relatively low-key.
[13:16:28] <sumanah> pf_moore: could you talk a little more about that?
[13:16:41] <sumanah> (You know how new I am)
[13:17:17] <sumanah> Thanks for the reminder about python-announce, btw, now I have verified that https://mail.python.org/pipermail/python-announce-list/2018-March/011883.html about the PyPI beta has landed :)
[13:17:42] <pf_moore> sumanah: Well, I think (and I've not been involved in previous pip releases) that we've mostly just announced on distutils-sig in the past.
[13:18:22] <pf_moore> Because of the breaking changes to the internals in 10.0, I made a point of pre-announcing to give people who use internals a chance to fix their code (back in Oct)
[13:18:47] <pf_moore> And I've kept up the habit of announcing 10.0 milestones pretty widely.
[13:19:11] <pf_moore> In actual fact, though, the *end user* impact of 10.0 isn't that huge. It's developers who are more affected.
[13:19:26] <pf_moore> So the net effect is that I've probably over-sold 10.0 a bit...
[13:20:05] <pf_moore> I'll likely review the comms afterwards and try to work out a more reasonable level of announcement for future releases
[13:23:02] <sumanah> pf_moore: so as an American (I think you're a British person? sorry if I'm wrong) I probably tend to over-announce compared to you :)
[13:23:24] <pf_moore> sumanah: Yeah, reserved Brit here ;-)
[13:23:52] <sumanah> We shall meet in the middle of the Atlantic with a reasonable comms strategy :)
[13:24:18] <pf_moore> lol Paint it on the side of a trawler and hope? :-)
[13:26:54] <pf_moore> sumanah: I'm actually trying to aim for comms more like you've being doing with PyPI, IMO you've been doing a really good job and I think it'd be good for packaging as a whole if we engage better with the community than we have in the past.
[13:26:56] <sumanah> pf_moore: Hee! No, more like something that involves long-term channel-building
[13:27:19] <sumanah> That's a wonderful thing to hear, pf_moore. Thank you so much.
[13:28:13] <pf_moore> sumanah: Yeah that would be good. At the moment my biggest problem would be knowing where pip's target audience actually look for information. That's why I've gone as wide as possible. I think we miss too many interested people if we just focus on distutils-sig.
[13:29:15] <sumanah> pf_moore: so I have been asking similar questions -- where do you get your Python packaging/distro/PyPI-etc.-related info? -- when I talk to Python programmers these days
[13:29:32] <sumanah> pf_moore: a *serious number* of them talk about the r/Python subreddit
[13:30:32] <pf_moore> sumanah: That's really interesting to know. I did post the Python Insider post for the pip beta on there, as it happens, so I'll probably add a comment there about the beta2
[13:32:30] <sumanah> pf_moore: And quite a number of people told me that even though they basically never can make it to PyCon, they watch PyCon talks online, and that's a key way they get this kind of info
[13:32:59] <sumanah> this is not particularly scientific -- it's anecotal, or as some call it, artisanal data
[13:33:58] <pf_moore> Still probably more useful than just guessing based on experience with something like the relatively limited distutils-sig community :-)
[13:34:05] <sumanah> Yeah :)
[13:34:17] <sumanah> pf_moore: I'm about to email https://mail.python.org/mailman/listinfo/pythonmac-sig to tell them about pip 9.0.3 unless you want to do so
[13:34:46] <pf_moore> sumanah: No, you go ahead
[13:34:48] <sumanah> pf_moore: ok
[13:56:20] <sumanah> pf_moore: will you check https://github.com/pypa/warehouse/issues/3293#issuecomment-377934325 for accuracy before I send it?
[13:56:39] <sumanah> I'll probably also forward it to python-list/comp.lang.python
[13:57:07] <sumanah> dstufft: di_codes ^
[13:57:13] <sumanah> in case you have time & interest
[14:02:28] <sumanah> hi lghampton
[14:02:42] <lghampton> sumanah: hi!
[14:03:03] <sumanah> lghampton: so I believe the last time we spoke, you were working on the mystery of the "rel" attributes in hrefs in the simple API. I found out the answer
[14:03:17] <lghampton> sumanah: oh? what is it?
[14:03:24] <sumanah> lghampton: would you like me to just tell you the answer, would you like to try to work it out, would you like hints?
[14:03:37] <lghampton> can you tell me? :)
[14:03:40] <sumanah> Sure :)
[14:03:54] <sumanah> lghampton: so once upon a time, PyPI was an index, not a file host
[14:04:03] <sumanah> lghampton: it only pointed to files hosted elsewhere
[14:04:24] <sumanah> lghampton: and then (I am handwaving and simplifying here) there was a period where you could host the files on PyPI or elsewhere
[14:04:49] <sumanah> lghampton: and then we entered the era you now know, the era where all packages that PyPI points to are always files that it self-hosts
[14:05:29] <sumanah> lghampton: but in that middle period, the Era Of Mixed Hosting as I will now call it, when people called the API, it was useful for them to know *what kind of URL this was*
[14:06:05] <lghampton> sumanah: ah, that makes sense
[14:06:17] <sumanah> lghampton: so you see all the relationships listed in https://warehouse.readthedocs.io/api-reference/legacy/#get--simple--project-- ?
[14:06:38] <sumanah> lghampton: PyPI *had to screen-scrape other people's websites* to find packages to serve
[14:07:12] <lghampton> sumanah: that sounds hairy
[14:07:18] <sumanah> lghampton: this actually bit Leonard once: https://www.crummy.com/2011/08/27/0
[14:07:40] <sumanah> oh, I should say, pip was doing screenscraping
[14:07:44] <sumanah> is?
[14:08:33] <lghampton> sumanah: really?
[14:08:43] <sumanah> lghampton: this is the bit I am fuzzier on.
[14:08:53] <sumanah> lghampton: but now! now we are in the third era
[14:09:00] <sumanah> lghampton: (did you ever watch Babylon 5?)
[14:09:26] <lghampton> sumanah: no :)
[14:09:29] <sumanah> no prob
[14:09:49] <sumanah> lghampton: so my assessment, based on a conversation with Donald on Friday, is:
[14:11:07] <sumanah> lghampton: the /simple/<project>/ API no longer serves any <a> tags that deserve "rel" attributes other than "internal", so right now it's fine that those <a> tags no longer have a "rel" at all
[14:11:50] <sumanah> lghampton: I can see arguments for adding some of them back, like the homepage, BUT, instead of adding new stuff to this API, we should focus our energy on getting legacy shut down, and then rearchitecting a glorious new API that will sing and dance and have lots of cool things
[14:12:10] <lghampton> sumanah: ok, that sounds like a good plan
[14:12:15] <sumanah> lghampton: and therefore, I believe you should submit a PR removing all the "rel" stuff from https://warehouse.readthedocs.io/api-reference/legacy/#get--simple--project--
[14:12:36] <lghampton> sumanah: ok, that sounds good
[14:12:46] <sumanah> lghampton: rockin'. Would you like to start on that now?
[14:12:55] <lghampton> sumanah: sure thing :)
[14:12:57] <sumanah> :)
[14:13:00] <sumanah> !logs
[14:13:00] <pmxbot> http://kafka.dcpython.org/channel/pypa-dev
[14:17:34] <sumanah> lghampton: you might find it useful to link to or quote http://kafka.dcpython.org/day/pypa-dev/2018-04-02 in your PR :)
[14:17:58] <lghampton> sumanah: ok
[14:22:27] <sumanah> lghampton: after you submit that PR, could you review https://github.com/pypa/warehouse/pull/3273 and https://github.com/pypa/warehouse/pull/3487 , in particular on mobile?
[14:22:46] <lghampton> sumanah: sure thing
[14:23:03] <sumanah> lghampton: if you click around in Chrome or Firefox developer tools, there's a responsiveness mode where you can check how a screen looks on different preset emulated devices
[14:23:28] <lghampton> sumanah: should I also check it using my phone?
[14:23:52] <sumanah> lghampton: the emulator should work well enough for #3487, but to test the trove classifier functionality, I think it would be good for you to *actually* test on a phone
[14:24:10] <lghampton> sumanah: ok
[14:24:34] <sumanah> lghampton: #3273 involves the user selecting checkboxes and I think that's fiddly enough that I want someone to try actually tapping things with a real metatarsal on a real device :)
[14:24:57] <lghampton> sumanah: my metatarsals are at your disposal :)
[14:25:11] <sumanah> lghampton: swell :)
[14:25:22] <sumanah> wait, don't swell, then your hands won't fit in your gloves!
[14:25:40] <lghampton> sumanah: yes, that would be problematic today :)
[14:25:46] <sumanah> indeed :)
[14:26:03] <sumanah> ok I'm going to stop making awful puns and go do other stuff. see you online in about 40 min lghampton
[14:26:13] <lghampton> ok :)
[14:36:09] <tos9> So, warehouse+setuptools question if anyone happens to know
[14:36:27] <tos9> somewhere in warehouse (I can go look up where again) is some code for parsing setuptools metadata versions
[14:36:53] <tos9> I notice that it specifically enumerates which versions of setuptools metadata that it accepts (so, the recently released 2.1 version is now in there)
[14:37:12] <tos9> Is that really the right way indexes should be treating that (reject artifacts using PKG-INFO versions they don't recognize)
[14:37:49] <tos9> (The "X" to this "Y" is "artifactory is currently broken, because they don't have code to handle 2.1, and I figured 'warehouse definitely has a correcter way of doing that', but nope it seems to also do what I'd guess artifactory is doing :P"
[15:00:18] <di_codes> tos9: yeah, i think since any Metadata-Version change _might_ not be backwards-incompatible, it’s OK for indexes to be opinionated about what versions they support.
[15:07:25] <tos9> di_codes: ok -- so setuptools neither guarantees backwards compatibility, nor provides a python API that parses PKG-INFO that *does* promise backwards compat?
[15:08:01] <tos9> (I know this is I guess part of what metadata.json is trying to solve? but looks like that work is still awhile away from being a thing)
[15:11:19] <dstufft> tos9: historically the upload API sent the metadata information as a HTML form alongside the file
[15:11:26] <dstufft> and so we don't actually inspect the contents of the file
[15:11:40] <tos9> dstufft: Right I noticed that
[15:11:45] <tos9> dstufft: so warehouse pins the version, but doesn't read the file
[15:11:52] <dstufft> yea
[15:11:57] <tos9> isn't that a bit odd?
[15:12:20] <dstufft> hysterial raisins
[15:12:25] <tos9> or would you expect/want artifactory to also do that
[15:12:38] <tos9> dstufft: heh yeah I assume we are in a raisin field for all of this
[15:13:19] <dstufft> probably in a hypothetical upload 2.0 we'll drop passing information along side the files, and just like, inspect the files
[15:13:39] <tos9> dstufft: and then how should alternate indexes implement the same?
[15:13:52] <tos9> is PKG-INFO (or metadata.json) going to be The Way to serialize package data
[15:14:17] <tos9> and would setuptools provide a way to parse it, or would each index basically parse PKG-INFO under whatever guarantees setuptools provides
[15:14:34] <dstufft> tos9: Uh, probably If they want to be compatible the best thing to do is to do it like Warehouse does, BUT then can inspect the files too
[15:14:42] <dstufft> twine basically just inspects PKG-INFO (in a sdist)
[15:14:50] <tos9> oh, good point
[15:14:52] <dstufft> and then submits that data in the form
[15:14:53] <tos9> let's see if twine pins
[15:15:19] <dstufft> twine uses pkginfo to do it's parsing
[15:15:20] <dstufft> https://pypi.org/project/pkginfo/
[15:15:46] <tos9> a ha! ok cool, that's the thing I was expecting to exist, cool
[15:16:05] <tos9> (doesn't really help for artifactory which is java or whatever, but probably if they cared enough they'd use some subprocess that uses that or something)
[15:22:54] <tos9> hm, ok, seems like to me that devpi (for another data point) just looks at the post body and doesn't bother inspecting the file at all
[15:23:03] <tos9> at least on the server side, the client side uses pkginfo
[15:53:53] <sumanah> hi di_codes! I presume you're catching up on stuff from last week
[15:58:49] <di_codes> sumanah: indeed
[15:59:09] <sumanah> di_codes: I can wait to ping you about new stuff, then :) talk with you in ~1 hr
[16:21:48] <sumanah> waseem18: *wave* hope things are going well
[16:22:22] <waseem18> Yes sumanah :)
[16:23:20] <waseem18> Project I'm working on in my office is releasing this week. So I'm not able to take enough time to work on Warehouse/Twine
[16:24:47] <sumanah> waseem18: understood. Let us know when you pick things up around Warehouse & Twine again. :)
[16:25:01] <sumanah> waseem18: remind me, does it look like you'll be able to come to PyCon North America or EuroPython?
[16:26:13] <waseem18> I can't for Pycon NA. Hopefully EuroPython :)
[16:26:57] <waseem18> One or Two more days and I'll be free and will start working on Warehouse/Twine :)
[16:27:52] <sumanah> understood :)
[16:41:04] <lghampton> sumanah: I tested 3273 on my phone (I had some issues with getting the site to display on it on my home wifi but resolved them.)
[16:41:27] <lghampton> sumanah: is there anything else I should take a look at before I leave a comment?
[16:42:02] <sumanah> lghampton: lemme look at 3273 for a moment
[16:42:30] <sumanah> lghampton: what functionality did you test?
[16:42:58] <lghampton> I tested that the trove classifiers displayed correctly, and that I could select and un-select them on my phone (I can)
[16:43:31] <lghampton> sumanah: I also tested the select/un-select with the phone simulation in Chrome dev tools
[16:43:54] <sumanah> lghampton: that sounds good enough -- go ahead and comment
[16:44:00] <lghampton> sumanah: ok
[16:46:28] <sumanah> ok I think https://github.com/pypa/warehouse/issues/3293#issuecomment-377934325 is good enough to ship -- dstufft if you have a moment to look, that would be great
[16:47:58] <sumanah> also di_codes, we missed you, glad you are back.
[16:53:33] <di_codes> sumanah: glad to be back!
[17:53:19] <sumanah> agronholm: thank you for the wheel release!
[17:53:37] <agronholm> sumanah: np
[17:56:45] <lghampton> sumanah: I am going to go eat lunch
[17:56:53] <sumanah> sounds good! enjoy lghampton
[17:57:00] <lghampton> sumanah: thank you :)
[18:37:27] <lghampton> sumanah: I am back
[19:14:00] <lghampton> sumanah: Are you around?
[19:37:23] <sumanah> Hi lghampton
[19:37:28] <lghampton> sumanah: hi
[19:37:38] <sumanah> I was on a call just now, am back
[19:37:53] <sumanah> lghampton: I'll take a look at other open PRs, just a moment
[19:38:03] <lghampton> sumanah: ok
[19:38:34] <sumanah> waseem18: *wave* I saw your note on https://github.com/pypa/warehouse/issues/3486#issuecomment-378016668 -- great, looking forward to you working on it!
[19:39:10] <sumanah> waseem18: I'm not sure how autocomplete stuff would end up affecting caching, performance, and stuff like that, so it's probably going to be an iterative process, adding that feature
[19:39:21] <sumanah> where you share thoughts with us and we talk about pitfalls
[19:39:40] <waseem18> I was able to tweak Elastic search in generating suggestions. I'm trying to build some UI on top of the current Search bar so as to show the suggestion results
[19:39:42] <sumanah> lghampton: I see you have a change requested in https://github.com/pypa/warehouse/pull/3503
[20:05:29] <sumanah> dstufft: https://github.com/pypa/warehouse/issues/3293#issuecomment-377934325 -- would you cast an eye over this for accuracy before I post it various places?
[20:06:43] <sumanah> I feel like AnyTimeTraveler should re-join *before* they quit. You know, because they're a time traveler.
[20:10:30] <dstufft> sumanah: looks fine to me in a quick skim
[20:10:58] <sumanah> Thanks dstufft, will post to the Mac SIG and some other places
[20:21:17] <lghampton> sumanah: I responded to Dustin's comment, and I am going to sign off
[20:31:29] <sumanah> Thanks lghampton
[20:31:30] <sumanah> see ya
[20:31:43] <lghampton> sumanah: see you later! :)
[21:38:48] <sumanah> di_codes: there's a question about the search indexing in https://github.com/pypa/warehouse/issues/3471 that's directed at Ernest, and of course he is unavailable the next few days. May I ask you to look?
[21:43:59] <di_codes> sumanah: I don’t have great insight into how or when this will be fixed. it seems like <https://github.com/pypa/warehouse/pull/3470> is an attempt at fixing the issue, but it’s unmerged
[21:45:25] <sumanah> di_codes: ok, thanks, I'll comment there to say so
[22:08:28] <sumanah> now that I reread it, the title of that issue reminds me of "Do Androids Dream of Electric Sheep?"
[22:08:58] <aalmazan> hi guys! sorry, di_codes (assuming you're @di) didnt mean to push that commit earlier (#3378) and thanks for your patience with my PRs
[22:13:34] <di_codes> aalmazan: no worries at all!
[22:15:56] <sumanah> aalmazan: you are correct, di_codes is @di on GitHub
[22:45:38] <sumanah> Notes from today's Warehouse core developers' meeting are now up at https://wiki.python.org/psf/PackagingWG/2018-04-02-Warehouse .
[22:48:30] <sumanah> perhaps the most exciting thing coming out of that is that we have a schedule for April's launch/redirect and shutting down legacy PyPI. https://wiki.python.org/psf/WarehouseRoadmap is updated accordingly.