Log file Viewer
#pypa-dev logs for Tuesday the 26th of January, 2016
[01:51:01] <StevenK> dstufft: Can you look at https://github.com/pypa/packaging/pull/50 ? I'm not sure about some of the docs prose, either comment on the PR or fix it
[02:21:10] <pmxbot> Rather than re-use globals of setuptools.launch, build a unique namespace in which to launch the script. Prevents imports from occuring relative to 'setuptools' on Python 2. Ref #490.
[16:37:56] <dstufft> nlh: by the way, I adjust Warehouse deployment so merges to master go straight to both warehouse.python.org _and_ warehouse-staging.python.org
[16:42:40] <dstufft> So right now, TestPyPI serves two purposes - One is as a staging site for PyPI changes, largely because it's impossible to really test PyPI legacy, but I think the new code base we can just deploy straight to prod with no issues so that removes that purpose. The other purpose is as a test sandbox for people to upload things to try out, and I think that https://github.com/pypa/warehouse/issues/726 is a better solution to that overall
[16:44:56] <nlh> ok, well, with CI, surely the need for a staging site is reduced. And that issue does indeed look like a better solution :D
[16:45:30] <dstufft> nlh: I also plan to enable review apps, which once I do that I'll need to get you onto the Heroku project too
[16:47:40] <dstufft> The blog post? heh yea. I was confused this morning when a bunch of people were talking about warehouse
[16:49:39] <dstufft> I have download information being sent straight into google bigquery now where it can easily be queried, I just need to get warehouse querying it (which is a bit harder because a single query in BigQuery can take ~10-20s instead of ms for a normal DB so I have to do some async crap)
[16:52:16] <dstufft> right now you can't link to a specific tab, and I'm not sure how to trigger a "go fetch the stats because it takes a bit of time" HTTP request with the tabs
[16:52:40] <nlh> hmmm.. the only problem with this is that the tabs are quite far down the page, so it's going to require the user to scroll down every time they click on a different tab
[16:54:13] <nlh> re: fetching the stats - is it *really* wasteful to load them in the BG after everything else has loaded?
[16:54:22] <dstufft> mostly I need something so I know when the stats tab is open so a spinner can be displayed while we compute the stats (if the stats have been computed recently, we'll have them cached and will return really quickly, if we haven't it will take a few seconds)
[16:56:27] <nlh> doesn't look like it, though I am not opposed to replacing these tabs with something that does
[16:57:32] <nlh> I know you can do it with jquery ui tabs, though that might be a heavy hammer to swing
[17:00:27] <dstufft> nlh: Also RE: Translations, I just tweeted https://twitter.com/dstufft/status/692027602438090752 so we'll see... I don't think I personally know any Chinese users of PyPI (though I don't hardly go interrogating people about where they live so it's possible one of the folks who I don't know where they live, live in China)
[17:01:37] <dstufft> Unless we get some some response, I think I'm going to just say our translations support what l20n.js supports and leave it at that unless someone shows up to make it happen back further than what l20n.js supports
[17:03:12] <dstufft> Specially since only 0.59% of PyPI visits at all come from a non english using IE (of any version)... though there might be a chicken/egg thing there
[17:06:48] <dstufft> I'm committed to making it so the way l20n.js fails is that you just get the untranslated english content
[17:08:03] <dstufft> so it's not specifically tied to what versions of what we support for the design stuff itself, but I'll let you decide on that :) I'm happy to have the cut off be just about anywhere.
[17:08:56] <nlh> well, we'll see about ie8 for design... I want to do it, but it might not be part of 'become pypi'...
[17:11:39] <nlh> it's not a huge percentage, but it is quite a few users. My idea is not to make it 'perfect' - just to check that it works...
[17:15:44] <dstufft> I don't know if the current stuff works on IE8, but I assume it does becasue it's like 15 years old :D
[17:16:19] <nlh> that's one of the reasons I don't want to totally break ie8... imagine the complaints!
[19:49:55] <pmxbot> jaraco pushed 3 commits to setuptools (https://bitbucket.org/pypa/setuptools/) :
[19:49:56] <pmxbot> Adapt resolution of classes from importlib.machinery. Restores compatibility for PyPy3 where importlib.machinery exists but FileFinder and SourceFileLoader aren't implemented.
[21:25:19] <nlh> dstufft: lol... just read that comic. I think I might turn off twitter / not look at the issue tracker for a good week after the launch :P
[21:25:40] <nlh> and rely on you to email me if there is anything urgent. Unless, of course, you also turn off everything
[21:26:32] <dstufft> lifeless: oops, didn't see this hightlight, sorry. been dealing with warehouse-y stuff :]
[21:26:57] <dstufft> nlh: heh, as much as I'd like to think I'm going to hide... the truth is there's little chance of it because I am not good at making life choices
[22:40:48] <lifeless> Ivo: because packaging is about being the standard implementation, not about being a reimplementation of the entire stack
[22:41:36] <dstufft> it's not hard to bundle a packaging dependency inside things that need to bundle packaging, as long as we keep the general pip guidelines in mind
[22:41:53] <Ivo> when its forming the base for things building and installing things, it helps when it doesn't need building and installing itself
[22:44:20] <dstufft> Ivo: is it causing an issue? pyparsing dep is inside pip 8 and I hadn't seen anyone claiming an issue but I could be wrong :]
[22:45:42] <lifeless> Ivo: if we can't use those things, pip stops being about package installations and becomes a far bigger code base
[22:45:47] <Ivo> dstufft, could you see if you agree with the direction I'm guiding the submitter of https://github.com/pypa/pip/pull/3419 ?
[22:46:28] <Ivo> For NTLM proxying, I thought since it's small use case that it could work with an extras_require dependency
[22:51:09] <Ivo> lifeless, aren't they already in that chicken-egg problem when they go to get pip in the first place?
[22:52:44] <lifeless> Ivo: there are three routes to install pip - get-pip.py, which has everything bundled, git, which has everything bundled, or linux etc distributions
[22:53:23] <lifeless> Ivo: for the second, if they can't get through via git, they can't use it, and if they can, by vendoring, it will just work
[22:54:01] <lifeless> Ivo: for the third, same: if they can't do updates, they can't use that route,a nd if they can, they can fully install pip that way
[22:55:20] <dstufft> lifeless: I've been going down through the PR right now in github between emails and trying to figure out if my concerns are just fear of changing a big important part of the stack that has existed for, wella s long as Python packaging has, or if they are well founded concerns :]
[22:56:07] <lifeless> dstufft: please do grab me here if you want to talk - or even voice if that will help
[22:59:34] <dstufft> lifeless: Will do, I think it's mostly just being nervous around just making changes to a pretty underlying cog in our little rube goldberg machine but trying to make sure that's the case :]
[23:03:16] <dstufft> Ivo: I've read zero of the code but I'd say that if the NTLM auth is pure python and single source and licensed reasonablly (e.g., not GPL) it probably makes sense to just bundle it unless there's some other reason to specifically avoid bundling that particular piece of software
[23:05:38] <Ivo> across 3 packages, I think pure python but I can do a better check to be sure, including a pure python DES implementation :D
[23:08:08] <Ivo> well python doesn't really play well with the idea of static inclusion being separate from dynamic linking to begin with
[23:10:24] <Ivo> To my mind, just plopping a chunk of LGPL source code and importing it and using the API seems as compliant with "dynamically linking to" as python could hope to be
[23:11:12] <dstufft> I think ``pip install thingwithlgpl`` and importing it as probably closest to dynamically linking whereas copying the source wholesale is... I don't know what?
[23:12:08] <dstufft> I wonder if HPE has any lawyering types that are available for me to ask random licensing questions of hmmm
[23:12:35] <Ivo> in the end its the same source code, it's just sitting in a different folder (after you've either `pip install`'ed it, or had it included in pip's source code)
[23:14:40] <Ivo> If you were conservative about that situation I guess you could say that's an argument to do the extras_require mechanism?
[23:15:43] <dstufft> given the LGPL there it might make the most sense until some lawyer tells us if it's OK Or not
[23:16:03] <dstufft> https://twitter.com/bagder/status/692115125143257096 this accurately represents what we do here to a scary degree.
[23:20:14] <lifeless> dstufft: because fundamentally, even if a lawyer says 'this should be fine' - authors intent matters a lot.
[23:20:33] <lifeless> dstufft: its going to depend fairly heavily on the definition of mere aggregation, I suspect
[23:20:54] <Ivo> Requests is Apache licensed, so whenever it gets distributed, technically it requires that the distribution include the license is included with it
[23:22:20] <lifeless> dstufft: I can introduce you if you like, its probably time I fight with the vpn again
[23:23:43] <dstufft> setuptools_shim is the thing to turn a PEP.. uh I guess we don't have a number of rit yet, the build system one, into a legacy sdist?
[23:24:36] <dstufft> I'm going to go eat dinner, then when I come back, I think we'll bite th ebullet on this
[23:24:42] <lifeless> dstufft: setuptools_shim is an implementation of the setup.py interrface that backs onto the abstract build system
[23:25:01] <lifeless> dstufft: so its not a PEP thing; its how pip <9 would support things that have migrated to the abstract build system
[23:26:01] <dstufft> lifeless: remind me why we didn't just make a new extension for the filename instead of a shim?
[23:26:11] <lifeless> Ivo: current pretty rendered copy: https://github.com/rbtcollins/interoperability-peps/blob/18ccdd03b14475cc633545060f99a0f4050a2e38/build-system-abstraction.rst
[23:27:10] <dstufft> Ivo: No, you still need to execute code (and realistically, you can't get rid of that- building is fundamentally a aciton where you execute code)
[23:28:33] <lifeless> dstufft: if we ignore old pips, then anyone who migrates cannot be installed by a pretty large % of the Python universe.
[23:30:04] <lifeless> dstufft: filename extensions for setup.py or whatever don't prevent wrapping an implementation of the absraction in a thing called setup.py
[23:33:16] <dstufft> lifeless: unless they also publish a legacy sdist if they want to do that, in either case you're going to run into people who either care about backwards support or they don't. For the people who do, uploaing an old style package while they care about supporting the old version is doable (possibly with a shim) if someone doesn't care about support though (which I imagine at some point in the future will be the case) reusing .tar.gz creates
[23:33:16] <dstufft> a sort of crummy experience, you get confusing error to someone who isn't up on the packaging standards (https://caremad.io/s/vzQB0go6QM/) vs being told it can't find a distribution that is suitable (or just using an older version)
[23:34:30] <lifeless> dstufft: thats no worse that being told 'no distribution available' when pypi's web UI is showing the thing there.
[23:34:37] <Ivo> do you guys remember if there was any specific hard thing(s) to work on that were blocking pep 426 from getting out of draft?
[23:35:37] <lifeless> Ivo: all the stuff thats now in 508 needs to be stripped out. The enterprisy bits need to be made sane. The speculative stuff needs to be removed or broken out into thigns for individual discussion