Log file Viewer
#pypa logs for Tuesday the 21st of June, 2016
[03:22:42] <Eber> Hey guys, I'm using pip on Cygwin and after installing a package, I can't run it directly... I think I have to add something to my PATH in order to make it work. Does anyone knows where can I find more info on how to do that? Thanks!
[07:38:09] <ssc> Is it possible to put C-libraries (like QT or libhdf5) into wheel packages as you can do with conda?
[14:48:59] <tdsmith> you can put arbitrary files in wheels and configure your linker to do the correct magic
[14:49:06] <tdsmith> there's a tool that automates this for os x https://pypi.python.org/pypi/delocate
[14:56:01] <tdsmith> i can't speak to that authoritatively; pip and conda have different goals imo. there's been a lot of discussion on distutils-sig lately.
[15:02:30] <[Tritium]> I think its important to remember that conda is primarily developed by a company who's business model is "Python packaging is painful, lets sell it."
[15:03:22] <[Tritium]> emulating their tools, or feeding their tools, is PROBABLY not going to end with a pleasent packaging experience
[15:04:39] <gchristensen> I thought their primary business model was more about munging lots of numbers for banks and what-not
[15:05:21] <[Tritium]> gchristensen: they sell prepackaged numpy to the banks so they can do their munging
[15:06:49] <ssc> anyways, being able to install QT and libhdf5 as a non-root-user via cona is a huge benefit.
[15:07:45] <[Tritium]> pyqt5 has manylinux1 wheels that ship with the entirety of qt... pyside does the same i think
[15:08:38] <ssc> pyqt is not the only problem. we need to ship some none-python-stuff to people who are not sudoers on their machine. qt was just an example.
[15:09:11] <tdsmith> conda seems to be encroaching on general-purpose package management which is not a goal for pip
[15:09:15] <ssc> maybe we could convert conda packages to wheels, because both are basically just zips.
[15:10:29] <[Tritium]> There is only so far that a language package manager should go. conda is trying to manage your math environment (all languages), and pip is trying to manage your python environment. I dont think anyone is asking npm to pull in the world....
[15:21:59] <dstufft> [Tritium]: tdsmith ssc njs has a plan for enabling that that he calls pynativelib
[15:22:33] <thrawn01> Hey guys, I get "HTTPError: 499 Client Error: Client Disconnected for url: https://upload.pypi.io/legacy/" when using twince to upload my package.
[15:23:12] <[Tritium]> I'm not going to cry "out of scope!" when someone is writing the code... so...uh....i guess i was wrong!
[15:25:33] <dstufft> [Tritium]: to be clear, it's not something that pip or PyPI itself is going to support directly. His thing is a plan to layer that idea into what was already there
[15:26:26] <dstufft> certainly its' on the fringe cases of support, but given that Wheels end up being just zip files that get unpacked, there's nothing cramming you from shoving pretty much anything in there, though the use cases will always be optimized for Python packages
[15:26:49] <dstufft> vs something like Conda, apt. Homebrew, Choclatey, etc where the use cases are optimized differently
[15:27:49] <ngoldbaum> [Tritium]: the problems that conda solves are real ones - it's very rare in my world for a python package to depend only on other python packages.
[15:28:15] <ngoldbaum> and it's extremely convenient to be able to use a cross-platform package manager like conda to install those needed C dependencies
[15:28:23] <dstufft> there's some argument that given a cross paltform thing like Conda that something like pip isn't required, assuming you can convince authors to upload conda packages instead of something else
[15:29:14] <dstufft> but the vast bulk of what conda really provides is precompiled binaries, the package maanger itself isn't super interesting (similarly to the vast bulk of what pip provides is a huge library of things on PyPI, the tool itself isn't very interestig)
[15:30:20] <dstufft> some of that is due to the historical nature of CPython being glue code between C things
[15:30:43] <dstufft> whereas you're much more likely to see someone rewrite something in pure JS than you are to see someone bind to a C lib
[15:31:33] <ngoldbaum> it also doesn't help that extremely popular environments like OSX and windows don't have a proper package manager
[15:31:42] <dstufft> and a bit of that is the math people yes, we have a contigent of users who want to use highly optimized things written a long time ago in a galaxy far away where FORTRAN was king ;)
[15:32:55] <dstufft> Homebrew is pretty OK on OS X, though part of the problem isn't that they don't have a package manager that people can use, it's that they have N package managers with overlapping functionality and which one is "best" will vary depending on who you ask :D
[15:32:56] <[Tritium]> ngoldbaum: i...actually think 'pay continuem analytics to figure that out' is probably the best solution for that
[15:34:27] <tdsmith> as a scientist who administers systems i also feel qualified to say that scientists are not very good at systems administration
[15:34:40] <ngoldbaum> and i don't think continuum analytics is a bad actor here like you're making them out to be. for example, they didn't have to release conda under BSD. efforts like conda-forge are also helping making the conda ecosystem less dependent on CA for everything
[15:34:50] <dstufft> Outside of when I get somewhat grumpy from the rhetoric, I think what Anaconda provides (that's the "pay continium to figure that out, vs conda the package maanger, or the PyPI to their pip) is probably useful and I don't think that they should stop existing or whatever
[15:35:20] <[Tritium]> they just have no incentive (legally) to make the situation better in general
[15:36:00] <dstufft> I don't think CA cares much for what pip does, but that's OK because I don't care a whole lot about what conda is doing :)
[15:38:42] <dstufft> I don't think conda is a general purpose solution, but the problem conda is trying to solve is fundamentally simpler which lets them provide a better out of the box experience, given your problem falls within those constraints
[15:40:41] <dstufft> In that, 90% of the pain of Python's packaging comes from sdists and the need to treat source packages as real releases. This makes everything a lot harder, not jus because our format isn't great, but also because inherently building from source is a harder thing to do than installing an already built binary
[15:40:58] <dstufft> conda completely eliminates that need, which lets them give a much nicer base line experience
[15:42:26] <[Tritium]> dstufft: how many death threats do you think you will get if pip just stops getting sdists?
[15:44:44] <dstufft> on one hand, if we move to only install from wheel then our baseline becomes much better (once we get over the intiial pain anyways) but then we revert to the case where anyplace we don't have a compiled wheel for it doesn't work without extra pain
[15:45:26] <ngoldbaum> yes, there's no fallback, and you need extensive build infrastructure to do it properly
[15:45:50] <dstufft> (Inherently there's nothing in conda that would prevent it from working on FreeBSD, but it'd need to start producing FreeBSD packages for it to work)
[15:46:16] <dstufft> [Tritium]: I don't think we support FreeBSD wheels no, so someone using pip on FreeBSD will always need a compiler for anything but pure Python
[15:47:45] <[Tritium]> Hypothetically speaking only, freebsd is closer to mac and windows than linux... you can sanely build a 'freebsd i386' wheel and it should work on other freebsds.
[15:48:23] <dstufft> all that would require is someone to spend the time to spec out the platform tag
[15:48:43] <dstufft> I could see us maybe move to a platform specific definition of what pip does here
[15:48:55] <ngoldbaum> see also ARM linux, other BSD flavors, any other somewhat obscure OS/platform
[15:49:49] <dstufft> but we'll have to see, I'd prefer not doing that if we can just make it so that authors don't have any incentive not to release wheels
[15:50:31] <ngoldbaum> dstufft: out of curiosity, is there a plan for warehouse to incorporate access to build farms for supported platforms?
[15:50:37] <dstufft> [Tritium]: we legit have traffic to PyPI using pip from AIX and other obscure platforms
[15:51:19] <dstufft> ngoldbaum: I want to support it, I haven't thought hard about it yet though since there's much lower hanging fruit to tackle first
[15:52:15] <[Tritium]> ngoldbaum: there was a thread about this on the sig mailing list. The consensus was "Dont ask for things in a condecending, demanding way" and "for now, we can document how to use travis and appvayor for the 95% case"
[15:52:17] <ngoldbaum> good luck moving to warehouse, thanks for making the whole ecosystem more secure by default and easier to work with
[15:53:20] <tdsmith> the original email _was_ pretty bad but he probably didn't need to hear that three or four times
[15:53:51] <ngoldbaum> dstufft: yeah, for now we've avoided having a github mirror to avoid confusing new contributors or adding a responsibility to the project to manage tooling for a git-hg bridge
[15:54:05] <dstufft> it's easier to document right now for folks who can use those platforms and then add our own thing later on
[15:54:34] <dstufft> I'm sort of stretched to the breaking point so I can't take anything more on right now, hopefully killing legacy pypi and getting warehouse our will reduce that
[15:54:46] <[Tritium]> who knows, the psf or infrastructure team might be able to get donated builders from them
[18:55:56] <sveinse> I am wondering if I have found a bug in pip: https://bpaste.net/show/3a8a545ef603
[18:57:07] <sveinse> Installing and then reinstalling using the same version number in a package neither complains about being alredy installed, and it does not clean up the old files before installing the new
[19:03:58] <sveinse> ngoldbaum: Thanks. Is there a "triage" state for the issue there? Because I'm not at all sure it is a bug, and it would be nice to get confirmation that this is a plausable error
[19:04:31] <ngoldbaum> sveinse: not sure, just suggesting that so your issue doesn't get lost in the IRC noise