PMXBOT Log file Viewer

Help | Karma | Search:

#pypa logs for Tuesday the 24th of March, 2015

(Back to #pypa overview) (Back to channel listing) (Animate logs)
[05:52:32] <tdsmith> removing -lpython from build scripts is my new crusade
[05:53:56] <tdsmith> explicit linkages to libpython / Python frameworks on OS X mean you can't build against e.g. system python and import from e.g. homebrew python because it segfaults the interpreter
[05:54:34] <tdsmith> which causes all kinds of unnecessary headaches in homebrew-land
[07:14:18] <tdsmith> cmake-based build scripts are the worst offenders
[11:25:43] <fladd> hi there, quick question: Can PIP install .exe installers? Like easy_install can do?
[11:27:37] <Wooble> fladd: if I remember correctly it cannot.
[11:27:53] <fladd> mmh...so there is no way to install numpy with pip on windows...
[11:28:11] <Wooble> Why can't you use easy_install?
[11:28:26] <fladd> numpy is a dependency in my project
[11:28:39] <fladd> it should be downloaded and installed automatically
[11:29:55] <Wooble> It's certainly odd that they provide wheels for OS X, for which it's easy to get a sane build system, and not for windows :/
[11:30:38] <fladd> i don't know...
[11:30:47] <fladd> python packaging/distribution is really broken...
[11:31:10] <dstufft> ask them to publish wheels
[11:33:02] <fladd> what happens if I define a link to http://www.lfd.uci.edu/~gohlke/pythonlibs/ in dependency_links? Will those be preferred by PIP, or will PIP still try to build numpy on Windows?
[11:33:24] <dstufft> depends
[11:33:40] <dstufft> pip will pick the most appropiate file
[11:33:55] <fladd> and that is defined how?
[11:34:04] <dstufft> it prefers the newest version that matches your version specifier, then it prefers wheels, then it finally prefers sdists, then it fails
[11:34:06] <mgedmin> btw you can use wheel to convert an egg to a whl
[11:34:54] <fladd> so if it finds a numpy‑1.9.2+mkl‑cp27‑none‑win32.whl, it will prefer that?
[11:35:16] <dstufft> assuming there isn't also a numpy-1.9.3-tar.gz
[11:35:43] <fladd> i see...
[11:36:07] <dstufft> version number is more important than file format
[11:36:08] <fladd> and i assume there is no way I could tell pip with an option to install, that it should ignore source?
[11:36:14] <dstufft> IOW
[11:36:44] <dstufft> not currently, there's an open issue for it. Of course you can do something like pip install numpy==1.9.2 mythingthatdependonnumpy
[11:38:00] <fladd> and can I also do a pip install mythingthatdependonnumpy[numpy==1.9.2]
[11:38:02] <fladd> ?
[11:38:10] <fladd> it will be an extra dependency...
[11:39:15] <dstufft> No, you can't put the == inside of an extra, you'd have to specify it seperately
[11:39:32] <fladd> sigh...
[11:40:36] <fladd> maybe the easier solution is to write an install python script, which just downloads all dependencies from various sources I define and then installs that...
[11:40:50] <fladd> its ugly, but at least it would work
[11:41:07] <fladd> I guess the whole PyPI thing is just not there yet...
[11:47:54] <dstufft> *shrug*
[11:48:10] <dstufft> ~2 billion HTTP requests a month is pretty "there" ;)
[11:48:43] <mgedmin> herding cats is hard
[11:48:57] <fladd> well, if the biggest python packages don't support it, it is not there yet. Same as with Python 3...
[11:49:35] <fladd> we have a rather simple package we want to make available on PyPi, but even that is not possible
[11:49:40] <dstufft> Numpy supports PyPI just fine, they just aren't releasing Wheels for Windows
[11:49:48] <fladd> exactly!
[11:50:04] <fladd> no acerage windows user has a c compiler installed
[11:51:17] <dstufft> That's not a problem with PyPI, that's a problem with Numpy. I dunno why they aren't releasing Wheels for Windows, Wheels themselves work fine on Windows, many other projects are releasing them (and gohlke's page shows they work for Numpy too). It might have something to do with them wanting to provide different options for things like SSE2 vs SSE3 or something, I dunno.
[11:51:50] <fladd> if big package maintainers do not see the value of putting their compiled code (which they have for windows, there are installers!) into PyPI, it seems to me that they don't see PyPI as a proper way to distribute their code
[11:52:31] <mgedmin> it takes effort to move a big ship
[11:52:58] <mgedmin> imagine you're a maintainer of a popular package; it's built with a system of jury-rigged scripts
[11:53:17] <mgedmin> now pip/pypi introduces a new feature (wheels), but you're afraid to touch your scripts
[11:53:22] <mgedmin> or don't have the time
[11:53:31] <mgedmin> or don't understand them because some other volunteer created those
[11:53:38] <dstufft> eh, Wheels are a fairly newish thing. and there are a lot of package maintainers. Often times getting new things to happen is more a matter of inertia than not feeling like they aren't suitable (as mgedmin is saying)
[11:55:58] <dstufft> http://pythonwheels.com/ shows support for Wheels on PyPI for the top 360 (in terms of download #s), it's roughly 35% or so. Pretty decent considering it's new and some of these things haven't seen releases in years
[11:56:08] <dstufft> and it's getting better
[11:57:24] <dstufft> Some of those also don't have wheels because for compiled code, building wheels requires access to a Windows machine, and some folks may not want to do that
[12:05:21] <meshy> morning :)
[12:06:59] <ronny> re
[12:48:28] <ionelmc> fladd: ah come on it's not that hard to install a compiler
[12:48:46] <fladd> not for me, no, but for the poeple I make the software for, it is...
[12:48:55] <ionelmc> plus the pain goes away after a while
[12:49:04] <ionelmc> you get used to it
[12:49:43] <ionelmc> fladd: you can always build wheels yourself for the lagging 3rd party
[12:50:07] <ionelmc> you only need to figure out a way to get your customer to install your wheels
[12:50:18] <ionelmc> and there are few ways to do it
[12:51:15] <ionelmc> like a custom package index (devpi looked pretty interesting) or some custom install thingy (just put the wheels in a dir and use pip install --find-links
[18:21:20] <snx> Hi
[18:21:46] <snx> I am a new user to virtualenverapper
[18:22:25] <snx> How can i create a venv using python 3?
[18:22:53] <tos9> same as with virtualenv, virtualenvwrapper passes args along
[18:22:58] <tos9> well no it doesn't
[18:23:02] <tos9> but it uses the same arg
[18:23:15] <tos9> mkvirtualenv -p python3 whatever
[18:23:40] <jessamyn_> I'm mostly using py3 now, so I put it first on my path. What tos9 said will work if you don't want to do that.
[18:26:56] <tos9> it's the same even if Py3 is first on your path
[18:27:07] <tos9> unless you called your py3 `python`, which you shouldn't do for other reasons
[18:28:13] <jessamyn_> tos9: you are right, and I didn't do that, and now I"m trying to figure out exactly what I *did* do
[18:28:21] <mgedmin> when you activate a virtualenv created with python3 it ends up putting a 'python' executable in from of your $PATH that runs Python 3
[18:28:21] <tos9> jessamyn_: you probably installed virtualenvwrapper to your py3
[18:28:33] <tos9> jessamyn_: virtualenvwrapper defaults to the python you installed it to
[18:28:49] <jessamyn_> hmmm
[18:29:01] <jessamyn_> I didn't think I reinstalled virtualenvwrapper
[18:29:48] <jessamyn_> snx: ignore me, as I haven't yet figured out what I did
[18:32:16] <jessamyn_> ah, tos9 I think you are correct, I have it in python3
[18:32:19] <jessamyn_> sorry about that
[18:32:47] <jessamyn_> though... hm
[18:35:26] <jessamyn_> ah HA
[18:35:46] <jessamyn_> tos9: I aliased mkvirtualenv to 'mkvirtualenv --python=/usr/local/bin/python3'
[18:35:50] <jessamyn_> it all makes sense now
[18:37:39] <tos9> ah :)
[18:48:08] <snx> Got it
[18:48:16] <snx> Thanks a lot
[20:04:36] <tos9> dstufft: so I have (finally?) run up against PEP 440 I think
[20:04:54] <tos9> pip is refusing to install a tarball that it just downloaded via pip install --download
[20:05:28] <tos9> saying that no versions match, when there's a mylib-latest.tar.gz which it just downloaded from the URL -- it doesn't appear that passing --pre makes it happy either.
[20:09:42] <tos9> (trying to trim down to something reproducable)
[20:19:41] <ronny> dstufft: it seems the communication of pypa to the outside world needs to be improved
[20:20:07] <ronny> having a discussion with the sqlalchemy author atm, hes pointing out many issues that confuse the hell out of everyone
[20:21:44] <tos9> is that a new thing heh, packaging has been confusing since the beginning of time
[20:21:52] <tos9> less so now, but.
[20:23:41] <ronny> tos9: its a task that needs doing sooner rather than later
[20:23:53] <ronny> contributors wont come along if they are confused
[20:24:48] <dstufft> tos9: how are you trying to instal it, by directly referencing the tarball or by indirect reference?
[20:25:09] <dstufft> ronny: it does need improved yea
[20:26:01] <ronny> dstufft: btw, is there any reason why twine puts up more metadata than setup.py bdist_wheel upload
[20:26:29] <ronny> (the required distributiosn thing)
[20:26:58] <dstufft> setuptools doesn't send that information
[20:27:59] <ronny> i see, should it be added?
[20:28:45] <agronholm> yes, why doesn't it?
[20:28:55] <agronholm> it's not like this is a backwards compatibility problem?
[20:29:10] <ronny> agronholm: its a nobody did it problem
[20:29:24] <ronny> agronholm: but it provides good reasons to stop using biltin commands
[20:29:39] <agronholm> it sucks that I have to have a separate tool for upload stuff
[20:29:43] <ronny> setup.py upload/install/develop should be removed from developers hands ^^
[20:29:59] <tos9> dstufft: indirect reference
[20:30:04] <agronholm> but OTOH you can't just remove those things from setuptools
[20:30:09] <tos9> dstufft: by referencing the thing I put in #egg=
[20:30:29] <dstufft> tos9: you want mylib===latest then
[20:30:31] <ronny> agronholm: well, there is a reason why im working on a tool that removes setup.py from the developers hands
[20:30:32] <dstufft> or mylib==latest might work
[20:30:42] <tos9> dstufft: but there's no way of knowing that I need that, or is there?
[20:30:50] <agronholm> ronny: what tool?
[20:30:52] <tos9> dstufft: my URL didn't have that info in it, how hsould I know what version that tarball is
[20:30:55] <agronholm> do you have code yet?
[20:31:08] <tos9> dstufft: well it did in the URL itself, and in the resulting filename. I guess that's the answer.
[20:31:23] <ronny> agronholm: codename gumby elf, code hopefully at the end of the week, my recovery from the sinus op is stopping my work
[20:31:32] <tos9> (this is in a build script, which is increasingly accumulating hacks)
[20:31:48] <ronny> agronholm: i'll boast about it once it has develop, install and upload
[20:31:49] <dstufft> tos9: there's no way to know ahead of time when --download is going to give you no.. though i'm surprised it gave you a mylib-latest.tar.gz
[20:31:50] <agronholm> ronny: ok I'm looking forward to it
[20:31:57] <dstufft> I would have guessed it would skip that
[20:32:02] <tos9> dstufft: I pointed it at a http://example.com/foo-latest.tar.gz
[20:32:05] <dstufft> oh
[20:32:09] <dstufft> makes sense then
[20:32:16] <tos9> (that URL is in my requirements file. And now I want to install the resulting tarball)
[20:32:31] <dstufft> agronholm: I can remove those things from setuptools, in a way
[20:32:32] <tos9> and I was using #egg=<whatever> and finding the whatever.
[20:32:44] <dstufft> DONE