Log file Viewer

[07:07:54] <psi29a> cheers! thanks tomprince, apollo13 and dstufft!

[07:08:04] <psi29a> from Richard: "Thanks, Tommi. I've granted bret Owner access to the Ldaptor project on PyPI."

[07:08:33] <psi29a> Tommi, TV42, is the original author of Ldaptor

[08:23:47] <dw> hey. is it normal for 'pip wheel' to repeatedly rebuild a binary wheel it's already placed in the output directory? Version number does not change across runs.

[08:24:19] <dw> It's a binary wheel of SciPy, which hasn't changed upstream version, nor have any sources files, or the 'pip wheel' command line changed across runs

[08:25:32] <apollo13> dw: I think so yes, the purpose of pip wheel is to build wheels

[08:26:35] <mgedmin> AFAIU if you want to avoid rebuilding the same wheels, make sure pip wheel can find them (--find-links)

[08:26:58] <dw> mgedmin: this looks useful

[08:29:01] <dw> mgedmin: fabulous :) ok, so if I just '-f wheelhouse', then prior to pip wheel on requirements.txt, check requirements.txt timestamp is older than wheelhouse dir, if not, wipe out wheelhouse dir. this means builds are slow only when requirements.txt changes

[08:29:46] <mgedmin> I don't think you need to wipe the wheelhouse; AFAIU if pip finds a newer upstream version, it'll ignore a preexisting wheel

[08:30:32] <mgedmin> <voice class="dreamy">someday we'll have a persistent automatic wheel cache</voice>

[08:30:34] <dw> it's for jenkins' benefit too, it is archiving the artifacts

[08:43:00] <apollo13> mgedmin: don't we have that already with the download cache?

[08:43:34] <mgedmin> for source packages, yes

[08:43:42] <mgedmin> and for wheels when there are wheels on pypi

[08:43:44] <mgedmin> otherwise no

[08:44:09] <mgedmin> my wish is that pip install lxml in several different virtualenvs only builds lxml's C bits once on a particular machine

[08:44:17] <mgedmin> (linux so no lxml wheel on pypi)

[08:44:37] <doismellburning> ah lxml

[08:44:50] <doismellburning> I think I could heat my house on lxml alone

[08:45:19] <apollo13> mgedmin: ah okay, we have a cache with prebuilt wheels, after all I don't want to build on the machines

[08:45:36] <mgedmin> one day I'll figure sane app deployment

[08:46:06] <apollo13> so we have a few different things, at one the buildmachine just builds deps which are then installed

[08:46:08] <mgedmin> I'm still building everything on prod servers during pip install... often as root *whimper*

[08:46:08] <doismellburning> mgedmin: OS packages/VMs/containers; it's the only way

[08:46:13] <doismellburning> mgedmin: :((((((((

[08:46:38] <apollo13> https://github.com/jordansissel/fpm ;)

[08:47:25] <mgedmin> as a frugal person I like to sometimes pack several instances of the same app on the same server -- e.g. staging and prod

[08:47:39] <apollo13> so for djangoproject.com we have a deploy script which does a git checkout

[08:47:43] <mgedmin> this means different versions this means I can't use distro packages

[08:47:52] <apollo13> but the deploy user is not running as root

[08:47:55] <mgedmin> and my servers are too underpowered to allow me multiple VMs

[08:48:00] <mgedmin> docker seems promising

[08:48:12] <apollo13> mgedmin: so create my_package-dev my_package-prod

[08:48:35] <apollo13> distro packages can be used even if you have multiple deployments in many cases

[08:48:49] <apollo13> the end goal should be no code execution on the target for a deploy

[08:49:05] <apollo13> in the end this also means no call to pip install

[08:49:20] <doismellburning> mgedmin: http://12factor.net/build-release-run

[08:49:22] <mgedmin> there's sense in what you're saying

[08:49:34] <doismellburning> mgedmin: you want the "run an instance" stage to be as fast and as simple as possible

[08:50:01] <doismellburning> mgedmin: i.e. doing all the compilation / fetching / other work at build time only, once

[08:50:27] <apollo13> database migrations are a bit hard to do at build time though ;)

[08:52:33] <doismellburning> database migrations are evil

[08:52:34] <doismellburning> but yes

[18:22:13] <Yasumoto> semi-random question: has anyone worked on a 'wheel-building service' that people can deploy in their environments to build dists given a requirement?

[18:23:41] <dstufft> Yasumoto: you might want to look at wheel-builder I thin kit's called

[18:30:42] <Yasumoto> dstufft: orly? rad, will scope it out

[18:30:53] <Yasumoto> we're looking to put some time into a service like that

[18:31:04] <Yasumoto> and my ~evil scheme is to make it generic enough that we can open source it

[18:31:35] <Yasumoto> and then we'll be able to configure it to hit (potentially) jenkins slaves running on the various architectures we need

[18:32:16] <Yasumoto> once we get a design doc written up I was considering sending to distutils-sig if that seems appropriate (but hopefully we can just use/improve wheel-builder :)

[18:59:19] <xafer> hello, any blocker for https://github.com/pypa/pip/pull/2034 ?

[18:59:54] <dstufft> xafer: just haven't had time to review it yet

[19:00:19] <xafer> ok :)