Log file Viewer

[01:29:01] <lifeless> ok wheel cache fixed up with 988

[03:00:59] <lifeless> dstufft: welp, weare over-caching a little. :( http://logs.openstack.org/25/181525/1/check/check-pbr-installation-dsvm-upstream/587b1a0/console.html.gz#_2015-05-08_20_34_05_962

[19:39:23] <lifeless> \o/ successfully installed all of openstacks requirements. glitches reducing, time after time

[22:17:44] <dstufft> lifeless: over caching?

[22:35:53] <lifeless> dstufft: an autobuilt wheel is being built for nova

[22:36:03] <lifeless> dstufft: which is an install from a local git tree

[22:36:10] <dstufft> lifeless: ahh

[22:37:25] <lifeless> in other news, I can using the resolver branch install everything and it works. muhahahaha

[22:37:31] <lifeless> everything openstack that is

[22:38:32] <dstufft> lifeless: nice!

[22:38:53] <lifeless> time PYTHONPATH=. pip install -r ../requirements/global-requirements.txt

[22:40:38] <lifeless> a big chunk of the install time is going to be the crazy that is PyECLib

[22:41:02] <dstufft> what's that

[22:41:08] <dstufft> not familar with pyeclib

[22:41:26] <lifeless> https://twitter.com/rbtcollins/status/597517090992455680

[22:41:41] <lifeless> and https://twitter.com/rbtcollins/status/597523841494028289

[22:41:48] <lifeless> real 2m48.407s

[22:41:48] <lifeless> user 1m44.952s

[22:41:48] <lifeless> sys 0m26.829s

[22:42:12] <lifeless> http://paste.openstack.org/show/219300/

[22:43:38] <dstufft> lifeless: is that 3 minutes include installation? and is that with an empty HTTP cache?

[22:43:50] <lifeless> dstufft: hot cache, including autobuilt wheel cache

[22:44:10] <lifeless> dstufft: so we're seeing here is a) time to build pyeclib and its three supporting autoconf based C libraries

[22:44:24] <lifeless> b) wheel unpack and copy into place for ~280 items

[22:44:30] <dstufft> yea

[22:44:48] <dstufft> do you know offhand the time it takes for the same install set without the resolver branch?

[22:44:50] <lifeless> c) resolver iteration over the hot cache, including unpacking and introspection of failed versions

[22:45:02] <lifeless> easy enough to figure that out

[22:45:11] <lifeless> let me switch to develop, to have the same wheel behaviours

[22:45:33] <lifeless> the resolver is ~50s CPU

[22:46:50] <dstufft> that's not bad

[22:47:14] <dstufft> not for ~280 items anyways

[22:47:36] <lifeless> right, test setup for develop, running now

[22:48:10] <dstufft> I imagine for simple cases the resolver is essentially a wash for time as well

[22:48:26] <lifeless> yeah

[22:48:36] <lifeless> if there are no conflicts that require pulling down unused version

[22:48:39] <lifeless> then its trivial

[22:48:47] <dstufft> if all the requirements are open ended on the top then the latest version will always work and it should be 1 download per

[22:48:47] <lifeless> download one version of each package, install.

[22:48:57] <dstufft> yea

[22:49:12] <lifeless> so, this comparison is going to be a little wrong

[22:49:25] <lifeless> because the resolved version picked different versions of a few packages

[22:49:29] <lifeless> so we had wheel cache misses

[22:49:46] <lifeless> for docutils, chardet, pulp and restructuredtext-lint

[22:49:50] <lifeless> real 1m53.421s

[22:49:51] <lifeless> user 1m24.247s

[22:49:51] <lifeless> sys 0m21.092s

[22:50:12] <dstufft> so roughly a minute give or take

[22:50:33] <lifeless> 100% slower :)

[22:53:36] <dstufft> lifeless: this was with python 2 I'm guessing?

[22:54:13] <dstufft> (I ask because a cache miss on a built wheel for docutils on python3 is gonna really mess with the time, because 2to3 is slowwww)

[22:54:46] <dstufft> -> goes to watch some tv

[22:54:55] <lifeless> 2 yeah

[22:55:06] <lifeless> cached wheels built, testing now

[22:58:48] <lifeless> heh, those three things are bad versions or something, fail to build wheels:).... so real 1m57.541s

[22:58:51] <lifeless> user 1m25.864s

[22:58:54] <lifeless> sys 0m21.517s

[23:23:50] <lifeless> dstufft: hmm, ~/.cache/pip/log/debug.log doesn't actually have debug output.. is that expected?

[23:24:41] <lifeless> Resolved 293 versions requirements in 4108 steps.

[23:31:27] <lifeless> I probably need to run this under a profiler soon