Log file Viewer

[01:38:09] <dstufft> _habnabit: https://github.com/pypa/pip/pull/2122

[01:38:24] <dstufft> _habnabit: that will fix your problem with the ve/build dir

[01:38:29] <_habnabit> dstufft, oh neat

[01:39:12] <_habnabit> dstufft, thanks

[01:39:26] <tdsmith> ^ exciting!

[01:39:28] <dstufft> it'll be in pip 6.0, if you want it nowish you'll need to use the dev version (I'll merge once the tests pass)

[01:39:34] <_habnabit> 6.0???

[01:39:48] <dstufft> _habnabit: we dropped the leading 1

[01:39:52] <dstufft> because it was meaningless

[01:39:54] <_habnabit> ha

[01:40:02] <dstufft> I wanted to just go full out twisted and do YY.N

[01:40:06] <dstufft> but other folks didn't like that

[01:40:09] <dstufft> so this was a compromise

[01:40:10] <dstufft> :D

[01:40:14] <_habnabit> dang

[01:40:23] <_habnabit> i like year version numbers a lot better

[01:40:30] <_habnabit> how cna you not

[01:40:39] <dstufft> I like them more depending on what your project does

[01:40:59] <dstufft> like, X.Y makes a lot of sense for Django and Python and such, because they actually release X.Y.Z versions

[01:41:09] <dstufft> and maintain multiple releases at once

[01:41:14] <dstufft> pip doens't really do that

[01:41:18] <dstufft> onwards and upwards

[01:41:43] <_habnabit> fwiw the version scheme i use takes care of that; it's major.year.minor.patch

[01:41:48] <_habnabit> e.g. 0.14.0.0

[01:43:20] <dstufft> theortically we're going to follow semver-ish and cut more releases more often too so stuff doesn't linger on the develop branch forever

[01:43:25] <dstufft> not sure how well it's going to work out

[01:44:15] <dstufft> we shall see!

[01:44:28] <dstufft> Worst case we have a number of versions until we can't switch to date based :D

[01:44:38] <_habnabit> heh

[01:44:47] <_habnabit> unless you went full on 2014.1

[01:45:07] <dstufft> yes that is true

[01:45:16] <dstufft> we have a lot of versions to go before we can't do YYYY

[09:31:29] <doismellburning> yay semver <3

[20:06:33] <_habnabit> so i'm building a deb containing a pex file right now, and atm i just build-depend on python-virtualenv, make a virtualenv, and `ve/bin/pip install -U pip setuptools pex wheel`, then use that virtualenv to build wheels and the pex file. it doesn't seem ideal, but i don't think there's a deb for pex at the moment anyway. i guess i could use stdeb to make a pex deb, but would it really be worth it?

[20:08:38] <dstufft> I think "would it be worth it" is a very personal question on how janky you're willing to have your build process :D

[20:08:52] <_habnabit> haha

[20:09:04] <_habnabit> does this seem very janky right now?

[20:10:29] <dstufft> it's not the worst i've seen :D the biggest thing i'd say is you're depending on the security of PyPI (but you're probably doing that anyways with pex I guess?) so it may not matter

[20:10:34] <_habnabit> looks like there's not a python-wheel deb either

[20:10:50] <_habnabit> pex is pulling from our local pypi

[20:12:39] <_habnabit> would you think that ideal would be a build-depends on python-pex python-pip python-wheel? i guess i'd need to make a new python-pip deb too.......

[20:30:13] <nanonyme> FWIW what's a pex file?

[20:32:16] <dstufft> self contained executable (other than Python itself IIRC) that can hold a project + all dependencies

[20:32:18] <dstufft> or so

[20:32:56] <dstufft> came from twitter

[20:33:24] <dstufft> http://pex.readthedocs.org/en/latest/whatispex.html#whatispex

[20:33:49] <doismellburning> omg yes?

[20:34:12] <doismellburning> do they handle the whole relocatable virtualenv faff too?

[20:34:25] <dstufft> I haven't used them personally yet

[20:34:34] <dstufft> I think they are completely relocatable though yes

[20:35:05] <dstufft> I think the idea is you can treat them similarly as you would a go binary (or any other statically compiled binary), other than they require python installed

[20:36:05] <dstufft> not sure if they work on windows or not

[20:40:37] <doismellburning> you've possibly made my month

[20:40:38] <doismellburning> cheers

[22:08:37] <Yasumoto> _habnabit: cool, how's pex been working out for you lately?

[22:08:57] <Yasumoto> no one's setup any packages for it yet, but if you're interested in putting up a .deb, that'd be rad :)

[22:09:12] <_habnabit> Yasumoto, pretty well! i convinced our devops team to try using pex instead of dh-virtualenv for internal packages

[22:09:26] <_habnabit> Yasumoto, so unfortunately i won't be releasing it, but i have been enjoying the process

[22:09:44] <Yasumoto> doismellburning: there are still some places where things could be better, but for the most part the point is to indeed have a self-contained executable with all dependencies :)

[22:10:31] <Yasumoto> _habnabit: awesome! yeah, I've been using it for the last ~2 years, but only recently has it started getting more people to take a look at it which is cool :)

[22:10:33] <_habnabit> Yasumoto, though, i might be able to get a template git repo released that shows how to integrate pex with debian

[22:10:51] <Yasumoto> _habnabit: ideally you should be able to just pip install pex tho, right?

[22:10:55] <doismellburning> hah yeah dh-virtualenv would be useful, but fpm

[22:11:09] <Yasumoto> (or are you looking for staying within the dpkg-packaging ecosystem?)

[22:11:10] <_habnabit> Yasumoto, not sure what you mean. as part of the build process?

[22:11:30] <_habnabit> doismellburning, i hate dh-virtualenv :(

[22:11:43] <Yasumoto> yeah, like when you're saying 'make pex available on debian'.. is pip install not giving you a working pex?

[22:12:14] <_habnabit> Yasumoto, oh. no, pip install gives me a working pex, but i'm not about to pip install globally

[22:12:59] <doismellburning> _habnabit: really? how come?

[22:13:08] <doismellburning> right now I've got virtualenvs building inside chroots

[22:13:12] <doismellburning> and it makes me sad

[22:13:51] <_habnabit> doismellburning, the latest issue i've had with it is that it doesn't support extras at all, so basically half of the packages i want to package with it i can't

[22:14:21] <_habnabit> doismellburning, but prior to that, i was having issues with it interacting poorly with pip's build directory stuff

[22:14:35] <_habnabit> doismellburning, (which apparently is going to be fixed soon, but hasn't been released yet)

[22:16:21] <doismellburning> :(

[22:16:36] <doismellburning> I'm glad I didn't spend time trying to move to Proper Debian Packaging then

[22:16:36] <Yasumoto> _habnabit: ah, yeah, totally with ya :)

[22:18:22] <_habnabit> Yasumoto, so basically i'm just trying to figure out the least painful way to build a .pex file on a debian system. at the moment it seems to be complicated by not having debs for wheel, pex, or a recent pip, so using virtualenv in this way makes it more portable

[22:18:48] <_habnabit> Yasumoto, buuut it's certainly possible to make those debs and distribute them and requires them for building the .pex deb

[22:19:01] <Yasumoto> hm, interesting

[22:20:34] <Yasumoto> _habnabit: yeah, those would probably be the 'base-packages' you'd need

[22:20:40] <Yasumoto> then the rest can come from an internal devpi or something

[22:20:45] <Yasumoto> (if I'm following right :)

[22:21:12] <Yasumoto> (I mean you can also bootstrap those without debian packages too... but better if you can apt-get install and or use chef/puppet/etc to install as usual)

[22:21:13] <_habnabit> Yasumoto, yeah, that's right. if you're building a .pex file it's probably of a package that isn't already on pypi

[22:21:32] <Yasumoto> typically yeah

[22:21:46] <Yasumoto> I mean, you can also use pex to create an executable with dependencies that are already on pypi

[22:21:58] <Yasumoto> then use that to execute a separate script which has those dependencies setup

[22:22:04] <Yasumoto> (and then don't need to worry about virtualenvs)

[22:22:05] <_habnabit> true!

[22:22:28] <Yasumoto> for us, we've found it suuuuper useful when paired with http://aurora.incubator.apache.org

[22:22:32] <Yasumoto> (which runs on top of Mesos)

[22:22:43] <_habnabit> ah

[22:22:45] <Yasumoto> so different people can deploy different .pex files without worrying about conflicting versions

[22:23:01] <Yasumoto> (and we have an internal package store they download from at the start of their job)

[22:23:01] <_habnabit> yeah this is just going to be managed with upstart, haha

[22:23:40] <Yasumoto> gotcha- it might be a little overkill if you can use all the system-level stuff, but it's been good to distribute command-line tools etc as well

[22:23:55] <Yasumoto> (you can have one pex which is executable on OS X and linux, for instance too)

[22:23:56] <_habnabit> you're making pex files for cli tools, you mean?

[22:24:01] <_habnabit> oh whaaaat

[22:24:03] <Yasumoto> yeah, exactly

[22:24:08] <Yasumoto> heh

[22:24:13] <Yasumoto> so the Aurora client

[22:24:18] <Yasumoto> you add it to your PATH

[22:24:22] <Yasumoto> aurora job create <blahblah>

[22:24:35] <dstufft> yea

[22:24:37] <Yasumoto> both OS X and linux eggs/sdists/wheels are there

[22:24:45] <_habnabit> so re: universal pex files, does that require a pure-python package?

[22:24:45] <dstufft> pex has some really great potentional

[22:24:47] <Yasumoto> so it uses the right set depending on platform

[22:25:01] <Yasumoto> dstufft: if it was more than two peeps working on it ;)

[22:25:07] <Yasumoto> "working"

[22:25:22] <dstufft> Yasumoto: I would, but I'm already over subscribed by like a factor of 3

[22:25:26] <Yasumoto> _habnabit: nope, we use python-ldap which has native code

[22:25:31] <_habnabit> hm, i see

[22:25:37] <Yasumoto> dstufft: heh I know! keep fighting the good fight and stay focused :)

[22:25:50] <dstufft> I haven't evenr eally got a chance to play with pex yet :/

[22:25:59] <_habnabit> so how do you build a universal pex file with native code in it?

[22:26:13] <dstufft> I think it just includes all the wheels

[22:26:17] <Yasumoto> heh, yeah, too much good stuff

[22:26:17] <dstufft> (or eggs)

[22:26:22] <_habnabit> no eggs plz

[22:26:25] <dstufft> for the platforms you want

[22:26:38] <Yasumoto> yeah, there's a way to specify platforms

[22:26:39] <dstufft> and selects the right one based on what platform it's being run on

[22:26:42] <Yasumoto> yep

[22:26:44] <_habnabit> i see!

[22:26:54] <_habnabit> so how well does that work across linux distros?

[22:27:18] <_habnabit> (also, does the wheel name encode UCS-2 vs. UCS-4 python..?)

[22:27:37] <dstufft> Wheel doesn't handle ucs-2 vs UCS-4

[22:27:41] <_habnabit> dang

[22:28:20] <dstufft> wheel handles ABI differences on 3.x versions which have the stable ABI stuff

[22:28:25] <dstufft> I forget what 3.x version introduced it

[22:28:33] <dstufft> ABI differences with Python itself*

[22:29:08] <_habnabit> so it's not feasible to distribute a 2.x linux binary wheel, because of cross-distro differences in python build?