PMXBOT Log file Viewer

Help | Karma | Search:

#pypa logs for Monday the 31st of August, 2015

(Back to #pypa overview) (Back to channel listing) (Animate logs)
[13:31:32] <jokus> Hi, how is it possible to access to anything like apache outside of a virtualenv ? The reason I am asking is I have Deb 6 which only has Py 2.6. I need 2.7 and Django 1.8 ?
[13:32:04] <jokus> Hi, how is it possible to access to anything like apache outside of a virtualenv ? The reason I am asking is I have Deb 6 which only has Py 2.6. I need 2.7 and Django 1.8
[13:33:28] <mgedmin> there seems to be an echo
[13:33:53] <mgedmin> virtualenv will not help you install python 2.7; you need to get that from somewhere first anyway
[13:34:20] <mgedmin> (upgrading your distro, if possible, might be best solution)
[13:34:27] <jokus> oh
[13:35:09] <jokus> Thanks for the reply, but that is not allowed at the moment ! Its a production box.
[13:36:01] <jokus> So if i need to keep the current server as it is, how can I get 2.7 and django 1.8 installed on it so that it can run a django app ?
[13:36:22] <mgedmin> perhaps there's a python2.7 package in backports?
[13:36:44] <mgedmin> if all else fails, you can always build cpython from source
[13:37:24] <jokus> so compile py 2.7 and put it somewhere and let django know of it somehow ?
[13:38:37] <mgedmin> compile and install py2.7, create a virtualenv using -p python2.7, pip install django into that virtualenv
[13:45:35] <jokus> ok, will try that, also I don't think there will be py2.7 on backport
[17:02:04] <rknLA> !logs
[17:02:04] <pmxbot> chat-logs.dcpython.org/channel/pypa
[17:02:45] <rknLA> hi, i asked this last week and got no response.. is this a known issue, or something i'm doing incorrectly with my setup.py? chat-logs.dcpython.org/day/pypa/2015-08-26#15.42.23.rknLA
[17:23:40] <dstufft> rknLA: there shouldn't be any difference between those two installation options
[17:23:51] <dstufft> are you sure the git commit and the git tag have the same files?
[18:22:24] <rknLA> dstufft: yep, i'm sure the tag and the commits are the same
[18:36:12] <agronholm> has anyone noticed anything weird about the search in pypi as of late?
[18:36:31] <agronholm> like, why the heck does searching for "crossbar" give me this page? pypi.python.org/pypi/crossbarconnect/0.5.3
[18:36:42] <agronholm> and not a list of all projects with "crossbar" in them?
[18:37:06] <agronholm> especially when there's a project that matches the name exactly and has even downloadable artifacts?
[19:11:42] <sontek> Hey, did pip change the way its doing caching of wheels? My 2.6 tox seems to be getting wheels from 2.7 with the latest versions
[19:12:06] <sontek> and what are there cli args to tell it "no matter what don't use the cache"
[19:13:42] <tdsmith> --no-binary :all: will never use wheels, including the cache
[19:13:54] <tdsmith> i don't think there's a way to specify "allow pypi wheels but don't trust the cache"
[19:17:25] <sontek> It doesn't seem to care: paste.ofcode.org/tzjqWiE9npqfMWJdsVS9rC
[19:17:52] <sontek> Ahh there we go. Its --upgrade --no-binary :all: --force-reinstall
[19:18:41] <sontek> Ok, so I can confirm that pip is mixing wheels between environments, so if I install something from py27 it gets cached and then py26 gets it
[19:18:47] <sontek> even though its not marked universal
[19:28:44] <Ivo> sontek: what is it marked
[19:30:51] <sontek> Ivo: it has nothing declared
[19:33:14] <Ivo> hmm
[19:33:32] <Ivo> I search 'snowballstemmer' on pypi
[19:33:35] <Ivo> and it returns nothing
[19:33:40] <Ivo> even though that's a package
[19:33:53] <Ivo> pypi.python.org/pypi?%3Aaction=search&term=snowballstemmer&submit=search
[19:34:06] <Ivo> pypi.python.org/pypi/snowballstemmer
[20:47:15] <dstufft> sontek: by default wheel builds wheels as "py2 and py3" unless they have a compiled extension
[20:47:49] <dstufft> sontek: github.com/pypa/pip/issues/3025
[21:31:55] <sontek> dstufft: ok, so is there a way to solve this? I don't remember having this issue with earlier versions of pip
[21:32:33] <sontek> We have a bunch of builds in our CI system broken right now and its because if packages that says `if sys.version_info < (2,7): ..`
[21:32:49] <sontek> as far as I can tell this should be fine since none of the ones we are having issues with are marked as universal
[21:33:04] <sontek> and this isn't just py2 vs py3 its an issue within py2.6 vs py2.7
[21:33:58] <dstufft> sontek: the problem is universal means "py2 AND py3", but wheel assumes by default that if it's a pure python package it will work the same on all of either 2.x or 3.x depending on what the current version of Python is
[21:34:02] <dstufft> you have some options
[21:34:05] <dstufft> sec
[21:34:18] <dstufft> bitbucket.org/zzzeek/mako/issues/249/mako-wheels-have-computed-install_requires
[21:34:21] <dstufft> see my comment sthere
[21:34:34] <dstufft> you're basically running into the same problem
[21:37:47] <sontek> Yeah, my only problem is that this shouldn't be an assumption of pip unless there is a way to change that assumption. If you can't guarantee you know its compatible then don't make it compatible across versions
[21:38:09] <sontek> universal=1 is the flag to say "i'm compatible" so trying to make me universal even though I've explicitly said I'm not is extremely odd behavior
[21:39:12] <dstufft> it's not making it universal
[21:39:22] <dstufft> unverisal is a single wheel for 2.x _and_ 3.x
[21:39:44] <dstufft> wheel's defaults just don't work well for implicit caching given the history of things
[21:40:02] <dstufft> #3025 is about having pip override those defaults to something that would
[21:40:24] <sontek> I think it should just do safe caching, assuming things would be compatible across versions is something we all know wont work given the history
[21:40:31] <sontek> it should cache per runtime
[21:41:14] <sontek> My options right now (after reading your comment): 1. disable wheel cache 2. downgrade pip right?
[21:41:23] <sontek> What are these environment markers you are talking about?
[21:42:41] <dstufft> sontek: well, to be specific, are these packages you control or are they ones you're pulling in
[21:44:02] <sontek> currently they are ones I control but I assume once I get past those I'll discover ones I don't
[21:44:12] <sontek> This is not an uncommon practice
[21:46:47] <sontek> according to this: github.com/search?utf8=%E2%9C%93&q=version_info+in%3Afile+filename%3Asetup.py&type=Code&ref=searchresults 914,614 setup.py's are checking version_info
[21:47:14] <dstufft> for ones you control you can use environment markers, which are basically a static way to declare that you need certain dependencies on different versions of Python in a way that won't have this problem
[21:47:40] <sontek> dstufft: where are the docs for environment markers?
[21:48:21] <dstufft> for a more global solution it's disable wheel cache, downgrade, or submit a PR for #3025 (or wait for someone else to)
[21:49:01] <sontek> I think the first PR should be to just disable this assumption, it is using very poor judgement with the knowledge of what existing packages currently do
[21:49:16] <sontek> If people want better cache there can be an opt-in (probably using environment markers or something)
[21:49:46] <sontek> but by default pip shouldn't make this types of assumptions knowing they will break hundreds of thousands of projects
[21:50:31] <dstufft> well pip just relied on wheels defaults, but in hindsight wheel's defaults are tailored more towards a sitaution where an author is generating a wheel.
[21:51:25] <sontek> What is the newest version available that doesn't do this?
[21:51:57] <dstufft> I'm not sure we really have docs on environment markers right now, the syntax is here python.org/dev/peps/pep-0426/#environment-markers adn you just pass them to extras_require={":<env marker": [,,,]}
[21:52:05] <dstufft> <7
[21:52:13] <sontek> The odd thing is we have been generating wheels in our CI system for over a year now without issues, we just ran `pip wheel` on anything needing built
[21:52:20] <sontek> and *never* had this issue
[21:52:32] <sontek> It seems to be a new issue with the auto-cache pip stuff
[21:53:01] <dstufft> pip wheel and the wheel cache output the same wheel
[21:53:18] <dstufft> were you generate them using python 2.6 by any chance
[21:53:23] <dstufft> generating*
[21:53:36] <sontek> We generate them per env, like pip2.7 wheel, pip2.6 wheel
[21:53:41] <sontek> we don't use a global pip
[21:54:07] <dstufft> was the directory you passed to pip2.7 wheel per env too?
[21:55:00] <dstufft> If they were, then you were essentially working around this issue via that
[21:55:20] <dstufft> since the wheel cache would have been per env in that case, but pip's auto cache is global
[21:55:34] <sontek> nope, we had a global PIP_WHEEL_DIR=/tmp/wheelhouse and PIP_USE_WHEEL=True
[21:56:53] <dstufft> strange, were you building pip 2.6 wheels first?
[21:57:31] <sontek> Depends on the order they declare their tox, but I'm guessing mostly yes
[21:57:43] <sontek> since most people list things in order like py26, py27, py34, pypy
[21:58:36] <dstufft> yea, so you would have a created a wheel that works for all of Python 2 using Python 2.6, so you would have gotten the < 2.7 dependency installed, and it just would have _also_ installed it on 2.7 when you dind't need it
[21:58:37] <sontek> I think the issue though is that not everyone supports py26, so sometimes you'll get a project who runs on the same build agent that only supports 2.7 or only 3.4
[21:58:50] <sontek> so the py26 wheel isn't guaranteed to be first
[22:00:15] <dstufft> sontek: to be clear, I'm not suggesting the current situation is correct, just trying to explain why it's probably wasn't a problem for you until now
[22:01:08] <sontek> dstufft: Yeah, I understand. I just can't even imagine how it got in this situation. We went to great lengths in our CI system to never have a wheel from a different environment be used
[22:01:26] <dstufft> the real solution is fixing #3025 so that pip overrides wheel's defaults
[22:01:37] <sontek> and it seems odd for the developers of pip/wheel to be like "This will probably work" when it will obviously not work
[22:01:50] <sontek> I hate using the word obviously in software, but in this specific case it seems quite obvious
[22:01:50] <dstufft> I think we just need to pass a flag to python setup.py bdist_wheel
[22:02:36] <dstufft> sontek: so in wheels' case, the defualts of "py2" or "py3" assume that you have someone at the helm building the wheel who'll test it out and such. So that makes sense to me
[22:02:57] <dstufft> it just didn't occur to me when I merged in the auto wheeling support in pip how that default would interact with this case
[22:03:09] <sontek> dstufft: yeah that makes sense because there is no cache, so you are getting a fresh wheel
[22:04:01] <dstufft> I'm kinda stretched super thin so I'm kind of always playing catch up on PR reviews and fixes and stuff
[22:35:38] <sontek> dstufft: Are there projects anywhere I can look at that use these enviornment markers?
[22:35:52] <sontek> The PEP isn't very clear on how to use them, would love to see some examples
[22:54:56] <Ivo> dstufft: you have any intuition why pypi.python.org/pypi?%3Aaction=search&term=snowballstemmer&submit=search would return 0 results?
[23:07:36] <dstufft> sontek: um, twine uses an older form of them that works with pip < 6 (but requires duplicating)
[23:07:46] <dstufft> Ivo: yea, our search is fucked and i haven't been able to find time to fix it
[23:08:13] <dstufft> github.com/pypa/twine/blob/master/setup.cfg#L13
[23:09:44] <dstufft> trying to think of something that uses the newer form
[23:11:38] <dstufft> github.com/jaraco/setuptools/blob/master/setup.py#L153
[23:12:14] <dstufft> but you can omit the group name, so it's just :sys_platform=='win32' if you want it to always install them (assuming the enviornment markers match) and not just when the extra is selected
[23:14:59] <transit> When installing a dependency in a svn repository using a pip requirements file, how do you deal with authentication?
[23:19:38] <ronny> transit: thats svn specific, i doubt anyone here knows in detail
[23:19:41] <ronny> also gn8
[23:21:42] <transit> ronny: It definitely wouldn't be within the svn+http://... format then?
[23:52:37] <sontek> dstufft: so with twine, it works even though it does github.com/pypa/twine/blob/master/setup.py#L28-L32 ?
[23:52:57] <sontek> because it declares the metadata as well
[23:52:59] <dstufft> sontek: yea, because the setup.cfg completely overrides install_requires in a wheel
[23:53:30] <dstufft> you have to do that if you want to support "py2" wheels on pip < 6
[23:53:59] <dstufft> if you're OK requiring pip >= 6, you can reduce the duplication and just specify it in extras_requires