Log file Viewer
#pypa logs for Thursday the 22nd of January, 2015
[00:24:19] <Moult> Yasumoto: possibly http://stackoverflow.com/questions/27203967/python-3-4-pip-install-wheel-fails-on-yosemite-not-a-supported-wheel-on-this
[00:25:25] <Yasumoto> holy moly: Most of the current tests do string comparisons of these values which will not work properly with a two-digit version number ('10.10' < '10.9' --> True).
[00:34:38] <Moult> Yasumoto: i suspect your PEX_VERBOSE=1 == -v, whereas i am using -vvvv or whatever
[00:35:28] <Yasumoto> so the env variable is pretty much our only hope for bumping verbosity with that
[00:35:59] <Yasumoto> yeah, we're feeling pretty alpha-tester-y for http://aurora.incubator.apache.org
[00:36:22] <Yasumoto> (it's been in use at a few places for a while, but it's finally started getting some traction.. which means lots of changes)
[01:05:19] <Moult> Yasumoto: neither on python 3.3, hmm, you're on osx? perhaps this is a linux issue?
[01:23:59] <Yasumoto> I have a feeling you're unearthing a deeper issue than has been discovered before.. :)
[02:09:59] <Yasumoto> might be interesting if you can get a gentoo vagrant image or something to make repro-ing easier
[02:13:41] <Moult> Yasumoto: i would be willing to help debug the pex code if you could help walk me through setting it up
[02:15:39] <Moult> Yasumoto: keep in mind i only started seriously using python a while back, so i'm still rather new to the language :)
[02:24:07] <Yasumoto> Moult: I gotta step out for the evening, but if you file a gh issue, we can tackle it together there
[03:37:27] <Yasumoto> "# TODO(wickman) This implementation is technically incorrect but we need to be able to"
[03:37:27] <Moult> Yasumoto: so abis.extend(['cp%sm' % version, 'abi3']) would form abis such as "cp33m" and "cp34m"
[03:37:31] <Moult> Yasumoto: now, according to pep425, and according to what my logs are spitting out, the actual values should be cp33 / cp34 ie. no m suffix
[03:37:32] <Moult> Yasumoto: so removing the m suffix fixes it, but ...as i say i don't know the implications, and why the m was there to begin with
[03:41:35] <Moult> Yasumoto: as an example the PEP does mention "cp33m", with the m suffix, but let me do a bit more reading to see exactly when the m comes into play
[03:42:31] <Moult> Yasumoto: if you import sysconfig and do sysconfig.get_config_var('SOABI') what do you get?
[03:52:47] <Moult> Yasumoto: "Python implementations MAY include additional flags in the file name tag as appropriate. For example, on POSIX systems these flags will also contribute to the file name:"
[03:53:27] <Moult> Yasumoto: --with-pymalloc (flag: m ), <-- so that's what the m is for, and also "By default in Python 3.2, configure enables --with-pymalloc so shared library file names would appear as foo.cpython-32m.so . "
[03:54:07] <Moult> Yasumoto: i suspect that the m there is incorrect as it assumes that it will always be compiled --with-pymalloc
[03:57:35] <Moult> Yasumoto: none is valid too, which explains why yours works and mine doesn't, i suspect the way gentoo manages multiple pythons had a part to play in this
[03:59:17] <_habnabit> right, but my point is just that it's not applicable to future versions of python 3
[04:01:59] <Moult> so (excuse my beginner python) - do you folks think this is sufficient as a fix? abis.extend(['cp%s' % version, 'cp%sd' % version, 'cp%sm' % version, 'cp%su' % version, 'abi3'])
[04:06:01] <Moult> Yasumoto _habnabit any ideas if there is an alternative to adding all the permutations of dmu?
[04:19:37] <Yasumoto> (that pastebin looks like it'll work on most/all interpreters if i'm reading right)
[10:18:44] <underyx> sometimes these new builds of our own software have new dependencies listed in setup.py
[10:19:35] <underyx> but if I do --upgrade with no --no-deps, libraries already at the latest version will also be reinstalled
[10:19:54] <underyx> having lxml in there for instance adds an extra 10 or so minutes due to compilation
[10:20:31] <underyx> is there a way to not reinstall packages that were already at the latest version?
[10:23:35] <underyx> pip install --index-url internal.devpi.index --upgrade --no-deps internal-package-1 internal-package-2 internal-package-3 internal-package-4
[10:23:54] <underyx> this won't install any new dependencies, breaking our services, so that's no good
[10:24:02] <underyx> pip install --index-url internal.devpi.index --upgrade internal-package-1 internal-package-2 internal-package-3 internal-package-4
[10:24:37] <underyx> this will reinstall all dependencies, which makes deployment take around 100-1000x the time it should, which also isn't very good
[12:28:36] <The-Compiler> I have a virtualenv on Windows 7. Now when I start the virtualenv's pip and try to install something, it tells me it's already installed in my C:\Python34\Lib\site-packages. Why does that happen?
[12:29:12] <The-Compiler> Requirement already satisfied (use --upgrade to upgrade): pylint in c:\python34\lib\site-packages
[12:39:10] <The-Compiler> so the issue is probably pylint being installed system-wide on my Win7 machine, but not on my Win8 machine
[12:39:33] <The-Compiler> so if I run it with --upgrade, I'll get the newer version in the virtualenv, correct?
[12:40:33] <apollo13> not sure what happens in windows, but installing the same package in the venv and the system site-packages sounds like a bad idea
[12:41:36] <The-Compiler> Well, I'd like the venv to work no matter what is installed system-wide. And I have a library (PyQt) installed which I need in the venv as well, thus the --system-site-packages
[12:42:05] <The-Compiler> on Linux I just symlink the .so's - maybe I should just copy the relevant files on Windows as well, instead of using --system-site-packages
[12:44:19] <The-Compiler> I wish, this is my test bot (buildbot), so that'd mean "don't support windows" :P
[12:46:10] <mgedmin> virtualenv /tmp/first; virtualenv -p /tmp/first/bin/python /tmp/second --system-site-packages
[12:47:18] <mgedmin> /tmp/second/bin/pip install requests -U => installs requests 2.5 into /tmp/second
[12:47:28] <The-Compiler> Not uninstalling pylint at c:\python34\lib\site-packages, outside environment C:\Users\florian\buildbot\slave\win7\build\.venv
[12:47:39] <apollo13> mgedmin: -p onto a venv will result in the site-package from your global python, not from the other venv python
[12:48:13] <mgedmin> anyway it didn't fail and it didn't try to uninstall /usr/lib/python2.7/dist-packages
[12:48:14] <The-Compiler> so either I don't use --system-site-packages, or I ditch the systemwide installation I don't need anyways
[12:49:37] <mgedmin> python-daemon installs a buggy entrypoint that breaks pip install of any other package :/
[12:50:16] <Wooble> calston: what version of pip are you using? I had that same ImportError on a stock Ubuntu 12 that was fixed by upgrading pip.
[12:50:28] <mgedmin> calston: https://mail.python.org/pipermail/distutils-sig/2015-January/025649.html
[13:22:48] <ychaouche> I tried with pip uninstall mypackagename, it says : Cannot uninstall requirement infomaniak, not installed
[13:23:16] <ychaouche> On the other hand, pip freeze isn't showing any packages at all on this fresh virtualenv
[13:24:31] <mgedmin> it sounds as if you have a different virtualenv activated than the one where you installed the package
[13:25:35] <ychaouche> as I said, it's installed with setup.py develop (not install) and I can see the infomaniak-link in the virtualenv I am currently in
[13:27:01] <mgedmin> ychaouche, I always thought that 'pip install -e foo' was the same as 'cd foo && python setup.py develop
[13:27:25] <mgedmin> but maybe plain setup.py develop doesn't write the right metadata somewhere? I dunno...
[13:27:58] <ychaouche> mgedmin: how to tell pip to install the package from a path, not from Internet ?
[13:29:22] <ychaouche> hum, at this point I don't really understand everything about how all of this works. Still learning by practice :)
[13:29:44] <ychaouche> I think I will create a new virtualenv and try to install with pip this time and see if uninstall works fine.
[13:32:35] <ychaouche> So I installed with pip -e, and pip freeze is a little confused it seems : https://gist.github.com/ychaouche/c70afdf4e35035fdb5c7
[13:33:12] <mgedmin> yes, pip freeze really likes to output -e git:...#commithash URLs for editable packages
[13:38:31] <calston> mgedmin: okay well I recreated the virtualenv and installed the stuff again and it works, so I guess whatever package required it has dropped that
[13:43:52] <ychaouche> https://gist.githubusercontent.com/ychaouche/3cdf8240b070a9c80ad2/raw/b5abfcb12c6cec3104324dfdaeb0d98ae2d054bf/gistfile1.txt
[13:48:59] <ychaouche> It would be nice if the standard error message of touch was "can't touch this".
[13:51:31] <ychaouche> Ok. Next step is to figure out how to rewrite my module so that it finds the files it needs to read
[13:52:00] <ychaouche> I guess it has something to do with the cwd not set to the location of the script
[14:48:28] <ychaouche> So my script needs some non python files, like HTML files and one configuration file.
[14:51:15] <mgedmin> if you want pip install to work, keep them with the source code, maybe in a subdirectory
[14:51:26] <mgedmin> if you're building debian packages, feel free to move them to /usr/share, /etc etc.
[14:52:04] <mgedmin> config files? ~/.config/yourpackage.ini (well, $XDG_CONFIG_DIR if it's defined) or /etc/yourpackage.ini or (best) allow the user to specify the config file on the command line
[14:52:31] <mgedmin> don't require a config file in /etc, maybe users will want to pip install your stuff and use it when they don't have root on the machine
[15:00:20] <ychaouche> actually, i don't intend to release the final "thing", it's only to write things the right way on my machine then deploy them on one or two servers, and also to make smooth updates.
[15:01:09] <ychaouche> Since the "thing" I want to write will be a daemon, I also need to put things in /etc/init.d/ etc., so I don't know what's the right packaging to choose (debian or python)
[15:02:26] <ychaouche> but since the dameon is written in python, I don't know where to put the python files, so I thought of creating a python archive that would go in site-packages (dist-package in debian)
[15:04:08] <ychaouche> also the install script should issue some post-install commands, like updaterc.d and service infomaniak start.
[15:08:06] <DanielHolth> debian packaging will be better at installing things outside of site-packages
[15:13:47] <ychaouche> How bad should I feel if I write an install.sh script that does the pip install then continues with copying the files and calling other commands ?
[15:16:47] <tos9> Sigh. I'm trying to track down a bug, and I can't quite figure out what the correct combination is to reproduce it is.
[15:17:25] <tos9> I *think* the combination is 1) pip install --download 2) a private git repo 3) with no git tags 4) with a specifier that specifies a branch name
[15:18:10] <tos9> Which produces a traceback in pip's rmtree_errorhandler that's tryign to remove a file in the temporary repo that looks like .git/tags.12345, when by the time shutil.rmtree runs, that file doesn't exist.
[15:18:33] <tos9> (There is one called .git/tags though.) I don't suppose anyone has any insights or has seen this before?
[15:45:54] <pf_moore> tos9: Sounds like a race condition in rmtree_errorhandler. Maybe add a check if the file exists, and if not then just return. See if that fixes the issue.
[15:47:12] <pf_moore> As shutil.rmtree isn't (can't really be) atomic, that's probably the best we could do.
[15:50:05] <tos9> pf_moore: it does, but it's consistent -- it can't be atomic but pip could ignore the error presumably?
[15:53:43] <tos9> bleh, no it looks like that tags file is being renamed in there, it's not a new one
[15:54:18] <tos9> does pip shell out for git cloning? yes right? and I presume it waits for the process to exit?
[16:10:17] <twilder6629> We deploy on closed networks so we cannot use a package index unless we bring one up ourselves. Right now the solution is to ship a bunch of wheels in an installer and use --find-links and --no-index to satisfy deps. I just made a custom package and strangely cannot get it picked up when I call pip install on the dir it resides in.
[16:10:47] <twilder6629> pip install --find-links ~/ansible_wheels --no-index --pre mass-install is the call
[16:11:25] <twilder6629> ansible-1.8.2-py2-none-any.whl MarkupSafe-0.23-cp27-none-linux_x86_64.whl pycrypto-2.6.1-cp27-none-linux_x86_64.whl
[16:11:25] <twilder6629> ecdsa-0.11-py2-none-any.whl mass_install-2.6.1.2.commit.544f732dd4f3a6d8a65d9bc15476e05745f7b2c9-py2-none-any.whl PyYAML-3.11-cp27-none-linux_x86_64.whl
[16:11:25] <twilder6629> Jinja2-2.7.3-py2-none-any.whl paramiko-1.15.2-py2.py3-none-any.whl setuptools-12.0.4-py2.py3-none-any.whl
[16:12:14] <twilder6629> if i specify the package directly as the target of pip it installs correctly and hass the mass-install name associated with it
[16:15:21] <twilder6629> The package is built with setuptools rather than distutils and has a version of 2.6.1.2 and a dev tag of .commit.<git_commit_hash>
[16:15:22] <bowlofgrapes> hey, so remember that pulp plugin i was talking about to sync from PyPI? we released the first version of it today! http://pulp-python.readthedocs.org/en/0.0-release/ however, it doesn't have the sync feature just yet
[16:15:44] <bowlofgrapes> but you can use it to upload and publish python modules, and it publishes in a way that is compatible with pip
[16:40:40] <twilder6629> reference to the package by module name fails, reference by full path still succeeds
[16:41:01] <twilder6629> pip install --find-links file:///root/ansible_wheels --no-index --pre /root/ansible_wheels/mass_install-2.6.1.2.commit.544f732dd4f3a6d8a65d9bc15476e05745f7b2c9-py2-none-any.whl
[16:42:03] <twilder6629> whereas pip install --find-links file:///root/ansible_wheels --no-index --pre mass-install is the failing call
[16:43:43] <twilder6629> Could not find a version that satisfies the requirement mass-install (from versions: 2.6.1.2.commit.544f732dd4f3a6d8a65d9bc15476e05745f7b2c9)
[16:48:02] <dstufft> pip install --find-links path/to/wheel/s --no-index mass-install===2.6.1.2.commit.544f732dd4f3a6d8a65d9bc15476e05745f7b2c9
[16:49:51] <twilder6629> since i need other things that depend on this to work without having to specify versions in requirements
[16:54:52] <twilder6629> the purpose of the commit tags is to implement parallel build caching for unchanged packages
[16:55:21] <twilder6629> and whenever we build one, we upload to a local package index and tag it with the git commit of the last change to that package's directory
[16:55:36] <twilder6629> then when we build, if a package matching the last commit to one of ours exists in the package index
[16:56:35] <twilder6629> so the requirement is to be able to communicate a git commit in the version identifier somehow that we can use when installing a package, but at the same time be able to wildcard with just the package name
[16:57:11] <twilder6629> it mostly matters for things with lots of C extensions that take a while to build
[16:57:13] <dstufft> You can do ``pip install 1.0+foo``, and it'll install _only_ 1.0+foo, there's no wild card if you specify the package version like that
[16:57:55] <twilder6629> i was concerned that the local version would be discarded even if explicitly specified
[16:58:23] <dstufft> nope, but if you do ``pip install foo==1.0`` it might install 1.0+foo if it's available
[16:59:24] <twilder6629> basically deployment is a single set of commit tagged packages representing one commit in master
[16:59:44] <twilder6629> so there's no ambiguity when deploying since only one of each package exists in the deliverable
[17:10:05] <dstufft> ionelmc: not right now. I'm tying up loose ends on my current team and switching teams at work a little over a week
[17:15:28] <dstufft> ionelmc: possibly, I'm not sure what i'm doing this weekend. We were supposed to go to a concert for my daughter's birthday present but that got canceled.. so idk yet if we're doing to do sometihng else or not
[17:17:24] <twilder6629> dstufft, the change works, is tested, and is checked in. Thanks again! Best one character swap ever.
[21:50:43] <charettes> I've been following https://packaging.python.org/en/latest/distributing.html#universal-wheels to create an universal wheel for https://github.com/charettes/django-sundial but `python setup.py bdist_wheel --universal` keeps building me a platform specific one.