Log file Viewer
#pypa logs for Thursday the 16th of July, 2015
[09:57:00] <mgedmin> why do I care: virtualenv . && bin/pip install pytest && bin/py.test now runs all the tests twice: once from src/, once from local/src/
[10:06:29] <mgedmin> amazingly it even creates local/.git, which makes 'git clean -dfx' leave the local/ subdir alone
[10:21:53] <jmurphyau> using pip -t and PYTHONPATH as an alternative to virtual env: http://blog.zoomeranalytics.com/pip-install-t/
[10:27:19] <[Tritium]> https://bpaste.net/show/c870b693c6b9 << note that the stdlib, and platform specific locations are in sys.path, but all other paths, including all instances of site-packages (the important bit), is prefixed by the venv location. There is no polution of site-packages from the system python
[10:31:12] <jmurphyau> mgedmin: I got that to work using this: export PYTHONPATH=libs/; python -S -m coverage
[10:32:06] <ronny> jmurphyau: also note that pip will ignore what is installed in the target and anways install all deps
[10:32:30] <jmurphyau> and there are also other scripts/bins that virtual environment works with - that pip -t and PYTHONPATH wont, or will have trouble with
[10:33:33] <jmurphyau> im just trying to understand why virtual env is so wide spread and trying to sell it to myself since I now have to use it.. it just seem like extra layers/complexity to something that shouldnt be that hard
[10:35:05] <[Tritium]> virtualenvs actually get out of the way. They do not sit between you and python, they make a copy of python in such a way that python thinks the virtualenv is the real environment.
[10:35:41] <[Tritium]> its not really abstraction in the sense of a wrapper around python, if thats what you were thinking
[10:51:04] <jmurphyau> i think the fact you feel like you're in a different environment (which i guess you are) when you type 'activate' is the big thing for me.. having to type 'activate' to transform my shell into a python virtual environment - it's almost like you need to have one dedicated shell session for a python virtual environment.. and the activate_this.py file as well.. it all just seems a bit much
[10:52:16] <mgedmin> which is why I used to do 'virtualenv .' at the project root and then use bin/python ... or, more often, bin/somescript ...
[10:52:40] <mgedmin> this is becoming untenable as virtualenv (& pip) create all kinds of junk in the virtualenv root
[10:53:39] <mgedmin> anyway, if you run python (or scripts) using the full (relative) pathname, then you never need to "activate" the venv
[10:59:49] <mgedmin> omg that document is terrible about (not) explaining the purpose and conventions
[11:00:10] <mgedmin> anyway I see https://github.com/jpadilla/django-dotenv also uses .env for a different purpose
[12:37:53] <Tarffull> Hi! I have a problem with a virtualenvironment and django. When installing third party tools for django (django-nose, django-bootstrap3) with pip install -e git.. everything works great. But when i install it just pip install django-nose i get a DistributionNotFound error on a completely different app when trying to run my application. What is the difference between pip install and pip install -e and how can i avoid this problem?
[12:48:23] <mgedmin> Tarffull, it might help us answer if you pastebin the two shell sessions (where you use -e and where you don't)
[12:49:20] <Tarffull> mgedmin: I get that exception when running my app after install django-nose w/o -e.
[12:50:51] <mgedmin> https://pypi.python.org/pypi/measurements doesn't exist, is it a private package?
[12:52:59] <Tarffull> Yes it's a private package and has nothing to do with django-nose. That's why it's so strange.
[12:56:45] <Tarffull> python portal/manage.py test portal --settings=portal.settings_test --liveserver=localhost:5000-6000
[12:59:21] <mgedmin> stepping back: in order for pkg_resources.get_distribution('measurements') to work, the package needs to be installed
[13:00:16] <mgedmin> a directory called measurements-{version}.egg-info (or .dist-info in the brave new world?) somewhere on sys.path
[13:00:30] <Tarffull> Yes, the virtualenvironment is activated (i have regeneragted it multiple times). I guess you are asking if measurements is in pip freeze? Yes it is.
[13:00:38] <nedbat> I tried to build a wheel with 3.5b3, and it failed, and I don't know whether to file a bug against 3.5, or wheel, or what: https://gist.github.com/nedbat/f0beb65a55e05599a931
[13:01:09] <mgedmin> Tarffull, is there a measurements*.egg-info in /home/epontpe/ivy-portal/.venv/lib/python3.2/site-packages/ ?
[13:04:02] <mgedmin> either it's because you get a different versions (pypi has 1.4.1, if google's cache is to be trusted -- can't be bothered check pypi itself, it 503s me)
[13:04:19] <mgedmin> changes since the last release: https://github.com/django-nose/django-nose/compare/v1.4.1...master
[13:04:49] <mgedmin> i.e. maybe the problem is always there, just older django-nose fails to trigger it
[13:05:15] <mgedmin> so, can you import pkg_resources and pkg_resources.get_distribution('measurements') in a python shell?
[13:07:58] <Tarffull> I've tried printing ','.join(pkg_resources.Environment()) from the test and it is empty when django-nose is installed w/o -e.
[13:08:37] <mgedmin> nedbat, an assertion error is always a bug in the package that raises it, imho, so I'd file it
[13:08:52] <ionelmc> nedbat: you should either patch up the wheel code to see what's going on or use hunter (it has a fancy variable printer)
[13:08:59] <mgedmin> user-caused errors shouldn't be assertion errors, they should be something specific and with a clear error message
[13:16:54] <mgedmin> I'm not sure I could help you either way sorry -- not really familiar with pkg_resources internals :/
[13:28:09] <ionelmc> this prolly doesn't help at all right now but maybe don't use unmaintained software like distribute or 3.2
[13:36:51] <ronny> Tarffull: elevate it as a critical imprediment that will cost a few thousand if you have to work around it instead of an admin fixing it
[13:52:15] <Tarffull> Just installed a fresh virtualenvironment with python 3.4 and setuptools instead of distribute. Worked like a charm! :D
[16:06:14] <benjaoming> Hi all! Would anyone in here have an experience about collecting statistics for their PyPi releases? Am releasing an open source project, but the organization behind is a bit skeptical about releasing on PyPi because of the download counts being polluted by bots and mirrors :/
[16:09:46] <dstufft> benjaoming: no replacement for that, better stats is on the roadmap for PyPI but not in the near term. Also unless you make your thing completely uninstallable via pip you're going to get some mirrors and bots downloading it
[16:15:13] <benjaoming> dstufft: Thanks! I'm working with a charitable organization and their support relies partly on being able to prove the usage of the software, I can imagine that other projects face the same challenge. Would there happen to be a "honey pot" package available that can be used for correcting download counts? That would be an okay quick fix for us.
[16:16:12] <dstufft> benjaoming: not really, and it's hard for that to be made because you'll get a couple hundred downloads anytime you make a release basically from mirrors
[16:16:52] <Wooble> sticking google analytics on my documentation page was my solution, but I don't have to prove anything to anyone.
[16:18:09] <benjaoming> dstufft: yeah, I was thinking that the challenge could be solved because we know the release dates of both our own package and a given honey pot package. So we can use the latest release stats from the honey pot (1-2 days after release) to correct the first 1-2 days of stats at any given release of the real package.
[16:23:19] <dstufft> benjaoming: I mean, the download counts can only really be taken with a grain of salt anyways. A mirror only counts as one download although they may be really masking thousands of downloads or they could just be inflating the download count, pip caches downloads by default now, so you'll only get 1 download per machine unless they clear their caches
[16:28:02] <benjaoming> dstufft: The pip cache is welcome, I guess the most interesting question is: How many users does our software have? We will surely have the stats polluted by upgrades as well, so there's a lot of uncertainty in the puzzle... making it an interesting task to have bestowed :)
[16:31:51] <dstufft> benjaoming: yea, and CI can often be a big influencer as well. If someone is running pip with cache disabled or in throw away environments in CI you can get a ton of donwloads from the same source :/
[16:31:59] <benjaoming> It might also be that we shouldn't be thinking too much about "absolute" numbers in the counts, but more about relative counts.. the ups and downs over time. Like economics, there's a general inflation (the mirrors), purchasing parity (the upgrades and cache), financial speculation (the bots, those leeches) and then growth should be adjusted accordingly :)
[16:32:42] <dstufft> I've often thought that we should get rid of numbers on the pages and just put something that's relative or fuzzier than pure numbers
[16:32:58] <dstufft> I also think sometimes about how we could get something more like Debian's popcon
[16:33:27] <dstufft> but you have to decide what you mean by an installation then too :V is it 1 per machine? 1 per python version? 1 per virtual environment? :V
[16:34:19] <dstufft> benjaoming: not that it helps you in the short term, but if you have any particular insights you'd like to be able to see, you can open issues up on github.com/pypa/warehouse
[16:48:07] <benjaoming> dstufft: thanks for the pointer, I'll have a look and see if I can contribute -- but maybe the responsibility would be for pip and other clients to be more explicit about "who they are". E.g. pip could say "i'm being run by a normal user" in its user agent header. Pip could also tell PyPi that it's being run with an --upgrade flag.
[16:50:08] <dstufft> benjaoming: it's hard to do that though, because you don't know that the --upgrade flag really applies here, for instance: I do pip install --upgrade foo, and I upgrade from foo 1.0 to foo 2.0, foo 2.0 includes a _new_ dependencies on bar so this is the first time bar was installed, is that an "upgrade" or is it a new installation?
[17:01:17] <benjaoming> tdsmith: if you need more downloads for your package, you can try convincing other popular packages to depend on it.. kind of like how the academic citation system works :)
[17:05:48] <benjaoming> dstufft: yes I think the "most optimal way" (since there's no potential of a correct one) could be more information from pip, for instance reporting that the --upgrade flag was applied and to which of the downloads (dependencies) it was relevant.. then leave the interpretation of the statistics to the package owners. Pip could add a couple of extra HTTP headers every time it downloads?
[18:08:56] <wsanchez> Is there a way to convince virtualenv to copy all all of the system python into the environment? ie. so it's totally self-contained
[18:09:33] <dstufft> wsanchez: No, but if you want the entire Python you probably just want to compile your own Python with --prefix?
[18:10:13] <dstufft> one of the core ideas of virtual environment is that it's lighter weight then a complete copy
[18:15:14] <dstufft> wsanchez: there's also pex if you are OK with depending on Python itself being installed, but you don't want to install things into that thing or modify that global environment