pmxbot IRC Log Viewer

[01:54:24] <kevc> Are there any plans to make virtualenv support ordered lists in requirements.txt?

[01:54:57] <kevc> the issues raised on github all seem to just fizzle out, and I get stuck with using the | xargs workarounds

[01:57:52] <dstufft> kevc: "ordered lists" ?

[01:59:18] <kevc> install the packages in the order listed in requirements.txt

[02:00:02] <kevc> the issue is that quite a lot of packages have broken deps, and so pip install -r requirements.txt does not work

[02:01:20] <kevc> Using "cat requeriments.txt | egrep -v '^ *#' | xargs pip install" works

[02:03:46] <dstufft> kevc: have you tried with pip 6.1? I think that is fixed in 6.1

[02:06:04] <kevc> let me try that

[02:40:00] <kevc> seems it's still problematic. Maybe I should raise an issue on github, but I'm sure it'll get closed with the issue being a broken package rather than a virtualenv issue.

[02:41:57] <dstufft> kevc: I mean, at some level there's only so much we can do to work around broken packages :( Future changes to pip to speed things up will likely break anything explicitly based on ordering anyways

[02:48:08] <kevc> dstufft: sure, but I've never managed to get virtualenv working _without_ workarounds

[06:24:00] <mgedmin> is pip creating pip-selfcheck.json files in the root of every virtualenv now?

[10:58:46] <jhermann> my venvs are in .venv/name, dump all the data you want in there ;)

[10:58:57] <jhermann> mgedmin, and yes, seems so

[11:24:23] <apollo13> mgedmin: virtualenvwrapper and mkproject

[11:26:35] <mgedmin> here's why not: I want 'git clone && make' to work, but not touch anything outside the source tree (except for ~/.cache and such)

[11:26:51] <mgedmin> some of the things I work with use zc.buildout (which creates bin/foo for all local scripts)

[11:27:10] <mgedmin> some use virtualenv (with a makefile that creates it automatically and updates if setup.py/requirements.txt changes)

[11:27:35] <ionelmc> mgedmin: tox!

[11:27:47] <mgedmin> tox is excellent for testing

[11:27:57] <mgedmin> but not for using/debugging stuff

[11:28:03] <ionelmc> mgedmin: i use it for running/debugging

[11:28:10] <mgedmin> ooh, can you tell me more?

[11:28:42] <ionelmc> it works really well and i don't have to remember configuration and have crappy makefiles/bootstrap shell scripts

[11:28:50] <mgedmin> example usecase: work on webapp, 'make run', click on http://localhost:NNNN, open it in a browser

[11:29:09] <mgedmin> I suppose tox -e run-app could do that

[11:29:10] <ionelmc> mgedmin: all my tox envs allow overriding the command

[11:29:19] <ionelmc> standard practice

[11:29:29] <ionelmc> so i can run tox -e 3.4 -- python foobar.py

[11:29:30] <mgedmin> but doesn't tox ignore changes to setup.py?

[11:29:34] <ionelmc> instead of running suite

[11:29:58] <ionelmc> mgedmin: tox operates in 3 modes wrt setup.py: no install, sdist install, develop install

[11:30:38] <mgedmin> I don't want to remember to manually run tox.py when I git pull and my requirements.txt is changed

[11:30:45] <mgedmin> s/tox.py/tox -r/

[11:31:06] <ionelmc> mgedmin: i also *don't* use requirements.txt

[11:31:09] <ionelmc> :)

[11:31:17] <mgedmin> s/requirements.txt/setup.py/

[11:31:17] <ionelmc> i just stick the deps in tox.ini

[11:31:24] <ionelmc> the concrete deps

[11:31:32] <ionelmc> the abstract deps go in setup.py ofcourse

[11:31:55] <doismellburning> the thing that gets me - how do you ensure the concrete deps match the abstract deps

[11:32:14] <doismellburning> it's not like Gemfile.lock / Puppetfile.lock / etc. where they're generated from the abstract deps

[11:32:16] <ionelmc> doismellburning: abstract deps take precedence

[11:32:20] <mgedmin> '-e .' in requirements.txt

[11:32:40] <doismellburning> sure

[11:32:57] <ionelmc> mgedmin: think out of the box for a moment, imagine you don't have requirements.txt (that something workflow specific)

[11:33:10] <ionelmc> that's*

[11:33:12] <doismellburning> speaking as someone who's seen "we support Django 1.7!" when they had an ordering bug and setup.py was reinstalling Django<1.7, like, yeah

[11:33:15] <mgedmin> TBH I usually don't have requirements.txt

[11:45:07] <ionelmc> mgedmin: still not convinced to use tox?

[11:46:22] <doismellburning> I do for apps, never for libraries

[11:46:29] <doismellburning> (probably should though)

[11:46:58] <doismellburning> but to call it a concrete dependencies file is a bit generous; it's not actually generated in the way Gemfile.lock etc. is

[11:47:08] <ionelmc> doismellburning: did you check that pylibrary template?

[11:47:21] <ionelmc> doismellburning: yeah, you do it by hand :(

[11:47:38] <doismellburning> ionelmc: sorry what template?

[11:47:38] <ionelmc> tho i don;t have long list of deps for libraries

[11:47:54] <ionelmc> something is definitely wrong if you make a library that has hundred dependencies

[11:48:04] <ionelmc> doismellburning: cookiecutter-pylibrary

[11:48:15] <doismellburning> will take a look, thanks

[11:50:11] <mg-phone> !logs

[11:50:11] <pmxbot> http://chat-logs.dcpython.org/channel/pypa

[11:52:07] <mg-phone> ionelmc, do you need special magic in tox.ini to allow tox -e py34 -- command?

[11:52:29] <ionelmc> mg-phone: just {posargs:default test command}

[11:52:45] <ionelmc> that gets replaced if you specify -- stuff

[11:53:28] <mg-phone> A-ha! Nice trick

[14:16:59] <MrJones> hi

[14:17:04] <MrJones> what is a pypiserver/pip package

[14:17:19] <MrJones> I assumed it wants .egg, but apparently it doesn't and pip doesn't like to install .egg's from a custom pypiserver

[14:17:22] <MrJones> so what DOES it want

[14:17:31] <MrJones> https://pypi.python.org/pypi/pypiserver just mentions "packages" without ever explaining what that actually is

[14:17:48] <MrJones> a .zip of my project with setup.py or something similar? or what is a "package"

[14:19:22] <ronny> pip does not install eggs at all

[14:19:40] <MrJones> yes I just wrote that. (:

[14:20:00] <ionelmc> MrJones: have you looked over https://packaging.python.org/ ?

[14:20:04] <ronny> an sdist would be a package as well

[14:20:52] <ronny> MrJones: pip supports sdist (tarball/zip including a setup.py, usually made by sdist command) and wheels

[14:20:59] <MrJones> ionelmc: this guide introduces a Python "package", linked to: https://packaging.python.org/en/latest/glossary.html#term-distribution-package

[14:21:22] <MrJones> but that paragraph is absolutely uselessly vague

[14:21:30] <MrJones> if I click around more on "package" I suddenly end up with a module package

[14:21:59] <ronny> hmm, i use devpi client/server instead of that thing

[14:22:04] <MrJones> tl;dr: I'd say the packaging guide also really drops the ball at early introduction of a nice and simple definition of how a package looks like, unless I'm missing it (disclaimer: haven't read the whole guide yet)

[14:22:30] <MrJones> does it actually want a source distribution?

[14:22:39] <ionelmc> MrJones: have you looked over https://packaging.python.org/en/latest/distributing.html ?

[14:23:27] <MrJones> ok so

[14:23:49] <MrJones> maybe https://pypi.python.org/pypi/pypiserver should be updated to refer to package (source distribution), .egg, wheels, ... instead of package, .egg, wheels

[14:23:59] <MrJones> because that would help a lot figuring out what it actually means by "package"

[14:24:07] <ionelmc> MrJones: be aware that it's hard for maintainer to improve the guide without feedback (hard to imagine what's hard to understand for new people , since you already know how to do it)

[14:24:09] <MrJones> ionelmc: that is helpful, thanks. I guess sdist is what I want

[14:24:24] <MrJones> the problem is more the pypiserver help page I find now

[14:24:31] <ionelmc> MrJones: so if you have suggestions, it's always welcome

[14:24:33] <MrJones> since 'package' can mean a lot of things, including .egg I guess?

[14:24:56] <MrJones> it should really refer to package (source distribution), or clarify it means the sdist source distribution format of setuptools somewhere

[14:25:32] <ronny> problem is suggestions dont fix the docs unless they land at a maintainer, bugtracker or pull request ^^

[14:25:40] <ionelmc> MrJones: historically package meant `dir with __init__.py file`, while distribution meant an archive (like tarbal,zip, egg or whl)

[14:26:08] <MrJones> so does pypiserver actually mean a "dir with file" or an archive? the help page for pypiserver really sucks in that regard by its use of "package"

[14:26:12] <MrJones> couldn't be any more vague really

[14:26:21] <MrJones> wheels or .egg distribution is pretty obvious since that is a defined known format I suppose

[14:26:26] <MrJones> but "package"? plis

[14:26:30] <ronny> MrJones: btw, wh do you want that pypiserver thing?

[14:26:45] <MrJones> https://pypi.python.org/pypi/pypiserver I'm talking about this

[14:27:00] <ionelmc> MrJones: misused terminology, since package means something else everywhere else

[14:27:01] <MrJones> (just in case that wasn't clear)

[14:27:11] <MrJones> well then maybe that page should be fixed :D

[14:27:23] <MrJones> since this is the pip channel, I guess this is the correct place to remark pypiserver things too..?

[14:27:39] <ionelmc> MrJones: which page? how? make a pull reqeust with tentative changes - there's an edit button on th epage

[14:27:51] <MrJones> https://pypi.python.org/pypi/pypiserver this page

[14:29:19] <ionelmc> MrJones: https://github.com/pypiserver/pypiserver, click readme, click edit, make commit, make pr

[14:29:30] <ionelmc> easy peasy lemon squezy

[14:29:41] <MrJones> "pypiserver is a minimal PyPI compatible server. It can be used to upload and serve packages, wheels and eggs to pip or easy_install. The packages are stored inside a regular directory."

[14:29:45] <MrJones> that first sentence

[14:29:48] <MrJones> packages really just fall from the sky.

[14:29:56] <MrJones> no explanation what that is specifically, e.g. sdist packages

[14:30:27] <ionelmc> MrJones: i agree, and _you_ can fix it :-)

[14:31:05] <MrJones> well right now I gotta fix my egg-producing code to use sdist instead because otherwise our packaging server is quite useless offering eggs to pip

[14:31:08] <MrJones> :D

[14:31:52] <ronny> MrJones: btw, again, take a look at devpi, i suspect you might like it a lot more

[14:32:10] <ronny> MrJones: also take a look at building wheel packages

[14:32:23] <ronny> (those install faster and without needing a setup.py)

[14:32:30] <ionelmc> MrJones: aaah, anger is not to be wasted on anything but pull requests

[14:33:23] <MrJones> ronny: lol, I assumed a dev package index would be exactly just running pypiserver?!

[14:33:27] <MrJones> :D

[14:33:36] <MrJones> there's a separate devpi tool we missed? interesting

[14:33:51] <ronny> MrJones: devpi is a lot more flexible/neat

[14:34:47] <MrJones> well we don't need much tbh, pip just needs to use our local package source that's all

[14:37:02] <MrJones> hmmmmm

[14:37:04] <ronny> MrJones: before i introduced devpi at the company i work at, they said the same ^^

[14:37:33] <MrJones> so to be absolutely sure of this, "python3 setup.py sdist" and the resulting .tar.gz is what pypiserver calls a "package" and what pip should want for installing?

[14:37:38] <MrJones> or does pip not like that either

[14:38:05] <ronny> MrJones: that looks fine, asuming the setup.py is correct ^^

[14:38:17] <MrJones> alright

[14:38:39] <MrJones> well, I got that on my little pypiserver now (it's listed in the index), but I still get: No distributions at all found for unifrhermes

[14:38:44] <MrJones> (pip output)

[14:39:11] <MrJones> browsing the pypiserver's /packages/, I see unifrhermes-0.0.1.tar.gz

[14:39:30] <MrJones> so I'm not sure what is wrong at this point.. possible ideas appreciated (:

[14:41:14] <MrJones> oh it was just a wrong port :D

[14:41:21] <MrJones> kinda vague error message from pip about that.. heh

[14:51:56] <doismellburning> does #pypa have an opinion on "distributing applications via PyPI"?

[14:52:17] <doismellburning> s/applications/services/

[14:52:43] <doismellburning> e.g. bucky

[15:04:11] <ionelmc> doismellburning: are you asking from service conf POV (eg: init scripts/supervisord conf etc)?

[15:05:43] <doismellburning> I mean that I have a Python IRC bot, and someone is suggesting I should distribute it via PyPI

[15:06:47] <ionelmc> doismellburning: what are your concerns?

[15:08:05] <doismellburning> mostly "I feel requirements files best express application dependencies"

[15:12:17] <ionelmc> doismellburning: not sure i understand, are you concerned about where to put/how the concrete deps (the ones with the version pins) ?

[15:12:37] <ionelmc> and how that comes into play if you install said irc bot via pip?

[15:15:40] <doismellburning> ionelmc: I guess so, yes

[15:16:57] <ionelmc> doismellburning: there's no right answer but consider this: if you pin the versions (strictly) then you might as well just vendor the deps like pip or requests do

[15:17:29] <ionelmc> so i guess it boils down to "do you actually want that?"

[15:18:21] <doismellburning> vendoring's more effort and doesn't get me support from things like requires.io

[15:18:24] <doismellburning> but yes

[15:18:59] <ionelmc> well to be honest i always thought that a test-time thing

[15:19:27] <ionelmc> you have a tox.ini (or a req.txt) that you use to test the project, and you pin versions in there

[15:20:09] <doismellburning> massively disagree

[15:20:12] <doismellburning> this is an application

[15:20:18] <doismellburning> not a library

[15:20:33] <doismellburning> so when you "install it" it Should Just Work

[15:20:41] <doismellburning> i.e. pinned dependencies

[15:20:46] <ionelmc> well then, you don't install apps with pip

[15:20:52] <ionelmc> you use rpm/deb

[15:20:54] <doismellburning> this was my thought

[15:20:59] <doismellburning> cheers

[15:21:05] <ionelmc> and you can bundle the deps in the rpm

[15:21:13] <ionelmc> that's how i'd do it

[15:21:20] <doismellburning> agreed

[15:21:35] <ionelmc> actually, that's how i do it right now :)

[15:22:06] <ionelmc> i build the wheels for all the deps, put em in the deb/rpm and then install them in a virtualenv

[15:22:11] <ionelmc> and that's the deb/rpm

[15:22:30] <ionelmc> and the app is rigged to run from that venv

[16:34:27] <GoClick> How can I upgrade something from the command line to more than 1.5 but less than 1.6? eg it’s at 1.5.2 now and I’m good with 1.5.3 or 1.5.4 but not 1.6? I get setting it in the requirements file, but what about the command line?

[16:34:50] <ionelmc> GoClick: sure you can, but you probably need to quote arguments

[16:35:08] <GoClick> So what do I do?

[16:35:31] <GoClick> doesn’t —upgrade force latest? or do I go pip —upgrade “>1.5<1.6”

[16:35:36] <ionelmc> pip instsall -U "foobar>=1.5,<1.6"

[16:37:04] <GoClick> There’s my problem.

[16:54:22] <jhermann> ionelmc, frozen deps are ok / the right thing for an _application_ like an iorc bot

[16:54:35] <jhermann> (assuming they are maintained)

[16:55:04] <jhermann> libraries are quite different of course

[16:56:59] <ionelmc> jhermann: yeah kinda, but you can also look at it as a library (the service part)

[16:57:27] <jhermann> what service part?

[16:57:53] <ionelmc> depends on how you want your users to use it (eg: if you want them to also be use it like a library - so they can customize)

[16:58:41] <jhermann> that is better served by a plugin/hook concept, and entry points

[16:58:49] <jhermann> (again, for an _app_)

[19:33:26] <ronny> dstufft: ping?

[21:05:00] <tdsmith> even for apps pinned dependencies can be annoying if users aren't using virtualenvs religiously. probably makes life harder for e.g. debian packagers with a single global namespace for python module dependencies and poor support for multiple versions. homebrew mostly doesn't care since we give apps private pythonpaths

Log file Viewer

Help | Karma | Search:

#pypa logs for Thursday the 9th of April, 2015