Log file Viewer
#pypa logs for Tuesday the 25th of August, 2015
[08:51:07] <angular_mike_> I'm working on two packages: a and b. a depends on b. I would like to declare b as dependency in a's setuptools install script, but instead of it searching for b in available repos, etc., just fail if it isn't installed. Is that possible?
[08:54:31] <pjdelport> angular_mike_: You can say "pip install --no-deps" to prevent pip from installing dependencies.
[08:55:35] <pjdelport> But generally speaking, this is an install-time decision, not a packaging-time decision.
[08:55:48] <angular_mike_> pjdelport: the thing is, I have some dependencies that I do want to install from web
[08:56:27] <angular_mike_> mgedmin: I don't want the install wasting time looking for something that isn't there
[08:56:57] <angular_mike_> mgedmin: or even worse, finding and installing a completely unrelated package with same name
[08:57:24] <angular_mike_> mgedmin: btw, I thought setuptools used easy_install by default, not pip?
[08:58:23] <pjdelport> angular_mike_: If you don't want pip to look for the package, you should probably not list it as a dependency, and just document it in the README or somewhere as a manual dependency.
[08:59:06] <mgedmin> you could claim the name on pypi and not upload anything, to avoid the 2nd potential problem
[08:59:41] <mgedmin> that would be rude if the name were generic, but if it's some namespaced thing like mgedmin.privatestuff, I don't think anyone would mind
[09:01:59] <pjdelport> angular_mike_: Right, but if the dependency needs manual installation anyway, what else can you do?
[09:02:22] <pjdelport> You could make your setup.py print out a notice or warning if the package isn't installed, perhaps.
[09:03:05] <angular_mike_> pjdelport: would that involve it being looked up on web beforehand as well?
[09:05:05] <pjdelport> The way the entire Python packaging ecosystem is designed, there is not really any concept of declaring dependencies as an entirely separate thing to finding and installing them.
[09:05:35] <mgedmin> one of the suggested workarounds was setting up an internal package index (using devpi)
[09:06:08] <pjdelport> For proprietary packages, you'll generally want to point pip at your own package index / proxy, or use --find-links.
[09:07:14] <pjdelport> angular_mike_: To rephrase the question, what are you actually trying to protect against? Internal people forgetting to install the internal packages?
[09:07:42] <pjdelport> For that kind of error, a better solution is probably just to approach it with your deployment or CI scripts.
[09:08:13] <angular_mike_> pjdelport: my main motivation is saving people time by preemptivly warning them that the package won't work in their current environment
[09:08:18] <pjdelport> That is, instead of treating a "root" package as your starting point, which is responsible for pulling and checking everything in and erroring out, you can instead treat a requirements.txt as your root.
[09:08:57] <pjdelport> Which will point at the actual proprietary root packages, as well as any other "environmental" packages (like DBAPI drivers etc.) that you generally don't want to list directly in install_depends.
[09:12:41] <angular_mike_> mgedmin: it can sometimes be a real pain to double-check manually every requirement givent in the readme, that's why we have automated installation in the first place
[09:14:32] <rkuska> dstufft: hi donald, I have a question about name handling on pypi/pip/warehouse, as when someone types pip install django and Django get installed my guess is that pypi takes foo and Foo for a same project, right? (one can't register Foo when foo is already registered) do you plan to keep it that way for warehouse?
[09:15:35] <pjdelport> angular_mike_: I think the best approach here is to follow the "consenting adults" principle. Don't assume too much about the environment, and just check and raise an error explaining "You need to install package foo before continuing" at the code that actually needs that.
[09:16:52] <ronny> bascially - tooling sucks, the more facny you try to be, the more you make live hard for packagers an users
[09:17:31] <rkuska> dstufft: I am asking beucase of Fedora inconsistency in using lower/upper cased first letter when naming rpms and dnf being case sensitive when installing, I would like to make it consistent but firstly I need a statement from upstream
[09:28:30] <angular_mike_> pjdelport: honestly, looking up an installing packages from the web without any confirmation by default sounds exactly like magic at installation time to me
[09:29:14] <pjdelport> angular_mike_: Well, to be fair, that's more or less pip's entire stated purpose.
[09:35:06] <pjdelport> Invoking setup.py directly should probably be considered a legacy use case nowadays, predating the existence of pip and so on.
[10:07:41] <angular_mike_> It seems, they inherit from distutils, but the furthest I coud track them was to class distutils.cmd.Command(dist)
[10:08:33] <ronny> angular_mike_: all setuptools calasses are old style due to distutils inheritance
[10:10:12] <ronny> angular_mike_: btw, avout setup.py install like hell (by default due to setuptools "good idea at the time" it invokes easy_install)
[10:13:53] <angular_mike_> ronny: you know, I get about 50/50 people telling me to avoid me and telling me there's no point avoiding it
[10:16:55] <angular_mike_> ronny: pjdelport just now even said that invoking setup by default is to be considered legacy and it should be evoked via pip
[10:39:13] <ronny> angular_mike_: setuptools makes setup.py install broken (because bascially with setuptools its an alias for easy_install .)
[10:39:54] <ronny> (there are ways to make setup.py non-broken, but that breaks other things as well
[10:41:19] <angular_mike_> ronny: are you saying its doesn't add the required packages to Python path?
[10:42:25] <ronny> angular_mike_: it adds an egg of the package to sys.path, and will potentially install eggs of the dependencies
[10:47:17] <ronny> angular_mike_: typically egg installed ditributions and multi version isntalls is something you want to avoid (it creates problems down the line)
[10:50:21] <ronny> angular_mike_: thats because you never ran into the problems down the line ^^ once you see the first VersionConflict after subsequent updates you will be in for a surprise ^^
[10:50:59] <ronny> inless you have a good/legacy reason to do egg based multi version isntalls, avoid them
[11:16:09] <pjdelport> angular_mike_: The shorter informal summary is that the packaging situation used to much worse than now, and it's gotten a lot better and saner with the advent of modern pip and related tools, and you almost certainly don't want to open the Pandora's box and go back to the Bad Old Days.
[11:17:25] <pjdelport> (unless you look forward to learning all the different, interesting ways in which old things will bite you, like ronny describes :)
[13:05:31] <dstufft> rkuska: we apply a normalization to the names before we check for uniqueness (and when we're doing look ups)
[13:07:34] <dstufft> basically, names are case insensitive, are resitrcted to ASCII alpha numerics or - _ . must begin and end with an alpha numeric, and - _ . are all considered equivilant
[13:16:10] <dstufft> theuni2: I meant to email you and then forgot, when I migrated the database to Heroku and applied some migrations I somehow got bandersnatch stuck, but killing the state and doing a full refresh fixes it
[13:24:48] <dstufft> theuni2: no problem, sorry i forgot. I wasn't sure how you wanted to bes tcommunicate that with bandersnatch users
[13:26:44] <dstufft> theuni2: oh, btw: when Warehouse purges it's using celery with retries and acks_late, so once celery is live we shouldn't get into issues where a purge fails and the job doesn't notice or retry
[14:08:52] <angular_mike_> has anyone tried extending setuptools' setup() to support additional arguments?
[14:11:43] <dstufft> angular_mike_: there's a plugin interface you can do to add extra capabilities to setuptools
[14:13:09] <dstufft> http://ziade.org/2007/09/30/extending-setuptools-adding-a-new-command/ there's ane xample adding a new command
[14:14:12] <angular_mike_> dstufft: I've already made some custom command classes, but how do I add the arguments?
[14:19:10] <dstufft> distutils.setup_keywords is the name of the entry point, everything else I highlighted is an example of keywords that were added
[14:27:37] <MIG-> As a "final" setup, why does pip install --target copy libs from the configured purelib directory instead of the purelib directory ?
[14:28:09] <MIG-> Defaulting to purelib has resulted in failures noted in this issue https://github.com/pypa/pip/issues/3056