pmxbot IRC Log Viewer

[06:51:03] <ronny> lifeless: sup, is there any easy way to disable the wheel handling without disableing the cache handling? its turning into a mess

[06:52:10] <lifeless> ronny: --no-binary

[06:53:08] <lifeless> dstufft: ok so 7.1. :)

[06:53:12] <ronny> lifeless: that also prevented it from downloading pypi wheels

[06:54:58] <ronny> lifeless: is there avariant that works for env vars?

[06:55:26] <lifeless> it should be usable via env vars like everything else

[06:55:52] <lifeless> there was a bug in the cfg glue which we found out about after 7.0.0, I don't know if the fix for that was backported to 7.0.x or not

[06:56:04] <ronny> i'll experiment a bit

[06:56:07] <lifeless> ronny: you can disable individual packages

[06:56:13] <lifeless> ronny: with no-binary

[06:56:43] <ronny> lifeless: what i want is disable the wheel caching mechanism

[06:56:57] <lifeless> ronny: there's no explicit just-disable-that

[06:57:07] <lifeless> ronny: why do you want that?

[06:57:30] <ronny> lifeless: downloading pypi packages to push to a other system with different abi

[06:57:48] <ronny> (the install there must be networkless

[06:58:18] <ronny> and i really dont want to have to maintain dozens blacklists and whatsnot, i just want to have whats on pypi

[07:00:03] <lifeless> so you're using pip install -d ?

[07:00:11] <ronny> yes

[07:00:33] <lifeless> and you're getting ABI specific wheels in the output, without the matching sources ?

[07:00:37] <ronny> at some point it started to put wheels in with a abi tag, and it seems impossible to prevent that

[07:00:42] <ronny> exactly

[07:00:45] <lifeless> yeah, that is indeed the cache.

[07:00:50] <lifeless> hmm, there are two issues here

[07:01:03] <lifeless> one: PEP-426 environment markers make what you're doing potentially invalid.

[07:01:31] <ronny> you mean something like downloading on the wrong environment?

[07:01:34] <lifeless> That is, pip install -d can't guarantee (today anyhow) that you'll get all the dependencies needed to do such a networkless install from the other environment

[07:01:46] <lifeless> totally ignoring the ABI question

[07:01:54] <ronny> you mean the setup_requires issue?

[07:01:57] <lifeless> nope

[07:02:01] <lifeless> I mean PEP-426 markers

[07:02:12] <lifeless> which can say that a dependency only applies on some expression

[07:02:17] <ronny> those arent a problem atm, but its a good point

[07:02:24] <lifeless> which is a very small subset of python, but evaluated at install time

[07:02:25] <lifeless> so

[07:02:39] <lifeless> I think making 'install -d ...; rsync; install' work is a good feature to have

[07:02:44] <ronny> (it will probably come in the future, but since python versions and operating szstem are the same its much less of a problem

[07:02:53] <lifeless> I realise it used to work, and there are now two issues preventing it in different ways

[07:02:59] <ronny> i created an issue about creating a fetch command

[07:03:00] <lifeless> one is the ABI and not getting the sdists anymore

[07:03:04] <lifeless> the other is markers

[07:03:11] <lifeless> so we need to spider out all the deps

[07:03:17] <lifeless> and that then raises a problem

[07:03:23] <ronny> such a fetch command could probably be given a json dict of te target environment

[07:03:25] <lifeless> because sometimes those deps aren't evaluatable on the local environment

[07:03:30] <lifeless> e.g. you're running pip with python 2.7

[07:03:41] <lifeless> and one of the markers guards python 3 only packages

[07:03:53] <lifeless> which we can't even interrogate to get their dependencies (because no static metadata)

[07:04:41] <ronny> packagse that use extras and environment markers are halde-able

[07:04:52] <ronny> the rest i think is a acceptable loss

[07:10:19] <lifeless> extras would be ok

[07:10:40] <lifeless> environment markers and binary-abi-only distributions are I think unhandleable in the general case

[07:11:02] <lifeless> but we should be able to handle them in the specific case where the distribution is queryable even if its not desired for install

[07:11:15] <ronny> hmm, meh, im going to install pip~=6.0 on the ci job, im getting an attribute error wrt binary_only on a string object

[07:11:25] <lifeless> thats the bug

[07:11:34] <lifeless> it's definitely fixed in develop

[07:14:04] <ronny> lifeless: envoronment markers could be handled by adding option to pass an override to the fetch command (and in any case python versions are haldeable with some virtualenvs)

[07:20:27] <lifeless> ronny: only when the distributions referenced by the marker guarded sections are queryable

[07:20:32] <lifeless> ronny: which is what I said above

[07:22:21] <ronny> lifeless: i recall that after egg_info is done one can just read the extras file, but im getting the slight impression we are talking about different things

[07:27:34] <lifeless> if you can't run egg_info

[07:27:37] <lifeless> you can't read the file

[07:27:47] <lifeless> some sdists on pypi only work on 3.x

[07:27:51] <lifeless> others only work on 2.x

[07:30:33] <ronny> lifeless: thats fine by me ^^, if something breaks the tool can still abort

[07:30:54] <lifeless> right

[07:31:07] <lifeless> but its a limit until we can query an API for the data

[07:34:40] <ronny> lifeless: which api would that be, i recall the pacakges would need changes for it to work anyway

[07:37:01] <lifeless> there'sa thing in pypi where you can upload dependency data I believe; might be warehouse only

[07:37:04] <lifeless> anyhow, that.

[07:37:10] <ronny> lifeless: but well, my problem currently is not that something fails, but that pip makes my job much harder

[07:37:17] <lifeless> right, I get that

[07:37:34] <lifeless> OTOH since you dont'want ABI specific packages, you could do this

[07:37:40] <lifeless> download -d --no-binary;

[07:37:54] <lifeless> pip wheel --no-index -f $downloaddir

[07:38:22] <lifeless> then you'll have both the sources and your local wheels and can get both worlds

[07:38:26] <lifeless> or something like tht

[07:38:28] <ronny> lifeless: i want all wheels i have on pypi

[07:38:30] <lifeless> perhaps two download runs

[07:38:51] <lifeless> just offering ideas for workarounds; I recognise the issue

[07:39:06] <ronny> for now im just going with creating a virtualenv with pip 6.x

[08:12:04] <straycat_> lifeless, Okay in this case I'm not feeding the output into a package repo, I just need to, pretty much, obtain a set of tarballs for a given package and its dependencies

[08:13:09] <straycat_> Still, in the context of a --list-dependencies option for pip I can see why my approach is faulty

[13:32:34] <ronny> dstufft: can we quckly get a mechanism to opt out of the automatic wheel building and its cache?

[13:34:03] <dstufft> besides --no-cache-dir and/or --no-binary?

[13:34:36] <ronny> dstufft: bascially i want to have pip install --download work like it did before

[13:35:07] <ronny> dstufft: i used that to collect the packages for networkless installs on linux boxes

[13:35:38] <ronny> however now i get prebuilt wheels instead of the source packages

[13:35:55] <dstufft> ronny: --no-cache-dir should do that

[13:36:57] <ronny> dstufft: as far as i understood it disables all http caching?

[13:37:13] <dstufft> yes

[13:37:39] <ronny> hmm, i guess i'll have to cope with that

[13:48:04] <ronny> dstufft: what env var will prevent it, i did set PIP_NO_CACHE to 1 but that doesnt help

[13:48:20] <dstufft> PIP_NO_CACHE_DIR=1 I think

[13:50:24] <ronny> dstufft: now pip gives me a exception

[13:50:35] <dstufft> what's the except

[13:52:37] <ronny> dstufft: http://paste.pound-python.org/show/vZf4kze7rXLJMp5t90Pw/

[13:57:20] <dstufft> ronny: try PIP_NO_CACHE_DIR=off

[22:21:06] <lifeless> dstufft: morning

[22:21:14] <dstufft> lifeless: hiya

[22:21:37] <lifeless> dstufft: I presume you saw my chat w/ronny about caching :)

[22:21:42] <lifeless> dstufft: but I'm here to pester you about 7.1

[22:22:14] <dstufft> for -c?

[22:22:24] <lifeless> yeah

[22:22:35] <lifeless> if you can tell me what other stuff you'd like in 7.1

[22:22:43] <lifeless> I can see about rounding up some patches

[22:24:47] <dstufft> gimme a few to page out of messing with openshift and switch over to pip

[22:26:35] <lifeless> ok

[22:26:38] <lifeless> thanks!

[22:27:02] <lifeless> when you do, you can eyeball https://review.openstack.org/#/c/189934/ to see what I'm doing for now ;)

[22:45:41] <Nakato> dstufft: Would it be possible to get this in with 7.1? https://github.com/pypa/pip/pull/2894 And if there's anything I can do to help with that

[22:51:15] <dstufft> Nakato: Not that I don't trust you, but I generally pull those in myself just because Id on't have a reasonable way to review that code other than to just repeat the steps and then diff the directories- and since they tend to be largeish code drops in code bases I'm not as familar in, it's easier to sneak in some bad code

[22:53:48] <Nakato> Yea, it's about 500 new lines from that pull. Feel free to close it and do it yourself, I'm just looking to get the fix from https://bitbucket.org/pypa/setuptools/issue/380/environment-markers-dont-support-etc in.

[22:54:43] <pmxbot> jaraco pushed 2 commits to setuptools (https://bitbucket.org/pypa/setuptools/) :

[22:54:43] <pmxbot> Added tag 18.0 for changeset 0a49ee524b0a

[22:54:43] <pmxbot> Bumped to 18.1 in preparation for next release.

[22:59:00] <dstufft> lol

[22:59:10] <dstufft> jaraco *shakesfist*

[22:59:14] <dstufft> all dem releases

[23:03:25] <dstufft> Nakato: https://github.com/pypa/pip/pull/2924

[23:09:15] <Nakato> dstufft: Awesome *looks*

[23:10:48] <dstufft> does anyone here know about Jython? /cc jimbaker

[23:10:56] <dstufft> dunno if jim is around or not :)

[23:50:24] <Nakato> dstufft: A big thank you!

[23:53:32] <tchaypo> I’m still slightly scared that I have commits on pip

[23:53:45] <tchaypo> even tiny single-character commits to the docs

[23:56:45] <dstufft> tchaypo: imagine how I feel

[23:56:51] <dstufft> Nakato: np

[23:59:05] <tchaypo> dstufft: but you’re The Dstufft, surely it’s just normal to a super-human like you?

Log file Viewer

Help | Karma | Search:

#pypa-dev logs for Tuesday the 23rd of June, 2015