PMXBOT Log file Viewer

Help | Karma | Search:

#pypa logs for Thursday the 8th of January, 2015

(Back to #pypa overview) (Back to channel listing) (Animate logs)
[01:13:21] <prologic> So it's no longer possible to configure the location of pip's cache on disk and to have s shared cache?
[01:32:18] <prologic> hmm
[01:32:38] <prologic> this forces us to use a local pypi cache in that case if we want to share a "cache" amongst multiple users
[01:42:52] <prologic> Is there a sensible way to share pip's cache (without installing/using some kind of pypi cache?)
[01:42:59] <prologic> In pip >= 6.0
[02:14:25] <dstufft> prologic: --cache-dir ?
[02:15:05] <prologic> really?
[02:15:05] <prologic> hmm
[02:15:09] <prologic> lemme see :)
[02:15:12] <prologic> I must have missed that
[02:15:57] <prologic> --cache-dir <dir>
[02:15:57] <prologic> Store the cache data in <dir>.
[02:15:59] <prologic> Ahh awesome :)
[02:16:10] <prologic> so a "explicit cache dir" hasn't gone away
[02:16:11] <prologic> *phew* :)
[02:18:11] <dstufft> prologic: nah, we just changed the name to reflect the fact it's a generic ache dir and not just a download cache dir, and set it by default
[02:19:45] <prologic> sure
[02:19:48] <prologic> makes total sense :)
[02:19:57] <prologic> had me worried there a bit wrt pip 6.x :)
[06:40:18] <mgedmin> wsanchez, pip freeze > requirements-stable.txt is the thing to take all the installed packages (and their versions)
[06:40:46] <mgedmin> if you create a fresh empty virtualenv and pip install your package into it, you'll get exactly all the dependencies
[11:55:04] <bersace> Hi, how to check version comparison used by pip ?
[12:33:00] <pf_moore> Before I start hacking a new pip subcommand (and having my brain explode because the finder) is there really no way to ask pip "if I say pip install foo, what will you download?" - ignoring dependencies so we don't need to actually download and run setup.py
[12:33:47] <pf_moore> Some questions I commonly ask - what version will I get and will it be a wheel or a sdist
[12:35:12] <mgedmin> 'pip install -d /tmp' will, download the package but not install it
[12:35:20] <mgedmin> I've no ideas if it'll consider dependencies or not
[12:36:48] <pf_moore> hmm, yeah. Seems a bit of a waste to download it just for a query...
[12:39:17] <pf_moore> I feel the need for "pip query REQs" - displays project name, version, filename, download source. Might give it a go.
[12:39:59] <mgedmin> I missed pip's equivalents of apt-cache show/apt-cache policy a couple of times before
[12:46:37] <dstufft> pf_moore: I thought about doing a --dry-run flag on pip install, but the dependency thing is an issue :(
[12:47:35] <pf_moore> dstufft: Yeah, but I want to decouple it from install - that's got far too much other complexity
[12:48:01] <ionelmc> sounds to me like this should be a pypi feature
[12:48:08] <ionelmc> being able to query metadata ...
[12:48:15] <mgedmin> well, warehouse will store install_requires and extras_require in pypi metadata, right? eventually ;)
[12:48:20] <dstufft> pf_moore: I think it depends on what the question you're attempting to answer is
[12:48:21] <ionelmc> without downloading the actual stuff
[12:48:33] <dstufft> mgedmin: pypi already does that if you upload that information
[12:48:39] <pf_moore> Basically I often find myself wanting to ask "will pip install get me a wheel or a sdist?" If a sdist, I want to do pip wheel first
[12:48:40] <dstufft> generally that means you used twine to upload a wheel
[12:49:13] <dstufft> pf_moore: that sounds like a pip install --dry-run question to me, because otherwise you're going to be duplicating logic I think?
[12:50:26] <pf_moore> Well, if it produced a report like pip list does rather than the install output crap that I'd have to parse (I want to put this in a utility script)
[12:51:07] <pf_moore> Basically, I'm still trying to find a way to maintain a local wheel cache that doesn't involve doing it manually.
[12:51:18] <dstufft> pf_moore: there's an open issue about doing this in a format that chef and friends can use to see what a pip install would install
[12:51:26] <dstufft> sounds like similar use cases
[12:51:45] <pf_moore> pip wheel does a lot of it automatically, but doesn't tell me "the project's started uploading wheels now so you can stop"
[12:51:57] <pf_moore> dstufft: yes, it does
[12:52:36] <mgedmin> hm, I thought pip wheel foo was a no-op when foo was already a wheel
[12:52:50] <pf_moore> It downloads the wheel
[12:53:07] <pf_moore> (or did it stop doing that? Maybe I'm out of date there). Off to test.
[12:53:35] <pf_moore> No, it still downloads the wheel.
[12:54:01] <mgedmin> yeah, it copies the .whl file from my download cache to my wheel cache
[12:54:30] <mgedmin> I misunderstood "Skipping virtualenv, due to already being wheel" as "not doing anything at all, ,honest, guv"
[12:54:40] <pf_moore> Basically, we've got pip list and pip show that work on what's installed locally. I want something similar for what's *not* yet installed. And pip search is basically useless because it does a fuzzy search not a name lookup
[12:55:03] <dstufft> pf_moore: it's possibly you just want a pip wheel --lazy or something
[12:55:18] <dstufft> or pip wheel --skip-existing
[12:55:19] <dstufft> or something
[12:55:32] <mgedmin> sometimes I want 'pip, what version of foo do I have and is it the latest version of foo on PyPI or should I consider upgrading?'
[12:55:48] <pf_moore> dstufft: maybe - I might try to write that and then see if I no longer care about the rest :-)
[12:56:19] <pf_moore> mgedmin - that's pip list --outdated isn't it?
[12:56:28] <mgedmin> and sometimes I want 'pip, download the source tarball of the latest foo from pypi into the current working directory because I'm about to build a debian package out of it'
[12:56:38] <dstufft> mgedmin: pip list -o ?
[12:56:46] <dstufft> er
[12:56:50] <dstufft> I should read the rest of messages
[12:56:53] <mgedmin> --outdated doesn't answer the "which version I currently have" if it's also the latest on pypi
[12:57:26] <[Tritium]> Silly question, isnt the new pypi infrastructure supposed to make it so metadata can be fetched seperate from the package itself?
[12:57:26] <dstufft> seems like it could without much effort maybe
[12:57:33] <mgedmin> 'pip search foo' almost answers both questions, but it takes a while and produces a bunch of irrelevant matches
[12:57:51] <dstufft> [Tritium]: yes and no. the problem is that for sdists we simply don't have that information
[12:58:07] <dstufft> it's locked inside of a setup.py file that PyPI can't execute for security reasons
[12:58:39] <[Tritium]> that should be something that...twine...sends along with the sdist
[12:58:40] <mgedmin> 'setup.py register' could upload that info while it's uploading long_description etc
[12:58:53] <dstufft> and even if we could execute it, people put conditionals in their setup.py so it might only be valid for the platform we run, or the python we run, or it might be a randomized dependency list and it's only valid for one particular run
[12:59:14] <dstufft> the long term answer is sdist 2.0
[12:59:27] <mgedmin> conditionals in setup.py are a Bad Idea for many reasons -- I know since I use them extensively :)
[13:00:12] <[Tritium]> is there a document somewhere of what the new metadata standard is/will be?
[13:00:19] <mgedmin> I'll stop as soon as I'm sure the alternative syntax (extras_require={':somethingcomplicated': [...]}) is supported sufficiently well out in the real world
[13:01:02] <mgedmin> the best part is where detox races running setup.py egg_info on all the python versions at the same time, and then parses that egg_info that it (or some other python) wrote
[13:01:37] <dstufft> [Tritium]: PEP 426 is the first place to look
[13:02:57] <bersace> What is the best way to release an unofficial version of a project ?
[13:03:11] <bersace> for the version numbre
[13:03:25] <bersace> What is the best way to version an unofficial version of a project ?
[13:03:30] <dstufft> bersace: can you depend on pip 6+ and setuptools 8+?
[13:06:06] <bersace> nope
[13:07:07] <bersace> i tried X.Y.mycompanyZ, but pip prefer the original
[13:07:23] <[Tritium]> does --pre help?
[13:07:24] <bersace> and i don't know how to test version comparison
[13:07:36] <dstufft> bersace: pkg_resources.parse_version()
[13:07:43] <bersace> ok
[13:08:03] <bersace> but pip does not use the correct one
[13:08:29] <bersace> In [30]: V('0.7.3') < V('0.7.3.novapost1')
[13:08:32] <bersace> Out[30]: True
[13:08:35] <bersace> but pip installs 0.7.3
[13:08:55] <mgedmin> what I would do is pin the version explicitly
[13:09:07] <mgedmin> (pin all the versions explicitly, actually, for reproducible deployments)
[13:09:14] <dstufft> bersace: what version of pip are you using and what version of setuptools
[13:09:29] <bersace> m$ pip --version
[13:09:29] <bersace> pip 1.5.6 from /home/nova/.virtualenvs/rh2/local/lib/python2.7/site-packages (python 2.7)
[13:09:33] <[Tritium]> bersace: what mgedmin said, or use a repo url in a requirements spec
[13:09:53] <dstufft> if you're using pip 1.5 you probably need --pre, but pinning is probably the real right answer here
[13:09:57] <bersace> i use pip freeze to generate the requirements for deployement
[13:10:14] <bersace> pip freeze -r source.pip | sed … > prod.pip
[13:10:21] <bersace> after test success
[13:10:46] <[Tritium]> i like how pip skipped 2-5
[13:11:03] <bersace> dstufft: how to check setuptools versions ?
[13:11:13] <bersace> i don't find the distribution in pip freeze
[13:11:13] <dstufft> bersace: pip list
[13:11:24] <bersace> setuptools (3.6)
[13:11:30] <bersace> dstufft: thx
[13:12:06] <dstufft> [Tritium]: heh, we just dropped the leading 1
[13:12:14] <bersace> :)
[13:13:44] <mgedmin> ok, if you have a unpinned requirement list and you're using a CI to freeze the versions if they past test
[13:14:02] <pf_moore> dstufft: re pip wheel --lazy, the download happens in requirement_set.prepare_files, which is *way* too messy to pick out a "don't bother downloading existing wheels" codepath
[13:14:15] <mgedmin> how about adding 'thispackage >= 0.7.3.novapost1' to your requirements.txt, so pip will be forced to consider the "prerelease" version?
[13:14:35] <mgedmin> uh, I should maybe double-check if pip 1.5 will do that or if it'll fail with a "no released version found matching requirement ..."
[13:14:58] <pf_moore> dstufft: You made me read the requirements code again, now my ears are bleeding.
[13:15:07] <dstufft> pf_moore: sorry :(
[13:15:43] <pf_moore> np, I should know better - "here be dragons", or is it "abandon all hope, ye who enter"?
[13:17:30] <dstufft> pf_moore: little bit of column A little bit of column B
[13:17:54] <xafer> fwiw, I've some waiting PR around prepare_files that try to simplify it (but small steps)
[13:18:23] <pf_moore> xafer: cool
[17:45:10] <AlexMeng> is there a formal process for transferring ownership of a pypi package?
[17:46:32] <dstufft> AlexMeng: if the person who currently owns it wants to transfer they just need to add the new person under roles
[18:27:54] <wsanchez> mgedmin: eggdeps is way more useful because I can see why things are in the list. pip freeze just lists everything.
[18:28:05] <wsanchez> Bu yes, pip freeze is also usful
[23:41:13] <gnudestallman> If I use setuptools option, include_package_data = True, how do I access the data when it is installed?