pmxbot IRC Log Viewer

[05:38:49] <dstufft> tdsmith: ping

[05:39:05] <tdsmith> dstufft: howdy

[05:39:30] <dstufft> tdsmith: homebrew python installs pre-compiled binaries right?

[05:39:51] <tdsmith> for cpython itself? yeah, in the general case

[05:39:56] <dstufft> cool

[05:39:56] <dstufft> thanks

[05:40:26] <tdsmith> welcome!

[11:50:46] <nanonyme> Oh, crap, I was supposed to file a bug against something and I forgot what :(

[11:51:42] <mgedmin> if the bug recurs, you'll remember

[11:51:46] <mgedmin> if it doesn't, even better :)

[11:52:54] <nanonyme> Well, it's that thing where uninstalling pypiwin32 repeatedly fails

[11:53:34] <nanonyme> Well, rather results in a bogus failure on cleaning up temporary directory

[14:21:33] <nanonyme> Also why is it that egg->wheel breaks entry-points of scripts *every* frigging time for me :(

[14:24:44] <ronny> nanonyme: what is "egg -> whee" ?

[14:25:13] <nanonyme> egg->wheel

[14:25:23] <nanonyme> Ie wheel convert on an egg

[14:25:49] <ronny> how do you convert?

[14:26:01] <nanonyme> py -mwheel convert foo.egg

[14:26:18] <ronny> i wonder if it turns the egg_info into dist_info propperly

[14:26:21] <nanonyme> When I install the result, the stuff defined in scripts ends up nowhere reasonable

[14:26:36] <ronny> define nowhere reasonable?

[14:26:48] <nanonyme> I suspect it ends up in site-packages but not totally sure

[14:26:59] <nanonyme> I should probably generate a minimal test-case

[14:27:22] <ronny> nanonyme: maybe it is missing alltogether?

[14:29:20] <nanonyme> Yeah, could be that it was just duplicated under site-packages as a coincidence in one of the packages I tried it with

[14:29:54] <nanonyme> Huh

[14:30:12] <nanonyme> No, I think it really does end up installed, just in wrong place

[14:30:32] <ronny> then its not entrypoints

[14:30:32] <nanonyme> There's a site-packages/package-foo.dist-info/scripts, everything is under that

[14:30:46] <nanonyme> Well, it's defined as scripts in the original setup.py

[14:30:51] <ronny> seems broken, are thre actual entrypoints in the package?

[14:31:02] <ronny> a in does the dist info have the entrypoints?

[14:31:04] <ronny> *as

[14:31:24] <nanonyme> I don't know how to verify

[14:32:21] <nanonyme> Is it the metadata.json? That doesn't have anything about scripts at any rate

[14:32:55] <nanonyme> Time for a minimal test-case :)

[14:34:25] <ronny> nanonyme: when i make a normal wheel package, there is a entrypoints.txt in the dist-info

[14:34:43] <ronny> nanonyme: if that is missing, you got your reason for no sane scripts

[14:37:19] <ronny> nanonyme: yup, its missing entrypoints

[14:38:58] <nanonyme> The egg doesn't appear to have any metadata about scripts either

[14:39:06] <nanonyme> It just has a directory scripts and everything under that

[14:39:25] <nanonyme> I guess convert from wheel should dynamically create the metadata then?

[14:39:46] <ronny> nanonyme: sounds like the egg is a mess to begin with

[14:39:52] <nanonyme> (I wonder if this has some weird side-effects on what kind of package names you can use with eggs)

[14:40:10] <nanonyme> Yeah, I'd prefer we would not use eggs but I'm having difficulty convincing everyone to migrate away from it

[14:40:27] <nanonyme> If someone can make setuptools drop egg support, please do. Will make my job easier

[14:42:35] <ronny> hmm, wont happen easily

[14:44:05] <nanonyme> I know, hence why I'm trying to have automatic .egg->.wheel conversions so I won't at least have to suffer easy_install

[14:46:29] <ronny> well, for those packages, teach them entrypoints

[14:47:45] <nanonyme> You mean teach wheel to automatically translate everything under scripts directory as entrypoints :)

[14:48:00] <nanonyme> (only for eggs)

[14:50:19] <nanonyme> ronny, wait, I just build a bdist_wheel, it didn't have entrypoints.txt either

[14:51:03] <nanonyme> So can't work like you described

[14:51:10] <ronny> nanonyme: does the setup.py have entrypoints?

[14:51:33] <nanonyme> Yes

[14:51:37] <ronny> nanonyme: im not talking about automatic, im talking about snd patches for entrypoints upstream

[14:52:22] <ronny> nanonyme: i jsut made a wheel for pytest, it has a entrypoints text file

[14:52:40] <nanonyme> When I create bdist_wheel directly, scripts is put under .data. When I convert, it puts scripts under .dist-info

[14:52:52] <nanonyme> In neither case is there an entrypoints.txt

[14:53:03] <ronny> sounds wrong

[14:53:10] <ronny> where is that setup.py?

[14:53:55] <nanonyme> Just a sec, I'll add this crap in a public repo

[14:56:08] <nanonyme> ronny, https://github.com/nanonyme/wheel-test-case

[14:58:10] <ronny> nanonyme: yup, that cant work good

[14:58:18] <ronny> thats not a entrypoint

[14:58:35] <ronny> and the script is not entrypoint compatible

[14:58:54] <ronny> nanonyme: afair wheel has a install scripts command for that

[14:59:06] <nanonyme> Well, that's how 99% of packages work like that

[14:59:42] <ronny> nanonyme: well, the new correct way is entrypoints, which are not scripts ^^

[14:59:56] <ronny> and any well maintained python package i know uses them

[15:01:03] <nanonyme> ronny, also when I roll bdist_wheel out of that and install it, it works fine

[15:02:08] <nanonyme> Every single package I've seen out there uses scripts, not entrypoints

[15:02:46] <ronny> well, then you are in trouble ^^

[15:05:53] <nanonyme> You mean everyone out there is in trouble ;P

[15:06:40] <nanonyme> Why was entrypoints introduced in the first place and can you have both scripts and entrypoints defined without things breaking?

[15:06:42] <tomprince> ronny: twisted doesn't.

[15:08:11] <nanonyme> Okay, entrypoints totally isn't compatible with eggs, they just break

[15:08:21] <ronny> huh?

[15:08:33] <nanonyme> Or then I'm using the syntax wrong

[15:08:52] <ronny> nanonyme: how do you use the syntax?!

[15:08:59] <nanonyme> Yes, I am using it wrong

[15:09:34] <nanonyme> entry_points looks over-complexified to me tbh

[15:10:01] <ronny> nanonyme: entryppoints ensure nice executable names, windows support, windows gui support and more

[15:10:56] <ronny> tomprince: yes, and they made various messses because of that

[15:10:57] <tomprince> They also require pkg_resources at run-time.

[15:11:01] <ronny> (see the plugin namespace)

[15:11:20] <tomprince> What?

[15:11:37] <ronny> the twiested plugin namespace is one of the twisted packaging messed

[15:12:01] <tomprince> Plugins predate any alternative.

[15:14:03] <ronny> tomprince: and they will never be fixed

[15:17:40] <tomprince> It isn't clear that it needs to be.

[15:19:11] <StGambrinus> Anyone know if there's a way to get pip to actually ignore SSL errors? I've tried --trusted-host pypi.python.org install -i http://pypi.python.org/simple --allow-unverified and it still dies with an SSL verification failure.

[15:20:20] <ronny> StGambrinus: how the hell do you get ssl issues from pypi ?

[15:20:54] <StGambrinus> My guess: I'm stuck on OS X

[15:21:11] <ronny> what does your browser say?

[15:22:15] <StGambrinus> Chrome says green light, identity verified Python Software Foundation [US]

[15:22:54] <skay> hi all, I'm having trouble installing something from pypi with pip. the project has the versions I'm looking for, but pip doesn't find them. http://paste.ubuntu.com/10516055/

[15:22:56] <ronny> hmm, i see

[15:23:09] <ronny> StGambrinus: then unfortunately im out of ideas

[15:23:14] <skay> I have a workaround, but I want to understand what is going on. if the project needs to update how they upload to pypi I'd like to fix it

[15:23:59] <StGambrinus> ronny: can't really complain I guess - so am I

[15:24:25] <ronny> skay: they no longer upload actual tarballs, thats why modern pip wont find it

[15:24:44] <ronny> skay: make tem upload modern versions or user --allow-external

[15:25:05] <skay> ronny: when I tried --allow-external (scroll to bottom of pastebin) pip didn't seem to honor it

[15:25:20] <skay> ronny: and suggested that I use --allow-external

[15:25:39] <ronny> skay: their download link is also broken and no longer shows certain versions

[15:25:56] <ronny> skay: bascially they completely screwed up

[15:26:15] <skay> ronny: heh. okay, I'm a packaging newbie, but I'll try to figure it out and make a mr

[15:26:24] <StGambrinus> ronny: here's the exact blow-by-blow (less the stack trace) in case it inspires anything: https://dpaste.de/1KU0

[15:27:38] <ronny> skay: basically they are supposed to either supply correct download links (which they dont) or supply tarballs hosted on pypi (which they also dont)

[15:27:54] <ronny> skay: to work the best way, they should put tarballs onto pypi

[15:28:20] <ronny> but canonical has a history of not following sane standards for being what they are ^^

[15:31:12] <skay> ronny: I'm passing along the info to the launchpad team

[15:31:57] <skay> ronny: thanks for the quick help, I wasn't sure what was going on.

[15:32:55] <skay> ronny: I might open an issue with pip to see if there is another error message that could happen when a project has broken download links. the message that suggests using --allow-external when the invokation has allow-external already isn't the best message

[15:33:24] <ronny> yes, that could use improvement

[15:33:32] <ronny> please open an issue

[15:37:29] <ronny> bbl

[15:38:35] <StGambrinus> so... there is no way to get pip to stop verifying SSL certificates, suggestively named options or not?

[15:48:17] <nanonyme> I thought there was a switch for that

[15:48:35] <nanonyme> But yes, do fix the CA's

[15:49:02] <tomprince> --trusted-host I think.

[15:49:36] <nanonyme> The nicer way is to ship the CA and use --cert to point to it

[15:49:55] <StGambrinus> I'm using --trusted-host and --allow-unverified and using http

[15:51:16] <StGambrinus> nanonyme: yes, that's obviously the correct solution, though it will take some googling

[16:10:50] <StGambrinus> Should using --cert result in "No distributions at all"? https://dpaste.de/6bTo

[16:26:31] <StGambrinus> yeah, --cert doesn't work either (even after converting the crt file I get from digicert to pem). It's still refusing. I give up.

[17:12:45] <linovia> mmm, damn pip

[17:13:03] <linovia> since when did it check it is in its latest version ?

[17:14:31] <linovia> it's a very terrible thing to do because the check won't be against the private pypi which takes a lot of time when the internet access is restricted

[17:16:24] <ronny> linovia: can you report a bug that the version check des not honor the index server?

[17:17:12] <linovia> ronny: sure

[17:17:25] <linovia> going back home and I'll open it

[19:10:36] <dstufft> linovia: you can disable it, and it shouldn't retry and it should have a short timeout

[19:10:44] <dstufft> assuming you're on 6.0.8

[19:13:12] <linovia> dstufft: how would I do that ?

[19:13:36] <dstufft> --disable-pip-version-check or PIP_DISABLE_PIP_VERSION_CHECK=1

[19:13:42] <dstufft> or similar in the conf files

[19:13:46] <linovia> dstufft: thanks !

[19:15:09] <dstufft> not hitting a private index was a conscious choice, we tried to limit the negative effects of not being able to hit PyPI by not allowing it to retry, making it fail silently, and giving it a short timeout, and allow it to be disabled. but the idea is that we want people to be aware there is a newer version available, regardless of if they've uploaded it to their private index or not

[19:18:26] <linovia> dstufft: the thing is, at my client's office, they don't have the dns resolution which means any access to pypi times out after ~30 s :(

[19:19:06] <dstufft> linovia: hmm, I guess we need to handle a DNS timeout better

[19:19:07] <linovia> (They are using a proxy to access outside)

[19:26:43] <nanonyme> dstufft, btw, pip gives *very* poor output on errors without verbose mode

[19:27:06] <dstufft> nanonyme: what kind of errors

[19:27:29] <nanonyme> Eg that classic error where you try to connect to PyPI and SSL handshake fails because of too old SSL library

[19:27:39] <dstufft> right

[19:27:46] <nanonyme> pip just says no distributions found and you don't get a clue of what goes wrong without --verbose

[19:27:58] <mitsuhiko> why does install --download need to build sometimes?

[19:28:14] <dstufft> PEP 470 makes it possible for us to hard fail on broken links

[19:28:24] <dstufft> right now we don't have good indication if a link should or should not br broken

[19:28:27] <mitsuhiko> i have the problem that i use it to download tarballs and wheels, but apparently i need to be running that as the user that installed pip which i am intentionally not :-/

[19:28:54] <nanonyme> dstufft, imo it should loudly fail if index server can't be connected to at all

[19:29:08] <dstufft> mitsuhiko: it shouldn't, but it does run setup.py egg_info which means it could be doing anything in there (particularly setup_requires)

[19:29:18] <nanonyme> But reading PEP 470

[19:29:36] <mitsuhiko> dstufft: Executing /srv/fireline-builder/venv/bin/pip install --download /tmp/tmpuMIHXK-venv-tmp virtualenv

[19:29:38] <mitsuhiko> that's what i execute

[19:29:51] <mitsuhiko> then it executes this:

[19:29:51] <mitsuhiko> File "/srv/fireline-builder/venv/local/lib/python2.7/site-packages/pip/req.py", line 1527, in _make_build_dir

[19:29:59] <mitsuhiko> which fails with

[19:29:59] <mitsuhiko> OSError: [Errno 13] Permission denied: '/srv/fireline-builder/venv/build'

[19:30:19] <nanonyme> dstufft, ie even if one of the index servers can be contacted to for downloading the content, it's still in most cases useful to know if some others are unreachable

[19:30:35] <dstufft> oh

[19:30:36] <nanonyme> (since you might own those indexes)

[19:30:59] <dstufft> mitsuhiko: it uses the build dir to unpack the thing so it can run setup.py egg_info

[19:31:06] <mitsuhiko> i see

[19:31:14] <mitsuhiko> i would use --build but that seems deprecated :-

[19:31:32] <dstufft> mitsuhiko: what version of pip?

[19:32:15] <mitsuhiko> looks like 1.5.4

[19:32:38] <nanonyme> 1.5.4 sounds pretty old at this point :)

[19:32:42] <mitsuhiko> not sure why it uses this

[19:32:49] <dstufft> mitsuhiko: also in pip 6.0 we undeprecated --build and --no-clean because we realized it was over zealous to deprecate them

[19:33:15] <nanonyme> Btw, why not just use a temporary directory through tempfile for building?

[19:33:20] <dstufft> if you upgrade to pip 6.0.8 though --download wil use a randomized build directory

[19:33:32] <mitsuhiko> ah, because fucking debian

[19:33:37] <nanonyme> You can control that through TEMPDIR

[19:34:27] <mitsuhiko> i have everything about system pip interfering with my things

[19:34:39] <nanonyme> dstufft, considered trying to negotiate freeze exceptions for pip on various distros?

[19:35:06] <dstufft> nanonyme: I'm still trying to negotiate for them not to break things randomly

[19:35:09] <nanonyme> Hehe

[19:35:30] <dstufft> on RPM based systems you can get unscrewed with pip/setuptools/virtualenv/wheel via COPR

[19:35:35] <nanonyme> dstufft, was just wondering since many distros pull eg newest version of Firefox even when there's API/ABI guarantees

[19:35:50] <dstufft> I don't have something similar setup for debian yet

[19:35:54] <nanonyme> Same would imo make sense with pip. It changes often

[19:36:11] <dstufft> nanonyme: it's not a bad idea, I just don't know if I have the motviation to die on that hill yet

[19:37:17] <nanonyme> dstufft, do you think you'll end up unvendoring and forking pip's requirements, btw? Like, say, urllib3 and folks

[19:37:31] <nanonyme> I heard someone comment it might satisfy Debian

[19:37:38] <dstufft> unlikely, we explicitly did not want to fork them

[19:37:53] <dstufft> Debian has a better fix now

[19:38:02] <nanonyme> Hmm, what kind of a fix?

[19:38:19] <dstufft> well not better than vendoring (imo), but better than just unvendoring

[19:38:58] <dstufft> they generate .whl files for all of pip's deps, and they place them in /usr/share, and their pip is modified so that ti places those first on sys.path

[19:39:13] <dstufft> so it doesn't matter what version of foo you have installed, pip is going to internally import them from those wheels

[19:39:23] <nanonyme> Wait wait wait, will we end up digging ourself back into the pit with PEP470 where people publish things into public but personally managed sites and then we lose those projects from pip completely once the sites die? :/

[19:40:15] <dstufft> for people who want to host things externally we're alrady in that pit, just it's implicitly done by directly linking to files on PyPI

[19:40:25] <dstufft> this just makes the lines clearer

[19:40:44] <nanonyme> I was hoping we could just eventually nuke all such projects

[19:40:56] <nanonyme> Or convince them to publish to PyPI

[19:41:40] <nanonyme> Oh, okay, that Debian's solution sounds reasonably sensible

[19:42:12] <dstufft> there are always going to be people for one reason or another don't want to host on PyPI, and that's fine. We have tooling to enable it. Not hosting on PyPI limits how friendly it is for peolpe to install things from them, so I don't think people will take it lightly

[19:43:08] <nanonyme> Insofar I've seen many don't see PyPI as worth the effort and just put source tarballs in random places like Google code

[19:43:45] <nanonyme> Rather than facilitating them keeping the tarballs there imo PyPI publishing should be made at least as simple as publishing into personal sites :)

[19:44:33] <nanonyme> dstufft, did I already comment to you how I think PyPI publishing really needs to be separated from distutils so it can be properly upgraded?

[19:45:00] <dstufft> nanonyme: see twine (and eventually pip upload)

[19:45:21] <nanonyme> Hmm

[19:46:09] <nanonyme> Okay, guess twine solves the problem already then?

[19:48:16] <nanonyme> Hrm, I apparently can't make twine documentation

[19:49:00] <nanonyme> Ahh, figures, I was just missing deps

[19:51:17] <nanonyme> Ugh, that totally did not work

[19:52:40] <nanonyme> dstufft, you own twine, right?

[19:52:55] <nanonyme> How about making its doc-building play nice with readthedocs? ;)

[19:53:26] <nanonyme> Or does it actually *have* any documentation?

[19:55:13] <nanonyme> dstufft, basically my grutch with distutils PyPi handling is there is no way at all to prevent it from trying to send to public PyPI by default. That plus bugs

[19:56:12] <dstufft> um

[19:56:20] <dstufft> I don't think it has any real docs besides the readme

[22:04:40] <StGambrinus> Still trying to convince pip to allow me to install numpy and it's still failing with a certification verification error. I've tried --trusted-host, -i http://..., --allow-unverified and supplying the digicert root cert using --cert. Ideas?

[22:08:08] <dstufft> StGambrinus: pastebin the output?

[22:11:21] <StGambrinus> dstufft: Ok. https://dpaste.de/6bZH

[22:12:51] <dstufft> your problem is the fact that scipy uses setup_requires, and setup_requires gets handled by setuptools not pip

[22:14:21] <dstufft> I'm not sure if easy_install provides a way to work around that

[22:15:16] <StGambrinus> So it's entirely unrelated to the SSL errors?

[22:15:32] <dstufft> No, the SSL errors are coming from easy_install

[22:15:54] <dstufft> pip is able to successfully download numpy and scipy, and then it attempts to install scipy

[22:16:26] <StGambrinus> oh, that would explain why no matter what I throw at pip I get that error.. ok. thanks.

[22:16:35] <dstufft> and when doing ``python setup.py install`` for scipy, setuptools takes the setup_requires that scipy specifies and hits the network to try and pull it down

[22:16:49] <dstufft> a work around would be to install the things scipy needs first, before installing scipy

[22:17:21] <dstufft> (setuptools will see it's already installed and not try to hit the network)

[22:18:34] <StGambrinus> yes, will do. if nothing else it at least *feels* like progress.

[22:19:05] <nanonyme> dstufft, sounds like the setup_requires logic needs some rework

[22:19:21] <dstufft> there are ~plans~ for pip to take over the setup_requires when we invoke it

[22:19:31] <dstufft> but nobody's figured out how to do it yet

[22:20:10] <nanonyme> And making pip a shim that uses libpip

[22:31:55] <StGambrinus> I must say I find the whole installation layout hilariously avantgarde on OS X and one of these days I need to find all the python executables on my system. Anyway, thanks again.

[22:32:45] <doismellburning> StGambrinus: heh yes, py34 doesn't work for me and I think it's because my install is using half of System and half MacPorts or something

[22:32:57] <dstufft> <3's my pyenv

Log file Viewer

Help | Karma | Search:

#pypa logs for Tuesday the 3rd of March, 2015