PMXBOT Log file Viewer

Help | Karma | Search:

#pypa logs for Tuesday the 19th of June, 2018

(Back to #pypa overview) (Back to channel listing) (Animate logs)
[02:09:10] <FunkyBob> back again! :)
[02:09:34] <FunkyBob> am working to try to simplify some install stuff for a client... who uses cython a lot
[02:10:09] <FunkyBob> some things can't be cythonised [e.g. task entry points for celery... apparently just doesn't work]
[02:11:10] <FunkyBob> so currently we have a custom-subclass of Cython's build_ext that copies over certain stuff "manually' [using shutil] into build_lib
[02:11:52] <FunkyBob> but I'm wondering why this should even be needed... if I have a valid list of packages, and only some are listed in the ext_modules ... shouldn't regular setup copy the rest over?
[02:12:06] <FunkyBob> or is this a difference between modules, packages, etc....?
[02:31:17] <FunkyBob> ok... have just done some testing, and it seems my predecessors were misled on some of this :)
[02:55:22] <techalchemy> there are a lot of misconceptions about this stuff :p
[02:55:43] <techalchemy> there is not that much actual magic that occurs, only a small amount
[03:15:47] <FunkyBob> perhaps, sure...
[03:19:25] <FunkyBob> am on the cusp of writing my own packaging tool for Python, just so I can learn more of how it all works :)
[03:19:44] <techalchemy> I still don't know how it works
[03:23:01] <FunkyBob> I never feel I appreciate a tool until I've worked to implement my own :)
[12:25:43] <toad_polo> FunkyBob: There is a ton of work that needs to be done on setuptools.
[12:25:48] <toad_polo> If you want to jump right in.
[12:26:24] <toad_polo> I'd be happy to help you get started if practical experience is what you want.
[12:35:16] <FunkyBob> I'll add it to my queue :/
[18:52:16] <norrit> Trying to spin up a local pypa warehouse instance, it's up and running and accepting wheel packages using Twine, but I cannot get pip to install correctly. Currently it's entering a redirect loop as is my curl against the simple url.
[18:52:49] <norrit> I updated the max_upload size in warehouse/forklift/legacy.py and the url in dev/environment.
[18:53:41] <norrit> my .pip/pip.conf looks as follows (IP redacted)
[18:53:43] <norrit> [global]
[18:53:43] <norrit> index-url = http://localhost/simple/
[18:53:43] <norrit> [install]
[18:53:45] <norrit> trusted-host = localhost
[18:54:23] <norrit> I can't determine what kind of redirect is causing this cause it doesn't appear to be http/https
[18:56:29] <di_codes> norrit: what does `pip install -vvv &lt;packagename&gt;` give you?
[18:59:56] <norrit> sec, cleaning up output.
[19:01:07] <norrit> pastebin.com/raw/5jHry9Xa
[19:02:33] <ngoldbaum> redirect loop?
[19:02:41] <norrit> yup
[19:02:47] <norrit> curl exibits the same behavior
[19:03:23] <norrit> if it impacts the troubleshooting I am using linux subsystem on windows 10.
[19:08:12] <norrit> sec, trying with user/pass in config
[19:09:16] <norrit> nope, still seeing it.
[19:09:51] <di_codes> norrit: ah right, so, by default, running warehouse locally doesn’t use any file storage
[19:11:44] <norrit> Odd, did I miss something in the installation documentation?
[19:12:21] <norrit> Let me dig into that a bit.
[19:13:22] <di_codes> er, actually, that’s not quite true. looks like it might be misconfigured at the moment, one minute
[19:14:29] <norrit> FILES_BACKEND=warehouse.packaging.services.LocalFileStorage path=/var/opt/warehouse/packages/ url=http://<ip>/packages/{path}
[19:14:32] <norrit> if that hepls
[19:14:36] <norrit> helps**
[19:22:11] <di_codes> so, in production we have a separate host `<files.pythonhosted.org|files.pythonhosted.org>;` that a request for `<pypi.org/packages/../|pypi.org/packages/../>;...` will get redirected to
[19:22:55] <norrit> ahh, would potentially explain the files.example.com I saw on an earlier error I was getting.
[19:23:26] <di_codes1> but we don’t have that host in development (hence that url being `<files.example.com|files.example.com>;`) so the redirect goes nowhere
[19:23:36] <di_codes1> since you’ve changed it to be the same host, it will just give you a redirect loop
[19:23:42] <di_codes> but we don’t have that host in development (hence that url being `<files.example.com|files.example.com>;`) so the redirect goes nowhere
[19:24:17] <norrit> ah, okay, makes way more sense now. I can probably hack together a workaround on that.
[19:24:50] <di_codes1> looks like we once had the ability to store these files locally via `warehouse.packaging.services.LocalFileStorage`, but it doesn’t seem to be working now
[19:25:02] <norrit> so the dev/environment line for FILES_BACKEND is where that's defined correct?
[19:25:17] <di_codes1> yes
[19:25:19] <norrit> I can use separeate sub domains and a new nginx entry more likely than not.
[19:25:42] <norrit> I'll tinker, thanks for the direction on this.
[19:25:48] <di_codes1> i should take a step back and say: if you’re trying to use warehouse as a local/in-house pypi, there are better options
[19:26:07] <norrit> Yeah?
[19:26:09] <di_codes1> warehouse is really only designed to be pypi, we don’t really support using it that way
[19:26:25] <di_codes1> <pypi.org/project/devpi/>;
[19:26:31] <norrit> Okay, fair enough
[19:27:12] <norrit> I feel like I'm going to be fairly upset when this is much easier than what I was doing.
[19:28:02] <norrit> I feel like I'm going to be fairly upset when this is much easier than what I was doing. << If you missed di_codes
[19:29:09] <di_codes> norrit: depends on what you were trying to do, but yeah, seems like devpi out of the box is approximately what you’re looking for
[19:30:08] <norrit> Much appreciated. If things change I'll determine the best option. Hadn't even crossed my mind that there would be a streamlined testing tool
[20:00:43] <nik> When I'm uploading with `twine upload dist/*`, I get this error: RedirectDetected: "pypi.org/legacy" attempted to redirect to "pypi.org/legacy" during upload. Aborting...
[20:00:54] <nik> this results in my package not updating on pypi...
[20:00:57] <nik> anyone any idea?
[20:02:48] <ngoldbaum> is it possibly due to github.com/pypa/twine/issues/310 ?
[20:02:51] <di_codes> nik: what version of twine? what’s in your `.pypirc`?
[20:05:04] <nik> twine version 1.11.0 (pkginfo: 1.4.2, requests: 2.19.1, setuptools: 39.1.0, requests-toolbelt: 0.8.0, tqdm: 4.23.4)
[20:05:58] <nik> Ive tried both [pypi] repository: pypi.org/legacy and [pypi] repository: pypi.org/legacy in the config file
[20:06:26] <ngoldbaum> that should work, sounds vaguely like you're hitting a bug, but let's wait for di_codes to weigh in
[20:07:26] <nik> using a trailing / results in: HTTPError: 404 Client Error: Not Found for url: pypi.org/legacy
[20:14:48] <nik> my file: gist.github.com/nikvaessen/39d3e61ec78efbcc70302441b4a633f0
[20:14:54] <di_codes> nik: can you just remove the `repository` lines from your pypirc entirely and try?
[20:15:43] <nik> yup that worked
[20:16:10] <nik> thanks :)