Log file Viewer
#pypa-dev logs for Tuesday the 23rd of August, 2016
[19:16:26] <gnychis> could somebody here please explain to me how files_url is supposed to work in warehouse? I'm getting infinite redirects because routes.py routes "/packages" to files_url which points to FILES_BACKEND, which then in turn redirects to "/packages" again
[19:19:57] <gnychis> dstufft: in docker-compose.yml, there is the FILES_BACKEND configuration parameter which looks like this: "FILES_BACKEND: "warehouse.packaging.services.LocalFileStorage path=/app/data/packages/ url=http://<hostname>/packages/{path}" (fill in hostname)
[19:20:53] <gnychis> dstufft, then in warehouse/routes.py there is a route for "/packages" which looks like this: config.add_redirect("/packages/{path:.*}", files_url, domain=warehouse)
[19:21:54] <gnychis> so when I go to pip install something, I get a 301 status code, and says the file has moved, because I think the request is bouncing back and forth with that redirect
[19:22:22] <gnychis> the route comes in, which redirects to the FILES_BACKEND url, which then comes back in and redirects again, etc.
[19:26:50] <gnychis> dstufft: and I try to pip install it via: pip install mytestpackage -i --trusted-host http://172.16.3.28/packages
[19:27:27] <dstufft> gnychis: are you trying to host a copy of Warehouse yourself locally for your own internal uses?
[19:29:19] <dstufft> gnychis: So Warehouse doesn't really support that-- you're likely to be happier with something like DevPI or something, I think right now it assumes that your files and warehouse are hosted on different domains
[19:31:57] <gnychis> dstufft: I see. DevPI seems a little more outdated. Is there a good backend to host the files on that would work well with warehouse? It seems like the code has S3 support in it. But I would think that this kind of local file support should work, maybe it's something we can help contribute to?
[19:32:48] <dstufft> gnychis: you can just give it a different hostname I think, it uses the domain to differenate, but S3 is what production deploys using, the LocalFile backend only exists for development work
[19:33:52] <gnychis> dstufft, okay, we can try to give a different hostname and see if that allows us to make some progress with it.
[19:36:23] <dstufft> gnychis: FWIW the main reason we don't really support this is mostly to reduce maintenance burden of having to maintain multiple different ways of doing something except where aboslutely needed for development's case. You can also expect things to not really work correctly in some cases unless you're also using the VCL in the repo (maybe with Fastly, or maybe any old Varnish would work), we also don't have any focus on backwards compatabilit
[19:36:23] <dstufft> y of _running_ Warehouse itself, except for what we personally need to deploy it
[19:48:30] <gnychis> dstufft, I see... okay, all of that is understandable. I just did some digging around and found your VCL
[19:55:09] <gnychis> dstufft, when you say: "you can just give it a different hostname" ... if the redirect still exists in routes.py and the two different domains still point to the same host (IP), won't we still get this infinite redirect? basically, we are trying to avoid the redirect and just get the server to respond with the LocalFileStorage response
[19:55:39] <gnychis> and sincerely, thanks for your responses, because we've been beating our head off of this one for a few days now
[19:56:40] <gnychis> I'm thinking we would change the redirect to simply be a route to LocalFileStorage?
[19:57:11] <dstufft> gnychis: you'll need a hostname for warehouse (let's call it warehouse.local) and then you need another hostname for files (let's call it, filestorage.local), you would set WAREHOUSE_DOMAIN=warehouse.local and FILES_BACKEND: ... url=https://filestorage.local"
[19:57:29] <dstufft> then warehouse should only serve the redirect at http://warehouse.local/packages/
[19:57:44] <dstufft> and you'll need to setup nginx or something to serve the files at filestorage.local/packages/
[19:59:56] <gnychis> dstufft, okay, that's understood. We can have nginx serve the files under a different domain