pmxbot IRC Log Viewer

[10:10:20] <Furai> Hey is pip website down?

[10:12:26] <Furai> https://pip.pypa.io << it's not working correctly on my end.

[10:12:41] <Furai> Most of the pages are either blank or I just get HTML markup.

[10:19:56] <Riviera> Same here.

[12:19:31] <ztane> how am I supposed to download binary packages with pip download?

[12:19:54] <ztane> it seems it sometimes downloads binaries; often, not.

[12:20:02] <ztane> install -d actually worked :(

[14:44:09] <tuxx> hey guys..

[14:44:56] <tuxx> in order to save bandwidth, i would like to mirror certain pip packages on a local server

[14:45:25] <tuxx> so far i havent found proper tooling for that, does anybody know of anything which would allow me to mirror packages into a local directory?

[14:46:02] <tuxx> pip2pi seems to be mentioned a couple of times

[14:56:46] <tdsmith> tuxx: are you looking for devpi maybe?

[14:57:02] <tdsmith> recent versions of pip cache downloads locally for each user on each machine

[14:58:31] <tuxx> tdsmith: but pip cache wont generate an index for me right?

[14:58:36] <tdsmith> right

[14:58:43] <tuxx> tdsmith: okay, i'm trying devpi

[15:02:06] <tuxx> tdsmith: do i need to run devpi-server for this or can i just create the file structure in the docroot of my apache?

[15:02:59] <tdsmith> dunno actually!

[16:14:32] <ronny> tuxx: devpi will download packages into a strucutre you can serve via apache, for onvenience its suggested to leave devpi running and use the webserver for file delivery

[16:15:27] <tuxx> ronny: it seemes quite complex.. i gave up

[16:15:35] <tuxx> we'll probably just use a squid

[16:16:55] <ronny> tuxx: huh? you jsut run the devpi and set up about 20 lines of apache config

[16:18:33] <tuxx> ronny: "just run devapi"

[16:19:19] <ronny> tuxx: so setting up services is hard for you?

[16:19:32] <tuxx> ronny: if they have so many flags, then yea

[16:19:44] <tuxx> it has some user management and whatnot

[16:19:57] <tuxx> the documentation wasnt very to the point either

[16:19:58] <ronny> user management doesnt matter if you dont make users

[16:21:18] <ronny> http://doc.devpi.net/latest/quickstart-server.html <- imho that one is reasonably to the point (unfortunately im not aware of a apache config example, the nginx one should be easy to translate tho

[16:22:29] <tuxx> ronny: so devpi-server --gen-config <- i dont even want generates a bunch of configs

[16:22:54] <tuxx> and service files, which all seem to be related to devpi-server... i have to have devpi-server running as a daemon?

[16:23:20] <tuxx> why the do i even need a service/daemon to simply statically mirror a bunch of files?

[16:24:48] <ronny> if you want a full static mirror of pypy, and only that, then you might want bandersnatch

[16:24:57] <tuxx> i dont want a full mirror

[16:25:01] <ronny> devpi will actuvely mirror on a as needed basis

[16:25:09] <ronny> do you need something that does the mirroring

[16:25:46] <ronny> the major plus on devpi is, you can make spaces on it where you can upload own internal packages as well

[16:25:51] <tuxx> really... all i want is to allow certain hosts that dont have access to the public internet to install a handfull of pip packages

[16:26:12] <tuxx> ronny: yea that does sound like it could come in handy

[16:26:39] <tuxx> but really like wtf is the supervisor site config

[16:26:45] <tuxx> the documentation doesnt even explain it

[16:26:56] <ronny> tuxx: my approach to shipping packages to non-internet hosts was to use pip to download them on my box and then upload them via scp

[16:27:58] <tuxx> ronny: but dont i need a directory index?

[16:28:08] <ronny> pip has a option to not usa a index

[16:28:16] <ronny> and pip also has a option to look for packages in a folder

[16:29:58] <ronny> tuxx: the genconfig thing generates a bunch of files, one for supervisor, one for systemd, one for upstart, one for apple services

[16:30:04] <ronny> you dont need most of them

[16:30:08] <tuxx> i'm not familiar with supervisor

[16:30:11] <tuxx> but whatever

[16:30:22] <tuxx> i see the systemd service file

[16:30:36] <tuxx> anyways.. aparently i need to start devpi-server and then connect to it via a client

[16:30:51] <tuxx> still dont see how to add packages however

[16:31:38] <tuxx> and i dont think that i can just have it place files transparently in my documentroot and be done with it

[16:31:49] <tuxx> it seems you proxy requests to the devpi-server running on the local machine?

[16:32:49] <tuxx> ronny: any idea how i can feed it a requirements.txt file?

[16:35:22] <dstufft> tuxx: you can just stick some files you download on the FS and let apache/nginx/anything-that-can-generate-an-index serve it

[16:35:31] <dstufft> if you don't want anything fancy

[16:36:06] <tuxx> how can i "stick files"?

[16:36:25] <dstufft> put them in the filesystem

[16:36:25] <tuxx> dstufft: download them with pip or something into a flat folder?

[16:36:29] <dstufft> sure

[16:36:38] <tuxx> dstufft: how will the requirements.txt find the right files then?

[16:36:45] <tuxx> i mean pip install --requirements bla.txt

[16:36:55] <tuxx> i have things like Django==1.9.7

[16:37:01] <dstufft> download some files into a folder, serve that folder, add --find-links=https://yourhost.example.com/path/to/that/folder/

[16:37:22] <dstufft> you can toss a --no-index ont here a well if you don't want it to still look at PyPI itself

[16:37:23] <tuxx> ERROR: unknown command "download"

[16:37:25] <tuxx> pip download Django==1.9.7

[16:37:49] <dstufft> or you can put them in a structure like $ROOT/django/<django files> and use --index-url instead of --find-links

[16:37:57] <dstufft> tuxx: you need a newer pip

[16:38:06] <tuxx> i'd prefer a simple flat structure

[16:39:20] <tuxx> dstufft: when i do "pip downloiad psycopg2==2.6.2" i get:

[16:39:24] <tuxx> Error: pg_config executable not found.

[16:39:46] <dstufft> psycopg2's setup.py requires pg_config to be installed to execute

[16:39:48] <tuxx> dstufft: why on earth is it performing any kind of logic.. i just want it to download, not process/compile/whatever

[16:39:57] <tuxx> dstufft: why is it running setup.py? i just want the data blob

[16:40:11] <dstufft> tuxx: so it can discover psycopg2's dependencies andd ownload them as well

[16:40:21] <tuxx> ugh

[16:40:30] <tuxx> but i dont freaking want pg_config on this host :D

[16:40:50] <dstufft> you can pass --no-deps, I think that will suppress that

[16:41:06] <dstufft> of course, you'll need to download the dependencies yourself then for anything you download

[16:41:17] <tuxx> dstufft: the dependencies are in my requirements.txt

[16:41:25] <tuxx> dstufft: anyway.. --no-deps yields the same result

[16:41:41] <dstufft> hmm, I thought we skipped executing setup.py with --no-deps

[16:41:43] <dstufft> maybe Iw as wrong

[16:42:00] <ronny> tuxx: you dont feed devpi a file, you just use it as pypi server, and it will transparently download whats missing

[16:42:27] <tuxx> ronny: okay, so i guess apache is going to have to interact with it somehow, so the apache config might be interesting indeed

[16:42:52] <ronny> tuxx: the usual case is to just set a directory alias to the devpi file store

[16:43:27] <tuxx> ronny: but how can it be informed of on-demand requests if apache just serves from a directory?

[16:43:50] <tuxx> ronny: apache is going to have to forward requests to devpi-server isnt it?

[16:44:57] <ronny> tuxx: the nginx config is set up so that missing files get passed to the devpi instead of doing a 404

[16:45:19] <tuxx> ronny: hmm

[16:45:24] <ronny> tuxx: so it tires for a file, and if there isnt, it ask the devpi, the devpi will then run the download, return the content and store the file

[16:45:26] <tuxx> i'll try to google for an apache config

[16:45:33] <tuxx> yes makes sense

[16:46:07] <tuxx> or i cld setup a seperate vm for this

[16:46:16] <tuxx> and use nginx :)

[16:47:57] <ronny> apache has this strange fallbackressource thing

[16:48:08] <tuxx> http://riceball.com/d/node/1489

[16:48:16] <tuxx> i found this.. not sure its what i need

[16:58:39] <ronny> tuxx: its missing the bit about trying to serve the files from the fs first

[16:58:57] <tuxx> ronny: honestly, i cld even live without that

[16:59:15] <tuxx> it cld download it each time again.. the main problem is the internet access restriction

[16:59:18] <tuxx> not so much the caching

[16:59:50] <ronny> tuxx: nah, devpi also serves files for its own cache

[16:59:58] <ronny> its just faster to let the webserver do it

[17:00:10] <tuxx> ronny: obviously

[17:00:34] <tuxx> devpi is a tool from hell

[17:02:19] <tuxx> a squid wld be easier than this

[17:06:46] <ronny> tuxx: well, when i did my first setup i was done in ~5 minutes

[17:06:57] <ronny> but i do use systemd and nginx for it

[17:08:01] <tuxx> yea well i cld consider setting up a dedicated host for this

[17:08:05] <tuxx> using devpi + ngix

[17:08:16] <tuxx> anyway, thx for the help ronny & dstufft

[17:08:20] <tuxx> i'm off for today

[17:08:27] <ronny> enjoy

[21:30:48] <JimmyNeutron> I'm getting this warning when installing using pip3: You are using pip version 8.1.1, however version 8.1.2 is available.

[21:31:06] <JimmyNeutron> However, when I run pip3 --version it says

[21:31:14] <JimmyNeutron> pip 8.1.2 from /home/foung/.local/lib/python3.5/site-packages (python 3.5)

[21:31:19] <JimmyNeutron> Any idea why?

[21:33:34] <nedbat> is there a way to invoke "pip uninstall" so that it won't fail if the package isn't installed in the first place?

[21:44:12] <apollo13> nedbat: pip list|grep your_pkg && pip uninstall your_pkg ;)

[21:44:51] <nedbat> apollo13: yeah, i figured something like that was the way. thanks for the shorter form than I would have come up with.

Log file Viewer

Help | Karma | Search:

#pypa logs for Monday the 11th of July, 2016