PMXBOT Log file Viewer

Help | Karma | Search:

#pypa logs for Thursday the 9th of June, 2016

(Back to #pypa overview) (Back to channel listing) (Animate logs)
[00:05:17] <TRManderson> I have fully PEP440 compliant versions (e.g. project-1.0.0.dev0+hash) for my packages, and pip won't recognise packages with versions similar to my example when using -i or --find-links
[00:05:21] <TRManderson> Is this intentional? Can I get around this while retaining my version structure somehow?
[00:06:14] <TRManderson> It was suggested I cc dstufft
[00:06:19] <dstufft> TRManderson: what version of pip?
[00:06:41] <TRManderson> 8.1.2, I believe
[00:06:43] <TRManderson> will double check
[00:06:58] <TRManderson> yep
[00:07:25] <dstufft> TRManderson: what's the output if you add a bunch of -vvvv,
[00:07:36] <dstufft> can you pastebin it
[00:09:04] <TRManderson> sure
[00:09:19] <TRManderson> it appears to be correctly detecting versions then just choosing a different one, putting it in to pastebin now
[00:10:14] <TRManderson> http://pastebin.com/1pn8Mtr8
[00:10:37] <TRManderson> only included partial output - some sensitive information in further output
[00:11:47] <dstufft> TRManderson: does adding --pre change the behavior?
[00:12:40] <TRManderson> yes
[00:12:47] <TRManderson> changes it to the behaviour I expected
[00:12:48] <TRManderson> thank you
[00:13:27] <TRManderson> Turns out I just wasn't reading the docs right
[09:50:33] <zmo> hi o/
[09:51:07] <zmo> is there a way to do a pip install of a package from pypi without using subprocess?
[09:51:17] <zmo> I mean doing it in pure python
[09:56:08] <apollo13> zmo: no supported API, no
[09:59:09] <zmo> grmpf :-(
[09:59:24] <zmo> apollo13 - is this a design choice or just a thing that happened to be like that?
[10:01:11] <apollo13> a little bit of both
[10:01:29] <apollo13> pip is already complex enough, having users muck around with it's internals would hardly be helpful
[10:04:21] <zmo> apollo13 - well, offering a public API that's just between the CLI interface and the real internal code would be nice to have, so that then we could call the same commands, but get python objects and iterators instead!
[10:04:38] <apollo13> zmo: that is easier said than done
[10:04:48] <apollo13> what about handling the interactive choices pip provides etc etc…
[10:04:51] <zmo> it feels just wrong to get through the shell to call another python thing
[10:05:07] <apollo13> I wish I had your problems (no offense) :D
[10:05:26] <apollo13> at the end of the day, using a shell here is probably the easiest and nicest way
[10:05:49] <apollo13> zmo: it would not just be some python objects and iterators, but also callback functions and whatnot
[10:06:00] <zmo> yup, sure, that's not a horrible thing
[10:06:19] <zmo> and even for stuff that requires interactivity, I'm pretty sure there's a way to impose a strategy
[10:06:26] <apollo13> yes, callbacks ;)
[10:06:28] <zmo> "fail if it's asking, or yes to all, or no to all"
[10:06:34] <zmo> like for apt
[10:06:45] <zmo> (I don't remember having pip asking me much stuff, though)
[10:07:00] <apollo13> mainly when removing packages afaik
[10:07:16] <zmo> ah ok
[11:17:02] <dstufft> zmo: One of the key problems is that invoking pip introspects the state of a Python environment, and the mechanisms it uses to do that are caching-- so once pip modifies that environment, if you don't start up a new process it's possible (and likely tbh) that pip will be operating based on a stale view of the world
[12:46:43] <carlesc> hi there, I was wondering, is it possible to create a pip package that contains pre-built shared libraries for multiple platforms? I want to avoid the user having to compile them since they require a lot of dependencies to build
[12:54:12] <carlesc> I read the following thread on SO, but it seems that there's still quite a few loose threds: http://stackoverflow.com/questions/31380578/how-to-avoid-building-c-library-with-my-python-package
[12:54:35] <carlesc> just like anyone else I would package both 32 and 64-bit shared libraries for all 3 platforms (Win, Linux, OS X)
[14:54:25] <count> hi, help :)
[14:54:32] <count> $ time python -c 'import xmlrpclib ; server = xmlrpclib.ServerProxy("http://pypi.python.org/pypi") ; print server.package_releases("pip")'
[14:54:52] <count> just made 10 tries: 8x 502 bad gateway, 1x 42s, 1x29s; hitting the endpoint prod.python.map.fastly.net
[14:56:06] <dstufft> count: try https://
[14:56:11] <count> i.e.: pypi XMLRPC sees to be broken on at least one endpoint
[14:56:29] <count> the problem is that this is hardcoded in &%$@!! Puppet with 10s timeout, with no way to elegantly change it
[14:56:48] <count> this means it's currently broken for everyone installing pip packages via Puppet via that endpoint
[14:59:23] <count> (can't change the puppet code for anyone else to point to https://, nor mine :()
[16:16:55] <count> how/where do I report this correctly?
[16:20:47] <count> seems like the XMLRPC interface is lacking proper monitoring :)
[16:21:26] <count> at least this was apparently fixed by the Puppet guys in their very very latest release 13 days ago: https://github.com/puppetlabs/puppet/commit/152299cc859fc74343c697841848086d4e41b6f8
[16:21:30] <count> *sigh*
[16:50:34] <dstufft> count: we've been under a bit of a DoS, it appears from people using puppet on EC2 and we're working to rate limit/block the IP that's degrading our service ATM
[16:50:41] <dstufft> the XMLRPC api is really old, and creally crappy
[16:55:22] <count> dstufft: oh boo. *sigh*
[16:55:55] <count> dstufft: yeah, I figure the bit about old and crappy. do you have an IP (range) for the AWS hosts? hope it's none of the services from our organzation :D
[16:56:43] <count> dstufft: I've no idea who decided when and why to integrate freaking puppet with hardcoded _EVERYTHING_ against it
[16:57:24] <dstufft> count: 54.236.1.18, 54.236.1.12
[17:00:01] <count> dstufft: hah, that's Pinterest
[17:00:31] <dstufft> count: are you Pinterest? :]
[17:01:23] <count> dstufft: nah, I'm running Puppet infrastructure at HERE ;)
[17:11:18] <EWDurbin> count dstufft i definitely agree that monitoring around xmlrpc requests is lacking. :/
[17:11:44] <EWDurbin> wonder if i can convince pingdom to create a well formed xmlrpc request for some additional monitoring
[17:11:50] <count> https://whois.arin.net/rest/customer/C04719695
[17:13:18] <count> EWDurbin: yes, you should be able to. want me to write a cURL commandline for you?
[17:13:39] <EWDurbin> sure! i'd appreciate the leg up that'd give me
[17:13:49] <EWDurbin> just need a request that will have a stable response
[17:15:09] <dstufft> EWDurbin: we can add xmlrpc commands too if that's helpful
[17:15:53] <count> EWDurbin: curl -v -X POST http://pypi.python.org/pypi -H 'Content-type: text/xml' -d "<?xml version='1.0'?><methodCall><methodName>package_releases</methodName><params><param><value><string>pip</string></value></param></params></methodCall>"
[17:16:04] <count> EWDurbin: basically my call:
[17:16:15] <count> $ time python -c 'import xmlrpclib ; server = xmlrpclib.ServerProxy("http://pypi.python.org/pypi") ; print server.package_releases("pip")'
[17:18:29] <count> EWDurbin: not sure whether 200 OK is the right thing to check for, or </methodResponse> .. a changing version might not be clever
[17:19:15] <count> EWDurbin: taking up dstufft on the extra XMLRPC call (like 'health', which does some internal check for sanity etc) might be clever, but should excercise the end-point end-to-end, I'd say
[17:19:39] <EWDurbin> for sure, i think making a real call will be sufficient to determine health
[17:35:31] <count> okay, thanks for the help everyone :) I'll fall back into silence and shall eventuall drop out of the channel
[20:19:09] <harijay> I am using pip 8.1.2 from /usr/local/lib/python2.7/dist-packages (python 2.7) . SOme of my packages ( “argparse , “wheel” , “wsgiref” ) dont get listed with pip 8.1.2 but were listed fine with the OS ( Ubuntu 14.04) supplied pip 1.5.4
[20:19:58] <dstufft> harijay: do you mean pip list or pip freeze
[20:20:07] <harijay> pip freeze
[20:21:03] <harijay> All of these are not in a virtualenv but installed in OS root level.
[20:21:12] <dstufft> harijay: pip freeze --all
[20:21:19] <harijay> Ill try that...
[20:22:53] <harijay> #dstufft That revealed wheel but “wsgiref” and “argparse” are still hidden to pip freeze —all
[20:23:09] <dstufft> harijay: oh, wsgiref and argparse are hidden because they're part of the stdlib
[20:24:28] <harijay> Oh I see…because argparse was not before…so then why does wheel get revealed only after —all. Also if I say pip install argparse it still says its installed, I assumed as a separate package
[20:28:27] <dstufft> harijay: --all exposes stuff hidden b/c it is installed by default in a virtualenv
[20:28:44] <dstufft> and yea you can still installa rgparse, but the stdlib argparse shadows it