PMXBOT Log file Viewer

Help | Karma | Search:

#pypa logs for Wednesday the 18th of February, 2015

(Back to #pypa overview) (Back to channel listing) (Animate logs)
[02:34:42] <SamuelMarks> hi
[02:35:44] <SamuelMarks> p0 deps on p1 deps on p2. Installing p0 does not bring in p2. All deps are on github not pypi.
[02:36:18] <SamuelMarks> Tried providing: "--allow-external" and "--allow-unverified" arguments in my requirements.txt, to be left with an error: "You must give at least one requirement to install"
[02:37:06] <_habnabit> SamuelMarks, allow-external and allow-unverified are unrelated to dependency resolution
[02:37:17] <_habnabit> SamuelMarks, both flags take arguments, though; they're not booleans
[02:40:17] <SamuelMarks> _habnabit: So do I need to mention the package names twice? - Here's my 1-line requirements.txt: https://github.com/bettertutors/user-api/blob/master/requirements.txt
[02:45:13] <SamuelMarks> _habnabit: --allow-external --allow-unverified https://github.com/username/package/archive/master.zip#egg=package
[02:46:12] <SamuelMarks> Should that be?: --allow-external package_name --allow-unverified package_name https://github...
[02:47:15] <_habnabit> SamuelMarks, that's what it has to be if you want that, yes
[02:47:39] <_habnabit> SamuelMarks, but it's unlikely that you actually want allow-external/unverified
[02:49:51] <SamuelMarks> _habnabit: I just want to be able to bring my packages in from github as dependencies without pushing them to pypi first
[03:08:51] <SamuelMarks> _habnabit *ping*
[03:10:54] <_habnabit> SamuelMarks, you don't need allow-external/unverified for github dependencies
[03:11:20] <_habnabit> SamuelMarks, just change your requirements.txt to `git+https://github.com/bettertutors/sql-models`
[03:11:45] <SamuelMarks> Okay, how do I include my github dependencies? - I've tried git+, I've tried zip links, I've tried install_requires+dependency_links
[03:11:56] <_habnabit> SamuelMarks, as i said above
[03:12:20] <SamuelMarks> _habnabit: Don't I need egg name? - And that's only a dependency of user-api...
[03:12:27] <_habnabit> SamuelMarks, you don't, no
[03:18:23] <SamuelMarks_> _habnabit: Tried that, but left with that third dependency (sql-models) not getting installed
[03:18:48] <_habnabit> SamuelMarks_, 'third dependency'?
[03:19:22] <SamuelMarks_> I install rest-api, that brings in user-api. user-api should bring in sql-models.
[03:19:38] <_habnabit> SamuelMarks_, where's rest-api?
[03:19:53] <SamuelMarks_> It's a bottle app that merges all my api's together
[03:20:08] <SamuelMarks_> *APIs
[03:41:36] <SamuelMarks> _habnabit *ping*
[03:56:03] <SamuelMarks> _habnabit *ping*
[04:46:20] <SamuelMarks> p0 deps on p1 deps on p2. All are on github, not pypi. When I install p0 it doesn't bring in p2... http://stackoverflow.com/q/28540839
[05:25:31] <SamuelMarks> Packages] p0 depends on p1, which in turn depends on p2. p2 isn't getting installed, everything else is. http://stackoverflow.com/q/28540839
[06:59:53] <SamuelMarks> hi
[07:02:25] <SamuelMarks> How do I get a dependent packages dependencies installed from github? - It's failing when I use install_requires+dependency_links xor when I use requirements.txt.
[07:21:16] <SamuelMarks> How do I get a dependent packages dependencies installed from github? - It's failing when I use install_requires+dependency_links xor when I use requirements.txt.
[10:07:54] <SamuelMarks> hi
[10:07:56] <SamuelMarks> p0 depends on p1, which in turn depends on p2. p2 isn't getting installed, everything else is. Tried rewriting setup.py: https://gist.github.com/SamuelMarks/45a998a83dd60ddbadbc
[10:10:35] <doismellburning> SamuelMarks: "rest-api depends on user-api" er, not according to https://github.com/bettertutors/rest-api/blob/master/setup.py#L11
[10:10:56] <SamuelMarks> doismellburning: That one has a https://github.com/bettertutors/rest-api/blob/master/requirements.txt
[10:11:16] <doismellburning> and?
[10:11:17] <SamuelMarks> Also see: https://github.com/bettertutors/rest-api/commits/master/setup.py, I've tried with install_requires
[10:11:46] <SamuelMarks> doismellburning: See this commit, for example: https://github.com/bettertutors/rest-api/commit/da1829c311633aeceef4f1f3ca3b59002bd5cf6d#diff-2eeaed663bd0d25b7e608891384b7298
[10:12:22] <doismellburning> right
[10:12:27] <doismellburning> that looks better
[10:12:50] <doismellburning> (though I never really grokked dependency_links)
[10:12:51] <SamuelMarks> Yes, but then it couldn't find the package on github and just raised an error
[10:13:19] <doismellburning> (and I think you actually may have wanted a url to the repo, not the zip, but I can't say I use d_l enough to be sure of that)
[10:14:12] <SamuelMarks> doismellburning: Tried with git link, with repo, with zip, with tar
[10:14:21] <SamuelMarks> and all with/without egg
[10:14:49] <ghickman> the invocation for dep links is a bit odd
[10:15:00] <ghickman> let me dig up an example
[10:16:29] <SamuelMarks> Thanks ghickman
[10:16:54] <doismellburning> oh yeah
[10:19:23] <ghickman> so where you have the install requires here https://github.com/bettertutors/rest-api/blob/master/setup.py#L11 you want to add the name of your dep (user-api)
[10:19:34] <ghickman> then you tell pip where to look for it
[10:19:46] <ghickman> like this: https://gist.github.com/ghickman/9db2bb9847b844b43c91
[10:20:02] <ghickman> those are examples of args to setup() btw
[10:20:48] <ghickman> note the dev version in install requires and the egg specifier in dep links
[10:20:50] <ronny> SamuelMarks: i strongly suggest local scripting and/or a devpi server - a combination of both is probably the most practical, afair you have lost some days to thos by now
[10:20:59] <ghickman> they have to match
[10:21:25] <ghickman> +1 to a dev pi server and putting the package on there
[10:21:34] <doismellburning> ghickman: oh cripes
[10:21:41] <doismellburning> ghickman: this may explain why I've never had d_l work
[10:22:37] <ghickman> my example is good for pulling from a vcs in a setup.py but you can make your life much easier with a package server and making sure pip uses that (devpi will fall back to The PyPI if it can't find it locally)
[10:23:27] <ghickman> doismellburning: yea it's some dark voodoo to get working (and wasn't designed for this situation but only thing I've found to work for sub package in setup.py with no package server)
[10:28:43] <SamuelMarks> ghickman: I'm pretty sure I've tried what you've suggest already. Will do it again
[10:29:21] <SamuelMarks> ghickman: Wait, what's the -dev? - Do I now need to keep to semver and include that at the end of my version strings?
[10:29:33] <ghickman> you could well have had an issue with the version/egg part
[10:29:49] <ghickman> it's the magic which links the req to the dep part
[10:30:48] <ghickman> SamuelMarks: you shouldn't need to change your package to use this
[10:31:15] <ghickman> it makes the vcs checkout look like it is the "dev" version
[10:31:55] <ghickman> oh shoot, I forgot a disclaimer (sorry): this breaks if pip finds a package on the pypi with the same name
[10:32:18] <SamuelMarks> Hmm, didn't know that... so this is the string: "git+https://github.com/bettertutors/rest-api.git#egg=bettertutors_rest_api-dev"?
[10:32:31] <ghickman> that should work
[10:32:32] <ronny> reminds me that i should mention hgdistver and setuptools_scm
[10:32:33] <SamuelMarks> ghickman: Yeah, dw about that, I namespace my packages
[10:32:53] <ghickman> you might want to swap _ for - in the name if you have issues
[10:33:07] <ghickman> pip does normalisation on names which I can never remember
[10:36:01] <ronny> package names get normalized to dashes i think
[10:42:09] <SamuelMarks> ghickman: "Could not find any downloads that satisfy the requirement bettertutors-sql-models==dev (from bettertutors-user-api==0.2.13)"
[10:42:47] <ghickman> hmmm
[10:42:56] <ghickman> what does your setup.py look like now?
[10:44:19] <doismellburning> ghickman: where'd "dev" come from?
[10:44:44] <ghickman> it can be anything you want but needs to match the egg section in dep links
[10:45:41] <ghickman> we use dev to signify it's not been packaged
[10:45:48] <ghickman> but you could use anything
[10:46:03] <doismellburning> oh right
[10:46:12] <ghickman> I'm looking for the doc I got this from but badly failing at google today
[10:46:40] <SamuelMarks> ghickman: Tried all combinations of ==dev, -dev, and without
[10:47:01] <ghickman> hmmm
[10:47:20] <ghickman> the package server idea is starting to look much easier
[10:47:27] <doismellburning> lol
[10:47:29] <ghickman> I remember having the same issue
[10:47:59] <SamuelMarks> Just want to reduce the deployment dependencies, so would rather not have to setup my own package server
[10:48:09] <ghickman> yea fair poing
[10:48:13] <ghickman> point*
[10:50:42] <SamuelMarks> Your welcome to experiment with my github repositories, they're all open-source + don't depend on having a database installed/accessible + work on Windows and *nix
[10:50:46] <SamuelMarks> *You're
[11:14:15] <SamuelMarks> Okay, wrote in an issue to their queue: https://bitbucket.org/pypa/setuptools/issue/349
[13:58:25] <ionelmc> oh man
[13:58:44] <ionelmc> why do people keep doing stuff like this https://github.com/bokeh/bokeh/blob/master/setup.py#L362-L372
[13:59:42] <ionelmc> and they are complaining that tools are fragile and borken ....
[13:59:52] <ionelmc> ffs
[14:01:57] <tos9> pip uninstall foo => AssertionError: Egg-link does not match installed location of foo
[14:02:00] <tos9> Sigh.
[14:17:50] <ronny> lol
[15:39:23] <[Tritium]> Can I suggest flag for pip? --just-do-it - it implies --allow-* for everything in the install command
[15:39:59] <[Tritium]> or --iknowwhatiamdoing or --nowarentee
[15:44:07] <[Tritium]> shouldnt --allow-external imply --allow-unverified?
[15:48:16] <dstufft> [Tritium]: You can have externally hosted but verified things
[15:48:36] <dstufft> however --allow-unverified does imply --allow-external, because you can't have an "internally" hosted unverified thing
[15:49:04] <[Tritium]> Good to know
[16:15:09] <ronny> re
[16:15:13] <ronny> dstufft: ping?
[16:15:49] <ronny> dstufft: im wondering abut introducing package repos and teaching pip/virtualenv to add them via metaimport/configuration
[16:16:28] <ronny> (reducing virtualenvs to python with slightly different site configuration)
[17:52:35] <Guddu> Please help
[17:53:05] <Guddu> I tried to install reportlab after installing setuptools the commands were as follows
[17:53:06] <Guddu> pip install setuptools-12.1-py2.py3-none-any.whl --no-index --find-links /GDD_Prerequisites
[17:53:06] <Guddu> pip install reportlab-3.1.44.tar.gz --no-index --find-links /GDD_Prerequisites
[17:53:06] <Guddu> The reportlab install however gives this error ---> http://dpaste.com/1HHXT4R
[17:53:06] <Guddu> What could be the issue?
[17:53:06] <Guddu> pip-6.0.8-py2.py3-none-any.whl is what i used
[17:53:08] <Guddu> This is on RHEL 6.5
[17:53:10] <Guddu> I also tried the following as suggested by stufft but there was no error
[17:53:12] <Guddu> import pdb; pdb.set_trace()
[17:53:14] <Guddu> import setuptools
[17:53:16] <Guddu> What i noticed is that easy_install behaves differently on both the machines. Could that somehow be related to this?
[17:54:06] <Guddu> For example, on my test server i have this
[17:54:07] <Guddu> [root@localhost ~]# easy_install --version
[17:54:07] <Guddu> setuptools 12.1
[17:54:17] <Guddu> But on client's server i get an error when i execute this command
[17:54:29] <Guddu> easy_install --version
[17:54:30] <Guddu> usage: easy_install [options] requirement_or_url ...
[17:54:30] <Guddu> or: easy_install --help
[17:54:30] <Guddu> error: invalid command 'easy_install'
[17:54:47] <Guddu> Please guide
[17:56:57] <ionelmc> Guddu: it appears you don't have setuptools
[17:58:05] <Guddu> ionelmc, Thanks for the response. Please look at http://dpaste.com/2NJ0CA6
[17:58:19] <Guddu> The last few lines suggest that setuptools import was working fine.
[17:59:59] <ionelmc> Guddu: can you try this: python -c "from setuptools import setup, Extension"
[18:00:13] <Wooble> Guddu: what does head /usr/bin/easy_install look like?
[18:01:38] <Guddu> Wooble, Thanks for the response Its at http://dpaste.com/3KKY4HN
[18:02:09] <Guddu> ionelmc, I can try that but for doing that I will have to disconnect and connect to VPN and then reconnect back here. I will be back with that information.
[18:03:17] <Wooble> Guddu: huh. That's a truly weird error message to get.
[18:03:27] <Guddu> Wooble, ionelmc If this helps...I noticed that the checksum of /usr/bin/easy_install is exactly the same on the server where it fails and the server where it works
[18:04:03] <Wooble> Guddu: yeah, the script is fine, must be something in the package itself acting weird.
[18:04:27] <Guddu> Wooble, What else can i do? Shall i compare other files piecemeal?
[18:05:09] <Guddu> Finding all setuptools* gives me thing list
[18:05:21] <Guddu> ./usr/lib/python2.6/site-packages/setuptools-12.1.dist-info
[18:05:21] <Guddu> ./usr/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg-info
[18:05:21] <Guddu> ./usr/lib/python2.6/site-packages/setuptools.pth
[18:05:21] <Guddu> ./usr/lib/python2.6/site-packages/setuptools
[18:06:11] <Guddu> What is beyond my comprehension is that message "error: invalid command 'easy_install'"
[18:06:23] <Guddu> I am so dooooomed :(
[18:06:29] <Wooble> Guddu: looks like https://bitbucket.org/tarek/distribute/issue/38/distribute_setuppy-fail-invalid-command
[18:06:45] <Guddu> I practised my installation 3 times before delivering it to the client. Still it failed :'(
[18:06:53] <Guddu> Thanks Wooble Checking that
[18:07:33] <Guddu> Wooble, So multiple copies of setuptools you mean?
[18:07:53] <Wooble> That could be it. You certainly don't want that 0.6 version hanging around.
[18:09:06] <dstufft> Guddu: this is RHEL?
[18:09:31] <ionelmc> RHELL
[18:09:33] <ionelmc> :-)
[18:09:43] <Guddu> Wooble, What could i do then? This is how i installed pip and setuptools at first place
[18:09:43] <Guddu> python get-pip.py --no-index --find-links /GDD_Prerequisites
[18:09:44] <Guddu> pip install setuptools-12.1-py2.py3-none-any.whl --no-index --find-links /APP_Prerequisites
[18:09:52] <Guddu> dstufft, Yes. RHEL 6.5
[18:09:53] <dstufft> Guddu: Does https://copr.fedoraproject.org/coprs/pypa/pypa/ help at all? It's got pip, setuptools, virtualenv, and wheel setup as yum repos
[18:10:11] <dstufft> https://copr.fedoraproject.org/coprs/pypa/pypa/repo/epel-6/pypa-pypa-epel-6.repo is a .repo file for RHEL 6
[18:11:02] <dstufft> in particular installing setuptools via rpm instead of pip will make it so you don't fight the OS about who owns /usr/lib/python2.6/site-packages/setuptools
[18:11:45] <Guddu> dstufft, I can definitely use another RPM but the client is not willing to add a non-RED HAT Repo. That's the reason why I am doing a local install (--no-index --find-links /APP_Prerequisites )
[18:12:06] <Guddu> What i did must also work right? :(
[18:12:25] <Guddu> Or do i need to uninstall setuptools first?
[18:12:36] <ionelmc> Guddu: you can just download the rpms
[18:12:43] <ionelmc> and install locally
[18:12:45] <ionelmc> no repos
[18:13:37] <dstufft> https://copr-be.cloud.fedoraproject.org/results/pypa/pypa/epel-6-x86_64/python-setuptools-12.1-1.fc21/python-setuptools-12.1-1.el6.noarch.rpm
[18:13:39] <Guddu> ionelmc, That's right but using .whl file and pip should also have worked
[18:14:03] <dstufft> Guddu: the problem is with setuptools and trying to install a new version over a really old version, it gets confused
[18:14:05] <ionelmc> Guddu: you have the older setuptools around, prolly causing issues
[18:14:19] <ionelmc> clean it up, use the rpm
[18:14:26] <dstufft> and you don't want to use pip to isntall over top of something the OS has installed
[18:14:42] <Guddu> dstufft, ionelmc Shalll i try cleaning up the older one? Is there a standdard way of doing so or just remove files manually?
[18:14:45] <dstufft> because if redhat releases an update for the python-setuptools RPM, it's going to install on top of the pip installed things
[18:16:08] <Guddu> So what is one solution for now? I use that RPM suggested by you and nothing else?
[18:16:13] <Guddu> Installling that will solve the issue?
[18:16:22] <Guddu> Or do i need to unistall something first?
[18:16:34] <ionelmc> Guddu: only one way to find out :)
[18:16:51] <Guddu> The problem is i cannot try in client's environment.
[18:16:58] <ionelmc> i'd try to uninstall/remove/purge the old python-setuptrolls package first
[18:17:13] <Guddu> Like i said. I did a rehearsal 3 times on diffferent VMs before delivering the instructions to the client ;(
[18:17:39] <ionelmc> reproduce the issue in test vm then
[18:18:24] <Guddu> ionelmc, I will try that and come back with results. Logging on to Client's VPN now. Thanks for your guidance.
[18:38:11] <Guddu> ionelmc, I checked and on th eserver with problem i see the following when i do pip list | grep setuptool
[18:38:17] <Guddu> setuptools (0.6rc11)
[18:38:21] <Guddu> But on my test server i see this
[18:38:25] <Guddu> setuptools (12.1)
[18:38:46] <Guddu> I am going to do a pip uninstall and then try to install again and see if that makes any difference
[19:28:52] <Guddu> ionelmc, Wooble I am back. I installed setuptools again and it solved the issue. Look at this session transcript.
[19:28:53] <Guddu> http://dpaste.com/2PKREY8
[19:29:11] <Guddu> Its as if it needs to be installed twice not once to be working properly
[19:30:24] <Guddu> And the second time around it works
[19:34:47] <Wooble> Guddu: that's... even more weird.
[19:34:59] <Guddu> :(
[19:35:51] <Guddu> I don't know what else to do Wooble I will just include one more step to check if easy_install returns the version correctly and if not then install again. That should be alright right?
[19:36:55] <Wooble> Well, if it work, it works. :)
[20:03:58] <Guddu> Wooble, :)
[21:08:01] <SamuelMarks> hi
[21:09:53] <SamuelMarks> How do I install dependencies from git? - I've tried `install_requires`, `dependency_links`, requirements.txt and combinations of those. More info: https://groups.google.com/d/topic/comp.lang.python/q5yrtlNnJBg
[23:12:51] <SamuelMarks> hi
[23:12:54] <SamuelMarks> How do I install dependencies from git? - I've tried `install_requires`, `dependency_links`, requirements.txt and combinations of those. More info: https://groups.google.com/d/topic/comp.lang.python/q5yrtlNnJBg
[23:38:16] <ronny> SamuelMarks: they are not nicely supported, how many more days do you want to loose over this
[23:41:21] <SamuelMarks> ronny: As many as it takes until I get this all working. Would prefer not to lose modularity over this...
[23:42:20] <ronny> SamuelMarks: strong suggestion is to go for something like devpi for testing, it takes about 5 minutes to set up and use a local instance
[23:42:34] <ronny> (and it acts as caching for pypi)
[23:43:46] <ronny> or use develop install in order for development
[23:44:25] <ronny> i do that for a lot of local projects where i need about 5 things instaleld in order, i test them on a devpi but the lcoal install just used develop installs of the folders
[23:45:16] <ronny> SamuelMarks: as far as i cna tell you burned about a week trying to be very purist with something tht has 2 pragmtic and quick working solutons, that when used in combination keep your modularity intact and testable
[23:46:09] <SamuelMarks> ronny: Perhaps I will use devpi, will keep my question open though in-case someone knows the solution
[23:47:19] <ronny> SamuelMarks: as far as i can tell, that one is unimplemented and wont get implemented anytime soon
[23:47:33] <doismellburning> today I had to deal with a local djangopypi2 install and it was the cause of so much sadness
[23:48:01] <ronny> SamuelMarks: i strongly suggest just to use sevelop installs for local testing and devpi upload + devpi test for testing scenarios'
[23:48:22] <ronny> *Develop
[23:50:33] <ronny> doismellburning: wow, it looks broken on first looks
[23:53:05] <SamuelMarks> ronny: I could hack together a solution and send it through to pip in a PR. Maybe something which downloads all the dependency links then checks the package name to see if they equal one of the install_requires, if they do, install
[23:54:05] <ronny> SamuelMarks: my preference is not to pile even more hacks into setuptools/easy_install, its a mess as is ^^
[23:54:40] <SamuelMarks> But isn't my use-case a ridiculously common one?
[23:55:24] <ronny> SamuelMarks: with local development, people use develop installs since ages, and devpi instances more recently
[23:55:58] <ronny> so the use case is solved simply by typing a few commands and being pragmatic
[23:56:08] <SamuelMarks> But then I need to setup/utilse servers just to develop
[23:56:19] <ronny> yes, and?
[23:56:34] <SamuelMarks> Whereas all my data is already online, and should be accessible through the regular setup process
[23:56:53] <ronny> listing the repos inside a pip requirements file will also work
[23:57:17] <ronny> but with local developlent, the developer typically wants controll over thecheckouts, what project is on what branch and so on
[23:57:31] <ronny> you dont get that if you let things like pip/easy_install manage that
[23:57:43] <ronny> also if stuff develops in turn you need setup installs anyway
[23:58:27] <ronny> as far as i can tell being pureist about the topic is neither efficient nor usable or practical
[23:58:58] <SamuelMarks> Well my usual process is to recreate a virtualenv on each deployment. Though sometime I care, and actually set it up to only upgrade
[23:59:06] <SamuelMarks> options.upgrade = True
[23:59:52] <ronny> for deployments i build whhels of all involved packages and push them into a folder from which i do the installs