Log file Viewer

[10:57:21] <mpeterson> hello, I have a question. I have a project with a requirements.txt file inside it's root folder that uses a vcs url for one of the requirements, if I run pip install -r project_root/requirements.txt it succeeds. Now if instead I run "pip install -e project_root" it says it cannot satisfy the requirement

[10:57:48] <mpeterson> the line in question of requirements.txt is "-e git+https://git.openstack.org/openstack/ceilometer@master#egg=ceilometer"

[10:58:10] <mpeterson> is there something I'm missing? :/

[11:11:46] <mgedmin> what does project_root/setup.py say about install_requires=[...] ?

[11:12:13] <mgedmin> pip install [-e] <directory> doesn't look at requirements.txt, it looks at setup(..., install_requires=...)

[11:16:41] <mpeterson> mgedmin: first it didn't have an install_requires but then I added where I read the requirements file with pip.reqs.parse_requirements to a variable and passed to that parameter

[11:17:39] <mgedmin> I do not believe "-e git+https://.." is valid syntax for install_requires

[11:17:49] <mgedmin> you'll have to fix that

[11:17:58] <mgedmin> (I assume pip.reqs.parse_requiements() doesn't do that for you)

[11:18:31] <mgedmin> install_requires should say [... 'ceilometer' ...] or possibly [... 'ceilometer >= minimum-version', ...] (because you must be using git master for a reason!)

[11:18:45] <mgedmin> btw pip developers do not like when people import stuff from inside pip

[11:20:51] <mgedmin> afaiu in pip 10 everything will be hidden in pip._internal, so there's no temptation to import stuff from pip

[11:24:48] <mpeterson> mgedmin: I can show you exactly the problem actually... Initialize a folder as git repo and then run "git fetch https://git.openstack.org/openstack/networking-odl refs/changes/70/557370/1 && git checkout FETCH_HEAD"

[11:25:25] <mgedmin> I'd rather look at the code on the web

[11:25:25] <mpeterson> mgedmin: once it's checked out you can run "pip install -e ."

[11:26:29] <mpeterson> mgedmin: that could be a bit more complex because this is a patch that hasn't been merged but https://review.openstack.org/#/c/557370/ is the patch and the repo is https://github.com/openstack/networking-odl/

[11:27:25] <mgedmin> on, pbr

[11:27:53] <mgedmin> does pbr support git+https:// URLs in requirements.txt?

[11:28:07] <mgedmin> if the docs don't say, file a bug report to inquire

[11:28:59] <mgedmin> https://bugs.launchpad.net/pbr/+bug/1677243 and https://bugs.launchpad.net/pbr/+bug/1373623 imply that pbr tries to support this

[11:29:02] <mgedmin> but there are bugs in the area

[11:29:07] <mgedmin> maybe you need to file a new pbr bug?

[11:29:42] <mgedmin> but wait

[11:29:47] <mgedmin> let's step back a bit

[11:29:56] <mgedmin> can you pastebin the error you get when you pip install -e . ?

[11:30:24] <mgedmin> I now think pbr does everything right so all you need is to pass a --find-links to pip install so it'll be able to find a ceilometer

[11:30:52] <mgedmin> (in theory a setup.py can define its own find_links, but pip forbids that by default, for security and speed reasons)

[11:32:09] <mpeterson> mgedmin: I was looking at the code at https://github.com/openstack-dev/pbr/blob/master/pbr/packaging.py and it seems to support it...

[11:32:15] <mpeterson> mgedmin: I'll pastebin the error, yes

[11:33:01] <mpeterson> mgedmin: https://pastebin.com/V2GVxW8B

[11:33:13] <mgedmin> ah, ok, I was right

[11:34:03] <mgedmin> so, when you pip install -e ., pip will look for all the requirements on pypi (or other indexes as defined in your global pip configuration)

[11:34:17] <mgedmin> this is by design

[11:34:32] <mgedmin> if you want pip to look in other locations (like random git repositories), you'll have to tell pip that

[11:35:08] <mgedmin> one of the ways is to use pip install -e https+git:// (or via -r reqs.txt that includes such a url)

[11:35:21] <mgedmin> but this kind of external url cannot go through a setup.py/install_requires....

[11:36:13] <mgedmin> I think there's a pep to allow that kind of thing, with a different syntax (ceilometer @ https://github.com/...)

[11:36:16] <mgedmin> PEP-508?

[11:36:32] <mgedmin> but I don't believe current versions of setuptools/pip support it yet

[11:36:50] <mgedmin> and I'm not sure pbr can convert editable git repo urls from requirements.txt into this format

[11:36:54] <mgedmin> anyway, this is the future

[11:37:07] <mgedmin> to solve your problem

[11:37:12] <mgedmin> you can do several things

[11:37:21] <mgedmin> the best would be to get a ceilometer release to pypi

[11:37:50] <mgedmin> other alternatives include documenting to the users to pip install -e git+https:///...ceilometer _before_ they pip install -e .

[11:38:12] <mgedmin> maybe there's a way to specify --find-links in a way that can use github's repositories so pip install -e . --find-links <some magic url here> would find ceilometer?

[11:38:33] <mgedmin> (usually --find-links expects an HTML page with links to downloadable .tar.gz sdists or wheels)

[11:38:40] <mgedmin> (maybe you can find such a page on github?)

[11:39:27] <mgedmin> possibly using an internal package repository (devpi) would be an option? you could upload ceilometer there

[11:44:55] <mpeterson> mgedmin: awesome, I think with this information I'll be able to solve the problem. I do agree that getting ceilometer in pypi is the way to go and will rise that in the mailing list as well... In the meanwhile your insights will allow me to find a solution to this problem, thanks!

[11:46:08] <mpeterson> mgedmin: it should be documented though, that there is a difference in behavior of -r vs -e, no?

[11:46:22] <sumanah> !logs

[11:46:22] <pmxbot> http://kafka.dcpython.org/channel/pypa

[11:50:59] <mgedmin> I'm sure it is documented... somewhere

[11:53:44] <sumanah> mpeterson: I agree that it would be nice to double-check :)

[14:34:50] <sumanah> in 90 minutes I'm going to be on the OpenNews community call https://opennews.org/what/community/calls/ talking about the new PyPI, in case any of you want to call in.

[14:48:09] <tos9> Nice!

[15:24:58] <sumanah> thanks tos9!

[16:17:47] <sumanah> tos9: hope it was fun!

[16:36:25] <tos9> sumanah: yeah! I put it on my calender for the future as well

[16:36:45] <sumanah> tos9: it's like a really synchronous podcast.

[16:37:00] <sumanah> (hosts of call-in shows everywhere go "aaaahhhh" at me saying that, I'm sure)

[16:40:08] <tos9> Heh

[16:40:34] <tos9> sumanah: Yeah I do quite a few of this style with other groups within my industry (ad tech, so close enough to journalism)

[16:40:51] <tos9> Or certainly I get to hear from NYT people way more often than I might have thought

[16:40:54] <tos9> But yeah, interesting

[16:40:57] <sumanah> tos9: https://publiclab.org/events has some too

[16:41:03] <sumanah> I was in their call earlier this week

[16:41:33] <tos9> Hm, cool, will have a look at that too

[16:41:40] <sumanah> tos9: the ones you're in - are any of them open enough that I could ask to speak to them to announce PyPI stuff?

[16:41:54] <tos9> sumanah: They're less tech focused generally

[16:42:01] <sumanah> right

[16:42:22] <sumanah> tos9: feel free to email me or ping me later if you think of any

[16:42:28] <tos9> sumanah: Nothing immediately occurred to me yeah but will do

[16:42:30] <sumanah> tos9: thanks

[16:42:40] <tos9> sumanah: Certainly the NY Python meetup group might be a good idea

[16:42:46] <tos9> Have you done that already / thought about it?

[16:42:50] <sumanah> tos9: yeah we are tentatively planning something

[16:42:55] <sumanah> for late April

[16:43:13] <tos9> Ah cool -- are you speaking to Jon? Or James / other (just out of curiosity)

[16:46:38] <sumanah> tos9: Laura Hampton is taking lead on this so I am not sure of all the names involved!

[16:52:29] <tos9> Ah

[18:48:31] <raydeo> hey folks, encountering an issue I haven't seen before uploading an sdist to pypi

[18:48:35] <raydeo> https://paste.ofcode.org/zLQrwjZtU4YLVEA6eRsWvj

[18:49:07] <raydeo> the wheel uploaded fine but not the sdist... just an hour or two ago I uploaded a wheel/sdist for a different project using the same twine

[18:49:31] <raydeo> twine version 1.10.0 (pkginfo: 1.4.1, requests: 2.18.4, setuptools: 38.5.2,

[18:49:31] <raydeo> requests-toolbelt: 0.8.0, tqdm: 4.19.6)

[18:50:26] <raydeo> ah well it worked with twine 1.11.0 ...

[19:00:00] <sumanah> Hi raydeo

[19:00:16] <sumanah> raydeo: that's odd. I have to step away but EWDurbin could you take alook?

[19:00:24] <EWDurbin> sup

[19:00:26] <sumanah> thanks for the report raydeo

[19:00:31] <raydeo> fwiw I don't care anymore now that the new twine worked ;-)

[19:00:47] <sumanah> oh! you're welcome ;-)

[19:00:49] <sumanah> (I released it)

[19:00:54] <raydeo> but I use twine a lot and hadn't seen any issues so certainly curious

[19:02:08] <raydeo> that was on python 3.6.0 anyway, but we can probably ignore it

[19:02:31] <EWDurbin> weird...

[19:03:18] <raydeo> I couldn't find anything to make twine more verbose

[19:05:32] <EWDurbin> yeah, not sure if that's a server or a client thing. looks to be a twine issue afaict

[19:05:44] <EWDurbin> warehouse would not just swallow a TypeError