Log file Viewer

[01:16:06] <dustfinger> Where do I find instructions on installing a local source tree of setuptools in edit mode so that I can debug an issue in setuptools?

[01:16:06] <dustfinger> This is what I have tried so far.

[01:16:06] <dustfinger> Given a local project called my_project and a local source tree of setuptools I tried the following: cd my_project && python -m venv venv && source venv/bin/activate && pip install --upgrade pip && cd ../setuptools && python bootstrap.py && cd ../my_project && pip install -e ../setuptools/ && python -v -m pep517.build . > build.log 2>&1.

[01:16:06] <dustfinger> But that resulted in errors in build.log related to unknown version https://paste.gg/p/anonymous/c39d732e8bfc4a719ba928b97e167fe8

[01:28:33] <dustfinger> I would like to be able to install setuptools in edit mode in a venv for this project https://github.com/BloggerBust/test_pep_420 so that I can debug an issue that I am having. In the example I gave above my_project is really test_pep_420.

[01:29:40] <sangy> dustfinger: there's this line that I think may be worth digging https://paste.gg/p/anonymous/c39d732e8bfc4a719ba928b97e167fe8#build-log-l374

[01:34:19] <dustfinger> Thanks. I noticed that as well. I will definately dig in to that error tomorrow morning (need sleep now). I thought that maybe there was some official development setup guide hiding somewhere that I missed :-P That line is definately the clue to follow.

[01:49:06] <sangy> I've honestly done it before, but I don't recall the steps. What you did does seem familiar though, so I'd say you're on the right track :)

[11:03:52] <dustfinger> sangy: Thanks, that is basically what I wanted to know. I am going to dig into that version error now. It seems strange because both my app, and of course setuptools, has a version defined. It is curious what is the cause of that issue.

[11:05:51] <dustfinger> Ahh, the entire PKG-INFO is missing.

[11:40:18] <dustfinger> I found out what was causing the problem.

[11:42:10] <dustfinger> Somehow I ended up with a setuptools.egg-info in my test_pep_420 project root. It only contained entry_points.txt I deleted that, build, dist and setuptools/setuptools.egg-info, re-created my venv, re-bootstrapped setuptools and reinstalled it in edit mode. Now it installs without error.

[11:42:41] <dustfinger> Not sure what I did to get that malformed egg-info directory in my project root, but at least I can move on.

[12:45:38] <dustfinger> If a project sets pyproject.toml build-backend = "setuptools.build_meta" and pep517 is used to build the project like this: python -v -m pep517.build . Does setuptools/setuptools/backend_meta.py Distribution class get used? I stuck some print statements in there, but they never get called.

[12:45:48] <dustfinger> I guess I should look at the pep517.build source now.

[14:58:18] <dustfinger> hmm pep517._in_process is loading setuptools.build_meta in a temp directory, but it is not the buiild_meta from my local setuptools. It must be loading a build_meta module that was compiled into something else I am guessing. Maybe pep517 itself. I didn't provide a build_path because I was not sure what to set for that and I was not sure that it was needed.

[14:58:38] <dustfinger> time for a coffee break :-P

[15:31:57] <toad_polo> dustfinger: Depends on which `pip` you are using I think.

[15:32:49] <toad_polo> Basically `setuptools` builds itself, andPEP 517 had no support for that for ages, until it was amended to allow in-tree backends.

[15:33:32] <toad_polo> Right now `setuptools` declares a build-time dependency on itself, so it may be building using the latest wheel.

[15:35:58] <toad_polo> We also did declare an in-tree backend, not sure which takes precedence, I assumed it was the in-tree one that did.

[17:37:59] <dustfinger> toad_polo: Thanks. I am just going to look through the soruce now and see if I can figure it out. Hopefully backend-url it will be happy with an egg-link since I installed setuptools in editable mode.

[17:39:26] <toad_polo> dustfinger: What are you looking to do again?

[17:42:54] <toad_polo> dustfinger: If I do `git clone git@github.com:pypa/setuptools.git && cd setuptools && tox -e py37` it works right out of the box.

[17:43:00] <toad_polo> Maybe that's a good starting point for you?

[18:06:04] <dustfinger> I have a test project called test_pep_420. The project contains a single implicit namespace package. The namespace package contains two child packages with modules. When I install the package it causes the nspkg.pth content for my implicit namespace to be malformed. I wanted to install setuptools so that I could debug the setuptools.build-backed and find out what the issue is.

[18:10:15] <dustfinger> I can install setuptools fine in edit mode, but when I build my project the pep517 the build_meta that is used by pep517/_in_process is not the build_meta in my local source tree of setuptools

[18:17:00] <dustfinger> I got it. I just made a hard link to setuptools from within my project. backend-path can't be a path outside the project root and it wouldn't expect the egg-link I had in my venv site-packages. a hard link at the project root to my local setuptools did the trick.

[18:17:12] <dustfinger> Now I can try to debug this and find out what is going on.

[18:18:48] <dustfinger> s/hard link/symbolic link/. Is there a better way to test the behavior of setuptools.build_meta in a project without using a link?

[18:24:31] <dustfinger> from test_pep_420 project I installed my clone of setuptools like this: pip install -e ../setuptools/. That produced an egg-link in test_pep_420 site-packages. Is there something else I needed to do? For now I am using the symbolic link just to get this tested.

[18:54:59] <dustfinger> Well, I have a fix, but I dout it is the right one.

[18:55:07] <dustfinger> I need more time to think about this.

[18:55:56] <dustfinger> just grabbing something to eat. back soon...

[18:57:03] <dustfinger> This fix is for https://github.com/pypa/pip/issues/6055 by the way. It is the issue I describe here on my little toy project that reproduces the issue: https://github.com/BloggerBust/test_pep_420

[19:04:00] <toad_polo> dustfinger: Oh, I would drop `namespace_packages `'

[19:04:11] <toad_polo> I thin that does something undesirable.

[19:13:30] <dustfinger> Why? Do you have a link to any discussions that explains the issue?

[19:55:50] <toad_polo> dustfinger: I don't, but I also don't know what the purpose of that argument is supposed to be, and I think it like injects a .pth file into your install or something similarly nasty.

[19:56:09] <toad_polo> Why have you included it?

[19:57:34] <toad_polo> Note the absence of namespace_packages in this documentation: https://setuptools.readthedocs.io/en/latest/setuptools.html#find-namespace-packages

[19:58:35] <toad_polo> (By the way I strongly suggest using a src/ layout if you use find_namespace_packages, or just in general, but especially in that case, because otherwise it may include random directories as "packages")

[19:58:37] <dustfinger> toad_polo: I created two python packages that do not have any entry points. They are essentially libraries. One depends on the other. I created a third pyhthon package that has an entry point and makes use of the two libraries. The library packages are namespace packages.

[20:00:05] <dustfinger> I am using find-namespace-packages via setup.cfg. In the [options] section I add namespace_packages and under that I add packages = find:

[20:00:24] <toad_polo> That is wrong.

[20:00:24] <dustfinger> According to the docs that either calls or otherwise satisfies find-namespace-packages.

[20:00:51] <toad_polo> Use find_ns: and don't populate namespace_packages.

[20:01:50] <toad_polo> Oh, it's `find_namespace:`

[20:06:19] <dustfinger> I see that https://setuptools.readthedocs.io/en/latest/setuptools.html#options. I must have read it wrong the first time and never looked back. I thought it said that :find also handled find_namespace. Not sure why I thought that now. I will make the change and see how it does.

[20:07:57] <dustfinger> omg that was it!

[20:08:03] <dustfinger> toad_polo: Thank you so much!

[20:08:12] <dustfinger> Yeesh!

[20:08:22] <dustfinger> I can't believe I went down that rabbit hole.

[20:17:44] <dustfinger> I know what append. When I saw pip #6055, given that I had exactly the same error, I assumed that I had exactly the same issue. I diagnosed my issue based solely on the symptoms and went looking for the root cause in the source code. I didn't think to re-read over the documentation to make sure I hadn't made a configuration mistake :-S

[20:22:19] <toad_polo> Eh, the documentation is horrible and out of date anyway.

[20:24:57] <dustfinger> :-P I am going to take a 10 min break. Thanks toad_polo!

[23:00:24] <dustfinger> I have a simple example project that I am using to learn more abut pep-420, setuptools etc. My setup is declarative and does not have a setup.py. Here is my setup.cfg https://github.com/BloggerBust/test_pep_420/blob/master/setup.cfg.

[23:00:24] <dustfinger> I would like the project to be installed using pep517. At the moment I am manually installing pep517 in the venv using pip. After that, I use pep517 to manually build my project with python -m pep517.build .

[23:00:24] <dustfinger> Is there a way for me to have the build requirements installed with a standardized command given a declarative setup?