Log file Viewer
#pypa logs for Monday the 3rd of June, 2019
[14:29:18] <palate> I am using setuptools (setup.py), and I have some resources that I want to include into my package. However, those resources are not in the repo, so I'd like to download them at "packaging time", I believe. The alternative would be to have a script like `download_resources.sh` and expect users to call that before they run setup.py. Does somebody have an opinion about that?
[14:31:26] <palate> I have seen I could override the build step and have it download my resources. But I'm not sure if it is a good idea because people wanting to use the project without packaging it will have to download the resources separately anyway
[16:20:54] <toad_polo> palate: I would put it in the build steps such that the resources are included both in the sdist and the wheel when those are built.
[16:22:14] <toad_polo> It may not be a great idea because it means building sdist / wheel might be flaky or not reproducible, but at least all distributions of the package will contain the file.
[16:22:41] <toad_polo> I did not recommend telling people to invoke setup.py anyway. I personally never invoke it.
[16:23:22] <toad_polo> Though obviously it is invoked as part of the build by my build front-end (pip, tox, pep517.build)
[17:03:38] <ngoldbaum> palate: subclassing the setuptools build class is probably the way to go, although agreed with toad_polo that you probably don't want to make your build reproducible like that
[17:03:54] <ngoldbaum> i've also seen pacakges downloading needed data files at the time of the first import after installation
[17:04:10] <ngoldbaum> e.g. for a package that needed to download export-controlled nuclear reaction rate data that can't be distributed in the package
[19:05:52] <palate> 1. I am not sure I get your point about the reproducibility. You mean that if the download fails, then the build fails, right?
[19:07:03] <palate> 2. I don't need to call setup.py when installing because I can do it with pip install, which invokes setup.py. But when building a wheel, I need to call setup.py, right?
[19:08:16] <palate> ngoldbaum: I wouldn't know where to download the resource, if done at the time of the first import. Could I downloaded in the place where the package is installed? That may be the best, because in my case, the resource is a binary (a server) that people may or may not want to start. So it could make sense to download it only the first time they try to use it.
[19:17:57] <di_codes1> Anyone here understand what this user is trying to do and might be able to answer their question? <https://stackoverflow.com/questions/56316435/pip-not-installing-project-with-makefiles>
[19:29:18] <ngoldbaum> i think you can build wheels without invoking setup.py these days but i'm not sure offhand
[19:42:18] <palate> ngoldbaum: and about the download at runtime, would you have an idea where I can install that? I'm not sure it will go at the same place as if I include the resource in the package, will it?
[19:46:03] <ngoldbaum> i'd define a folder where your users can configure where it should be downloaded
[19:56:15] <ngoldbaum> in that case your original approach is probably best, installing it as package data
[20:42:27] <catern> hi, I've developed a small C library and a much larger Python native extension that uses that library. while the library is useful on its own, I'm thinking it would be good to include the library's source directly with my native extension's source when I upload to PyPI, so that it can be easily built for arbitrary platforms without me having to create wheels for them. is this a known approach? have people done this before?
[20:52:30] <sumanah> just in case stuff is slow today and you don't get answers here I want to invite you to also ask on the mailing list: https://mail.python.org/mailman3/lists/distutils-sig.python.org/
[21:38:03] <toad_polo> You can also build a wheel with pip: `pip wheel -w dist/ --no-deps .` is the equivalent of setup.py bdist_wheel, though I prefer pep517.build
[21:39:00] <toad_polo> The reproducibility thing is that you are checking in a reference to a resource, which means that the same command invoked on the same repo may have different results depending on what that reference points to.
[21:45:51] <toad_polo> This kind of thing comes up when your project breaks and you want to see what the first broken version is.
[21:47:56] <toad_polo> If it's broken because of something about data included by reference, you will get a misleading picture by trying to install the old versions. Similarly, if the resource you are downloading moves, the old versions will be unusable, because they are pointing to a missing resource.