PMXBOT Log file Viewer

Help | Karma | Search:

#pypa logs for Tuesday the 27th of October, 2015

(Back to #pypa overview) (Back to channel listing) (Animate logs)
[19:24:06] <michaelstandley> Does anyone know of a pip equivalent of bundler-audit? https://github.com/rubysec/bundler-audit
[19:41:51] <tos9> if such a thing existed, it couldn't be done before install
[19:41:56] <tos9> (I haven't heard of one though)
[20:11:57] <Demiurge> hi
[20:12:19] <Demiurge> can anyone help understand how to use setuptools to make linking a non-system library work?
[20:13:14] <Demiurge> i have a library that seems to be building successfully to objectfiles and linking into a .so library file
[20:13:33] <Demiurge> however, its part of my module, and is shared between different swig interfaces of the module
[20:13:51] <Demiurge> and it seems to be a problem to link to it and import it
[20:14:12] <Demiurge> unless i do some horrible hack such as hardcode the absolute path in setup.py
[20:16:56] <ngoldbaum> Demiurge: don't have anything constrctive to say beyond that you are hitting the nasty edges of the pip/setuptools system...
[20:18:07] <Demiurge> haha
[20:18:20] <Demiurge> i feel like ive fallen off the edge
[20:18:21] <Demiurge> actually
[20:18:27] <Demiurge> i was hoping i was just lost
[20:18:56] <ngoldbaum> so the .so is a C extension that you've built with SWIG?
[20:19:04] <ngoldbaum> not an external library installed by e.g. a package manager?
[20:19:05] <Demiurge> some are
[20:19:12] <Demiurge> there is a normal _module.so
[20:19:15] <Demiurge> that improts fine
[20:19:18] <Demiurge> but there is a liblib.so
[20:19:29] <Demiurge> thats the one that is the library that needs to be linked from _module.so
[20:19:50] <ngoldbaum> ah, and LD_LIBRARY_PATH is probably out of the question
[20:19:53] <Demiurge> yeah
[20:20:03] <ngoldbaum> pathelf to change the rpath maybe?
[20:20:05] <ngoldbaum> that's how conda doesit
[20:20:06] <Demiurge> i think i have two roads
[20:20:11] <ngoldbaum> *patchelf
[20:20:18] <Demiurge> figure out how rpath, but then i need from the setuptools
[20:20:21] <Demiurge> to know where the final path is
[20:20:26] <Demiurge> the install path
[20:20:33] <Demiurge> and how to link against the lib in temp build path
[20:21:56] <Demiurge> i did see a great thread on how conda uses rpath on osx
[20:22:10] <ngoldbaum> and actually, have you considered using conda for this?
[20:22:13] <Demiurge> i still havent figured out how it works
[20:22:32] <ngoldbaum> (not to annoy any of the real pypa people in here)
[20:22:33] <Demiurge> not sure how to use conda for this, i do want this to be pypy
[20:22:43] <ngoldbaum> oh, nm, that won't work probably
[20:22:50] <Demiurge> yeah
[20:22:54] <ngoldbaum> you're using a SWIG-wrapped module from pypy?
[20:22:59] <Demiurge> oh
[20:23:01] <Demiurge> i meant pypi
[20:23:02] <ngoldbaum> I thought that pypy wanted people to use CFFI
[20:23:03] <Demiurge> sorry
[20:23:03] <Demiurge> heh
[20:23:04] <ngoldbaum> oh ok
[20:23:05] <ngoldbaum> haha
[20:23:09] <Demiurge> yeah
[20:23:40] <ngoldbaum> honestly the way conda does this is sort of black magic
[20:23:51] <ngoldbaum> but also I try to avoid directly linking against a C library from python if at all possible
[20:24:02] <Demiurge> yeah, it seems like it would help if i figure it out, but no luck so far
[20:24:10] <Demiurge> they have way more other requirements and dependencies
[20:24:18] <Demiurge> i have a pretty simple ONE extra level of indirection
[20:24:51] <Demiurge> if i could just build in the final install location and link there, and know that path
[20:24:54] <Demiurge> i could get things to work
[20:25:02] <Demiurge> and i think its assumed that those are not relocatable
[20:25:08] <Demiurge> so that would be fine
[20:25:20] <Demiurge> i wonder if 'inplace' config option does that
[20:25:27] <ngoldbaum> well you can build in your source directory with "pip install -e ."
[20:25:35] <ngoldbaum> but that's not useful if you want to upload to pypi
[20:25:40] <Demiurge> yeah
[20:25:56] <Demiurge> i also want to use it with venvs
[20:26:02] <Demiurge> actually need to
[20:26:16] <Demiurge> because its being actively developed and used at the same time
[20:26:18] <ngoldbaum> yeah, i think the only way to do that is to manually patch the rpath in your .so
[20:26:26] <Demiurge> i see
[20:26:32] <Demiurge> so, like a postinstall command?
[20:26:37] <ngoldbaum> which is a bit hairy since you're editing binary values in object code
[20:26:41] <ngoldbaum> yup
[20:26:47] <Demiurge> if it works...
[20:26:48] <ngoldbaum> that's actually what conda build does
[20:26:54] <Demiurge> its something i can improve layer
[20:26:56] <Demiurge> later+
[20:27:04] <ngoldbaum> they even use an extra-long path to leave room in the binary
[20:27:05] <Demiurge> any example of this, besides conda? i didnt see that in conda
[20:27:18] <ngoldbaum> let me see if i can find it in the conda-build package...
[20:27:59] <Demiurge> haha that is nasty tho
[20:28:02] <Demiurge> if its a binary pathc
[20:28:07] <ngoldbaum> https://github.com/conda/conda-build/blob/62894e75be02e4d048d6180ce9de250a46cd37fa/conda_build/post.py#L303
[20:28:07] <Demiurge> and not using install_name_tool
[20:28:10] <Demiurge> like osx does
[20:28:23] <ngoldbaum> https://github.com/conda/conda-build/blob/62894e75be02e4d048d6180ce9de250a46cd37fa/conda_build/post.py#L229
[20:28:34] <ngoldbaum> yes, it is nasty
[20:28:35] <Demiurge> ok
[20:28:40] <ngoldbaum> conda is kind of a hack...
[20:28:54] <Demiurge> well, when the ecosystem is so old
[20:29:36] <Demiurge> ok ill look at using these. any idea what the license is?
[20:29:46] <Demiurge> oh i think they do this for binary redistribution too?
[20:30:06] <Demiurge> yeah, i think this might work, but isn't necessary for me, because i can build
[20:30:35] <ngoldbaum> the license os BSD
[20:30:41] <Demiurge> ok
[20:30:48] <ngoldbaum> yeah, I guess you just need to set the rpath at build time
[20:30:59] <ngoldbaum> the trick is to extract the necessary final install paths somehow from setuptools
[20:31:07] <ngoldbaum> which likely means reading setuptools/distutils source code
[20:31:09] <Demiurge> yes!
[20:31:20] <Demiurge> ok
[20:31:24] <Demiurge> lol
[20:31:30] <Demiurge> yeah thats why i came here first ;)
[20:31:44] <Demiurge> the old style classes already bit me a bit
[20:31:45] <ngoldbaum> maybe one of the pypa people will show up later
[20:31:59] <Demiurge> i like how when i have an extension
[20:32:01] <ngoldbaum> there's also https://groups.google.com/forum/#!forum/pypa-dev
[20:32:04] <Demiurge> i modify one of the lib dirs
[20:32:13] <Demiurge> and it modifies all other extensions, like its immutable
[20:35:37] <Demiurge> wow
[20:35:40] <Demiurge> those coda functions
[20:36:19] <Demiurge> definitely would be my last resort
[21:16:17] <Demiurge> https://docs.python.org/2/distutils/configfile.html
[21:16:25] <Demiurge> In certain peculiar cases, though, modules are built right in their installation directory, so this is conceivably a useful ability.
[21:16:26] <Demiurge> WHICH