PMXBOT Log file Viewer

Help | Karma | Search:

#pypa logs for Friday the 27th of February, 2015

(Back to #pypa overview) (Back to channel listing) (Animate logs)
[09:04:09] <nanonyme> Hmm, can I use stdafx.h's with setup.py-build extensions?
[11:33:03] <nanonyme> pf_moore, glyph, poke: https://pypi.python.org/pypi/pypiwin32 ; where do I file bugs about how pypiwin32 is packaged? Uninstalling it fails on Windows. Looks like you guys are marked as package owners
[11:44:08] <pf_moore> nanonyme: don't really know tbh. What's the problem?
[11:45:13] <pf_moore> Is it the wheel or the exe/src that fails?
[14:10:12] <nanonyme> pf_moore, http://www.fpaste.org/191354/42504603/
[14:10:23] <nanonyme> I installewd through pip so the wheel
[14:14:50] <nanonyme> pf_moore, my guess is the file is loaded due to some import by Python
[14:15:13] <nanonyme> This wouldn't obviously happen on Linux as Linux lets you remove files in use by people
[14:26:11] <pf_moore> nanonyne: ah, OK. It's a bug in pip/utils/appdirs.py which uses pywin32 if present. Could you raise a pip bug please? If you want a temporary fix, go to the bottom of that file and remove the bit that uses pywin32 for _get_win_folder.
[14:26:26] <pf_moore> dstufft: ping
[14:26:40] <dstufft> pf_moore: pong
[14:27:11] <pf_moore> dstufft: see the above comment from nanonyne. I'l be able to fix this but not till after the weekend
[14:27:27] <pf_moore> You mentioned wanting to do a new release - don't know if it's worth getting this in
[14:27:38] <pf_moore> Basically it means pip can't uninstall pywin32
[14:28:05] <pf_moore> (not many people use pywin32 because there's no official wheels, but obviously some people do...)
[14:28:14] <dstufft> I can wait
[14:28:26] <pf_moore> s/use pywin32/install pywin32 with pip/
[14:28:38] <pf_moore> cool - I'll look at it on Monday
[14:28:57] <pf_moore> TBH I think we just remove the pywin32 option and use ctypes always. Can't see why we need 2 options.
[14:29:43] <mathieui> is there a guideline for which kind of software versions should be uploaded to the cheeseshop? (e.g. only stable versions, or anything we want?)
[14:29:43] <dstufft> works for me, I don't know the meaningful difference between pywin32 and ctypes
[14:29:54] <dstufft> mathieui: "depends"
[14:30:21] <dstufft> mathieui: setuptools/easy_install will install anything you upload by default, so if you have 1.0 and 2.0.dev0 uploaded, it'll install 2.0.dev0
[14:30:27] <mathieui> ok
[14:30:37] <mathieui> so it’s up to what the maintainer wants
[14:30:51] <dstufft> that's generally not what you want, and because of that, pip (since 1.4) ignores pre-release versiions by default
[14:31:21] <dstufft> unless they are the only thing available to install, or use use the --pre flag, or you use a version specifier that explicitly includes one (like >=1.0.dev0)
[14:31:36] <dstufft> but yea, totally up to the maintainers
[14:32:26] <mathieui> and another question, is there a specific classifier for python sub-versions only (e.g. "Programming Language :: Python :: 3.4 :: Only")
[14:32:42] <mathieui> I didn’t see stuff like this in the classifiers list, so I assume there isn’t
[14:33:37] <mathieui> (and it gets more complicated when you support only 2.7 AND 3.4)
[14:33:46] <dstufft> no, but classifiers aren't strict limits, they are generally designed to be hints rather than explicit constraints
[14:33:52] <mathieui> right.
[16:15:17] <nanonyme> pf_moore, pywinauto32 I think uses pywin32 and we're kind of stuck with it
[16:15:43] <nanonyme> Oh, it doesn't. Interesting
[16:16:33] <nanonyme> Now I'm really curious to find an actual consumer for it
[16:17:58] <nanonyme> How about a package that provided pywin32 API using ctypes as a transitional package if ctypes is supposed to have all of it anyway?
[16:21:34] <ronny> hmm
[19:30:37] <_habnabit> dstufft, so i could probably make this better, but here's my current approach for distributing a cffi application: https://github.com/habnabit/passacre/blob/master/setup.py https://github.com/habnabit/passacre/blob/master/passacre/_libpassacre.py
[19:31:12] <_habnabit> dstufft, the LIBPASSACRE_NO_VERIFY stuff is probably the worst part
[19:31:12] <dstufft> _habnabit: will take a look shortly, gotta run to the corner store real quick
[19:31:17] <_habnabit> dstufft, np
[19:32:29] <_habnabit> actually, now that i think about it, i could split this into two modules: one that has the FFI object, and one that calls verify on that FFI object
[19:32:40] <_habnabit> then setup.py would only have to import the first module
[20:00:51] <dstufft> _habnabit: your setup.py isn't going to run if cffi isn't already installed (or rather, it's going to run but without any modules)
[20:02:26] <dstufft> which you wouldn't notice in dev, because when you import the module it's going to get implicitly compiled
[20:02:59] <dstufft> this is part of the reason why my method disables implicit compile all together, because the whole stack is really fragile in ways that are not obvious when you have implicit compiling masking it
[20:11:19] <_habnabit> dstufft, what do you mean by "isn't going to run"?
[20:11:51] <dstufft> if CFFI isn't availab.e and you run ``setup.py install``, it's going to attempt to import cffi, fail with an importerror, and ext_modules will be []
[20:12:05] <_habnabit> dstufft, right
[20:12:06] <dstufft> then the rest of the setup.py wille xecute and install without compiling
[20:12:44] <dstufft> if they install as sudo but don't import until you're on a idfferent user, you'll get gcc errors
[20:13:31] <_habnabit> disabling implicit compilation is probably a good idea
[20:13:33] <_habnabit> i hadn't thought about it
[20:14:39] <_habnabit> apparently i closed my window that had cffi tabs in it without meaning to
[20:14:39] <_habnabit> whoops
[20:15:38] <_habnabit> dstufft, so the only thing i need to do to disable implicit compilation is not call ffi.verify() ?
[20:15:59] <dstufft> No
[20:16:02] <dstufft> sec
[20:16:27] <dstufft> https://github.com/pyca/cryptography/blob/master/src/cryptography/hazmat/bindings/utils.py#L119-L120
[20:16:53] <_habnabit> heh okay
[20:17:04] <_habnabit> you really have to touch the _compile_module too?
[20:17:22] <dstufft> depends on the version of cffi you're using
[20:18:01] <_habnabit> hmm okay
[20:18:25] <_habnabit> is it newer or older versions that require _compile_module?
[20:19:23] <_habnabit> oh, hm :|
[20:19:49] <_habnabit> i think the way i set this up was relying on pip to install cffi before it actually tried running setup.py for installation of my package
[20:19:58] <_habnabit> but pip seems to be doing a different order now
[20:24:41] <dstufft> pip's ordering is a best effort of installing deps before things that depend on them
[20:24:48] <dstufft> best effort if wrong
[20:24:54] <dstufft> "half assed" is more correct
[20:25:05] <_habnabit> i see
[20:25:08] <dstufft> but until we get a real dep solver half assed is about all that's reasoanble
[20:25:23] <_habnabit> but if my install_requires has cffi in it, how is it possible for my package to be installed first?
[20:25:43] <dstufft> because install_requires is poorly named
[20:25:50] <dstufft> it's really runtime_requires
[20:25:59] <_habnabit> i see
[20:26:10] <_habnabit> so, aside from setup_requires, is there any way to specify this?
[20:26:36] <dstufft> there's a project from Daniel Holth, but I think it's worse than just using setup_requires
[20:26:48] <_habnabit> haha
[20:26:50] <dstufft> https://bitbucket.org/dholth/setup-requires
[20:27:27] <_habnabit> oh wow
[20:27:29] <_habnabit> that's nasty
[20:28:20] <dstufft> _habnabit: fwiw unless you're having fun going down this rats nest, I suggest copying what cryptography does (documented in my blog post) is the most sanity preserving way. There's a lot of niggly edge cases that we only really found out from having people do things (and there's still more we haven't yet solved, like we apparently break cx_freeze somehow...)
[20:28:40] <_habnabit> dstufft, i really really don't like what cryptography does :(
[20:28:48] <dstufft> which part?
[20:29:26] <_habnabit> keywords_with_side_effects and LazyLibrary
[20:30:02] <dstufft> keywords_with_side_effects is more or less requires if you use setup_requires unless you want something like ``setup.py --version`` to download and compile CFFI
[20:30:07] <dstufft> required*
[20:30:21] <_habnabit> yeah, that's why i was trying to avoid setup_requires
[20:30:38] <_habnabit> i guess i can't avoid it though
[20:30:44] <dstufft> yea, there's not really a better answer to that :(
[20:31:04] <dstufft> LazyLibrary you can avoid by splitting up the definition and the actual attempt to load into different modules
[20:31:13] <_habnabit> okay, i already did taht
[20:31:20] <dstufft> like you're doing
[20:31:21] <dstufft> yea
[20:31:58] <dstufft> I went that way because I felt like LazyLibrary was less fragile than depending on not having an import that triggers the library loading in the import path of the FFI Verifier object
[20:32:07] <dstufft> but it's not the only way to solve it for sure
[20:32:24] <_habnabit> i'll play around with this a bit and then push what i come up with
[20:32:26] <dstufft> the main thing is you just need to make sure that you don't trigger the case that calls load_library when executing setup.py
[20:32:32] <t4nk848> Does anyone know about installing a package from a subdirectory of a git repository?
[20:32:45] <dstufft> which is easy to tell if you are if you disable implicit compilation, because you'll get a runtime error instead of it compiling silently
[20:32:57] <_habnabit> yeah, i'm going to change that right now
[20:33:09] <_habnabit> but is it newer or older versions of cffi that require _compile_module?
[20:33:35] <dstufft> um
[20:33:38] <dstufft> I think it's older versions
[20:33:42] <_habnabit> oh okay good
[20:33:56] <dstufft> I didn't investigate super hard, this cold was originally from pynacl where I used _compile_module
[20:34:03] <_habnabit> aha
[20:34:09] <dstufft> then when I moved it to cryptography later on that wasn't working
[20:34:22] <dstufft> so I just added the compile_module and that worked and I left the old one
[20:35:22] <_habnabit> come on bitbucket why can't i click the 'blame' button
[20:36:06] <_habnabit> amazing; safari and chrome can't do it, but firefox can
[20:37:19] <_habnabit> drat; it looks like load_library calls _compile_module
[20:37:25] <_habnabit> i guess i'll clobber both then
[20:43:41] <_habnabit> dstufft, this kind of stuff makes it kind of hard to sell cffi over writing C extensions or using ctypes ;p
[20:44:38] <dstufft> _habnabit: this is why I've said that it might not be reasonable to recommend CFFI to random programmers yet, even though I think it's superior in general
[20:45:08] <dstufft> at least until CFFI gets these things fixed
[20:48:07] <dstufft> _habnabit: armin is on and off working on something better that is inspired by CFFI but which outputs .c files instead of being called by a python API (and then a python API thing to actually load/expose the .so you compile from the .c)
[20:48:25] <_habnabit> dstufft, yeah, you showed me it before, and it's pretty cool
[20:48:27] <dstufft> which is essentially the position that the cryptography hacks put you in
[20:48:42] <dstufft> just you don't get to access the intermediate C
[20:49:10] <_habnabit> dstufft, i'm still promoting cffi whenever i can on #python though
[20:49:50] <dstufft> _habnabit: yea, I do too
[20:49:58] <dstufft> I feel a little bad about it when I have to link to my blog post tho
[20:50:02] <_habnabit> heh
[20:50:10] <_habnabit> i link to my own blog all the time ;p
[20:50:59] <dstufft> well not in the linking to myself thing, but in the "yea I'm advising you use a thing which involves all this bullshit you have to do to package it sanely, I'm sorry"
[20:51:12] <_habnabit> ah yeah
[20:51:41] <dstufft> some of this I'd like to get into CFFI itself
[20:51:46] <dstufft> like a flag to disable implicit compile
[20:51:58] <dstufft> maybe a flag for a LazyLibrary too
[20:52:39] <dstufft> I think they already have a better module name (though I could be wrong)
[20:53:33] <dstufft> that would make it so you could do something like ffi.verify("...", lazy=True, compile=False)
[20:53:35] <dstufft> or something
[21:07:28] <jlkinsel> hey guys - running into a strange issue - getting pip to install a requirement...it downloads 99% then exits with MemoryError https://gist.github.com/jlk/07b02f01cef197d9301f
[21:07:57] <jlkinsel> this is a brand new VM, this usually has worked in the past (earlier this week)
[21:10:15] <jlkinsel> pip 6.0.8 on Ubuntu 14.04
[22:23:29] <_habnabit> dstufft, er, does setup_requires not pull in transitive dependencies? i have setup_requires=['cffi'] and it didn't install pycparser
[22:24:20] <dstufft> _habnabit: it should, but setup_requires doesn't install things normally, it puts it into your current dir (or .eggs in your cur dir depending on python version)
[22:26:28] <_habnabit> hm
[22:26:52] <_habnabit> i looked at the log tox wrote out, and it looks like pycparser was installed, but then setup.py failed to import pycparser anyway
[22:27:01] <_habnabit> what the heck :(
[22:40:39] <_habnabit> does tox have an option for teeing command output to the terminal instead of requiring me to dig through to find which log file i want?
[22:44:04] <pjdelport> Does -v help?
[22:47:37] <_habnabit> pjdelport, i have it cranked up to -vvvv but it still only shows me logfile names
[22:54:03] <dstufft> _habnabit: I feel like I've seen this problem before, but I can't recall what it was
[23:15:38] <_habnabit> dstufft, it resolved itself, somehow :T
[23:15:54] <_habnabit> dstufft, what determines whether a .eggs directory is created?
[23:16:14] <dstufft> _habnabit: vs dropping a bunch of .egg's in curdir?
[23:16:19] <_habnabit> dstufft, yeah
[23:16:25] <dstufft> setuptools version
[23:16:32] <_habnabit> oh okay
[23:16:36] <dstufft> I think .eggs is 7+