Log file Viewer

[02:08:36] <ngoldbaum> brian_: there's also conda and conda-forge, since of course there's a completely separate packing environment that people use...

[02:09:29] <hadim> hey guys, I need a Python C/C++ extension expert. I am compiling a **pure** C extension with setup.py Everything works fine under Linux but under Windows (using mingw32) I got the following error : "Cannot export PyInit__fista_decon_utilities: symbol not defined". Looks like it's looking for a function with PyInit but my extension is **pure** C without any Python related code and also I don't understand why it works fine under Linux. Any idea ?

[02:10:19] <tdsmith> my joy after discovering singularity and discovering that it was installed on my university's HPC cluster was tempered by the realization that the suid-root helper binary singularity needs os installed on a filesystem mounted nosuid on all the worker nodes

[02:10:33] <tdsmith> s/os/was

[02:12:00] <ngoldbaum> tdsmith: yeah, after looking at it closer it kinda feels like it's reinventing ideas from docker a bit? Perhaps typical of lab projects being NIH-driven...

[02:12:58] <ngoldbaum> hadim: mingw with cygwin or something? I'm not sure, I think most people use the visual C compilers on windows

[02:14:13] <hadim> ngoldbaum, I am trying to use a closed env with anaconda without the need to install visual C compiler. I use the "m2w64-toolchain" package from "conda forge".

[02:14:45] <hadim> But why is python looking for PyInit function on Windows and not on Linux ? My C code obv does not have such a function and it works fine under Linux.

[02:14:52] <ngoldbaum> ah, i'm not sure either, the packages i maintain on windows for conda-forge use the MSVC toolchain

[02:16:40] <hadim> ngoldbaum, Is the MSVC free to use ?

[02:16:45] <hadim> toolchain*

[02:16:58] <ngoldbaum> yes, for python2.7 there's a binary you can get specifically for python

[02:17:17] <ngoldbaum> other verisons use newer versions of MSVC that you can download separately...

[02:17:22] <hadim> I need a toolchain different for 2.7 and 3.5 ?

[02:17:32] <ngoldbaum> appveyor has this setup already on conda-forge

[02:17:39] <ngoldbaum> i didn't have to mess with that when i added my packages

[02:18:02] <ngoldbaum> hadim: yeah on windows i think the deal is that C extensions need to be compiled with the same compiler that build the interpreter?

[02:18:19] <ngoldbaum> that may not be correct, i fully cargo-cult windows stuff :)

[02:18:33] <ngoldbaum> https://wiki.python.org/moin/WindowsCompilers

[02:18:33] <hadim> oh god... Each time I try to use Windows it remembers me why I hate it...

[02:18:38] <hadim> thank you ngoldbaum

[02:19:02] <ngoldbaum> the main problem is the version of MSVC on python2.7 is so old that it's not c99 compliant

[02:19:11] <ngoldbaum> so it will barf on any code that's not c89

[02:19:23] <ngoldbaum> for my projects this was mostly trivial stuff but it might be painful

[02:19:36] <tdsmith> ngoldbaum: it's docker for multitenant environments; docker trivially exposes root to anyone who has privileges to launch a container

[02:19:39] <ngoldbaum> i don't think there's a fully working non-msvc toolchain on conda-forge yet...

[02:20:08] <ngoldbaum> tdsmith: i have a similar setup on my local cluster fully using docker

[02:20:16] <ngoldbaum> the sysadmin exposes a port that i ssh into

[02:20:21] <ngoldbaum> and i have root in a docker container

[02:20:40] <ngoldbaum> i think there are security concerns though, he only does it with semi-trusted users

[02:21:01] <hadim> ngoldbaum, i was able to compile lapack on conda forge recently https://github.com/conda-forge/lapack-feedstock/pull/3

[02:22:37] <ngoldbaum> i guess there's no other way to get a fortran compiler that's usable by a community effort?

[02:22:51] <ngoldbaum> continuum analytics uses the intel compilers iirc

[02:22:55] <ngoldbaum> on windows

[02:23:00] <tdsmith> some talk about this on distutils-sig lately

[02:23:56] <tdsmith> iirc continuum has a license to use the fortran compiler they use that isn't generally available or something

[02:24:04] <ngoldbaum> it'd be nice if we could use a GNU toolchain on windows, C89 is annoying...

[02:31:35] <hadim> do you know if packages compiled with mingw (such as lapacl) will be compiled by Python extension compiled by MSVC

[02:31:36] <hadim> ?

[02:48:04] <hadim> ngoldbaum, I still have the same error with MSVC : LINK : error LNK2001: unresolved external symbol PyInit__fista_decon_utilities

[02:48:21] <hadim> Windows is still looking for this PyInit function that will never exist...

[04:32:36] <raydeo> dstufft: I pushed a new version to pypi and pypi.org is not showing it but pypi.python.org is (hupper v0.3.1)

[04:32:39] <raydeo> it's been a few hours at this point

[04:32:59] <dstufft> raydeo: pypi.org/project/hupper/ you mean?

[04:33:04] <raydeo> yeah

[04:33:14] <dstufft> fixed

[04:33:21] <dstufft> it probably ogt caught when I was upgrading celery

[04:33:29] <raydeo> gotcha

[04:33:57] <raydeo> fwiw hupper probably replaces celery's now-removed autoreload feature ;-)

[04:34:29] <raydeo> I dunno, I don't use celery, but I saw something named that was removed

[08:59:48] <[Tritium]> does download_url do anything in pypi anymore?

[09:00:06] <[Tritium]> well... pip more than pypi i guess

[10:20:21] <ronny> [Tritium]: i recall pip is starting to completely ignore those since it creates a mess so often (maybe there still is an opt-in)

[12:18:11] <dstufft> [Tritium]: No

[12:18:21] <dstufft> [Tritium]: the only thing it does is get presented in the PyPI/Warehouse UI

[12:18:42] <[Tritium]> noted

[12:18:45] <[Tritium]> thank you

[12:19:31] <dstufft> (if it's helpful, PEP 470 is the one that changed this)

[12:20:52] <[Tritium]> since it affects the web ui, im not going to send the PR i was going to, but it does, so i wont

[12:33:17] <dstufft> [Tritium]: what PR is that?

[12:34:09] <[Tritium]> Someone published a brand new project with a lot of effort paid towards generating a link to a github source tarbal

[12:38:20] <dstufft> ahh

[14:09:19] <goldbox> hi all

[14:11:00] <goldbox> pip is on fire:(

[14:13:03] <ronny> goldbox: huh?

[14:24:26] <dstufft> goldbox: you'll need to give more details than that

[14:35:43] <goldbox> ronny: pip not works :(

[14:36:27] <goldbox> dstufft: i setup pip.ini, but pip still connets to pipy.python.org when i search, what's wrong?

[14:37:02] <goldbox> dstufft: pypi.python.org is censored by the disgusting China Great FireWall :(

[14:37:06] <dstufft> goldbox: what OS are you on, what is the full file path where you put pip.ini, and what does pip.ini contain

[14:37:22] <goldbox> dstufft: windows 7 x64

[14:37:25] <dstufft> goldbox: huh, PyPI is blocked by the GFW? I wasn't aware we were

[14:37:40] <goldbox> the pip.ini file is in userhome/pip/pip.ini

[14:38:18] <goldbox> dstufft: gfw is asshole, the python.org is also blocked:(

[14:39:21] <dstufft> goldbox: can you pastebin the file contents somewhere?

[14:41:06] <goldbox> dstufft: http://pastebin.ubuntu.net/23442133/

[14:42:02] <dstufft> Ah search, search currently uses a different protocol than pip install does, and that protocol isn't mirror-able :(

[14:42:31] <dstufft> I'm not aware of anyplace to search PyPI's data besides PyPI itself, if China is blocking that you may not be able to use ``pip search``

[14:43:35] <goldbox> dstufft: pip search must have to connect to pypi.python.org, right?

[14:44:58] <dstufft> goldbox: it needs to connect to something that provides the search API, pypi.python.org and pypi.org are the only two sources of the search data that I am aware of, you could try ``pip search -v -i https://pypi.org/pypi sqlite`` and see if pypi.org is blocked for you

[14:52:31] <goldbox> dstufft: also not works, is that possible to build my own search server?

[14:53:09] <dstufft> goldbox: it's possible, but you'll have to write the code to do that from scratch AFAIK, I don't think there is anything out there that supports it right now

[14:53:53] <goldbox> dstufft: are you sure about this? all search requests should be send to pypi's official site?

[16:34:41] <xafer> latest devpi provides the search api if that can help

[16:36:20] <tdsmith> so i pip installed a tarball that has pyyaml==3.11 in its install_requires and i got pyyaml 3.12 in my virtualenv instead

[16:36:50] <tdsmith> i think one of the other dependencies probably also depends on pyyaml with a looser (or incompatible?) spec

[16:38:13] <dstufft> tdsmith: yea this is a known problem with pip right now

[16:38:19] <dstufft> we don't actually have a dep resolver

[16:38:28] <dstufft> so it just uses the first "pyyaml" dependency it finds

[16:38:35] <dstufft> dpendency spec*

[16:42:59] <tdsmith> kk, thanks

[16:43:12] <tdsmith> how on earth did this happen, though: https://pypi.org/pypi/lazr.authentication/json

[16:43:19] <tdsmith> 0.1.3 tarball in the 0.1.2 releases section

[16:43:27] <tdsmith> empty 0.1.3 block

[16:48:43] <tdsmith> i see i have complained about this before https://bugs.launchpad.net/lazr.authentication/+bug/1609128

[16:59:15] <tdsmith> okay i'm also mad at the people who pinned versions in their setup.py and then did not subsequently confirm that pip installing their thing gave them the right versions

[16:59:42] <tdsmith> Five Yaks Later

[17:09:36] <dstufft> tdsmith: the upload API is sending all of the metadata + a file

[17:09:55] <dstufft> and there is not currently much validation that the file and the metadata match

[17:11:12] <ngoldbaum> presumably changing that will break lots of people's package upload scripts?

[17:12:38] <dstufft> ngoldbaum: it is a mysterry

[17:13:13] <dstufft> Largely it hasn't changed becuase it rarely matters much, sometimes there are some weird edge cases but mostly people do it correctly and haven't spent the time on it yet