pmxbot IRC Log Viewer

[00:08:12] <stephenmcd> My Python package has a setup.py which modifies the requirements list during install-time, is this still possible with wheel installs?

[00:08:31] <dstufft> stephenmcd: No, but you can do conditional dependencies

[00:08:39] <dstufft> why are you modifying it at runtime

[00:09:06] <stephenmcd> it only installs pillow if both PIL or pillow aren't installed

[00:09:12] <dstufft> oh

[00:09:16] <dstufft> I don't think that's possible :(

[00:09:46] <stephenmcd> will I go to hell if I stop supporting wheel installs?

[00:10:24] <dstufft> No, but you'll make folks sad. PIL is effectively unmaintained, why not just use Pillow?

[00:11:36] <stephenmcd> no good reason really, I'll probably just make the dep explicit. Thanks a lot.

[02:19:37] <Ivo> dstufft: thought of building wheels for pynacl?

[16:09:07] <Ivoz> dstufft, jezdez: how should one debug https://github.com/pypa/virtualenv/issues/564#issuecomment-34992966 ?

[17:40:14] <Ivoz> python windows paths can go clog a drain

[17:43:22] <Ivoz> dstufft: can you think why/how the virtualenv wheel has two pip wheels in virtualenv_support?

[17:43:55] <dstufft> Ivoz: huh?

[17:44:25] <Ivoz> dstufft: download the virtualenv wheel from pypi and extract it

[17:45:53] <dstufft> well that's not right....

[17:46:47] <Ivoz> :D

[18:21:37] <Ivoz> GustavEnk: sat you pfmoore?

[18:22:59] <Ivoz> Aha, well I've submitted https://github.com/pypa/virtualenv/pull/566 which seems to somehow substep it

[18:26:05] <GustavEnk> Yep, that's me - sorry, didn't think to say that :-)

[18:26:48] <Ivoz> Makes me wonder... all this time 1.11 has been out and feb 14th is the first time someone runs it on windows?

[18:27:11] <Ivoz> sort of seems to be a logical conclusion to draw

[18:27:38] <GustavEnk> Nope, I use it on Windows all the time.

[18:27:51] <Ivoz> Why does your windoez work then

[18:28:12] <GustavEnk> Wish I knew... There's something far more screwy going on here. Argument parsing's a nightmare of reparsing the same stuff on Windows.

[18:28:28] <Ivoz> I have an install of Py27 on this machine, but I just did a completely fresh install of 3.3.4 on x64 Windows 8.1 and I got the problem

[18:28:50] <GustavEnk> Your patch looks sensible, but I do remember having trouble getting parsing right.

[18:29:02] <Ivoz> GustavEnk: another fix I found was pythonpath = pythonpath.replace('\\', '\\\\')

[18:29:07] <GustavEnk> Hmm, I use Powershell not cmd, wonder if that might be relevant?

[18:29:14] <Ivoz> GustavEnk: try cmd

[18:29:28] <Ivoz> but I really did not want to put that in someone's code willingly

[18:29:33] <GustavEnk> Yeah, I hate that replace fix - it just feels so like randomly adding backslashes till things work :-)

[18:29:55] <Ivoz> I bet you'll encounter it on cmd.exe

[18:31:03] <GustavEnk> Nope, worked ok on a quick test.

[18:31:09] <Ivoz> ....bah

[18:31:25] <Ivoz> what python version you running?

[18:31:28] <GustavEnk> But again, that's an uninstalled copy. Don't know if that affects things

[18:31:53] <GustavEnk> 3.3

[18:31:58] <Ivoz> patch?

[18:32:13] <GustavEnk> 3.3.2

[18:32:42] <GustavEnk> I'd be very sad if the Python patchlevel makes a difference, though...

[18:33:16] <GustavEnk> I definitely think it's somehow related to multiple levels of argument parsing.

[18:33:47] <Ivoz> it'd be pretty funny if the patchlevel made a difference, in a tragic way

[18:34:07] <GustavEnk> I could try to do some intensive investigation of the arg parsing idea, but I won't have the time till the weekend

[18:34:16] <Ivoz> well travis says my patch works on ubuntu... ^_^

[18:34:41] <Ivoz> GustavEnk: what you mean by 'arg parsing idea'?

[18:35:00] <GustavEnk> Sorry, typing too fast.

[18:35:05] <GustavEnk> What I think is going on is

[18:35:34] <GustavEnk> Windows doesn't have argc/argv as a fundamental concept when spawning commands

[18:35:44] <GustavEnk> The raw command line is what the OS API works with

[18:36:03] <GustavEnk> The C runtime or application code splits it up into argv

[18:36:21] <GustavEnk> So subprocess does a nasty dance of taking an argv

[18:36:28] <GustavEnk> joining it up into a command line

[18:36:34] <GustavEnk> then calling CreateProcess

[18:36:51] <GustavEnk> and the created process breaks it up again

[18:37:10] <GustavEnk> Getting that right is a complete nightmare, but generally very effective.

[18:37:33] <GustavEnk> I suspect we're hitting a corner case here somehow, but it'll take some work to identify it

[18:38:45] <GustavEnk> Your patch may well help simply because it simplifies the number of levels in the dance we're doing here

[18:38:56] <Ivoz> GustavEnk: http://pastie.org/8730600 wooo

[18:39:44] <GustavEnk> Can't use single quotes for args on Windows.

[18:40:12] <GustavEnk> Welcome to the black magic world of Windows command line. I hope you brought some sacrificial goats :-)

[18:41:14] <GustavEnk> (It's actually simpler than bash, but who's arguing. For really arcane try Powershell :-))

[18:43:36] <Ivoz> so this actually fixes it as well

[18:43:39] <Ivoz> '"import sys, pip; sys.exit(pip.main([\'install\', \'--ignore-installed\'] + sys.argv[1:]))"',

[18:45:06] <Pancho_Villa> Would shlex.quote help to generate that subcommand?

[18:45:25] <GustavEnk> @Ivoz Actually, not for me :-(

[18:45:43] <GustavEnk> @Pancho_Villa probably not, the rules are different...

[18:45:50] <Ivoz> GustavEnk: you said yours was always working?

[18:46:29] <GustavEnk> Till I tried your patch...

[18:46:40] <Ivoz> the PR or that line above?

[18:47:01] <GustavEnk> The PR

[18:47:05] <Ivoz> jf4389jf 3498fj3498f j

[18:47:17] <GustavEnk> But I'm just quickly hacking an existing git clone, so don't put too much faith in that - I need to do a proper test TBH, and I don't have time just now.

[18:47:28] <Ivoz> nw

[18:47:46] <Ivoz> Pancho_Villa: did you manage to test my pr for 1.11?

[18:48:09] <GustavEnk> Actually, that ending backsklash you pointed out is showing for me now, so I've got something to investigate. So progress has been made :-)

[18:48:21] <Pancho_Villa> I'm working on it, I'm a total newbie and trying to figure out how to switch to your PR and test it

[18:48:57] <Pancho_Villa> I cloned your repo at ivoz/virtualenv, and switched to remotes/origin/patch-1 but the change isn't there

[18:49:19] <GustavEnk> I've gotta go now (teatime :-)) but I'll try to do some tests in the next few hours and report back.

[18:49:28] <Ivoz> Pancho_Villa: if you want, you can just reinstall virtualenv 1.11, and then delete/copy/paste these lines into C:\Python33\Lib\site-packages\virtualenv.py https://github.com/Ivoz/virtualenv/commit/5ecbfa833489c0d6b1a12eed4e279ca6252c2fa5

[18:49:57] <Pancho_Villa> kk, trying that now

[18:54:34] <Pancho_Villa> That worked, activate and deactivate bat files are there, and work as expected

[18:54:41] <Ivoz> kewl

[18:55:40] <Pancho_Villa> thanks for the quick turnaround

[19:42:37] <GustavEnk> @Ivoz OK, I see what's wrong with your patch - having double quotes triggers weird behaviour in subprocess.py.

[19:43:16] <GustavEnk> @Ivoz @Pancho_Villa could you try https://github.com/pypa/virtualenv/pull/567 and make sure it fixes things for you?

[19:43:38] <Pancho_Villa> sure thing, brb

[19:46:02] <Pancho_Villa> I'm new to working with PRs, where can I find your repo/branch?

[19:46:28] <GustavEnk> Sorry, go to https://github.com/pfmoore/virtualenv/tree/doublequote

[19:46:43] <Pancho_Villa> ah, thanks

[19:46:47] <GustavEnk> Or clone https://github.com/pfmoore/virtualenv and checkout branch doublequote

[19:51:01] <Pancho_Villa> same error as the original

[19:51:37] <GustavEnk> Ah, interesting!

[19:52:54] <GustavEnk> Could you paste the output of the line from "virtualenv foo -vvv" that says "Running command .... import sys, pip..."

[19:53:05] <GustavEnk> I want to see if it has any odd backslashes.

[19:53:21] <GustavEnk> For me it says

[19:53:30] <GustavEnk> Running command C:\Work\Scratch\virt...o\Scripts\python.exe -c "import sys, pip; sys.exit(pip.main())" install --ignore-installed setuptools pip

[19:54:17] <Pancho_Villa> OSError: Command C:\Users\Administrat...y\Scripts\python.exe -c "import sys, pip; sys.exit(pip.main())" install --ignore-installed setuptools pip failed with error code 1

[19:54:41] <Pancho_Villa> one sec lemme get the full output of virtualenv foo -vvv

[19:55:23] <Pancho_Villa> ok here it is:

[19:55:24] <Pancho_Villa> C:\Users\Administrator\virtualenv>python -m virtualenv -vvv fooey

[19:55:24] <Pancho_Villa> Creating fooey

[19:55:24] <Pancho_Villa> Using base prefix 'C:\\Python33'

[19:55:24] <Pancho_Villa> Creating fooey\Lib

[19:55:24] <Pancho_Villa> Symlinking Python bootstrap modules

[19:55:25] <Pancho_Villa> Copying to fooey\lib\os.py

[19:55:25] <Pancho_Villa> Cannot import bootstrap module: posix

[19:55:26] <Pancho_Villa> Copying to fooey\lib\posixpath.py

[19:55:26] <Pancho_Villa> Ignoring built-in bootstrap module: nt

[19:55:27] <Pancho_Villa> Copying to fooey\lib\ntpath.py

[19:55:34] <Pancho_Villa> wait a sec

[19:56:11] <GustavEnk> ok, cool. BTW, what happens if you copy that line exactly as quoted (but put the full path where the ... is) and run it from the command prompt?

[19:56:39] <Pancho_Villa> k, one sec

[19:57:47] <GustavEnk> I'm hoping it downloads & installs pip & setuptools (won't use the wheels because we don't have the env vars set up).

[19:58:03] <GustavEnk> Or maybe I'm hoping it won't, might help more that way!

[19:59:47] <Pancho_Villa> C:\Users\Administrator\virtualenv>c:\Users\Administrator\virtualenv\fooey\Scripts\python.exe -c "import sys, pip; sys.exit(pip.main())" install --ignore-installed

[19:59:47] <Pancho_Villa> Traceback (most recent call last):

[19:59:48] <Pancho_Villa> File "<string>", line 1, in <module>

[19:59:48] <Pancho_Villa> ImportError: No module named 'pip'

[20:00:17] <GustavEnk> Ah, yes, my venv got an installed pip yours didn't.

[20:01:03] <Pancho_Villa> yeah, which is the original problem, virtualenv gets installed no problem. The activate/deactivate, pip, and virtualenv exes don't get installed.

[20:01:13] <GustavEnk> I think what I'll do is add some extra debug level reporting to my PR and ask you to try it again. We might get some further info that way.

[20:01:19] <Pancho_Villa> kk

[20:07:12] <Ivoz> GustavEnk: using your pr without the doubequotes

[20:07:13] <Ivoz> http://pastie.org/pastes/8730863/text

[20:08:41] <Ivoz> with my pr, with doublequotes, this is the successful callout to pip

[20:08:45] <Ivoz> Running command C:\Users\Ivo\testfoo\Scripts\python.exe -c "\"import sys, pip; sys.exit(pip.main())\"" install --ignore-installed setuptools pip

[20:08:54] <GustavEnk> @Ivoz Yeah, something very odd here. @Pancho_Villa got the same but I consistently don't.

[20:09:09] <Ivoz> GustavEnk: still think you should test 3.3.4

[20:09:12] <GustavEnk> Staring at the code and pondering atm...

[20:09:32] <GustavEnk> You're probably right. Not hard to upgrade.

[20:09:38] <GustavEnk> ok, let me see

[20:10:23] <Ivoz> if I use single quotes, i get similar success:

[20:10:26] <Ivoz> """'import sys, pip; sys.exit(pip.main())'""",

[20:10:44] <Ivoz> results in successful:

[20:10:46] <Ivoz> Running command C:\Users\Ivo\testfoo\Scripts\python.exe -c "'import sys, pip; sys.exit(pip.main())'" install --ignore-installed setuptools pip

[20:11:26] <Ivoz> GustavEnk: i suggest trying both of those and see if either works for you

[20:12:23] <GustavEnk> Yuk. BTW, just noticed the PC I'm on now (home not work) is 3.3.2. Upgrading now, then I'll try.

[20:13:26] <GustavEnk> But now way am I committing something that was developed by adding random quotes till it worked. Don't tell anyone but that's how I got to the original version ;-)

[20:13:48] <Pancho_Villa> ha!

[20:14:51] <GustavEnk> You want professionalism? Hire a million monkeys, I hear they just finished Shakespeare's plays...

[20:17:25] <GustavEnk> Whoa! @Ivoz you were right. This is a 3.3.4 problem

[20:18:37] <Ivoz> either with " ' x ' " or ' " x " ' it works for me

[20:18:50] <Ivoz> (without the extra spaces)

[20:20:33] <GustavEnk> And yet we now know that they don't work the same on 3.3.3. I'll have to see if I can knock up a version that works on both old and new versions.

[20:20:50] <GustavEnk> A diff of subprocess.py isn't shedding much light on what changed, either...

[20:20:57] <GustavEnk> OK, leave it with me.

[20:56:31] <GustavEnk> @Ivoz @Pancho_Villa looks like this is a bug in Python 3.3.4. See http://bugs.python.org/issue20621

[21:04:21] <Ivoz> GustavEnk: you C+Ped the same txtlog in your report instead of different ones

[21:04:28] <GustavEnk> I'm not at all sure why the multiple backslash or random quote variations are fixing things, but one thing at a time

[21:12:22] <GustavEnk> @Ivoz Ouch thanks for spotting that...#

Log file Viewer

Help | Karma | Search:

#pypa logs for Thursday the 13th of February, 2014