pmxbot IRC Log Viewer

[01:52:58] <dstufft> am

[01:52:59] <dstufft> man*

[01:53:02] <dstufft> this bug

[01:53:15] <dstufft> Alex_Gaynor: so it appears that the deadlock thing

[01:53:20] <dstufft> only occurs on one single test

[01:53:25] <dstufft> if I add pytest-timeout

[01:53:30] <dstufft> the rest of the test suite completes fine

[01:54:39] <Alex_Gaynor> Wtf

[01:55:32] <dstufft> *restarts it to see if it fails on the same test*

[01:55:44] <Alex_Gaynor> Cool that pytest timeout is a thing though

[01:55:57] <dstufft> test_config_file_override_stack

[01:56:10] <dstufft> result = script.pip('install', '-vvv', 'INITools', expect_error=True)

[01:56:18] <dstufft> Alex_Gaynor: yea

[01:56:21] <Alex_Gaynor> I don't see how my patch breaks anything :/

[01:56:24] <dstufft> me neither

[01:57:21] <dstufft> I might keep pytest-timeout around

[01:57:30] <dstufft> even after this is solved

[01:57:44] <dstufft> getting a timeout on a single test is nicer than having travis kill the entire test run

[01:58:27] <Alex_Gaynor> Given pip's test suite hits the internet that's a pretty good idea

[01:58:46] <dstufft> we should fix the whole hit the internet thing at some point too

[01:58:47] <dstufft> :/

[02:03:44] <dstufft> same test

[02:03:44] <dstufft> test_config_file_override_stack

[02:04:20] <dstufft> same spot too

[02:04:20] <dstufft> result = script.pip('install', '-vvv', 'INITools', expect_error=True)

[02:04:24] <dstufft> which is hilarous

[02:04:32] <dstufft> becuase I don't think the code you changed is even executed in that path

[02:08:31] <Alex_Gaynor> Cool computer

[02:10:13] <dstufft> intttttesting

[02:10:36] <dstufft> https://github.com/pypa/pip/blob/develop/tests/functional/test_install_config.py#L129-L130

[02:10:47] <dstufft> wonder if a work around for an old bug is crewing somrthing up

[02:11:00] <dstufft> though I can't think how

[02:24:32] <dstufft> Alex_Gaynor: next thought: pipes are evil and instead of using pipes i'm going to redirect shit to files

[02:24:48] <dstufft> https://github.com/pypa/scripttest/compare/use-tempfiles

[02:24:50] <dstufft> trying that

[02:31:14] <dstufft> nope

[02:31:16] <dstufft> :[

[02:33:05] <Ivoz_> dstufft: https://travis-ci.org/pypa/scripttest/jobs/28555783#L41-L42 you MONSTER

[02:33:19] <dstufft> wat

[02:33:25] <dstufft> oh

[02:33:38] <Ivoz_> :P

[02:33:43] <dstufft> I'm just trying to figure out why the fuck this test deadlocks

[02:34:07] <Ivoz_> is that that thing you were having trouble with alex before?

[02:34:22] <dstufft> yea

[02:36:36] <Ivoz_> dstufft: it deadlocks on your machine, or just travis?

[02:36:55] <dstufft> I've repro'd it on my local machine, but I Can't today

[02:41:58] <Ivoz_> dstufft: I'd maybe try just adding a PR for the things that *don't* bork first? Just to get them out of the way

[02:42:35] <dstufft> I have no idea what borks

[02:42:44] <dstufft> https://github.com/pypa/pip/pull/1900/files this is the PR

[02:43:05] <dstufft> it shouldn't even be touching code that the process that is deadlocking is touching

[02:44:44] <Ivoz_> I need to get ma nix installed again

[02:45:13] <Ivoz_> got a new ssd, put windows on it for stuff, but haven't moved my arch partitions over yet

[02:51:14] <Ivoz_> dstufft: in https://github.com/dstufft/pip/blob/cleanup-compat/tests/functional/test_install_config.py#L135 we shouldn't be expecting_errors=True in all those script.pip calls, right?

[02:51:30] <dstufft> lol who knows

[02:51:34] <dstufft> our tests are so bad

[03:02:09] <Ivoz_> so from https://travis-ci.org/pypa/pip/jobs/28556384 it seems somehow scripttest/ script.pip isn't creating a new environment for pip in consecutive runs?

[03:04:17] <Ivoz_> no, that makes sense...

[03:04:24] <Ivoz_> script is the same object all the time

[03:04:34] <Ivoz_> so that test doesn't make sense then

[03:09:05] <Ivoz_> So then it seems to me the script.pip() calls in https://github.com/pypa/pip/blob/develop/tests/functional/test_install_config.py#L135 at least need virtualenv.clear() in between them

[03:09:17] <Ivoz_> or e.g,, need their enviornment reset each time

[03:09:26] <Ivoz_> / recreated

[03:10:12] <Ivoz_> and therefore I don't understand how they ran beforehand... but anyway...

[03:55:31] <dstufft> w t f

[03:55:38] <dstufft> Alex_Gaynor: looks like this change is what does it https://github.com/dstufft/pip/commit/5517314856b12c61b8d73f967c3e17c2bf60d64b

[03:55:43] <dstufft> Alex_Gaynor: I'm narrowing it down further

[03:56:10] <Alex_Gaynor> dstufft: good branch name.

[03:56:32] <dstufft> I've been applying your branch one file at a time heh

[03:56:41] <dstufft> https://github.com/pypa/pip/pull/1901

[04:15:38] <dstufft> Hmm

[04:15:45] <dstufft> looks like this line makes it fail - https://github.com/dstufft/pip/commit/1b132e7595879a7fd235339616f129370f582aea

[04:29:36] <Ivoz_> dstufft: what exactly does that commit cause to fail?

[04:30:04] <dstufft> Ivoz_: in one of the tests, a particular invocation of pip deadlocks

[04:30:34] <dstufft> I'm seeing if this line causes it to fail too - https://github.com/dstufft/pip/commit/b104ddca87922a3d217d8565dc9c1f48758c3e38

[04:30:43] <Ivoz_> you can repro locally?

[04:30:51] <Ivoz_> (the fail)

[04:31:03] <Ivoz_> / deadlock

[04:32:15] <dstufft> not today I can't

[04:32:18] <dstufft> I could last week

[04:33:50] <Ivoz_> bugger

[04:34:46] <dstufft> looks like that line causes it to fail too

[04:35:11] <dstufft> which I don't understand

[04:35:27] <dstufft> because b() is literally def b(s): return s.encode("utf8")

[04:36:05] <dstufft> so b(entry + endline) should be the same as (entry + endline).encode("utf-8")

[04:36:41] <dstufft> sorry, def b(s): return s.encode("utf-8")

[04:36:58] <r1chardj0n3s> yea but that's not the same thing as b'entry + endline' for whatever the value of those two things is

[04:37:17] <r1chardj0n3s> if b() causes some exception, could that deadlock?

[04:37:28] <dstufft> b() is the one that works

[04:37:37] <r1chardj0n3s> oh, and b'' fails!?

[04:37:45] <dstufft> https://github.com/dstufft/pip/commit/b104ddca87922a3d217d8565dc9c1f48758c3e38 that fails

[04:37:57] <dstufft> which is jsut b(entry + endline)

[04:37:58] <r1chardj0n3s> erk

[04:38:08] <dstufft> vs (entry + endline).encode("utf-8")

[04:38:12] <r1chardj0n3s> yah

[04:38:14] <dstufft> ltierally the only difference is a function call

[04:38:17] <dstufft> literally*

[04:38:54] <r1chardj0n3s> that makes no sense to me

[04:38:57] <r1chardj0n3s> (hth)

[04:39:22] <r1chardj0n3s> it's a transient error though, yes?

[04:39:31] <dstufft> it happens every time on travis

[04:39:42] <r1chardj0n3s> but not reproducable locally

[04:40:01] <r1chardj0n3s> this is the pip test suite?

[04:40:14] <r1chardj0n3s> can I try it out?

[04:41:00] <dstufft> yea it's cleanup-compat-wtf on my pip fork

[04:41:07] <dstufft> though I just piushed another change to try something

[04:41:16] <dstufft> oh

[04:41:20] <dstufft> and it only happens on PYthon 3.3

[04:41:23] <dstufft> the deadlock that is

[04:41:44] <r1chardj0n3s> ok, lemme clone your thing

[04:41:56] <dstufft> https://github.com/dstufft/pip/commit/d87e21e7102c386b31dec3bdf755f58a4bb6b7f4 this is what I just pushed

[04:42:04] <dstufft> to see if any function call fixes it

[04:42:26] <r1chardj0n3s> what OS are you testing on?

[04:42:33] <dstufft> Travis is Ubuntu 12.04

[04:42:41] <dstufft> I'm using OSX Mountain Lion

[04:42:50] <r1chardj0n3s> ok, Imma try on 14.04 py3.3

[04:43:37] <r1chardj0n3s> durn, 14.04 is py3.4 ;)

[04:45:08] <r1chardj0n3s> (building 3.3)

[04:45:24] <dstufft> lol

[04:45:26] <r1chardj0n3s> am also watching PTL webinars, so not just twiddling thumbs ;)

[04:45:27] <dstufft> that works

[04:45:34] <dstufft> https://github.com/dstufft/pip/commit/d87e21e7102c386b31dec3bdf755f58a4bb6b7f4 this works

[04:45:42] <dstufft> https://github.com/dstufft/pip/commit/b104ddca87922a3d217d8565dc9c1f48758c3e38 this doesn't

[04:46:18] <r1chardj0n3s> um

[04:46:48] <Ivoz_> dstufft: you could try rewriting that method to be less forgiving: http://pastie.org/9329002

[04:47:19] <dstufft> if it was a timing thing it'd be intermittent right?

[04:47:32] <dstufft> like surely travis timing isn't so consistent that it'd happen 100% or 0%?

[04:49:14] <dstufft> by the way when I was tracing this last week, pip itself didn't deadlock

[04:49:25] <dstufft> pip got the entire way to the end of it's own code

[04:49:30] <dstufft> to where it was calling sys.exit(0)

[04:49:41] <dstufft> thne it just never exited

[04:49:52] <dstufft> however os._exit(0) would cause it to exit

[04:50:18] <Ivoz_> so replacing sys.exit(0) with os._exit(0) also rectified the failure?

[04:50:27] <dstufft> yes

[04:50:43] <Ivoz_> da fuck

[04:50:54] <dstufft> someone suggested threading...

[04:50:58] <dstufft> except we don't use threads

[04:51:08] <dstufft> and threading.enumate() didn't list any except the main thread

[04:51:09] <r1chardj0n3s> dstufft: python always uses threads

[04:51:13] <dstufft> enumerate*

[04:51:30] <dstufft> r1chardj0n3s: right, but there wasn't a second or third thread keeping things from exiting is what I mean

[04:51:33] <dstufft> it was just the main thread

[04:53:20] <r1chardj0n3s> dstufft: right, sorry I'm just throwing noise into the pool at the moment

[04:55:40] <r1chardj0n3s> I'm gonna see if I can reproduce the issue anyway

[04:55:53] <r1chardj0n3s> can I isolate to a single test or does it need the entire suite

[04:56:26] <r1chardj0n3s> ooh, I get failures in test_freeze.py

[04:56:32] <r1chardj0n3s> and test_install

[04:56:36] <r1chardj0n3s> whee!

[04:56:47] <dstufft> r1chardj0n3s: I was able to reproduce last week with a single test

[04:56:54] <r1chardj0n3s> dstufft: k

[04:57:21] <dstufft> tox -e py33 tests/functional/test_install_config.py::test_config_file_override_stack

[04:57:46] <r1chardj0n3s> I've got a bunch of failures ;)

[04:58:05] <Ivoz_> paste a log?

[04:58:10] <Ivoz_> unless you know why

[04:58:33] <r1chardj0n3s> Ivoz_: I'll be looking into it once the run is complete

[04:58:52] <Ivoz_> kk

[04:59:07] <Ivoz_> it could be you don't have all the dcvs tools...

[04:59:24] <Ivoz_> or not

[05:04:36] <dstufft> https://github.com/dstufft/pip/commit/f2ac328067b4159e5fc91136ee49940e5d616492 fails too

[05:04:46] <r1chardj0n3s> Ivoz_: yah, "FileNotFoundError: [Errno 2] No such file or directory: 'svnadmin'" a lot

[05:05:36] <r1chardj0n3s> Ivoz_: I assume that's what you were referring to?

[05:05:54] <r1chardj0n3s> "pip.exceptions.BadCommand: Cannot find command 'svn'"

[05:05:56] <Ivoz_> dstufft: my main try with that was changing the logic away from thjje lines.remove() call to what I wrote

[05:06:10] <Ivoz_> r1chardj0n3s: we should really make those conditional tests...

[05:06:57] <Ivoz_> basically, ja, if you install git & hg & svn & bzr it'll stop whining

[05:07:12] <r1chardj0n3s> yep

[05:07:17] <Ivoz_> or you could just test the specific tests we're interested in atm

[05:07:33] <r1chardj0n3s> yeah, I've narrowed it down for now

[05:08:28] <Ivoz_> dstufft: I was wondering whether the passed ValueError was hiding something going wrong

[05:09:18] <r1chardj0n3s> Ivoz_: I was concerned about that as well

[05:09:25] <r1chardj0n3s> I can't reproduce a failure here, sorry

[05:09:49] <r1chardj0n3s> platform linux -- Python 3.3.5 -- py-1.4.20 -- pytest-2.5.2

[05:10:03] <Ivoz_> so it only fails on travis worker bots now?

[05:10:15] <r1chardj0n3s> yay CI!!! \o/

[05:10:19] <Ivoz_> r1chardj0n3s: what commit were you testing

[05:10:33] <r1chardj0n3s> Ivoz_: I tried a bunch of dstufft's variations

[05:12:23] <r1chardj0n3s> am now on "latest" 'Don't allow ValueError'

[05:12:55] <Ivoz_> r1chardj0n3s: I guess dstuff was looking to see if https://github.com/dstufft/pip/commit/b104ddca87922a3d217d8565dc9c1f48758c3e38 failed for you in particular

[05:13:21] <r1chardj0n3s> oops, I didn't actually try the b() variation

[05:16:20] <dstufft> looks like removing the ValueError has the same results

[05:16:22] <dstufft> deadlocked

[05:17:19] <dstufft> Alex_Gaynor: I hate this deadlock bug you found

[05:17:24] <Alex_Gaynor> dstufft: sorry :/

[05:17:32] <Alex_Gaynor> I'll try not to find a bug next time

[05:18:03] <dstufft> Alex_Gaynor: I'm so confused, it passes if I had a noop function call but deadlocks if I don't

[05:18:08] <Alex_Gaynor> wat the wat

[05:18:22] <dstufft> https://github.com/dstufft/pip/commit/b104ddca87922a3d217d8565dc9c1f48758c3e38 == deadlocks

[05:18:32] <dstufft> https://github.com/dstufft/pip/commit/d87e21e7102c386b31dec3bdf755f58a4bb6b7f4 == passes

[05:18:39] <Alex_Gaynor> nice

[05:19:45] <r1chardj0n3s> dstufft: just to clarify, I'm *just* running tox -e py33 tests/functional/test_install_config.py::test_config_file_override_stack

[05:20:18] <r1chardj0n3s> but I've gone back through every commit and none of them fail here

[05:20:20] <r1chardj0n3s> yay travis

[05:20:43] <r1chardj0n3s> now I *am* running 14.04 and not 12.04 but...

[05:20:55] <Ivoz_> 14.04 ftw

[05:21:04] <Ivoz_> it has relevant versions of python packages again

[05:23:15] <Ivoz_> well richard's pytest platform string is exactly the same as travis', so that's sad

[17:14:09] <dstufft> Alex_Gaynor: hum, so here's an interesting thing

[17:14:58] <dstufft> Alex_Gaynor: if I configure it so that subprocess.Popen() doesn't have stdout=subprocess.PIPE and stderr=subprocess.PIPE and stdin=subprocess.PIPE... then on every python but python 3.3 (the one that's deadlocking) I get output in py.test with -s

[17:15:30] <dstufft> could the underlying process be using the wrong fds for stdout/stderr/wahtever which is blocking it somehow?

[17:15:36] <Alex_Gaynor> no idea.

[17:21:01] <dstufft> Alex_Gaynor: I get worried when a problem confuses even you :|

[17:29:45] <Alex_Gaynor> dstufft: you should see the memory leak dreid and I are chasing

[19:46:56] <ionelmc> dstufft: pytest redirects output via dupfd into a temp file by default

[19:47:11] <ionelmc> that's why it gets your subprocess output too

Log file Viewer

Help | Karma | Search:

#pypa-dev logs for Friday the 27th of June, 2014