pmxbot IRC Log Viewer

[03:39:50] <lifeless> tchaypo: Nakato: checking in

[03:41:47] <tchaypo> lifeless: good morning

[03:43:24] <tchaypo> I’ve been having a frustrating week, but I’ve managed to reproduce the behaviour inside pdb (which took all of 3 seconds once I figured out that it only happens the second time through) so now I’m poking at the code

[03:43:42] <lifeless> can I help at all with your furstation ?

[03:43:45] <tchaypo> it looks like this is a simple case of an assumption that used to be safe having changed

[03:54:23] <tchaypo> I don’t suppose you can remember what the comment at https://github.com/pypa/pip/commit/fd05fb62#diff-abf8962da0a79184dd2c4bf311fc9b57R344 means?

[03:55:15] <lifeless> yes

[03:55:16] <lifeless> so

[03:55:48] <lifeless> I suspect you're processing constrained already installed but not scheduled to install requirements there

[03:56:03] <tchaypo> I’m guessing there’s meant to be some kind of decision made earlier

[03:56:07] <tchaypo> looking back up the stack trace

[03:56:16] <lifeless> its a few lines earier

[03:56:20] <lifeless> that the method referenced is called

[03:56:32] <tchaypo> https://www.irccloud.com/pastebin/D4Qfhn99/

[03:56:37] <lifeless> InstallRequirement is a horrid pile of accumulated state

[03:56:57] <tchaypo> the handler in _walk_req_to_install() ?

[03:57:21] <tchaypo> I’m guessing that handler is supposed to either be pointed at _check_skip_installed

[03:57:23] <lifeless> no, not up the call stack

[03:57:27] <lifeless> few lines earlier in that function

[03:57:28] <tchaypo> or _prepare_file

[03:58:10] <lifeless> hmm so, req_to_install.constraint triggers return [] at the top of _prepare_file

[03:58:13] <tchaypo> I just parsed “constrained already installed but not scheduled to install"

[03:58:16] <lifeless> what requirement is is that is triggering this

[03:58:35] <lifeless> and what is its __dict__

[03:58:38] <tchaypo> that’s something that’s listed in constraints, and is already installed, but we haven’t seen it listed as a requirement of any of the packages we parsed yet

[03:58:42] <tchaypo> oslo.serialization

[03:59:08] <tchaypo> if you mean the __dict__ of the req_to_install

[03:59:09] <lifeless> this is btw some of the worst code in pip; I have a branch that throws it all away

[03:59:09] <tchaypo> {'comes_from': '-c upper-constraints.txt (line 135)', 'req': Requirement.parse('keystonemiddleware===1.6.1'), 'target_dir': None, '_ideal_global_dir': None, 'markers': None, '_wheel_cache': <pip.wheel.WheelCache object at 0x10cfb4ad0>, 'editable_options': {}, 'use_user_site': None, 'source_dir': None, 'satisfied_by': None, '_egg_info_path': None, 'isolated':

[03:59:09] <tchaypo> False, '_link': None, 'uninstalled': None, 'as_egg': None, 'conflicts_with': None, '_temp_build_dir': None, 'editable': False, 'update': True, 'pycompile': True, 'install_succeeded': None, 'constraint': False, 'extras': (), 'options': {}}

[03:59:26] <lifeless> ok, so its been converted to a thing to install

[03:59:44] <tchaypo> *blink*

[03:59:48] <tchaypo> triple-equals?

[03:59:57] <tchaypo> ('keystonemiddleware===1.6.1’)

[04:00:10] <lifeless> yup, see PEP 4440

[04:00:12] <lifeless> erm 440

[04:00:20] <lifeless> hey, js has ====

[04:00:28] <lifeless> and I proposed a ==== to dstufft the other day in here

[04:01:35] <lifeless> this req can't be triggering it

[04:01:38] <lifeless> the assertion is

[04:01:50] <lifeless> assert req_to_install.satisfied_by is None

[04:02:02] <tchaypo> d'oh

[04:02:03] <tchaypo> one sec

[04:02:07] <lifeless> and the state dict has

[04:02:08] <lifeless> 'satisfied_by': None,

[04:02:41] <tchaypo> {'comes_from': '-c upper-constraints.txt (line 180)', 'req': Requirement.parse('oslo.serialization===1.6.0'), 'target_dir': None, '_ideal_global_dir': None, 'markers': None, '_wheel_cache': <pip.wheel.WheelCache object at 0x10cfb4ad0>, 'editable_options': {}, 'use_user_site': None, 'source_dir': None, 'satisfied_by': oslo.serialization 1.6.0

[04:02:41] <tchaypo> (/Users/james/.virtualenvs/pip-2888/lib/python2.7/site-packages), '_egg_info_path': None, 'isolated': False, '_link': None, 'uninstalled': None, 'as_egg': None, 'conflicts_with': None, '_temp_build_dir': None, 'editable': False, 'update': True, 'pycompile': True, 'install_succeeded': None, 'constraint': False, 'extras': (), 'options': {}}

[04:03:45] <tchaypo> “arbitrary equality"

[04:03:49] <tchaypo> that’s a lovely name

[04:04:06] <lifeless> ok so

[04:04:10] <lifeless> this has been processed already

[04:04:16] <lifeless> thats what that assertion aims to check

[04:04:32] <tchaypo> oh. so we just want to pass?

[04:04:36] <lifeless> _check_skip_installed is called two lines lower down

[04:04:44] <lifeless> no - we want to figure out WTF we're seeing it twice

[04:05:07] <tchaypo> ah.

[04:05:18] <tchaypo> okay. First thing I’m going to do is enhance the comment

[04:05:46] <tchaypo> second thing I’m going to do is set a breakpoint to see if we can catch it the first time around

[04:05:51] <lifeless> if you go up the stack to _walk_req_to_install

[04:05:57] <lifeless> and print discovered_reqs

[04:06:05] <lifeless> and self.requirements.values()

[04:06:09] <lifeless> and self.unnamed_requirements

[04:06:15] <lifeless> you might be able to spot it as a dupe

[04:06:40] <lifeless> I'm going to wager a guess

[04:06:52] <lifeless> that its being added because has_requirement says false for constraints

[04:07:19] <lifeless> and then we're returning it

[04:07:28] <lifeless> or something

[04:08:51] <tchaypo> https://www.irccloud.com/pastebin/LaOVfc94/

[04:08:56] <tchaypo> yeah, looking back at the output

[04:09:10] <tchaypo> https://www.irccloud.com/pastebin/4MC1y91i/

[04:09:21] <lifeless> oh

[04:09:25] <lifeless> I bet I know

[04:09:35] <lifeless> so at the top of this routine I put the early bail

[04:09:47] <lifeless> which protects us in this order:

[04:10:14] <lifeless> constraint, thing-to-install-that-depends-on-constraint, the-constraint-again-after-conversion-to-regular

[04:10:19] <lifeless> but if the order is

[04:10:50] <lifeless> thing-to-install-that-depends-on-constraint, constraint-from-requirements.values, constraint-cause-added-to-discovered_reqs

[04:10:51] <tchaypo> I’m not sure what you mean by “protects us in this order"

[04:11:00] <lifeless> then we're going to hit this case

[04:11:37] <lifeless> we have a structure 'Requirements'

[04:11:45] <lifeless> which has all the constraints and requirements-to-install in it

[04:11:50] <lifeless> we iterate that in order

[04:12:00] <tchaypo> when you say “this routine” you mean prepare_file() ?

[04:12:06] <lifeless> _prepare_file

[04:12:27] <lifeless> things are added as discovered by returning a list of new reqs

[04:12:41] <lifeless> a constraint which something we're installing depends on is added like that

[04:12:56] <tchaypo> the early bail is line 396, where we check req_to_install.constraint ?

[04:12:56] <lifeless> because *if* we already visited it, we wouldn't get its dependencies

[04:13:00] <lifeless> yes

[04:13:11] <lifeless> so say we have A depends on B

[04:13:12] <lifeless> and B

[04:13:24] <lifeless> and we pip install -c c.txt A

[04:13:29] <lifeless> and c.txt contains B

[04:13:39] <lifeless> I think this will trigger the assertion the first time around

[04:13:56] <lifeless> that should let us write a test for this

[04:14:30] <lifeless> in tests/functional/test_install.py ( I think, check where I added tests for this in my commit and do something similar)

[04:14:44] <lifeless> there are existing test packages that will be close enough to permit this

[04:15:25] <tchaypo> this is in pip?

[04:15:46] <lifeless> ah

[04:15:49] <lifeless> this won't hit it

[04:15:51] <lifeless> but I know why

[04:16:01] <lifeless> the constraints are added first

[04:16:08] <lifeless> but if c.txt contains

[04:16:10] <lifeless> A

[04:16:11] <lifeless> B

[04:16:14] <lifeless> then it will hit it

[04:16:22] <lifeless> you should be able to trivially modify my test

[04:16:32] <lifeless> yeah

[04:16:45] <lifeless> once we can reproduce in test we can figure out what to do about it more sanely

[04:18:30] <tchaypo> I don’t see tests on https://github.com/pypa/pip/commit/fd05fb62

[04:18:47] <lifeless> uhh

[04:18:49] <lifeless> how did you get there?

[04:18:54] <lifeless> git log, search for constraint

[04:20:10] <tchaypo> https://github.com/pypa/pip/commit/bb0b429a49cba9f5d5fca9c59e017c80f1649743

[04:20:41] <tchaypo> okay, I need to step away from computer for a few mnutes, back soon

[04:20:48] <tchaypo> if Nakato shows up, ask her how the pizza was

[04:22:09] <lifeless> yes, bb

[04:22:25] <Nakato> tchaypo: lifeless, o/

[04:58:24] <tchaypo> lifeless: so contraints needs to be list A and B, and requirements.txt also needs to list A and B?

[04:58:34] <lifeless> no requirements.txt

[04:58:51] <lifeless> constraints.txt lists A and B, and we ask to install A, and A depends on B

[05:18:33] <lifeless> I am thinking I might reopen trello for this

[05:18:42] <lifeless> it would make the sourcing of outs easier

[05:18:48] <lifeless> thoughts?

[05:36:55] <lifeless> Nakato: do you have a trello account?

[05:37:01] <lifeless> tchaypo: I've used your tchaypo account

[05:37:10] <Nakato> lifeless: Yes, chroniclinux@gmail.com

[05:38:02] <tchaypo> Sure

[05:42:30] <lifeless> https://trello.com/b/n6jGWb3l/requirements

[06:02:38] <lifeless> Nakato: tchaypo: ok, I've done a quick brane dump

[06:02:44] <lifeless> its probably incomplete

[06:03:03] <lifeless> but if we burn those down I think we'll be sitting pretty happy. The 'solver' list is a future thing

[06:40:34] <tchaypo> pip can't proceed with requirements 'TopoRequires==0.0.1 from file:///tmp/pytest-5/test_package_in_constraints_and_dependencies0/data/packages/TopoRequires-0.0.1.tar.gz (from -c /tmp/pytest-5/test_package_in_constraints_and_dependencies0/workspace/scratch/constraints.txt (line 2))' due to a pre-existing build directory

[06:40:34] <tchaypo> (/tmp/pytest-5/test_package_in_constraints_and_dependencies0/workspace/tmp/pip-build-fefgEO/TopoRequires). This is likely due to a previous installation that failed. pip is being responsible and not assuming it can delete this. Please delete it and try again.

[06:40:38] <tchaypo> that’s not what I was expecting

[06:41:06] <tchaypo> also i just noticed this line

[06:41:07] <tchaypo> assert not cwd or not run_from, "Don't use run_from; it's going away"

[06:41:11] <tchaypo> which amused me

[06:59:12] <tchaypo> lifeless: the ‘can’t proceed’ above

[07:33:31] <lifeless> tchaypo: this is with a new test?

[07:33:52] <lifeless> tchaypo: put the code somewhere I can look at it and I will tomorrow first thing (or perhaps earlier - I'll be very disrupted tonight)

[07:34:09] <tchaypo> Yep, will do after German class

[07:34:20] <tchaypo> I was trying to do it beforehand but ran out of time

[16:55:35] <tomprince> ronny: Can I steal the guess-next-version code from setuptools_scm for use in a public domain project?

[18:33:02] <ronny> tomprince: why not just use setuptools_scm?

[20:04:11] <tomprince> ronny: setup_requires

[20:04:26] <tomprince> Although, it looks like I'm actually going to take a different take anyway.

[20:37:53] <ronny> tomprince: why is setup requires a problem, it hapens only in sdist, and can be mitigated for os packages

[20:38:03] <ronny> tomprince: and wheels dont have the problem

[23:23:35] <lifeless> anyone know why distutils2 considered tests_require deprecated

[23:23:36] <lifeless> ?

[23:23:51] <lifeless> d2to1 has it in 'backwards_compat' :/

[23:37:28] <lifeless> dstufft: ^

[23:39:34] <dstufft> lifeless: no idea, afaik it's not deprecated though

[23:39:51] <lifeless> yah

[23:40:02] <lifeless> I'll just hold my nose while I muck with this code for now

Log file Viewer

Help | Karma | Search:

#pypa-dev logs for Thursday the 18th of June, 2015