Log file Viewer

[00:06:41] <qwcode> lifeless, ok, so "pip install --no-install -d" in your branch now unpacks and downloads, whereas before, it just downloaded

[00:06:59] <lifeless> qwcode: yeah, which AFAICT was actually a regression from earlier

[00:07:40] <qwcode> lifeless, it's hard to make sense of what combining those options should do, but the change seems ok to me.

[00:07:46] <lifeless> qwcode: I discovered this when tracking down the failure to install from my implicit wheels branch, which was due to the temporary source dir not being deleted.

[00:07:49] <lifeless> qwcode: thanks

[00:15:58] <lifeless> so https://github.com/pypa/pip/pull/2618 should be good to go now, I've just shuffled things a little so one more build is going to be needed

[00:43:11] <lifeless> yay - https://github.com/pypa/pip/pull/2618 is green.

[00:43:24] <lifeless> (And I have confidence that it should be green now :))

[00:49:06] <dstufft> lifeless: sorry was taking a nap, sounds like you got it sorted?

[00:55:43] <lifeless> dstufft: yeah its all ready to review and merge

[00:57:39] <dstufft> lifeless: awesome, I might cut a release before merging it just to get the topo sort out and a few other things out there before a release that holds this

[00:57:46] <dstufft> will take a look soon

[01:00:20] <lifeless> I think I want a new data structure

[01:00:35] <lifeless> I'm going to do some exploratory refactorings (new branch)

[01:01:01] <dstufft> "<lifeless> I think I want a new data structure" -> stranger things have been said

[01:01:09] <lifeless> req_set

[01:01:12] <lifeless> and req_install

[01:01:20] <lifeless> are very coupled to the idea of one installation pass

[01:01:26] <lifeless> but setup-requires are doing to do many

[01:01:36] <dstufft> yea

[01:01:39] <dstufft> makes sense

[01:02:05] <lifeless> I don't want to install numpy 15 times if there are 14 things I'm installing all of which setup-requires numpy

[01:02:27] <lifeless> I want to install it once and use it 14 times, then install it into the final target once

[01:03:22] <dstufft> I had previously thought about making it so that there was a single setup requires tmp dir, than did shit like /tmp/numpy-X.Y, and /tmp/pbr-X.Y, and it would short circuit additional setup uses to add /tmp/numpy-X.Y if it was already temp installed to that location

[01:03:51] <lifeless> righ

[01:04:25] <lifeless> and python path shenanigans to bring it all together

[01:04:55] <lifeless> I want an API like 'give me a PYTHONPATH I can use that has requirements X, Y, Z on it'

[01:05:18] <dstufft> yea

[01:05:21] <dstufft> that seems reasonable

[01:05:37] <lifeless> I want to be able to clean it up afterwards

[01:06:06] <lifeless> future iterations: I want to be able to utilise the already installed requirements

[01:06:53] <lifeless> future iterations: I want to be able to use this when running egg-info to calculate deps, for cases where install-requirements does require turing complete blah.

[01:07:08] <lifeless> initial implementation static data only, of course, because sanity

[01:07:29] <lifeless> this seems separate to req-set to me

[01:07:42] <lifeless> it needs to be able to hold multiple numpy-X and numpy-Y versions, for instance.

[01:08:11] <dstufft> yea

[01:08:22] <lifeless> but ther'es stuff req-set does that this will want

[01:08:28] <lifeless> thus - refactoring

[01:08:39] <lifeless> I'd also like to get something thats a bit easier to reason about

[01:09:50] <lifeless> possibly via some functionally-styled code (separate pure and impure things, avoid updating hidden state...)

[01:11:48] <dstufft> ok, i'm gonna go look at PRs and see if what looks mergeable for a 6.1

[01:11:58] <dstufft> then I'll merge those and cut a 6.1

[01:12:09] <dstufft> then I'll bump to 7.0.dev0 and look at 2618

[01:16:59] <lifeless> are unit tests allowed to do IO ?

[01:17:09] <lifeless> e.g. making temp dirs etc

[01:17:17] <lifeless> or should that be in functional, even if its not calling the pip CLI ?

[01:18:16] <lifeless> dstufft: ^

[01:19:00] <dstufft> lifeless: they can do IO yea, our definition of unit is a little wrong, it's really more like "does this create a virtualenv and need to invoke hte CLI and thus is it really slow" or "is it fastish"

[01:19:18] <lifeless> I don't subscribe to the 'unit === no IO' but I know others do

[01:19:20] <lifeless> so I like to ask :)

[01:21:56] <dstufft> lifeless: if you have a moment to look at https://github.com/pypa/pip/pull/2636, it looks right to me but since you're more familar with the refactors to this i'd love a double check

[01:23:44] <lifeless> the comment in the compat file is confusing

[01:23:49] <lifeless> should 'not' be considered installed ?

[01:24:51] <dstufft> lifeless: the list of stdlib packages was originally there in order to filter out standard library things that look like they are isntalled

[01:25:10] <dstufft> e.g. on some platforms argparse shows up in pip list natively, so that list is there to let us filter it out

[01:25:47] <lifeless> could this break things that use Requires(argparse > x.y) ?

[01:27:07] <dstufft> it just won't install argparse if they do that, Pytho puts the standard library before the site-packages, though I suppose it could break something if someone was doing some hacks to try and explicitly get argparse from site-packages

[01:29:22] <dstufft> maybe not worth the risk for an edge case

[01:30:11] <lifeless> I dunno. I have a suggestion on the code on the review for you

[01:34:35] <dstufft> lifeless: thanks, updated with your changes

[07:28:40] <dstufft> lifeless: https://github.com/pypa/pip/pull/2641 removed some of the crap that you noticed that was duplications :)

[09:45:23] <lifeless> dstufft: cool; uhm, I hope you merged my branch first :)

[09:46:07] <lifeless> gnight

[10:24:51] <straycat> dstufft, hey, i just noticed you closed my pr, any feedback you have for how to improve that pr would be greatly appreciated :)

[10:33:33] <dstufft> straycat: will take a look shortly, gotta take my daughter to the bus soon

[10:34:02] <straycat> awesome thanks :)

[13:21:37] <dstufft> lifeless: Ok, I've got to lay down for a bit. I've messed around with the wheel building PR a bit. I rebased it onto develop (which exposed a few issues with the PR), then i pulled out a few of the more indepenent commits and merged them on their own so that the diff itself of this PR is smaller and easier to read. You can see it at https://github.com/pypa/pip/pull/2649. I've commented there with the issues that I'm aware of now. If you want

[13:21:37] <dstufft> to continue working on it you can take my branch (or redo it if you want) and go from there

[21:06:56] <xafer> hello dstufft, could you restart the broken builds on https://travis-ci.org/pypa/pip/builds/57544212 ?

[21:17:05] <lifeless> dstufft: thanks

[21:32:21] <lifeless> dstufft: I am amuse at the argparse failure :)

[21:35:05] <dstufft> xafer: restarted

[21:35:15] <dstufft> lifeless: yea, it was certainly something I hadn't considered, ah well

[21:40:34] <xafer> thanks :)

[21:41:35] <xafer> seems better now

[21:44:31] <dstufft> lifeless: updated the ticket with some information, feel free to take it back. I just wanted to play with it and start to review it. I identified some issues in my PR that I noticed while doing that

[21:45:44] <xafer> dstufft, any chance for a 6.1.2 with https://github.com/pypa/pip/pull/2659 in a near future ?

[21:49:48] <dstufft> xafer: did this break with 6.1.0 but work with 6.0.x?

[21:50:20] <lifeless> dstufft: question fo ryou

[21:50:27] <dstufft> lifeless: sure!

[21:50:29] <lifeless> dstufft: you kept the installing_wheels flag

[21:50:36] <lifeless> dstufft: but discarded the use of it in unpack_url

[21:50:42] <lifeless> dstufft: I am confuse :)

[21:50:53] <xafer> dstufft, it worked on 6.0.8 yes

[21:50:56] <dstufft> lifeless: I probably just didn't notice.

[21:51:13] <dstufft> if the rebase didn't conflict I didn't touch it

[21:51:48] <lifeless> dstufft: this is likely another cause of failures

[21:52:06] <lifeless> dstufft: I'm redoing the rebase with your branch as a reference, so that my internal state is synced :)

[21:52:31] <dstufft> lifeless: other than pulling out the isolated-ish changes I tried to only make changes that were needed to cleanly apply. It's possible (maybe even likely!) that I didn't do it completely correct

[21:52:42] <dstufft> oh and I added a changelog entry

[21:52:51] <dstufft> feel free to toss my work or whatever makes the most sense to you

[21:52:55] <lifeless> kk

[21:53:25] <dstufft> xafer: I can probably do a 6.1.2 in a bit

[21:53:36] <dstufft> might try waiting till later tonight to make sure no other issues pop up

[21:54:13] <dstufft> lifeless: fwiw, aside from the issues I've noticed, it does generally work which is pretty awesome

[21:55:05] <dstufft> lifeless: hopefully my messing around didn't make things too hard for you :/

[21:55:23] <lifeless> dstufft: was useful

[21:55:30] <lifeless> dstufft: good to get some of the primitives merged

[21:56:33] <xafer> sure, no problem, it's just to know if I should patch our sytem (certainly pin pip to 6.0.8) or if I should just wait for the fix to land :)

[21:57:01] <xafer> In the mean time, good night

[21:58:16] <dstufft> lifeless: yea, makes the diff easier to review when the misc refactoring stuff is gone

[22:06:46] <lifeless> pushed up 2618 again

[22:09:12] <sigmavirus24> lifeless: how can you just push pull requests around like that? You're a monster =P

[22:09:13] <dstufft> lifeless: I'll close my PR then

[22:09:32] <lifeless> sigmavirus24: github gives little alternatives