[11:32:07] <pthiem> ivo, yeah in the case of the manifest everything should be utf-8 encodable, but the are a couple other things in write_file outputs
[11:32:37] <pthiem> (should be = have already been weeded out)
[11:32:54] <Ivo> dstufft: i see last commit as 15 hours ago
[11:37:52] <dstufft> I don't like Hg so I don't know a lot about it
[11:38:25] <pthiem> ivo: it at least recognizes them, though I seems that I have to identify them by hash
[11:38:27] <Ivo> we need less hg-for-svn-developers and more hg-for-git-developers
[11:38:32] <dstufft> (infact I dislike hg enough instead of trying to use branches, I just store multiple patch files and I use patch to apply them to "switch" branches
[11:41:04] <Ivo> does travis have post-build hooks?
[12:07:01] <pthiem> jaraco, I was taking a look at that regressions you mentioned
[12:09:37] <jaraco> pthiem, Glad to hear it. To be sure, I'm not blaming your PR or work on 3.7. It's quite possible, likely even, that I made changes to your PR that were implicated.
[12:09:43] <Ivo> jaraco: is there a particular reason you include setuptools.egg-info/ under version control?
[12:10:24] <jaraco> Ivo, it's because setuptools can't build without having setuptools present (and its entry points).
[12:10:49] <jaraco> So without having them in the source code, a virgin Python can't install setuptools from the repo.
[12:11:14] <Ivo> jaraco: it can't install itself with distutils?
[12:15:06] <agronholm> so API incompatible changes were made?
[12:15:44] <jaraco> 4.0 removed an expectation about how scripts are installed by setup.py develop. If someone depended on the behavior to copy the files as text, this change would violate that expectation.
[12:16:41] <agronholm> I myself also try to follow the semantic versioning thing
[12:16:46] <jaraco> It's a subtle change, and potentially could have been considered a bugfix or simple enhancement... but it felt more like a contractual change to me (though small).
[12:17:24] <agronholm> it still feels like setuptools is getting releases a bit too often
[12:17:33] <Ivo> setuptools supposed to run on python 3.1 any more?
[12:20:58] <pf_moore> virtualenv 1.11.6 bundles setuptools 3.6 which "feels old". I know it's just a pip install -U away, but hey.
[12:21:41] <dstufft> jaraco: FWIW I'm not one to tell you how to run setuptools or anything :) just saying that it's a problem with frequent releases, the flipside to infrequent releases is that stuff gets locked away fixed/added but unreleased for some period of time
[12:21:42] <agronholm> also, you probably wouldn't have to bump the major version so often if you consolidates API breaking changes over a longer time period
[12:23:10] <jaraco> agronholm, but then every client would have to absorb all breaking changes at once. Having multiple releases means each client has control over which breaking releases to accept and when.
[12:23:37] <pf_moore> My understanding of semver is that API-breaking changes are bundled up to make a major releases
[12:23:45] <Ivo> 4.x isn't nearly as different to 2.x as 2.x is to 0.x though
[12:23:55] <jaraco> Right. So if we're using semver, we have to shake the perception that a major version bump means a lot of changes. It doesn't mean that at all, but only means that some contract was broken since 3.x.
[12:24:23] <dstufft> well semver states that any breaking change is bumped, although I think in practice that tends to mean changes are bundled together like that for a lot of projects, it doesn't have to mean that
[12:24:24] <agronholm> jaraco: then clients have to adapt so many times
[12:24:37] <pf_moore> Yeah, but doesn't it *also* mean that upgrades through a major version carry a risk?
[12:24:49] <agronholm> "If even the tiniest backwards incompatible changes to the public API require a major version bump, won't I end up at version 42.0.0 very rapidly?"
[12:24:54] <agronholm> "This is a question of responsible development and foresight. Incompatible changes should not be introduced lightly to software that has a lot of dependent code. The cost that must be incurred to upgrade can be significant. Having to bump major versions to release incompatible changes means you'll think through the impact of your changes, and evaluate the cost/benefit ratio involved."
[12:25:01] <jaraco> dstufft, that's a consideration I hadn't made. I presumed, perhaps incorrectly, that most systems did not upgrade to the latest automatically. By adopting semver, setuptools declares that it expects its users to give deference to that adoption.
[12:25:39] <pf_moore> Also, if you don't bundle major changes, people lose access to bugfix changes over a major release
[12:26:01] <agronholm> jaraco: are you going to do any 3.x setuptools releases?
[12:26:03] <jaraco> agronholm, I don't see any problem with version 42.0.0. It's a bit more to type, but negligibly so.
[12:26:03] <pf_moore> That's less friendly for users who want to only auto-update painless releases
[12:26:34] <dstufft> jaraco: setuptools and ensurepip pin to a specific setuptools version, so if they are using a virtual environment they'll at least start out with a particular version, but it's not unusual for something to trigger it to get updated
[12:26:40] <agronholm> or do I have to upgrade to 4.x to get bugfixes
[12:27:19] <jaraco> pf_moore, people _can_ lose access to bugfix changes as necessary. And the same is true for 0.x releases as well, no?
[12:28:15] <jaraco> agronholm, you're right, I mis-edited.
[12:28:27] <jaraco> I meant to say bug fixes can be back-ported as necessary.
[12:28:49] <pf_moore> OK, but there's no process of doing so ATM, is that right?
[12:28:52] <agronholm> if they WILL be backported to lower major versions, then it's not such a big problem
[12:28:58] <pf_moore> It's possible to argue (not that I am) that Python 3.4 should bundle setuptools 2.x only because going to 3.x or 4.x breaks Python's compatibility requirements.
[12:28:59] <jaraco> And if there are essential fixes (i.e. security fixes) that need to be back-ported to 1.x, I still plan to support that.
[12:29:05] <dstufft> jaraco: it's also true that most people don't have control over what their end users have installed, so more "breaking" versions means a larger matrix of backwards incompat versions to test (or at least be aware of)
[12:29:37] <dstufft> pf_moore: pip/setuptools are exempt from that
[12:30:17] <pf_moore> dstufft: I know (hence I wasn't suggesting it) but it's an easy example of how setuptools' versioning impacts what other tools can bundle.
[12:30:44] <pf_moore> If virtualenv had strict compatibility rule like Python, we'd have the same question there with 1.11.x
[12:30:55] <jaraco> dstufft, the alternative is not to fix issues that have backward-compatible releases or to implement those changes but not declare the compatibility concerns, which is what most other projects do.
[12:31:19] <jaraco> (not declare the compatibility concerns using semver, I mean)
[12:33:58] <dstufft> jaraco: you mean backwards incompatible?
[12:35:39] <dstufft> jaraco: sure, I mean you can also hold off on backwards incompat changes until you have a couple of them to bundle together. It's a larger pain, but it's condensed to one episode instead of multiple episodes
[12:36:06] <dstufft> jaraco: FWIW I think you should do setuptools however you think is appropiate, so please don't feel like i'm trying to tell you how to run it :)
[12:36:50] <dstufft> jaraco: I do know that i've gotten some feedback (particular from openstack) where they are feeling somewhat exasperated about how frequently setuptools is released (or more specifically, how frequently setuptools is released that breaks something for them)
[12:37:16] <Ivo> pthiem: no idea how that error occurred.. but it occured with pypy somewhere completely random -_-
[12:37:34] <jaraco> dstufft, In my defense, I did release a beta release of 3.7 with a tweeted announcement.
[12:37:46] <dstufft> jaraco: I also think this is _really_ hard to figure out what's going to break things for people, becuase of the nature of the situation where you have distutils, then setuptools layered on top of it and then pip on top of that
[12:38:15] <dstufft> and none of the boundaries between those things are particularly well defined :(
[12:38:23] <jaraco> Sometimes, it seems the only way to get feedback is to release and await feedback, and then try to respond quickly.
[12:40:48] <dstufft> jaraco: I don't actually know what the right answer is or anything :) I just am more or less parroting what i've seen a few people describe recently
[12:40:56] <pthiem> jaraco, maybe some of the zipcach stuff added post 3.7?
[12:47:03] <dstufft> openstack fixed their CI just now by blacklisting setuptools 3.7+ from their mirror (they don't install from PyPI in their CI) but it's stil broken on folk's laptops and such
[12:54:26] <pthiem> jararco, actually i'm not sure how unicode from the file list would be migrating to the requirements. I could try to bisect it there (have work), but there are some simple patches which paport to fix the issue.
[12:57:52] <pmxbot> jaraco pushed 6 commits to setuptools (https://bitbucket.org/pypa/setuptools/) :
[12:57:52] <pmxbot> Use compat's StringIO. Should fix bitbucket #213
[13:19:28] <Ivo> dhellmann: you might have noticed ppl complaining about a setuptools bug relating to pdb which broke virtualenvwrapper, you might consider uploading a wheel for it which would fix that in the future :)
[13:19:51] <dhellmann> Ivo: I was going to offer to help add tests to setuptools, actually
[13:20:09] <Ivo> dhellmann: I'll take that any day of the week
[13:20:25] <dhellmann> is there a test suite that tries installing packages? I'm looking at https://github.com/jaraco/setuptools/tree/master/tests and don't see much
[13:20:29] <dhellmann> is that the right place to look?
[13:20:39] <dhellmann> ah, I see more in https://github.com/jaraco/setuptools/tree/master/setuptools/tests
[13:20:46] <Ivo> I suspect adding some really-unicode-using package to test installing might help with regressions
[13:21:28] <Ivo> dstufft: any type of emails in particular?
[13:21:39] <dstufft> Ivo: the ones where it says "so and so did a thing on this ticket"
[13:21:42] <dstufft> they reuse the same message id
[13:21:50] <dstufft> so mail.app collapses it into the old message
[13:21:55] <dstufft> even though the content is the same
[13:22:26] <jaraco> dhellmann, I would very much welcome a test for this recent regression.
[13:22:42] <jaraco> (or anything else that helps maintain contracts on which projects rely)
[13:22:52] <dhellmann> jaraco: I'm thinking "integration" style tests that actually install packages. Does that make sense?
[13:22:52] <dstufft> http://d.stufft.io/image/012V181p031E this is the only email I've gotten as far as Mail.app is concerned, but I've gotten in like 8 times (even though those 8 times were actually different messages with different contents, but the same message id)
[13:23:47] <Ivo> dhellmann: scripttest might help.. I've been wanting to do that with virtualenv as well
[13:25:13] <jaraco> dhellmann, that does make sense, if it can be done in a robust and reliable way.
[13:25:44] <dhellmann> jaraco: it would tie the tests to pypi availability, which might cause some issues on occasion
[13:26:15] <dhellmann> it wouldn't need to be super complex, though: given a list of dist names and python package names, create a virtualenv, install the dist, active the virtualenv, try to import the package
[14:38:14] <dhellmann> jaraco: When you have a few minutes, please let me know if https://github.com/jaraco/setuptools/pull/8 is heading in the right direction. The tests that are commented out fail, even though I can install those packages, so I think I'm missing some setup work.
[14:45:02] <Ivo> dhellmann: you could really just copy the .hgignore
[15:20:21] <dhellmann> Ivo: ok, I can look at that. I'm more worried about whether what I'm doing to set things up is correct, and about the fact that some of those tests fail for reasons I don't understand. I can update the fixture to use tmpdir and monkeypatch after we work that out.
[15:21:44] <dhellmann> Ivo: I mean the tests I have commented out because they don't work for me locally. I guess I can uncomment them to show the failures in travis.
[15:21:50] <Ivo> dhellmann: same failure I got before, I think its related to another commit after 4.0 which needs to be fixed