[12:35:36] <techtonik> nanonyme: and also embedding 500k of html5lib with 2.8Mb in vendor dependencies
[12:36:00] <tomprince> techtonik: There are good technical reasons for that.
[12:36:49] <techtonik> Kuba: i think it is a valid bug request
[12:37:04] <nedbat> techtonik: don't expect it to change
[12:37:28] <techtonik> that's why i proposing a better name for this monsta =)
[12:37:48] <nedbat> techtonik: propose all you like. Just don't expect it to change :)
[12:38:15] <techtonik> nedbat: how about making pip modular?
[12:38:29] <techtonik> nedbat: I have a proposal for minipip core
[12:38:35] <nedbat> techtonik: I would like a pip API
[12:38:46] <tomprince> packaging is the start of that.
[12:39:04] <techtonik> nedbat: API needs usage scanarios first
[12:39:11] <tomprince> There are enough other issues, that an API isn't a priority.
[12:39:24] <tomprince> And modularizatioin isn't on the table at all.
[12:39:59] <techtonik> where is the priority table, by th way?
[12:40:42] <tomprince> I don't know if there is an explicit public one.
[12:42:01] <techtonik> ephemeral priorities is a showstopper
[12:46:27] <tomprince> Not at all. Things are improving. Quite rapidly, even, compared to pre-pypa days.
[12:48:02] <techtonik> It doesn't look so bright from outside. For example, virtualenv is failing on big percentage of Windows and there is no hope on the horizon.
[12:48:42] <techtonik> nedbat: showstopper means that people are excluded from improving parts that are not on priority list
[12:49:03] <techtonik> nedbat: and because there is no priority list for them
[12:49:10] <techtonik> nedbat: the show is stopped
[12:49:35] <nedbat> techtonik: it would be easier to contribute if you knew where the priorities were, I understand that. Maybe ask here?
[12:50:19] <techtonik> nedbat: well, the problem is that I need to match this list with my priorities
[12:50:39] <nedbat> techtonik: sure, which you would have to do if there were an explicit priority list also, right?
[12:51:00] <techtonik> nedbat: right, but it will be much faster
[12:51:12] <techtonik> nedbat: my priority list is to make pip modular
[13:00:48] <nedbat> techtonik: also, keep in mind: pip isn't going to change that.
[13:01:12] <techtonik> checkout of PyPy takes 40 minutes over ssh here
[13:01:26] <nanonyme> Well, that's understandable, PyPy is rather large
[13:02:10] <tomprince> There are good, solid reasons, why pip vendors lots of things, and that is not going to change.
[13:02:24] <techtonik> nanonyme: or maybe I am using 4k SSH certificate?
[13:02:35] <techtonik> because 1k is already insecure?
[13:03:18] <techtonik> but updating pip is not the bottom of the list also
[13:04:08] <nanonyme> techtonik, PyPy is dozens and dozens of megabytes to checkout. It's not really comparable to pip
[13:04:14] <techtonik> the bottom is that virtualenv doesn't unzip pip that is already in vendors
[13:04:32] <nanonyme> If you're checkouting PyPy every time, there's your problem
[13:05:23] <tomprince> virtualenv looks in a number of places for pip and setuptools. You could put the pip that you have in any of those places to get it to work. (I don't have a list handy).
[13:05:25] <techtonik> nanonyme: with vendor dependencies in repository it can grow very fast
[13:05:58] <techtonik> tomprince: this doesn't work with 1.11.6
[13:06:05] <techtonik> tomprince: it requires wheel
[13:06:19] <techtonik> tomprince: it can't install pip from tzr.gz
[13:07:09] <techtonik> and also virtualenv 1.11.6 does't wotk on my system
[13:07:31] <nanonyme> I'm getting more and more the impression this is about loads of XY problems
[13:08:05] <tomprince> Well, why do you need that tar.gz?
[13:08:37] <tomprince> Also, my understanding is that you have a slightly atypical windows setup, and that there is a fix, that just hasn't been released yet.
[13:10:11] <techtonik> nanonyme: right, nedbat proposed to try matching priority lists here, so it is not that easy
[13:10:48] <nedbat> techtonik: it sounds like you have a problem to fix. You should fix it without trying to reinvent pip
[13:12:49] <nedbat> techtonik: i'm sorry, I don't have any advice for you
[13:13:57] <tomprince> techtonik: Like I said, there is a fix for the virtualenv issue. So far, you seem to be the only person who has run into the issue, and the pypa team hasn't had a chance to make a release.
[13:14:08] <tos9> Writing a wheel installer that isn't pip is probably a bit more of a reasonable task.
[13:14:21] <tomprince> Being very noisy in here, complaining about things doesn't seem like it is likely to motivate them to do it sooner.
[13:14:34] <tomprince> wheel can unpack wheels, I think. :)
[13:14:57] <techtonik> tomprince: no, I am not the one - https://github.com/pypa/virtualenv/issues/660
[13:15:48] <tos9> tomprince: I knew it cna unpack them ... but apparently it can install them too. So never mind then :)
[13:15:58] <techtonik> well, I am complaining becaue my priority list is very emotional after 4 dayz =)
[13:17:29] <techtonik> tomprince: Iam not motivating or asking you to do something sooner
[13:24:12] <techtonik> If there was a priority list..
[13:24:52] <techtonik> ..I would think - "ok, the pypa team is focused on that, so lets' not bother them".
[13:25:19] <techtonik> ..and do this stuff in the meanwhile..
[13:25:29] <nedbat> techtonik: you're saying that if there was an explicit list that said, "We are working on stuff other than techtonik's bug", that you would be happy?
[13:25:50] <techtonik> nedbat: yup, but let me finish.
[13:26:29] <tomprince> techtonik: None of the people you have been talking to here in the last ~30 minutes are actually part of pypa (I don't think), so aren't able to answer your question about what there priorities are. (And those will vary from person to person too)
[13:26:32] <techtonik> ..but when I receive a reply "it is not on priority list"..
[13:27:19] <techtonik> ..it sounds like "we are going to fix it and we don't need any help"..
[13:27:45] <techtonik> ..so a person can not participate. Show stops.
[13:28:13] <nedbat> techtonik: when I hear, "it isn't on the priority list," I think, "they aren't fixing it, so I will try to." Send them a patch.
[13:28:52] <techtonik> tomprince: not knowing who is on the team and what is everybody's priorities is another problem, but it is more internal to the team I'd say
[13:29:22] <tomprince> Well, there is https://github.com/orgs/pypa/people
[13:31:00] <tomprince> I know dstufft is working on PEP470 right now. And may also be involved in SSL work?
[13:32:11] <tomprince> And probably other stuff, including his day job.
[13:34:31] <techtonik> nedbat: well, that's a difference in a mindset.. some of my friends are working in a corporation where priority level 3 means it will never be fixed, so why would you invest your time if it won't be reintegrated anyway?
[13:35:06] <techtonik> tomprince: sometimes I think that dstufft is working alone
[13:35:54] <techtonik> why there are thousands of Python developers who can't help?
[13:36:07] <nedbat> techtonik: packaging is not what most people want to work on.
[13:36:18] <nedbat> techtonik: also, you are asking questions that we don't really know the answers to.
[13:36:37] <techtonik> nedbat: but it is the stuff that most people want to work right
[13:36:49] <techtonik> nedbat: and that is enough for motivation.
[13:36:53] <nedbat> techtonik: as tomprince has pointed out, there are nine people in the pypa org, that's pretty good.
[13:37:23] <techtonik> nedbat: and how active they are? how many time can they dedicate to that?
[13:37:29] <tomprince> techtonik: dstufft is in the lucky position that rackspace lets him spend part of his work time on packaging work. I don't think any other members of pypa are in that position.
[13:37:31] <nedbat> techtonik: how would I know that? :)
[13:37:48] <nedbat> techtonik: this conversation is getting off-track I think.
[13:37:56] <tomprince> So, obviously he is going to contribute more than people who don't have that luxury.
[13:38:17] <nedbat> techtonik: i have no idea what the word "pulse" means here.
[13:38:28] <techtonik> i might know the answers if you want to go a bit offtopic
[13:38:57] <techtonik> tomprince: the point is to allow other people to contrinute more in less time
[13:39:01] <tomprince> nedbat: github pulse (I guess), for a project (or maybe an organization). It is a summary of recent activity.
[13:39:15] <techtonik> tomprince: not to get more people with more time
[13:39:38] <nedbat> techtonik: i know you have theories about how to get more people to contribute. I don't work on pypa, so convincing me is pointless.
[13:39:54] <techtonik> nedbat: i didn't know that github implemeted it, but it is right - some general idea what is going on
[13:40:24] <techtonik> nedbat: i am not convincing anybody, i just have ideas )
[13:40:29] <tomprince> What point? People are free to contribute however they wish.
[13:42:03] <techtonik> tomprince: the point is that people hit Norris Number and stop
[13:42:13] <techtonik> tomprince: and the project stalls
[13:51:55] <nedbat> techtonik: that can be frustrating, I understand.
[13:56:07] <techtonik> nedbat: but understanding doesn't fix things. working with the causes fixes things, or at least a proof that something is going on. the cause are interesting though, but require a ful anbstraction from the pypa team
[14:01:04] <techtonik> nedbat: for example, what is the problem to release a virtualenv with a bugfix right now?
[14:01:56] <nedbat> techtonik: releasing code is a complicated process sometimes, and they may be waiting for other bugs to be fixed. You'd have to ask the maintainers.
[14:03:18] <techtonik> nedbat: I asked dstufft, he didn't reply. So, the cause now looks like dstufft doesn't reply. Is it the real cause?
[14:03:22] <nedbat> techtonik: or they might just be busy with other priorities.
[14:03:31] <nedbat> techtonik: of course, I have no idea.
[14:04:54] <techtonik> nedbat: Ok. The cause is that known maintainers are busy. How to fix that?
[14:05:09] <nedbat> techtonik: offer to help them with things they need help with?
[14:17:15] <techtonik> nedbat: 200 issues vs 233 pull requests
[14:17:25] <tomprince> techtonik: You have brought a lot of negative energy here. That isn't likely to cause the result you want.
[14:17:34] <nedbat> techtonik: i'm not sure what your point is.
[14:17:42] <techtonik> there is only one secret to that - rapid workaround
[14:18:16] <techtonik> nedbat: the faster the cycles, the more people are kept interested, not only the core team
[14:18:26] <nedbat> techtonik: you seem to be saying that people need to work harder on the problems you care about.
[14:18:31] <nedbat> techtonik: that won't win you friends.
[14:18:47] <techtonik> nedbat: where did I said that?
[14:19:08] <nedbat> techtonik: you are saying that the virtualenv team should release faster.
[14:19:42] <techtonik> nedbat: where the "work harders" stuff here?
[14:20:09] <nedbat> techtonik: I don't want to get into hair-splitting with you. You aren't happy with how the maintainers are doing their work, right?
[14:21:15] <nedbat> techtonik: you've have a reputation of critiquing the way projects are run, without providing solutions that people want.
[14:23:45] <techtonik> nedbat: shorter cycle means just release bug fix ASAP, and that's it. i don't want to spoil the fun for volunteers if they are happy with their activities. i want to say that people (like me) will be happier not waiting for them to complete their own deeds
[14:24:14] <nedbat> techtonik: sure, that's kind of obvious.
[14:24:26] <nedbat> techtonik: what's not obvious is how to make it happen.
[14:24:52] <nedbat> techtonik: you have proposals for solutions ("so they need a tool that can help them communicate better"), which don't sound like solutions to the maintainers.
[14:26:09] <techtonik> nedbat: if by "providing solution" you mean "implementing them" then I am not the person capable of coding warehouse 2.0 - i'd rather patch PyPI, but PyPI maintainerrs were strongly against that.
[14:26:58] <nedbat> techtonik: I said, "proposals for solutions"
[14:27:31] <techtonik> nedbat: I get that you mean. Things like "public priority list" is not useful for them at all.
[14:28:18] <techtonik> nedbat: It is more a nuisance and too much energy, and it is not fun for volunteer work. Something like this?
[14:28:37] <nedbat> techtonik: perhaps. I don't know why they don't have a public priority list.
[14:31:07] <techtonik> nedbat: I can guess that there is no feedback from users that this list is useful to them.
[14:31:58] <techtonik> nedbat: For me the importance of actual roadmap is very obvious, but it is impossible to explain.
[14:32:15] <techtonik> nedbat: It is important for coordination.
[14:33:07] <techtonik> nedbat: But sometimes it is hard to get that list right.
[14:33:24] <techtonik> nedbat: So the solution was just to arrange bugs in order.
[14:33:27] <nedbat> techtonik: i'm not saying a public roadmap is a bad thing. I'm saying I don't know why there isn't one.
[14:33:45] <nedbat> techtonik: and I'm saying that I don't know if it will make virtualenv development go faster
[14:36:16] <techtonik> I think that if every person from #pypa team could just prioritise the bugs he or she is working on (or willing to join if there would be some sprint) in a list, that could make a big difference. Together with independent bugs decomposition, dependency assignment and list merging.
[14:38:29] <techtonik> This way everybody could just say - "well, I am focused on this, so I don't know who can help you, but you can see that there are some people working in related issues they're subscribed to".
[14:40:35] <techtonik> This will enable people to join and work in parallel. Also, if there is no pulse on person's top issue, then it might be that he is busy and development is stalled.
[15:31:39] <dstufft> I don't personally have a public roadmap because a) the time spent maintaing such a list could be spent actually fixing thigns and b) my list of prioities changes quite often based on what's happening in the greater ecosystem around me
[15:34:26] <[Tritium]> something something more manpower something something later
[15:41:39] <techtonik> dstufft: so how do you think what is the reason why bug fix releases don't happen for virtualenv?
[15:47:36] <dstufft> techtonik: well releases *do* happen. There was an issue where our branching strategy made it confusing about what was going to be released when, and some things got stuck on th enext major release instead of the next bug fix release. We've simplified things greatly though.
[15:47:54] <dstufft> There's now only two branches and after the next release we'll probably drop that down to one
[15:56:00] <[Tritium]> I am going to assume that means 'one canonical branch' cause something something develop fature in own branch something something merge into master
[15:57:28] <dstufft> previously we had master, develop, and then one for each release series (X.Y)
[15:57:51] <dstufft> now we have master and develop, and that'll switch to just master once we do the next release
[16:35:37] <nanonyme> dstufft, I quite like the idea of having one branch and tagging releases through git. Then you can create maintenance branches on-demand easily
[16:36:07] <nanonyme> (not sure if it's even necessary in the context of your project to have a maintenance branch)
[16:49:34] <[Tritium]> You really only need them if you have multiple supported versions
[16:55:33] <techtonik> dstufft: why releases don't happen as soon as they are needed?
[17:07:42] <techtonik> speaking of XY problems, I decided not to try bootstrap pip with virtualenv, and looking how much stuff python-pip fetched as dependencies, it seems that it was the right way
[17:08:48] <[Tritium]> >.> I never install python-pip willingly. `python get-pip.py --user`
[17:12:51] <techtonik> now I am getting thsi error - https://github.com/pypa/virtualenv/issues/656
[18:34:02] <techtonik> is that because pip depends on it?
[19:14:50] <cob> hi, why don't we get a new command for pip, related to freeze, that fixes versions to minor instead of patch... eg. >=1.7,<1.8
[19:16:22] <cob> in order to get builds run against the freshest patches, methinks it'd help version bumping be more agile
[19:20:38] <nedbat> cob: you can edit the requirements file to do that right?
[19:21:27] <cob> nedbat: sure, but that wouldn't get everyone running builds with freshest patches quickly, causing entire ecosystem to evolve more quickly
[19:21:47] <nedbat> cob: how often do people use pip freeze to create a requirements file? I edit mine by hand.
[19:22:32] <cob> i dunno really, i'm just guessing it'd advertise this notion of freezing requirements to minor instead of patch
[19:22:40] <nedbat> cob: your idea also depend on version number meaning something very reliable.
[19:38:15] <nedbat> cob: hmm, i'd prefer to choose to upgrade, and be able to test
[19:39:54] <cob> so run production with patch frozen, but have development frozen to minor
[19:58:54] <cob> i think it'd help a few things... motivate more tests and ci, optimism about python ecosystem, facilitate contributions, enforce semver patch version to what patch really is, perhaps more
[20:00:34] <cob> it's just a thought, i dunno if a pep would be appropriate
[20:01:19] <nedbat> cob: you don't need a pep to add a feature to pip
[20:03:59] <nanonyme> [Tritium], the tagging allows you to decide later if you or someone else decides multiple supported versions is a must-have thing
[20:04:22] <nanonyme> Erm, rather it allows to post-pone decision of foo. Anyway
[20:19:53] <dstufft> cob: a flag to freeze is probably more appropiate than a whole new top level command
[20:31:54] <nanonyme> Might as well have a flag that drops out version information completely and just keeps package names while at it
[20:41:37] <nanonyme> I dunno, my mind is filled with bazillions of corner-cases with each idea
[20:46:44] <cob> i think --format is cumbersome, but it could be an added flag
[20:48:32] <cob> pip list, --freeze, and --freeze-minor are the only truly useful ones i can think of
[20:49:09] <nanonyme> I meant, --format would have allowed you to tweak output. Obviously there would have been a default
[20:49:49] <nanonyme> FWIW does anyone really use freeze as is? I've always found it rather clunky and end up rewriting the versioning into a range of versions