pmxbot IRC Log Viewer

[01:26:12] <Alex_Gaynor> wat.

[02:30:45] <kuramanga> Evening, is there any official/unofficial info on when PyPI will be replaced by warehouse? dstufft?

[02:31:12] <dstufft> Whenever it's got all of the features implemented that PyPI has which we aren't getting rid of

[02:31:48] <kuramanga> dstufft: is there a list of those features anywhere? I had a look through the docs but couldn't find any info.

[02:32:27] <kuramanga> dstufft: is the JSON data URI for packages being deprecated? How about the two RSS feeds?

[02:32:44] <Alex_Gaynor> kuramanga: https://github.com/pypa/warehouse/milestones/Become%20PyPI

[02:33:00] <kuramanga> Alex_Gaynor: thanks

[02:33:31] <dstufft> that's not an exhaustive list

[02:33:47] <kuramanga> dstufft: no problem, I guessed that may be the case.

[02:33:49] <dstufft> that's the list of things I Could think of when I made the milestone

[02:34:02] <dstufft> you mean /pypi/<foo>/json?

[02:34:12] <kuramanga> dstufft: yeah

[02:34:30] <dstufft> That is not being removed, I think we're techincally deprecating it where that just means "we're going to have a better API, you should use it instead" but won't remove it

[02:34:44] <dstufft> I don't believe we are removing the RSS feeds either

[02:35:17] <dstufft> Mostly we are removing things either people aren't using or which are generally causing issues

[02:35:17] <kuramanga> dstufft: good to know, it seems to be missing currently, I tried several combinations using the current warehouse.python.org example and couldn't get a working one, hence the question.

[02:35:31] <dstufft> https://warehouse.python.org/pypi/Django/json ?

[02:35:45] <kuramanga> Ah it's pypi/<foo>

[02:36:01] <dstufft> yes because that's what the URL is on PyPI :)

[02:36:04] <kuramanga> The project/<foo> was the confusing bit, I tried json/<foo>

[02:36:08] <dstufft> ah

[02:36:15] <kuramanga> And project/<foo>/json

[02:36:16] <dstufft> there will be a better API that'll be somewhere

[02:36:21] <dstufft> dunno where yet

[02:36:25] <kuramanga> No worries.

[02:36:31] <dstufft> but that one will stay there and continue to exist

[02:36:39] <kuramanga> Good to see you've picked up the JSON API empty string annoyance.

[02:36:52] <kuramanga> Currently I find empty strings and convert them to None with Yarg.

[02:37:25] <kuramanga> I'm asking this stuff though because I'm giving Yarg the ability to search using the newer Warehouse version, 'cos I'm bored.

[02:37:47] <dstufft> Ideally Warehouse should be a drop in replacement for end users, but realistically I'm sure it's going to break shit when we do because I'm sure people have all sorts of random wierd shit about the current PyPI encoded into their apps

[02:38:04] <kuramanga> dstufft: I imagine so.

[02:38:28] <kuramanga> dstufft: With the huge numbers https://pypip.in is currently handling, I'm sure I'll be submitting some bug reports. =P

[02:38:57] <dstufft> :D

[02:38:58] <kuramanga> That's why I want to get myself familiarised with warehouse now.

[02:39:16] <dstufft> I haven't been working on it lately, been focusing more on getting a new pip finished

[02:39:20] <kuramanga> Because Yarg is the library that powers pypip.in now so I want to be as ready and stay on top of it as I can be.

[02:39:20] <dstufft> then will go back toWarehouse

[02:39:36] <Alex_Gaynor> dstufft: how close are we to pip 1.6?

[02:39:42] <dstufft> 6.0 now!

[02:39:50] <Alex_Gaynor> whatever

[02:40:20] <kuramanga> Any new fun stuff in pip? Are we gonna have a nice new Node-like format for dependencies? So I can finally finish my dependency manager without wanting to kill myself?

[02:40:53] <dstufft> Alex_Gaynor: the last thing I really care about getting in to it is PEP 440 which there are patches for both setuptools and pip, those are almost done

[02:41:05] <dstufft> I have a test job queued on Openstack to test that

[02:41:10] <dstufft> via devstack

[02:41:17] <dstufft> kuramanga: nothing like that yet

[02:41:17] <kuramanga> If we could just like, have dependencies in one place dstufft I will be a happy boy.

[02:41:44] <dstufft> automatic http cache now

[02:41:47] <dstufft> which is on by default

[02:41:51] <kuramanga> And find a way to stop people doing annoying stuff like open('requirements.txt').read().split()

[02:41:51] <dstufft> um

[02:41:55] <dstufft> retries for network errors

[02:42:13] <dstufft> virtualenv and site wide config files

[02:42:14] <kuramanga> Oh, nice.

[02:42:18] <kuramanga> Retries will be very cool.

[02:42:34] <kuramanga> Flaky internet connection on a train while trying to pip install is pretty annoying.

[02:42:50] <kuramanga> *flakey (sp?)

[02:42:53] <dstufft> it doesn't handle 503 errors from Fastly (I want to make it do that, but it doesn't yet)

[02:42:59] <dstufft> but connection errors and stuff it should auto retry

[02:43:03] <dstufft> 3 times I think

[02:43:04] <dstufft> configurable

[02:43:15] <kuramanga> Nice. It uses a vendor version of requests right?

[02:43:18] <dstufft> yea

[02:43:36] <kuramanga> I thought I'd seen that when I was browsing the source a few months back, good move.

[02:43:38] <dstufft> totally revamped the logging infrastructure to use the stdlib logging so a bunch of unicode issues are gone now if that's your kind thing

[02:44:06] <kuramanga> What about doing something like I did with Yarg, which is replace a bunch of the look-up stuff as a library?

[02:44:41] <kuramanga> 'Cos the new warehouse API seems to cover a lot of what I had done for the current PyPI, making it unified would be sweet and mean one less lib for me to maintain.

[02:44:52] <dstufft> as things become defined standards they'll be implemented in packaging

[02:45:03] <kuramanga> Good to see this finally happening!

[02:45:13] <dstufft> https://warehouse.python.org/project/packaging/

[02:45:27] <dstufft> pip will start using that with the PEP 440 stuff ^

[02:45:39] <kuramanga> Nice.

[02:45:56] <kuramanga> I'll have a look tomorrow when I wake up, see if there's any low hanging fruit I can kill off for warehouse.

[02:46:16] <kuramanga> And maybe start talking to jezdez again about working on the wheel build farm I wrote a proposal for.

[02:47:11] <kuramanga> dstufft: has anyone else been talking about or working on a wheel farm or has it just been my random convo with jezdez, to your knowledge?

[02:47:28] <dstufft> It's something I would very much like to do as an official ofering of PyPI

[02:47:43] <dstufft> but higher prioroity things at this moment

[02:48:02] <kuramanga> Yeah, that was what my proposal was for. Jannis and I were talking when I was bored and he said write a proposal for how it could be done, so I did.

[02:48:31] <kuramanga> Nothing official but nor was it something as a third party service.

[02:48:38] <kuramanga> Would be sweet to have one though.

[02:49:32] <kuramanga> This was just something I did when bored, it's not detailed or anything but - https://kura.io/2014/05/29/python-platform-and-version-dependent-wheel-build-bot-proposal/

[02:49:34] <dstufft> Alex_Gaynor: just merged the PR to get rid of the /simple/$name/$version/ urls and deprecate the fallback to /simple/ heh

[02:49:52] <Alex_Gaynor> dstufft: which paths is this fewer requests in?

[02:50:02] <dstufft> anytime you use ==

[02:50:12] <dstufft> one fewer request per ==

[02:50:15] <Alex_Gaynor> so pip install "Django==1.4" now does one fweer?

[02:50:16] <Alex_Gaynor> nice!

[02:50:19] <Alex_Gaynor> !m dstufft

[02:50:19] <pmxbot> you're doing good work, dstufft!

[02:50:26] <dstufft> also I merged the normalization thing

[02:50:29] <dstufft> a little bit ago

[02:50:44] <dstufft> so ``pip install dJanGo`` takes one request always now

[02:50:53] <dstufft> well one to discover and one to download

[02:51:09] <Alex_Gaynor> dstufft: 43 == requirements for the pycon site, should be a nie speed up!

[02:51:29] <dstufft> plus http caching on by default

[02:51:33] <dstufft> this should be a zoom zoom release

[02:51:37] <kuramanga> dstufft: are there any plans on the board for package case sensitivity?

[02:51:45] <dstufft> kuramanga: not sure what you mean

[02:52:03] <kuramanga> dstufft: well, for example on PyPI Django != django

[02:52:15] <dstufft> yes it does

[02:52:18] <kuramanga> You have look up with the correct case.

[02:52:37] <dstufft> well techincally it's sometimes it does

[02:52:38] <dstufft> heh

[02:52:45] <kuramanga> =D

[02:52:48] <dstufft> for uniqueness constraints we apply a cannonicalization thing

[02:53:13] <dstufft> and for the /simple/ API we always redirected things to their "real" name, as long as the name you entered would cannonlize to the same thing

[02:53:14] <kuramanga> I see. Fair enough.

[02:53:34] <kuramanga> Is this a new addition?

[02:53:40] <dstufft> the redirect? No

[02:53:52] <dstufft> /simple/DjAnGo/ would redirect to /simple/Django previously

[02:53:53] <kuramanga> Hmmm.

[02:54:10] <kuramanga> It used to break. Maybe it was a long time ago. Ignore that then.

[02:54:12] <dstufft> recently we switched it to the redirect takes you to the cannonical form instead of the form that the author entered

[02:54:29] <dstufft> so that /simple/DjAnGo/ and /simple/Django/ redirect to /simple/django/

[02:54:36] <kuramanga> Gotcha.

[02:54:43] <dstufft> this means that pip can now normalize before making the request and not have to rely on redirects

[02:54:54] <dstufft> You might be thinking of /pypi though

[02:54:55] <kuramanga> Yeah, hence the fewer requests you previously mentioned.

[02:55:04] <kuramanga> Well, I just tried it with /pypi

[02:55:06] <kuramanga> And it works

[02:55:20] <dstufft> previously on /pypi you <had> to type it in with the exact name or you'd get an error

[02:55:29] <kuramanga> Yeah, but that's now changed?

[02:55:44] <dstufft> at the same time I made the change to redirect /simple/ to the cannonical form I made it so /pypi/ would redirect to the "real" form

[02:55:50] <kuramanga> 'Cos it certainly is working now, which is great and means I can change pypip.in documentation.

[02:56:07] <kuramanga> I <3 you for that change, I really do.

[02:56:33] <dstufft> with Warehouse I might end up changing that so that /project/{name}/ is always the cannonical form and it just uses the "real" name that the author added in the actual body of the page

[02:56:34] <dstufft> I dunno

[02:56:53] <kuramanga> As long as I can get the same JSON data no matter the case, I'll be happy.

[02:57:17] <dstufft> but yea, anyplace where it doesn't consider things equivilant is more or less a bug and should be fixed

[02:57:31] <kuramanga> 'Cos right now the documentation for pypipin has a stupid part explaining what case sensitivity is and I know from the server logs that people have had to try a few times to get it right...

[02:58:41] <dstufft> yea that should be fixed now

[02:59:04] <kuramanga> It is indeed, just tested it out and it's all grand.

[03:01:28] <kuramanga> Final question, the difference between requires= and install_requires= in setup.py, I guess that's something to do with distribute vs. setuptools? Plans to remove? Need a ticket put somewhere so it's written? If so, then where would you like it?

[03:02:19] <Alex_Gaynor> requires= does nothing, never use it

[03:02:26] <Alex_Gaynor> install_requires does what you want

[03:03:05] <kuramanga> Alex_Gaynor: in the setup.py itself yes, but I believe requires= is what PyPI itself is using to display what dependencies a package has on the website.

[03:03:09] <dstufft> No

[03:03:12] <dstufft> nothing uses requires=

[03:03:14] <dstufft> ever

[03:03:19] <dstufft> wellll

[03:03:22] <dstufft> Ok I lied slightly

[03:03:24] <kuramanga> Are you sure?

[03:03:32] <dstufft> PyPI will display a requires=, but it's not what you want

[03:03:47] <dstufft> requires=[] does not specify project names

[03:03:54] <dstufft> is specifies _importable_ names

[03:04:03] <kuramanga> Really?

[03:04:05] <dstufft> yes

[03:04:09] <dstufft> and it verifies that

[03:04:19] <dstufft> so it will fail if you try to say put django-wahtever in there

[03:04:42] <Alex_Gaynor> Quality Features.

[03:04:46] <kuramanga> 'Cos to get https://pypi.python.org/pypi/yarg to say on the website that it requires requests

[03:04:49] <kuramanga> I had to do https://github.com/kura/yarg/blob/master/setup.py

[03:05:03] <dstufft> you could also use install_requires and upload with twine

[03:05:09] <dstufft> (you should upload with twine anyways)

[03:05:16] <kuramanga> I could use twine, I really should...

[03:05:25] <kuramanga> I have it installed and I always mean to use it, but I keep using make pypi. =X

[03:05:35] <Alex_Gaynor> update make pypi to do teh right thing.

[03:05:36] <Alex_Gaynor> QED>

[03:05:47] <kuramanga> Alex_Gaynor: you're right, I know. =(

[03:06:00] <dstufft> PyPI supports requires-dist, which was the distutils2 style replacement to install_requires

[03:06:03] <kuramanga> But then it means I have to be careful about being in the right envs and having the right stuff and awergsdgsgr

[03:06:06] <dstufft> Wheels use Requires-Dist

[03:06:17] <kuramanga> Ok cool, fine.

[03:06:25] <dstufft> so uploading a Wheel with twine will generate the correct metadata

[03:06:27] <kuramanga> But you see, I was right, it does do what I thought it does!

[03:06:43] <dstufft> but setuptools could also send that data if you wanted to

[03:06:54] <dstufft> the problem is it'd be wrong if it did anytime someone used a conditional

[03:06:55] <dstufft> :|

[03:06:56] <Alex_Gaynor> setuptools should stop doing things :-(

[03:07:08] <dstufft> yes I think setuptools should do less things

[03:07:11] <dstufft> focus on being a build tool

[03:07:13] <kuramanga> setuptools also does annoying stuff with certain values.

[03:07:22] <kuramanga> I forget if it's the documentation url or bug tracker

[03:07:39] <kuramanga> But if you set them in setup.py it totally ignores them and you have to set them on PyPI itself.

[03:07:42] <kuramanga> =D

[03:07:46] <dstufft> bug tracker url

[03:07:52] <dstufft> because it was never added tod istutils ever

[03:07:57] <kuramanga> Fine, make pypi now uses Twine, happy?!

[03:07:57] <dstufft> it was a thing that got added to PyPI

[03:08:01] <dstufft> yes!

[03:08:13] <kuramanga> now I have to do it for everything. >_>

[03:08:18] <dstufft> I really need to get around to making ``pip publish`` instead of twine

[03:08:33] <dstufft> so people jsut have it instead of needing to download a seperate thing

[03:09:12] <kuramanga> And also make it so I don't need to use python setup.py bdist_wheel then twine upload

[03:13:39] <kuramanga> On the plus side, I found somewhere I was still creating eggs and killed that off.

[13:56:23] <pmxbot> jaraco pushed 1 commit to setuptools (https://bitbucket.org/pypa/setuptools/) :

[13:56:23] <pmxbot> Merged in mesocody/setuptools (pull request #72)

[13:56:25] <pmxbot> jaraco pushed 4 commits to setuptools (https://bitbucket.org/pypa/setuptools/) :

[13:56:25] <pmxbot> Fix exclude list on python 3.2+

[13:56:25] <pmxbot> Commit the fix we did when testing python3

[13:56:25] <pmxbot> Chase upstream

[13:56:25] <pmxbot> Merged in mesocody/setuptools (pull request #72)

[15:02:41] <pmxbot> jaraco pushed 17 commits to setuptools (https://bitbucket.org/pypa/setuptools/) :

[15:02:41] <pmxbot> Normalize syntax

[15:02:41] <pmxbot> Extract method for generating exclude names

[15:02:41] <pmxbot> Generate the filenames more directly.

[15:02:41] <pmxbot> Extract calculation of base path

[15:02:41] <pmxbot> Add comment

[15:02:41] <pmxbot> Construct exclusions as a set

[15:02:41] <pmxbot> Add docstring for get_exclusions. Just return the set as it is a sized container.

[15:02:41] <pmxbot> Extract method for calculating namespace packages for single_version_externally_managed

[15:02:41] <pmxbot> Extract method for computing parent packages of a package

[15:02:41] <pmxbot> Extract path calculation for paths

[15:02:41] <pmxbot> Rewrite package traversal as a generator expression

[15:02:41] <pmxbot> Reindent

[15:02:41] <pmxbot> Use itertools.product for a cross-product of two iterables

[15:02:41] <pmxbot> Reorganize imports

[15:02:41] <pmxbot> Incorporate the exclusion path in the _exclude function.

[15:02:41] <pmxbot> Return the exclusions directly

[15:02:41] <pmxbot> Move inline function into an instance method and rename for clarity.

[15:16:04] <pmxbot> jaraco pushed 4 commits to setuptools (https://bitbucket.org/pypa/setuptools/) :

[15:16:04] <pmxbot> [Fix/Typo] Fix missing parenthesis in egg_info.py

[15:16:04] <pmxbot> Update changelog

[15:16:04] <pmxbot> Merge Pull Request #78 correcting regression in Pull Request #76.

[15:16:04] <pmxbot> Update changelog

[15:17:36] <pmxbot> jaraco pushed 2 commits to setuptools (https://bitbucket.org/pypa/setuptools/) :

[15:17:36] <pmxbot> make order of lines in top_level.txt deterministic

[15:17:36] <pmxbot> Merged in moben/setuptools (pull request #79)

[15:17:36] <pmxbot> jaraco pushed 1 commit to setuptools (https://bitbucket.org/pypa/setuptools/) :

[15:17:36] <pmxbot> Merged in moben/setuptools (pull request #79)

[15:28:05] <pmxbot> jaraco pushed 2 commits to setuptools (https://bitbucket.org/pypa/setuptools/) :

[15:28:05] <pmxbot> Fix issue #118: Prevent the egg-info directory from being redundantly included in the list of modified files.

[15:28:05] <pmxbot> Update changelog

[15:31:19] <pmxbot> jaraco pushed 2 commits to setuptools (https://bitbucket.org/pypa/setuptools/) :

[15:31:19] <pmxbot> Adds monkeypatching for msvc9compiler.find_vcvarsall() to look for a standalone compiler installation and improves the error message for missing VC installation.

[15:31:19] <pmxbot> Merged in stevedower/setuptools (pull request #83)

[15:31:21] <pmxbot> jaraco pushed 1 commit to setuptools (https://bitbucket.org/pypa/setuptools/) :

[15:31:21] <pmxbot> Merged in stevedower/setuptools (pull request #83)

[15:34:32] <pmxbot> jaraco pushed 2 commits to setuptools (https://bitbucket.org/pypa/setuptools/) :

[15:34:32] <pmxbot> Update changelog

[15:34:32] <pmxbot> Resave with excess whitespace removed

[21:13:32] <pmxbot> jaraco pushed 4 commits to setuptools (https://bitbucket.org/pypa/setuptools/) :

[21:13:32] <pmxbot> Move monkey patch to a separate module

[21:13:32] <pmxbot> Protect against import errors.

[21:13:32] <pmxbot> Extract patched functions as top-level functions

[21:13:32] <pmxbot> Create global import more traditionally. Now the patched functions are available directly as well.

[23:29:21] <pmxbot> jaraco pushed 11 commits to setuptools (https://bitbucket.org/pypa/setuptools/) :

[23:29:21] <pmxbot> Remove excess whitespace

[23:29:21] <pmxbot> Remove unused import

[23:29:21] <pmxbot> Move stable import into stdlib imports section

[23:29:21] <pmxbot> Grab winreg module from distutils.msvc9compiler

[23:29:21] <pmxbot> Skip tests if msvc9compiler isn't available.

[23:29:21] <pmxbot> Extracting environment patcher

[23:29:21] <pmxbot> Correct test assertion

[23:29:21] <pmxbot> Meant 'items'

[23:29:21] <pmxbot> Avoid passing None values to os.environ.

[23:29:21] <pmxbot> Add docstring

[23:29:21] <pmxbot> Always restore os.environ even if an exception occurs.

Log file Viewer

Help | Karma | Search:

#pypa-dev logs for Friday the 26th of September, 2014