PMXBOT Log file Viewer

Help | Karma | Search:

#pypa-dev logs for Friday the 3rd of June, 2016

(Back to #pypa-dev overview) (Back to channel listing) (Animate logs)
[00:44:05] <robhudson> `make shell` is a nice helper. used that a lot already.
[00:46:13] <dstufft> robhudson: :D
[00:46:18] <dstufft> that prod DB is almost ready
[00:46:33] <dstufft> DELETE FROM statements are expensive when you have a bunch of foreign keys and a big DB
[01:31:23] <robhudson> views patched, working locally. Pretty confused by all the stubbing and why those fail in the tests now. I might throw this up in a PR and continue tomorrow.
[01:31:35] <robhudson> I’ve not used the pretend library before
[01:41:19] <dstufft> robhudson: ah, stubs are pretty easy actually, you're just making fake objects, so like ``pretend.stub(foo=1)``, creates a fake object, with an attribute foo that is equal to 1
[01:41:36] <dstufft> tests aren't run yet so I don't see specific failures yet :]
[01:43:46] <robhudson> ok. I’m heading back to hotel but can look later.
[01:46:26] <dstufft> robhudson: no problem!
[01:46:39] <dstufft> postgresql is still trying to sanity this DB :(
[01:46:44] <dstufft> it's at the lest step though!
[01:46:58] <dstufft> trying to update a huge table to set an IP address and user account to something different
[02:25:27] <dstufft> robhudson: commented, overall looks great. this thing is still trying to run `UPDATE journals SET submitted_by = 'dstufft', submitted_from = '127.0.0.1';` on 2 million rows and it's about time for me to go to bed :(
[02:25:49] <dstufft> I'll let it run overnight and have it ready for you in the morning if you're still interested in it then
[02:26:05] <robhudson> dstufft: no problem. Yeah, sounds good. I may do more tomorrow.
[02:26:15] <robhudson> thanks for the comments
[02:27:09] <dstufft> robhudson: no problem! Hopefully they don't come across as too nitpicky or anything :)
[02:32:15] <robhudson> dstufft: not at all. Great comments. I’ll update after awhile and see if I can fix the stubs. thanks.
[12:28:19] <dstufft> heh
[12:28:26] <dstufft> cleaning the prod dB only took 7 hours
[12:28:27] <dstufft> o.O
[13:00:01] <ronny> dstufft: is it a python script that walsk the objects?
[13:04:45] <dstufft> ronny: nah, just a .sql
[13:05:03] <dstufft> https://github.com/pypa/warehouse/blob/master/dev/clean.sql
[13:07:57] <ronny> dstufft: truncate all the tables?
[13:08:13] <ronny> dstufft: what takes the most time in that actually?
[13:08:40] <dstufft> ronny: they're interdependent so you have to do TRUNCATE .. CASCADE, but that cascades into tables we don't want to truncate
[13:08:58] <dstufft> https://github.com/pypa/warehouse/blob/master/dev/clean.sql#L22 and https://github.com/pypa/warehouse/blob/master/dev/clean.sql#L45 are the big two
[14:31:41] <jaraco> dstufft: Does twine allow for environment variables or a project-level config file to define config?
[14:33:15] <jaraco> In particular, I’m trying to address https://github.com/pypa/setuptools/issues/589, but Travis CI uses Twine, so I’m looking for advice on how to signal to Travis’ invocation of twine to use another repo.
[14:35:06] <ionelmc> so setuptools now uploads to pypa.io? that instance still has the big scary red banner ...
[14:35:32] <jaraco> Reading the code, it doesn’t look like it’s possible… unless Travis exposes a hook to override the command-line option.
[14:35:48] <jaraco> ionelmc: Well, that was the intention, but it failed.
[14:36:09] <jaraco> Based on the advice here - https://bitbucket.org/pypa/pypi/issues/262/getting-500-error-when-uploading-python#comment-28236641
[14:36:15] <ronny> jaraco: https://docs.travis-ci.com/user/deployment/pypi#Releasing-to-a-different-index%3A ?
[14:36:39] <jaraco> oh!
[14:36:43] <jaraco> The hook exists.
[14:36:49] <jaraco> !rm ronny
[14:36:49] <pmxbot> Вы делаете хорошую работу, ronny!
[14:37:08] <ronny> you are welcome :)
[14:43:08] <ionelmc> jaraco: what do you mean it failed?
[14:43:36] <jaraco> I mean it uploaded to pypi.python.org/pypi and not pypi.io/pypi
[14:43:59] <dstufft> jaraco: no env vars or project level- though there's no reason it couldn't
[14:44:47] <jaraco> dstufft: I have a solution, but it might be nice if twine honored the distutils setting in [upload] repository.
[14:45:34] <dstufft> jaraco: fwiw, uploading to upload.pypi.io/legacy/ is probably 100% better now than uploading to pypi.python.org/pypi
[14:45:41] <dstufft> I'm planing on making a distutils-sig post about it
[14:46:17] <dstufft> I got reid of all the known caveats for doing so
[14:46:42] <jaraco> dstufft, so using https://pypi.io/pypi as the upload repository isn’t correct? It needs to have the upload name and the legacy path?
[14:47:26] <dstufft> jaraco: correct (it was there for a little bit, but I moved it so we can bypass fastly for uploads which solves the main intermittement 503 issue)
[15:52:19] <jaraco> @dstufft: Is there already a forum where I can express a concern/issue with removing documentation hosting?
[15:55:36] <jaraco> I know it probably shouldn’t be a surprise to me. Indeed, I seem to recall this being mentioned years ago.
[15:59:37] <jaraco> But it occurs to me that if index-hosted documentation is removed, that may add a non-trivial series of steps for any new project to add documentation, a feature that’s currently free (a no-op) for me by way of using a project skeleton that includes the directive to upload docs. I’m hoping there’s a plan or we can form one to make publishing
[15:59:37] <jaraco> documentation as trivially easy in the warehouse world.
[16:00:20] <dstufft> jaraco: distutils-sig or the warehouse tracker, either is fine depending on how wide of an audience you want
[16:00:43] <dstufft> RTD is pretty trivial too (even more trivial once you set it up, since it can auto build straight from master so you don't need to do an upload step)
[17:53:27] <jaraco> Does anyone know how to get RTD to build the latest master?
[17:54:33] <jaraco> I keep triggering a build of latest of setuptools, and it keeps building the second-to-last commit.
[17:54:45] <jaraco> i.e. https://readthedocs.org/projects/setuptools/builds/4067109/
[17:55:11] <jaraco> I want it to build this - https://github.com/pypa/setuptools/tree/ffddefca6a6974f8f02a992f132e3e7a1feadcea
[18:03:45] <jaraco> I suspect I’ve encountered a bug in RTD.
[18:09:07] <robhudson> dstufft: 100% test coverage? :) I think I found the conditional I’m not testing. I’ll add a test.
[18:15:07] <dstufft> jaraco: um, it should always build the latest
[18:15:14] <dstufft> robhudson: yup 100% :D
[18:23:42] <dstufft> robhudson: oh, it might be too late now, but did you want a copy of the DB? it finally finished over night
[18:26:42] <robhudson> dstufft: if it’s easy to send, sure. There may be a couple more changes I’d like to make.
[18:27:37] <dstufft> robhudson: PM'd it to you
[18:27:42] <dstufft> it's ~240M large fwiw
[18:28:47] <robhudson> thanks
[19:13:37] <robhudson> dstufft: I found an interesting case of something hard to search. pyramid_debugtoolbar. If you search “pyramid debug toolbar” you don’t find it on the first page and get a bunch of django stuff.
[19:14:16] <dstufft> huh
[19:14:34] <dstufft> I bet it's Elasticsearch stemming that into "pyramid", "debugtoolbar"
[19:25:43] <asmacdo> dstufft, I have been using twine as a dependency for my project, it is working wonderfully thank you!
[19:26:32] <asmacdo> dstufft, I wanted to make you aware that I am planning to package twine for fedora.
[19:30:59] <dstufft> asmacdo: awesome :]
[19:31:09] <robhudson> dstufft: yep, exactly: http://dpaste.com/38M8ESH Curious how we can make that better.
[19:31:44] <dstufft> robhudson: maybe the normalized_name shouldn't be analyzed at all? idk
[19:31:51] <dstufft> I'm not so good at this search thing :]
[19:33:04] <robhudson> Let me wrap up this PR before digging further. I just thought it was an interesting use case.
[19:46:03] <ronny> asmacdo: do yoi know rpm/packaging well?
[19:46:45] <ronny> oh, the right nat ;P
[19:47:37] <asmacdo> ronny, actually i dont. I work with some people who do though, so I'll learn as I go.
[19:49:16] <ronny> ok, i see
[19:49:46] <ronny> hmm, same problem as me then ^^ i want to get at it since a week now but actual work "gets in the way"
[19:51:20] <asmacdo> lol yes.
[21:30:46] <robhudson> dstufft: do things committed to master get auto deployed?
[21:34:36] <robhudson> looks like it does b/c prefix searching now works
[21:45:38] <dstufft> robhudson: yup
[22:23:05] <robhudson> dstufft: is it possible to get a `config` from the shell?
[22:23:32] <robhudson> e.g. cli methods get passed a config object
[22:23:48] <dstufft> robhudson: should just be ``config`` I think
[22:24:00] <robhudson> oh hey :)