pmxbot IRC Log Viewer

[00:46:33] <iElectric> dstufft: you around?

[00:46:52] <dstufft> iElectric: sup?

[00:47:11] <iElectric> https://travis-ci.org/pypa/warehouse/jobs/59078846

[00:47:20] <iElectric> trying to understand this failure

[00:47:27] <iElectric> it shouldn't be connected to my PR

[00:47:32] <iElectric> somesort of ordering issue

[00:49:56] <dstufft> iElectric: hm

[00:49:56] <iElectric> hmm it doesn't fail on master

[00:49:58] <dstufft> iElectric: that's odd

[00:51:32] <dstufft> I'm not sure how your change could cause anything like that

[00:51:41] <dstufft> can you reproduce it locally?

[00:52:19] <iElectric> ahh

[00:52:22] <iElectric> I think I got it

[00:52:27] <iElectric> let me double check

[00:54:16] <iElectric> dstufft: yeah so my test data somehow triggers that

[00:54:25] <iElectric> maybe because version sequence reaches 9, 10, 11

[00:54:35] <iElectric> and sorting fails?

[00:54:56] <iElectric> simple_index view does order_by(filename)

[00:55:17] <iElectric> that's not really proper version comparison

[00:55:43] <iElectric> but shouldn't those fixtures be rollback back for each test?

[00:56:08] <dstufft> they should be, perhaps it doesn't work though. I just copy/pasted some code

[00:56:22] <iElectric> they're not

[00:57:01] <iElectric> File(id=UUID('3e4b0aff-cdd1-423b-9219-0ec589441c41'), name='SeqtnJhVIxsG', version='10.0',

[00:57:10] <iElectric> so that should be 1.0

[00:57:22] <iElectric> since that test only created 3 versions

[00:59:18] <dstufft> is the ReleaseFactory leaking state or the database leaking between tests

[00:59:50] <iElectric> don't know yet

[01:00:02] <iElectric> so transaction should rollback per class or per method?

[01:00:11] <dstufft> per test case, so per method

[01:01:14] <iElectric> so it's probably ReleaseFactory leaking sequence between test

[01:01:23] <iElectric> and then order_by(filename) fails

[01:01:35] <iElectric> since -9.0 -11.0 -10.0

[01:01:51] <iElectric> is ordered different as a number and as a string

[01:02:23] <dstufft> I'm not sure that Releasefactory is supposed to get the sequence reset between tests, maybe it's just a bad test - The ordering of the links on simple doesn't really matter as long as it's stable which is why I just did by filename

[01:03:03] <iElectric> so maybe that test should sort on version name

[01:03:08] <iElectric> and then assert

[01:03:14] <iElectric> or filename, w/e

[01:03:59] <dstufft> yea

[01:04:02] <dstufft> probably makes sense

[01:04:57] <iElectric> yup that works.

[01:13:15] <iElectric> dstufft: any comment on https://github.com/pypa/warehouse/pull/502

[01:13:36] <iElectric> dstufft: all tests now pass for https://github.com/pypa/warehouse/pull/492

[01:13:44] <dstufft> iElectric: nope, just didn't look at it yet

[01:13:46] <dstufft> it looks fine to me

[01:13:52] <dstufft> I'm going to assume the nix commands are correct and merge it

[01:14:01] <iElectric> I'm using them

[01:14:24] <dstufft> will take a look at 492 shortly

[01:14:32] <iElectric> thanks!

[01:14:45] <iElectric> I'll tackle elasticsearch

[01:14:59] <dstufft> cool

[01:51:00] <iElectric> hmm.

[01:51:18] <iElectric> upgrading components doesn't seem to have any effect?

[01:51:20] <iElectric> docker-compose run web pip install --upgrade pip

[01:51:25] <iElectric> upgrades pip

[01:51:36] <iElectric> but not really, if you run it again it will repeat the same steps

[01:51:45] <iElectric> but pip remains as it was before the upgrade

[01:52:04] <dstufft> iElectric: it's like heroku, each invocation gets it's own container that then throws away

[01:52:21] <dstufft> iElectric: you can rebuild the primary container by doing docker-compose build

[01:52:52] <iElectric> oh so only directories that are marked as data can be mutated

[01:53:07] <iElectric> that's kind of neat

[01:53:55] <dstufft> iElectric: specifically the docker-compose.yml mounts the warehouse directory at /app/

[01:54:09] <dstufft> as a docker volume

[01:56:06] <iElectric> oh and then it redownloads everything

[01:56:26] <iElectric> I have to write a blog post how Nix would solve a bunch of Docker issues

[01:58:08] <dstufft> iElectric: well it has caching, which doesn't work on Warehouse's dockerfile because I optimized for smaller containers instead of less downlaods

[01:58:50] <dstufft> so I install everything I need to build warehouse, install warehouse, then purge everything I don't need to run warehouse in one atomic action

[02:00:42] <iElectric> how big is the base web image?

[02:01:50] <dstufft> not sure, I think docker ps or docker something will tell you, my environment is kinda ripped apart ATM so I can't easilylook

[02:21:23] <iElectric> got it

[02:21:24] <iElectric> ohhh

[02:21:34] <iElectric> postgresql supports events now

[02:21:55] <iElectric> I was thinking how to handle delete cascades to elasticsearch

[02:23:25] <iElectric> that's neat

[02:24:42] <dstufft> iElectric: oh, incase it wasn't clear, I'm fine using ORM hooks for elasticsearch since that's an external to the DB system

[02:25:38] <iElectric> but it's hard to handle all edge cases

[02:25:41] <iElectric> bulk update/deletes

[02:25:44] <dstufft> yea

[02:25:46] <iElectric> cascading stuff

[02:25:48] <iElectric> etc

[02:26:15] <iElectric> dstufft: you think we won't use any of that?

[02:26:21] <iElectric> I'm sure cascades will show up sooner or later

[02:26:47] <dstufft> not sure

[02:27:18] <dstufft> I don't know SQLA that well tbh still earning it

[02:27:48] <iElectric> that's why I said there is a benefit doing everything at ORM level

[02:28:18] <iElectric> makes it slower but also makes sure that you can get all the power out of SA

[02:29:37] <iElectric> also not sure if those ORM hooks handle a rollback

[02:34:41] <dstufft> well the ORM hooks can add an afterCommit hook to the global transaction

[03:02:14] <iElectric> right

[14:33:07] <iElectric> dstufft: what tables should we index to ES?

[14:33:24] <iElectric> I guess we'll end up indexing all of them

[14:34:29] <iElectric> and since we have elasticsearch we can use logstash as well :-)

[14:34:43] <willingc> iElectric: cool. what are you using ES for?

[14:34:57] <iElectric> mostly search

[14:35:05] <iElectric> so we can finally facet it

[14:35:15] <iElectric> (one rainy day)

[14:35:22] <willingc> For a packaging project?

[14:35:26] <iElectric> willingc: https://github.com/pypa/warehouse/pull/495

[14:35:57] <iElectric> lots of callbacks :)

[14:36:57] <willingc> Very cool. Have you used https://github.com/elastic/elasticsearch-dsl-py

[14:37:10] <willingc> Quite nice for prototyping

[14:38:22] <iElectric> oh nice

[14:39:01] <willingc> honza is helpful too if questions come up:)

[14:39:22] <iElectric> I'll take a look, thanks!

[14:39:28] <willingc> Enjoy :)

[14:39:37] <iElectric> was just wondering that we'll have to implement some higher level utils

[14:40:01] <iElectric> does dsl extract values from an object?

[14:40:08] <iElectric> or do you have to pass them in

[14:41:15] <willingc> iElectric: I honestly don't know. Just started playing with it after Elasticon and then got distracted with PyCon. Docs are decent for a newer project.

[14:41:53] <iElectric> I only see two examples - wouldn't call that docs :)

[14:42:06] <iElectric> oh there is more on RTD

[14:42:08] <iElectric> cool

[14:43:23] <willingc> https://elasticsearch-dsl.readthedocs.org/en/latest/

[21:28:26] <lifeless> morning morning morning

[23:20:20] <dstufft> lifeless: morning!

[23:21:56] <lifeless> dstufft: can pip install bdists ? [not wheels]

[23:22:14] <lifeless> dstufft: and if so, how do I tell that a particular archive is one?

[23:22:25] <dstufft> lifeless: no

[23:22:42] <dstufft> sdists and wheels only

Log file Viewer

Help | Karma | Search:

#pypa-dev logs for Sunday the 19th of April, 2015