PMXBOT Log file Viewer

Help | Karma | Search:

#pypa logs for Sunday the 1st of March, 2015

(Back to #pypa overview) (Back to channel listing) (Animate logs)
[13:27:08] <malinoff> hi, can someone help me? i have this line in my requirements.txt: git+git://github.com/malinoff/button.git@develop; pip install -r requirements.txt works fine
[13:27:33] <malinoff> but pip install -e . fails with "error in button-cli setup command: 'install_requires' must be a string or list of strings containing valid project/version requirement specifiers"
[13:37:45] <ronny> malinoff: pip style requirements are not supported in setup.py?
[13:38:26] <malinoff> ronny, pythonhosted.org/setuptools/setuptools.html#dependencies-that-aren-t-in-pypi
[13:40:41] <ronny> malinoff: the link is missing a egg version marker i think
[13:40:55] <ronny> malinoff: so pip will work, setuptools not so much
[13:41:06] <malinoff> ronny, i've tried with egg, not working
[13:41:20] <malinoff> and the general idea was not to include concrete version, so...
[13:41:28] <mathieui> malinoff, git+git://github.com/malinoff/button.git@develop#egg=button ?
[13:41:47] <malinoff> mathieui, and that too
[13:43:38] <ronny> malinoff: not sure a bout how to solve in detail (i'd just use a devpi and upload every commit as a .dev version)
[13:44:18] <nedbat> ronny, mathieui: malinoff tried using dependency_links as the docs say, with no success. I've never used dependency_links, do you have an example that works?
[13:44:46] <malinoff> ronny, well, i want travisci to work, so devpi is not even a solution in my case
[13:45:04] <mathieui> nedbat, no, I never tried it
[13:45:23] <mathieui> If my software needs something not in the cheeseshop, then it’s not stable enough to go on the cheeseshop, I guess
[13:45:56] <ronny> mathieui: cant travis use a devpi index server for dependency installs?
[13:46:12] <malinoff> mathieui, i'm developing bot button and button-cli at the same time; button is stable enough, but i'm playing with different options and solutions
[13:46:24] <ronny> whats the button bot?
[13:46:59] <malinoff> ronny, travisci can link to anything which is public :) and i'm a poor student :)
[13:47:07] <malinoff> ronny, button bot?
[13:47:19] <ronny> ah, s/bot/both/
[13:47:28] <malinoff> ah
[13:47:29] <malinoff> yes
[13:47:30] <malinoff> sorry :)
[13:47:31] <nedbat> malinoff: github.com/Mezner/pidgin-log-daemon/blob/1a158a7a54369971bd7cdafabfd19cb297443fec/setup.py this example might be real?
[13:47:49] <ronny> hmm, whatdoes button do better than make/invoke/fabric?
[13:48:08] <malinoff> ronny, it has no borders
[13:48:14] <ronny> borders?
[13:48:20] <malinoff> err
[13:48:22] <malinoff> well
[13:48:24] <malinoff> restrictions
[13:48:48] <ronny> define that please, to me that maeks no sense
[13:48:55] <malinoff> sure :)
[13:50:02] <malinoff> make/invoke/doit look to the task execution from the very simple position - command-line interface
[13:50:21] <ronny> what does that even mean?
[13:51:00] <malinoff> ronny, i'm not a native english speaker, please, give me some time to express my thoughts :)
[13:51:32] <ronny> oh, i see, i'll go slow then
[13:51:43] <ronny> malinoff: whats yor native language (mine is german)
[13:51:50] <malinoff> ronny, russian
[13:53:25] <malinoff> so if you look at make/invoke/doit docs, they spend a lot of time explaining how to declare tasks and how to run them from a shell
[13:53:51] <malinoff> and almost no time explaining how to run tasks from the python
[13:54:11] <malinoff> this is the first issue
[13:54:31] <ronny> i see what you mean
[13:54:56] <ronny> so bascially you want to clear the python side
[13:54:58] <malinoff> the second one is they have lots of concepts (like pre_tasks, post_tasks, etc), but they're really "extensions"
[13:55:39] <malinoff> let me explain a little bit further please :)
[13:56:02] <ronny> bascially i want to figure where this makes sense, and how it can be extended uppon (thinking about things like an internal deploy tool, tox, devpi)
[13:56:03] <malinoff> so they have a lot of concepts in their core, but at the same time they're really not extensible at all
[13:56:30] <ronny> brb, cutting onions
[13:56:48] <malinoff> ronny, ok, i'll finish my talk :)
[13:58:30] <malinoff> i've played with celery a lot, and i really like its task execution capabilities
[13:58:34] <ronny> :)
[13:58:51] <malinoff> but why we tied with remote execution?
[13:59:25] <malinoff> why can't we have such smart execution locally without code for daemonization, transport layer etc
[14:00:04] <malinoff> and here is my general idea - take the best from celery and build an extensible task execution framework
[14:00:24] <malinoff> in such way so it will be able to be extended for remote execution, for example
[14:01:37] <malinoff> this "extensible" concept was used even for command line interface
[14:02:03] <malinoff> the default implementation is quite verbose, like button -E tasks apply build
[14:02:21] <malinoff> but one can write a make-like cli implementation, e.g. bake build
[14:04:06] <malinoff> to summarize, button is a) general-purpose task framework with nice programming api, b) extremely extensible, so almost everything is possible and c) it has a very tiny codebase, so it is very easy to test, to install and to use
[14:05:31] <malinoff> of course, there's more, but my current english skills do not allow me to express them in adequate time frame :(
[14:06:09] <malinoff> nedbat, thanks, i'll try that
[14:07:23] <ronny> malinoff: i see
[14:07:45] <ronny> malinoff: i'll keep track on it, it seems like a potential good fit for the ci system i creates as part of my thesis
[14:08:04] <malinoff> ronny, i'll probably rename it
[14:08:38] <malinoff> ronny, i don't like "button", pytato (python task toolkit) looks a little bit funnier and a little bit more descriptive :)
[14:09:28] <malinoff> ronny, and the highest idea is to make a deployment tool using it
[14:09:56] <malinoff> ronny, i've used salt for a year, now i'm using ansible, and they are broken
[14:10:12] <ronny> malinoff: i jsut made a deployment tool at work that completely avoids taks toolkits
[14:10:28] <malinoff> ronny, can you share it?
[14:10:32] <ronny> insted i just steamlined the workflow and standardized
[14:10:42] <ronny> still working on getting the opensource permission
[14:11:25] <malinoff> ronny, i have a colleague that works on the workflow part
[14:11:51] <malinoff> i'm the one who code the foundation :)
[14:12:12] <ronny> hmm, for deployent tools i see absolutel no need for complex task frameworks
[14:12:21] <malinoff> ronny, button is not complex
[14:12:40] <ronny> malinoff: compared to what my deplotment tool does it adds an order of magnitude on complexity
[14:13:00] <malinoff> ronny, it would be really awesome to see what you've done
[14:13:14] <malinoff> ronny, can you describe concepts?
[14:13:19] <malinoff> tools used?
[14:13:41] <ronny> malinoff: only paramiko and pip
[14:13:46] <malinoff> ah, so, ssh
[14:14:08] <malinoff> that's not our case, unfortunately
[14:14:13] <ronny> bascially it comletely streamlines a partcula flow, which is upload all deps, make a venv for a version, unpack that, reconfigure the supervisor
[14:14:16] <malinoff> we have lots of servers without ssh access
[14:14:24] <malinoff> wait a second
[14:14:31] <malinoff> you mean, deployment of python apps?
[14:14:36] <ronny> yes
[14:14:38] <malinoff> ah
[14:14:50] <malinoff> I'm talking about a general purpose deployment system
[14:15:11] <ronny> deployment of servers is something where i'd pretty much go for a packaging based solution, and a provate package repo
[14:15:21] <malinoff> which is able to orchestrate hosts, do configuration management and installing software
[14:15:22] <ronny> yadt is a good example
[14:15:56] <malinoff> yes, packages are awesome, but you can't really do everything only with packages
[14:15:57] <ronny> malinoff: myy use case is servers at clients that have own server setup rules ^^
[14:16:15] <ronny> malinoff: see yadt, it does orchestration, config management and other stuff with packages
[14:18:51] <malinoff> >it is required that the hosts are accessible via passwordless ssh and provide a yadt minion
[14:19:00] <malinoff> I plan to have no such restrictions :)
[14:19:07] <malinoff> although, it looks nice
[14:19:11] <malinoff> thank you
[14:19:36] <malinoff> I really like the idea of agentless ansible
[14:19:43] <malinoff> but i hate the implementation
[14:20:03] <ronny> malinoff: btw, why is ssh no option?
[14:20:24] <malinoff> usually it is about security concerns
[14:20:31] <ronny> (i prefer ssh over having slave services)
[14:20:43] <malinoff> and ssh is quite slow
[14:20:49] <ronny> huh?
[14:20:56] <ronny> define slow?
[14:21:10] <malinoff> compared to salve-based solutions that use *mq
[14:21:18] <malinoff> slave*
[14:21:40] <ronny> i see, what you mean
[14:21:53] <ronny> but i dotn see how that matters
[14:22:01] <ronny> all that matters is the downtime
[14:22:13] <ronny> as soon as its zero, the rest hardly matters
[14:22:29] <ronny> also brb again, need to make a sauce for fish
[14:23:06] <malinoff> not exactly, if the deployment process with "no changes" report takes minutes, it is very annoying
[14:23:27] <malinoff> we all hate long operations, because they look like broken
[14:23:38] <malinoff> and some deployments are *really* long
[14:24:01] <malinoff> we deploy, oh, like 15 different projects in a week
[14:24:18] <ronny> i see waht you mean
[14:24:29] <ronny> deloyment with no changes should be a noop afair
[14:24:32] <malinoff> yes
[14:24:37] <malinoff> and ssh becomes a bottleneck
[14:24:57] <malinoff> also, ssh has no routing abilities
[14:25:25] <malinoff> so you can't send the same message to 10k clients quickly
[14:25:38] <malinoff> the sender will spin up some threads/processes
[14:25:46] <malinoff> which takes time and resources
[14:26:11] <ronny> sounds like you want to do a different kind of orchestration
[14:26:16] <malinoff> exactly!
[14:26:31] <malinoff> and i don't want N different tools for N different setups
[14:26:31] <ronny> with yadt you'd just fire up more vm's
[14:26:42] <malinoff> i want to have 1 extensible tool
[14:26:58] <ronny> wel, im curiious as to what you will nd up with, people have different needs
[14:27:06] <ronny> usually one tool to fullfill them all will be a mess tho
[14:27:29] <malinoff> that's why it is not a single tool :) it is a glue for extensions
[14:27:47] <malinoff> so adopting a new setup is easy as writing an extension
[14:28:07] <malinoff> jenkins is quite good at it
[14:28:26] <malinoff> (which is the second project i've been inspired of)
[14:29:32] <ronny> jenkins os a big pile of pain tho ^^
[14:29:44] <malinoff> jenkins os? Oo
[14:29:49] <ronny> *is
[14:29:51] <malinoff> ah
[14:30:13] <malinoff> probably because of java and tons of legacy code
[14:30:45] <malinoff> but it works perfectly for me, and every time i need a new functionality, i don't have to install a different tool
[14:30:58] <malinoff> all i need is to walk to the plugins page and press "install" button
[14:32:22] <ronny> malinoff: well, it has a bad awwaeness of modern scm and branches, and its really bad at build pipelines ^^
[14:32:39] <ronny> malinoff: but before i even try to make a own i'll tacle other items
[14:33:09] <ronny> (first i want the surrounding tooling for python pacakging and testing to be usefull enough
[14:33:44] <malinoff> ronny, we've tried almost all of deployment systems, 2 years is quite long time
[14:34:17] <malinoff> ronny, and as i described, existing task frameworks are tied with their ideas
[14:36:11] <ronny> malinoff: i see :) will yours be open source later?
[14:36:19] <malinoff> nedbat, it looks like dependency_links are completely ignored :(
[14:36:26] <malinoff> ronny, they are already
[14:36:42] <malinoff> although they're not quite useful for now :)
[14:36:49] <malinoff> the docs should be improved
[14:36:51] <malinoff> a lot
[14:37:24] <malinoff> and I really want to implement a complex tool on top of button just to show that it *can* be used for complex apps
[14:38:26] <malinoff> It would be awesome if there will be a native english speaker which is patient enough to talk with me :)
[14:38:36] <malinoff> and help me to improve the docs
[14:39:57] <ronny> malinoff: i cant make promises, but im curious
[14:40:11] <ronny> malinoff: for now, use pip for installing/testing and/or upload .dev versions
[14:40:27] <ronny> also take a look at devpi and setuptools_scm for tooling
[14:40:37] <malinoff> ronny, pypi.python.org/pypi/button
[14:40:47] <malinoff> ronny, as i said, i'll probably rename it
[14:40:57] <malinoff> and 0.5.4 contains bugs
[14:41:25] <malinoff> ronny, i think, my case with dev git versions won't repeat
[14:41:39] <malinoff> i'll try to make a workaround for now
[14:45:24] <malinoff> ronny, i want to use devpi in future to make an extension registry
[14:52:04] <ronny> k
[14:58:02] <malinoff> nedbat, can i thank you for coverage? :)
[14:59:03] <ronny> bbl, dinner
[15:00:22] <nedbat> malinoff: sure :)
[15:01:02] <malinoff> nedbat, so, thank you :) this is awesome tool!
[15:01:08] <nedbat> :)
[15:01:32] <malinoff> nedbat, i've been missing so many edge cases before coverage
[15:01:47] <nedbat> malinoff: i'm glad to hear it's helping
[15:01:52] <malinoff> yeah, A LOT
[15:02:49] <malinoff> and it is nice that coverage is the standard, so when I moved from nose to py.test, coverage stuff just keep working
[16:26:25] <malinoff> ronny, one more thing - i take testing seriously, i test button on linux, osx and windows automatically using TravisCI and AppveyorCI :) you won't find this in other similar projects
[16:43:36] <ronny> malinoff: its quit unfortunate all those things require git, i much prefer mercurial
[16:44:45] <ronny> oh
[16:45:15] <ronny> malinoff: nice, AppveyorCI support bitbucket
[16:51:28] <malinoff> ronny, well, github has integration with these services, so it is extremely easy to see if tests passed for a pr, for example
[16:52:12] <malinoff> these services have integration with github*
[17:03:48] <ronny> malinoff: well, i dont like all that social pressure to go git, its pretty much harrasment by now ^^
[17:05:27] <straycat> pretty easy to convert between hg and git so long as you don't use submodules
[17:14:40] <ronny> straycat: the conversion is less of a problem as the interop is (unlike hg git will not store any metadata)
[17:23:46] <malinoff> straycat, i'm happy with git, but it will make me even more happier if github would support hg
[17:24:03] <malinoff> straycat, just because I like github's interface more than bitbucket
[17:24:36] <malinoff> ronny, I'll create a bitbucket mirror just for you :)
[17:25:05] <malinoff> although it will be backed by git, of course
[17:48:42] <ronny> malinoff: no needed
[17:48:52] <ronny> malinoff: i can work fine with github, i just prefer hg as scm
[20:19:19] <jessamynsmith> hahaha oh, have I been there, pjdelport
[21:39:00] <pf_moore> nanonyne: did you raise that bug for pip? Do you have the issue number?
[22:29:14] <_habnabit> pjdelport, ikr; there's not a day i'm sad bzr lost
[22:29:28] <jessamynsmith> I never used any of the others, but I hear they were better
[22:29:39] <jessamynsmith> I think really it was githu
[22:29:43] <jessamynsmith> er, github
[22:29:43] <pjdelport> _habnabit: I'm a Mercurial fan, myself.
[22:30:04] <_habnabit> er, not a day i'm not sad
[22:30:08] <_habnabit> stupid double negatives
[22:30:12] <jessamynsmith> haha
[22:30:13] <pjdelport> Bzr is different in many ways; hg is simply a better git.
[22:31:02] <_habnabit> that might be the case, but i always thought bzr's abstractions were better
[22:31:13] <_habnabit> f.ex bzr's fundamental unit is a branch, not a repository