Log file Viewer

[01:02:29] <benjaoming> njs: that's a good idea!! I wanted to build an app that was able to do something useful, using the closest match in honeypot release history to the package release that you want stats for seems like a good rule-of-thumb... no sure if I'll ever have time to do the app though... but I did back when I made the honeypot bot :)

[01:04:21] <njs> benjaoming: I guess you could add it as a feature to vanity ('vanity --corrected' or similar)

[01:05:10] <benjaoming> njs: thanks yes, should do that mosdef

[01:09:59] <dstufft> in the future you'll be able to see exactly what is downloading a particular project :]

[01:12:27] <benjaoming> njs: I've created an issue for the project that concerns me, I hope we can find someone who can dedicate attention to Vanity from my description: https://github.com/learningequality/ka-lite/issues/4898

[01:13:43] <benjaoming> thanks for the advice! dstufft, would be great if PyPi had the ability to filter out well-know CI test servers :)

[01:14:03] <dstufft> it currently can't tell that

[01:14:29] <dstufft> I'm migrating the stats stuff to be inside of Google BigQuery

[01:14:34] <dstufft> where it will be publically queryable

[01:14:36] <dstufft> schema -> https://github.com/pypa/linehaul/blob/master/schema.json

[01:20:32] <njs> benjaoming: problem is that connection from e.g. travis-ci just looks like "connection from a VM hosted at AWS/Google"

[01:22:13] <dstufft> we don't make IP addresses public in the data set either

[01:22:34] <dstufft> we make some information about the machine (provided via the user agent) available, and then the region that the geoip resolved to

[01:26:25] <njs> if you knew which IP addresses were CI systems, you could add a "CI: true" tag before publishing the data

[01:26:32] <njs> but this is hopeless because you don't know that :-)

[01:27:27] <dstufft> I thought about adding a pip install --reason ci/ flag that travis could set as an env var PIP_REASON=ci

[01:27:44] <njs> huh, interesting idea

[01:27:51] <dstufft> but I'm not sure it'd make sense to treat travis special when there are thousands upon thosuands of jenkins out there that are also CI

[01:28:10] <dstufft> science (tm) is not my speciality

[01:28:18] <njs> well, if you want to treat travis special you could check for the TRAVIS_* envvars directly :-)

[01:28:21] <dstufft> and I don't know if I'd make the numbers less or more meaningful if I did that

[01:28:31] <njs> PIP_REASON= would be something that jenkins users could set too

[01:28:45] <dstufft> yea, I just assumed most of them would never actually set it :)

[01:29:38] <njs> hmm, travis sets CI=true and CONTINUOUS_INTEGRATION=true, which seems like the sort of thing that could be a de facto standard... I wonder if anyone else does that

[01:30:26] <njs> looks like appveyor sets CI=true but not CONTINUOUS_INTEGRATION

[01:30:49] <njs> circleci also sets CI=true

[01:33:19] <njs> it looks like Jenkins does not set CI=true or any other similarly generic things by default. one could check for JENKINS_URL I guess

[01:33:26] <njs> (list of their envvars: https://wiki.jenkins-ci.org/display/JENKINS/Building+a+software+project#Buildingasoftwareproject-below)

[01:33:38] <dstufft> njs: the problem with checking for Jenkings is jenkins is a lot more than just a CI server

[01:33:44] <dstufft> a lot of people use it as part of their deployment pipeline

[01:34:03] <njs> people do that with travis etc. too

[01:34:12] <dstufft> yea :[

[01:34:38] <dstufft> "why are you running an install" is sort of a tricky question to answer for someone

[01:34:51] <dstufft> but if we can't do it mostly automatically, then it won't get answered in the general case I think

[01:39:07] <njs> wow, the jenkins source code is very java

[01:41:42] <njs> anyway, I think it might be useful to have the number "N downloads, M of which were from CI servers" -- even if this M is still not exactly what you want, it is more useful than having no number at all :-)

[01:42:23] <njs> and ("CI" in os.environ or "JENKINS_URL" in os.environ) would cover a huge proportion of CI deployments

[02:05:21] <tdsmith> i kinda want to start doing this with homebrew browser strings

[02:05:36] <tdsmith> er, user-agent

[03:34:56] <tdsmith> and done, thanks for the tips https://github.com/Homebrew/homebrew/pull/49437

[04:24:33] <paroneayea> python-unittest2 depends on python-traceback2 which depends on python-linecache2 which depends on python-unittest2 :(

[04:24:36] <paroneayea> :*(

[04:24:47] <paroneayea> guess I'm not packaging latest unittest2 for guix thanks to a weird cycle

[04:25:55] <paroneayea> except oh no pastescript requires python-traceback2

[04:26:04] <paroneayea> ;_;

[04:27:51] <paroneayea> I can't avoid it!

[04:32:39] <Ivo> paroneayea: linecache2 only depends on unittest2 for testing, not build

[04:33:29] <paroneayea> Ivo: ah, hm!

[04:33:32] <paroneayea> that may mean home then

[04:52:49] <paroneayea> Ivo: yeah I can't seem to hack past it

[04:52:50] <paroneayea> sigh

[04:53:06] <Ivo> separate build process from test process

[08:28:08] <DRMacIver> If I wanted to get a dump of all (package name, release date, version numbers) triples from pypi, is there an easy way to do that?

[08:28:41] <DRMacIver> and if it involves asking nicely, who should I ask nicely? :-) dstufft?

[08:31:09] <njs> you can use the "simple" index api I think

[08:34:12] <DRMacIver> That doesn't appear to have the release date info on it

[08:34:15] <DRMacIver> Unless I'm missing something

[08:35:21] <njs> https://wiki.python.org/moin/PyPIJSON ?

[08:37:08] <DRMacIver> Yeah, just looking at that. I guess I *can* just get the package listing and do that for each package.

[08:37:12] <DRMacIver> Thanks

[08:41:22] <DRMacIver> (Side note: If this conversation is making you think "I bet DRMacIver is about to make a really bad life choice", you're not wrong)

[09:18:04] <ionelmc> DRMacIver: you're not trying to do something like this https://pypi.python.org/pypi/everything are you?

[09:18:48] <DRMacIver> I don't think so

[09:22:04] <DRMacIver> *checks* no, I'm not

[12:12:05] <odyssey4me> if we're in a situation where we need to pin pip to a major version (eg: >7,<8) then what companion packages should also be pinned? it seems that setuptools and wheel are closely related and should be on that list

[12:14:47] <nedbat> odyssey4me: don't you want to pin all of your dependencies?

[12:16:44] <odyssey4me> nedbat we kinda do that mostly, my question is an input into the pinning process

[12:17:24] <nedbat> odyssey4me: i would agree that setuptools and wheel should be on the list

[12:17:55] <odyssey4me> basically we set an upper constraints based on known working versions - but our testing seems to have missed that some packages are a bit broken, so I'm trying to close the gap

[12:21:01] <nedbat> odyssey4me: personally i prefer to pin to exact versions.

[12:21:13] <nedbat> odyssey4me: then at least I get to choose when I am experimenting with newer stuff.

[12:22:39] <odyssey4me> yeah, we like to keep our master branch moving ahead with new things so that we pick up and solve problems as they come up... our stable branches, however, need a good pin range to accommodate the vast number of packages we deal with and the variability we allow our users

[12:23:35] <odyssey4me> would it be fair to say that setuptools >19,<20 fits with pip >7,<8 ?

[12:59:42] <ionelmc> odyssey4me: i woulnd't call those "pins"

[13:00:02] <ionelmc> maybe "boundaries" ? :)

[13:01:00] <odyssey4me> :) constraints ?

[13:01:11] <ionelmc> you got it

[13:02:14] <ionelmc> using constraints basically means "i know what the maintainers are gonna do with the versions"

[13:02:37] <ionelmc> pins means "don't take any bets"

[14:32:31] <nedbat> odyssey4me: the idea of semantic versioning is very appealing, but it also seems very subjective to me, and I don't know that it works out in practice.

[15:59:07] <hynek> so…how exactly does pip's caching work? https://gist.github.com/hynek/8c0a19d07ad89ab3a2ab – it *claims* to store stuff in the cache but then it really doesn’t. and it will download everything again and again. :( somehow it never really worked for me on Linux.

[20:13:09] <dstufft> hynek: for the HTTP cache, do you have cache control headers and/or etag headers?

[20:14:03] <dstufft> hynek: pip has two caches, the HTTP cache which functions like, well a HTTP cache, it uses the url as a key, respects cache-control headers, will reuse the response if there is a etag header, etc

[20:14:41] <dstufft> the other a wheel cache which will is also keyed by an URL, (but this time, the URL of the .tar.gz that it would otherwise need to download if a wheel wasn't available)

[21:43:43] <mdeboard> Hi there, when I do `twine upload *.whl -r repo-name`, I get an "SSLError: No such file or directory". What am I doing wrong?

[21:45:47] <mdeboard> I mean, I grok that somehow, somewhere, there is a missing file, but..

[21:47:10] <mdeboard> Oh, actually the command I'm doing is `REQUESTS_CA_BUNDLE="/path/to/ssl/cert" twine upload *.whl -r repo-name`. The $REQUESTS_CA_BUNDLE file does exist

[21:52:55] <mdeboard> Any help would be appreciated. I'll take my answer off the air.

[21:53:50] <mdeboard> AHA wrong filename extension. Thanks Bob