Regarding my last boost:
- git cloning the Linux kernel (done; took a few minutes, 3.1 GB on disk)
- git log the lines changed per commit (doing, seems to be averaging a month every 2-3 seconds)
- merge the log into number of lines changed (added+deleted) each day (todo)
- plot daily activity in Linux kernel.
Does the Linux kernel follow the Wikipedia rise and decline pattern?
@22 I guess it does not.
However, another intersting statistics would be the number of independent contributors (that is, all people working for the same company count as one).
@Shamar Hey! Ah, sorry, I was going to put together some thoughts before bugging you about your recent paper but you found me first :). I found that daily editor counts on Wiki (over ~5 major languages) seemed to tell the same story as daily edits count (and even bytes changed), so if that link holds in the Linux kernel, the number of lines changed might be a useful sufficient statistic. (My research notes on Wiki are at https://github.com/fasiha/wikiatrisk)
I'm afraid this assumption does not hold in the Linux kernel.
My bet is that the number of independent contributors (with "independent" meaning "working for different companies or contributing for free") will tell a different story from the number of contributors.
Originally Linux was contributed by few people working in their free time, then a few companies joined and now a large percentage of the changes are collettively produced by several developers for each large company.
So while originally Linux was a driven by a large variety (in relation to the codebase size) of people needs, now it's mostly driven by few corporate needs (always in relation to the codebase size).
Despite the quality of the leaders, my fears is that this reduce the originality of the software (with the variety of people) and ultimately affect the quality of the project itself.
Obviously this is my uninformed bet, but frankly I'd be happy to be proven wrong.
@Shamar I appreciate your thoughts! Well, the first question I asked—does Linux kernel daily lines of code committed have a RAD (rise & decline) pattern—the answer is, maybe.
https://github.com/fasiha/KernelAtRisk/blob/master/linux.png
The git repo only has data starting 2005, when git was started, and the pre-git commits don't exist there. I'd guess the ramp up in 2005 is a transition-to-git period. Ignoring that, loc committed has been very flat over time. Which is a bit confusing, such stability.
@Shamar About the issues you raise… ok so the daily lines of code suggests a project in a stable equilibrium, fine. If the number of unique entities contributing (firms+individuals) is declining over time, I'm not sure if I would necessarily read that as a community in decline, since the kernel is powering ever more interesting and important architectures and workflows (ARM, containers, etc.). The main question I have is, where else do we see RAD and when does it indicate a dead community.
@Shamar
So I come to the topic of Wikipedia pre-disposed to Gwern's point of view (https://www.gwern.net/In-Defense-Of-Inclusionism), who blames the deletionist takeover of Wikipedia for the peak and decline in editors+edits in English, French, Japanese, Hebrew. (Russian/Arabic/Chinese show different trends). Your team shows a similar rise and fall among Wikias (sorry, only read the blog post, not the paper), so I'm left wondering where that leaves Gwern's thesis.
@Shamar I remember how awful the fights on English were between the inclusionists and deletionists—I had friends ask for help voting to keep their pages from being deleted, etc. It's possible that the timing of that fight just coincided with Peak Wiki. Or possibly the deletionist decimation of Wiki editors reduced the global status of Wiki editorship—Wikipedia might have seen steady (if not exponential) growth but the fighting reduced it to the standard rise+fall narrative of all human efforts.
@Shamar Have you noticed that, at least on English, editor counts stopped declining in 2014 and have started ticking up since then? A steady decline from 2007 to 2014 was replaced by a steady increase, of close to, slightly less than the same magnitude (on a log scale). I noticed it when looking at sliding windows of auto-correlations and I haven't checked if any other languages have that, but I was wondering if you'd noticed it and/or had any thoughts on it?
