Developer here, I subscribe to Github's official weekly newsletter and various other ones but always found that they did a poor job at highlighting what's trending. The "top trending" repos always seem to be most established projects like Bootstrap or ReactJs.
I'm using a peak detection algorithm to find which repos are spiking in traffic and rank the uptrend based on a normalized growth curve.
This filters out all the well established repos and actually uncovers new trends and launches. Hope you guys find it useful!
This is awesome. I had the same desire and even told folks at GitHub that improving this report is low hanging fruit. It's been on my list of ideas to hack on for awhile :-)
Ideally you (well, really I :-P) also want star/follow activity from people you follow bundled in any update that's also reporting the projects discovered by your peak detection algorithm over that same period of time. There'd likely be some overlap in the report but you could indicate that in your report. This also gives you more incentive to login with GitHub which is a little lacking right now.
Thanks for the feedback, the first thing on my list is definitely more personalization. I just wanted to get it out there and see if people liked it first.
Awesome product :) Is it possible to allow accounts so that user can browse starred repos from your web app ? I am suggesting this because the search feature's topics looks quite complete already, so it would be great to navigate hundreds of starred repos based on the topics, which is something I've been looking for.
Thanks, it's for starring a repo. Github ties it to that same permission scope. You can go to your Github integration settings and disable that scope just for a certain app if you want.
Hi DC2, actually I do cache some of the results, first 10 days and top 50 repos per day. I think I didn't expect people to scroll as far as they did. I'll up that cache later
I can recommend you to "split" your stream of repos in multiple streams [1], and then create a function to merge multiple streams into a single one.
This way you can gain:
1. detect spikes in less popular streams, and decide via the merging function how to compare them to spikes in popular streams.
2. personalization, where each user can potentially get its own stream combining the categories she's interested in.
[1] e.g., by language as also suggested elsewhere, by "category" if you can come up with some definition, potentially by source (e.g. big company) as well.
Why are you asking for "read and write all public repository data" & "read and write all user data."?
I understand the read, but not clear on the write
I think someone else mentioned this as well, but unfortunately Github ties the "staring a repo" to the write permission. I think I can try to separate it out so it can ask for that scope when a user wants to use that feature.
Thanks for the response! Github is definitely a little unclear on the permissions side of things. Might help clear things up if you mention what you are doing with the permissions somewhere.
Pff. That's a vicious circle. Then trending repos will get even trendier. Social networks are largely enough to expose that kind of "noise"/"buzz". I think that a good metric to find interesting projects that are starting or are not already under the spotlight would be to use the ratio: new visitors / stargazers. Quite hard to achieve but I really think it would be more accurate (and fair).
That's definitely something on my mind originally. But since I can't get traffic data from Github, the best thing I can do is to filter out the highly established repos and weigh new repos higher in the rankings. You can see that even with just 150+ stars gained for a brand new repo this system will show it at the top top of the rank despite other popular repos that might have gotten 700+ gains on that day.
However, that said I do plan to improve discoverability even more with traffic data since now I can atleast measure views on gitlogs.
Thanks, I actually did build something like that. My original inspiration was Google Trends (thus the UI) but the data wasn't very useful, traffic on github is very spiky, even on a log graph it didn't look good.
Hey thanks for the tip, it took me a while to find it but apparently react-router 1.x had an issue https://github.com/reactjs/react-router/issues/1661 and I had to upgrade to react-router 2.x I'll push out a fix soon
Right now if a repo is trending down (from the moving average) it will have a negative score and won't be showed at all.
The algorithm basically looks at the moving average, the new watchers gained and the number of total watchers. It'll calculate the the rate of change in new watchers and compare that against what it expects given the lifecycle of where the repo is at. So a brand new repo adding 150 stars per day will beat out an established repo that adds 250 stars per day.
I'm using a peak detection algorithm to find which repos are spiking in traffic and rank the uptrend based on a normalized growth curve. This filters out all the well established repos and actually uncovers new trends and launches. Hope you guys find it useful!
Some notable features, there's also a better interface to search topics on github as well as a curated topics section: http://www.gitlogs.com/resource-guides http://www.gitlogs.com/search
Appreciate any feedback,