How To Keep BSDataAnon's Forks Up To Date

[Jonskichov]

So currently things are set up so that the BSDataAnon user manages forks of the data file repos. When a user submits a file, it is committed to a new branch in the appropriate fork, and BSDataAnon issues a pull request back to the main repo.

The next step is how do we then keep BSDataAnon's forks in sync? In other words, when a data file admin accepts and merges in a pull request, BSDataAnon's fork will need to mirror that merge (or otherwise refresh it's fork) so that it matches what's currently in the main repo. This then leaves it ready for future file submissions.

The app can listen out for events in GitHub, so it's a question of what events to listen for and what to do upon receiving an event.

Option one: Make use of the "release" functionality. Once an admin has merged in one or more submissions, and deems the repo ready for release, they, erm, make a release.

1. The app is notified that a new release is created
2. BSDataAnon deletes it's current fork and recreates it from the new master in the main repo

This is nice because BSDataAnon's forks won't get out of hand with millions of branches. Down side is that when a release is done, any commits/pull requests from the fork will be lost (so no going back to include submissions prior to the release)

Option two: Listen out for pushes on the master branch.

1. App is notified that something has been pushed to master
2. BSDataAnon replicates that push on it's own fork (or pulls the change from the main repo)

This is nice because we don't loose any old pull requests/commits on the fork, but it could get to a point where the fork as loads of rejected file submissions as separate branches...

My preference would be the first option as it'd be easier to write (keeping the fork in sync with the master in "real time" would be hard to get right I think).

Thoughts (looking at you @amis92)? Further options on how to do this?

[HeWhoWatches]

I've been reading through the following pages but being new to Git am struggling to understand under what circumstances it's better to fork / branch:
https://help.github.com/articles/fork-a-repo
http://git-scm.com/book/en/Git-Branching-Basic-Branching-and-Merging

This model is very interesting and am wondering if it's something we could adapt?
http://nvie.com/posts/a-successful-git-branching-model/

To borrow one of my favourite movie quotes, can somebody take this bag of snakes and lay them out straight for me? 10 bonus points if you know the film without needing help from the internets.

[Jonskichov]

OK a problem. Because the Anon user doesn't have write access to the main repos (and it shouldn't have write access btw), it can't create a "hook" there (what's used to send notfications).

A solution: Turns out that deleting Anon's fork doesn't delete the related pull requests! So we can just nuke the fork every time!

[HeWhoWatches]

Ah, that would be this? http://developer.github.com/v3/repos/hooks/ - fair enough, it sounds like nuking the fork is the way to go :)

[Jonskichov]

I've fiddled around, and I think I have something workable.

BSDataAnon will compare the number of releases on it's fork to the number of releases in the main repo. If they differ, the fork is nuked and recreated.

This means the admins have control based on using the GitHub's release mechanism. They do a release, and the Anon user's fork is recreated to match the main repo at that point. This also means that future Anon submissions/pull requests will have the latest release as their parent.

I'm also going to modify the file download so that the files you get are those from the latest release (rather than just the head of the master branch). This way releases can be properly controlled.

[amis92]

I really like this.

As per nuking pullreqs, it's a love/hate part of git. It's hard to really nuke anything for eternity, except for forced push. And even thne, it might still live in refs.

I fully agree on using release mechanism. That would stop little changes to cause multiple(infinite) website repo refreshes.

On side note, we could agree to do releases on schedule (if anything is new), say weekly.

PS as for mail spam, I enjoy looking at progress. And of course I can unsubscribe these notifications any moment, so no worries.

[Jonskichov]

@amis92 I guess it's not a bad thing to have the old pull requests still there - they might come in handy. Or they can just be declined.

Glad this sounds like a good mechanism. Leveraging GitHub's processes seems to be the right way to go. Bear in mind that the decision to refresh the cache/re-fork the Anon repo is based on the number of tags - if that number of tags in the master is different to the number in the fork then the refresh takes place. So deleting (not sure if you can do that) a tag would cause a refresh. I don't see that being an issue, but if you can see any reason it might be then let me know.

And releases can be done as-and-when it's logical. No problem doing a release for just one merge, if it's super important :)

[Jonskichov]

This is all working as intended now :)