How We Moved the Scratch Wiki


The Scratch Team huddling around the Scratch Cat.
The Scratch Team huddling around the Scratch Cat.
Credit: Scratch Wiki

Over the past several years, Scratch has seen nothing but explosive growth, and in fact is listed as the 29th most popular programming language. However, this growth wasn't without challenges.

The Scratch Team at MIT have experienced maintenance pangs maintaining the sibling "Scratch Wiki". In response, they felt that MartinWollenweber would be a great patron because he hosts the international wikis. Discussion for a server transfer started in October 2017, and we've finished moving last month. (Although cosmetic changes are still ongoing!)

This is a monumental move for the Scratch Wiki. Grab some popcorn, relax, and listen to the story of how it happened.

The Transfer

As with any transfer, the first step is to analyze what's involved. What we found is that the backend technologies used were the same as ours: Apache and MariaDB. Also surprisingly, we found that the Scratch Wiki didn't require much to run. The only optimization it had was caching servers (and possibly a coffee machine for programmers).

The second step of any transfer is practicing for it. The Scratch Team provided a snapshot of server files a few weeks beforehand, and we were able to iron out issues and prepare file downloads ahead of time. So far so good.

The final step of any transfer is actually performing it!

The Scratch Team started by freezing the wikis, then pgp encrypting all necessary files for public cloud sharing. We, the admins, then performed all file updating on a local test environment before pushing to production. This mainly consisted of running the database dump through a Mediawiki upgrade.

After the server move, the admins celebrated with party horns and confetti! However, that is until we discovered some issues...

Upgrading Mediawiki

During the server transfer, we took the liberty to upgrade parts of the website. The largest change was upgrading Mediawiki (MW). The old site was running v.1.22, which was released way back in 2013! With a little love and care, we decided to move to MW v.1.28, the same version as the international wikis.

This obviously broke many extensions and our website skin to boot. To deal with this, we upgraded most extensions, removed some, and rewrote ConfirmAccounts and ScratchWikiSkin.

Another issue that occurred is that all links, transclusions, and categories broke. We initially guessed this was due to the DB upgrade. This was quickly fixed by running rebuildall.php and runJobs.php, which forced MW to re-parse every page. In reality, this was an issue with the job queue. (Ironically, we've had several issues with PHP's job queue as well!)

Requirements

The last interesting part of the transfer is the obligations it brought. As some say, there's no such thing as a free website.

The largest concern is that since MIT no longer controls the Scratch Wiki, that it and Scratch need to appear sufficiently different. Eventually that boiled down to two points: graphics and redirects.

For graphics, we need to use a different website skin from the main Scratch website. Thankfully the Scratch Team was polite enough to accept a color palette change. This is still to be implemented.

As for maintaining old links, MIT cannot hard redirect from the old url's to the new url's. Hence we've setup a compromise where the old url's display a link to our landing page. A nice redirect page is to be setup.

Today

The community is as active as ever, which is a good sign! Our servers have been able to keep up with the massively increased traffic, and we've only had one minor hiccup with PHP process limits.

The only potential issue we've yet to look into is massive traffic generated by crawlers. The Scratch Team warned us about this, but we haven't had major issues yet.

As always, we'll continue to monitor the situation as it evolves. We want to make sure everything is running smoothly for Scratchers around the world.