Deploying with oregano

Niklas Laxström

Outline

  • Me and my wiki projects
  • Deploying with Oregano
  • Running jobs
  • Short URLs - one step further

About the speaker

Niklas Laxström

Developing and translating MediaWiki since 2004 [1]

Software Engineering, Language team,
Wikimedia Foundation
since 2011 [2]

Doctoral student, Language Technology, University of Helsinki
since 2014 [3]

Portfolio

Deploying with Oregano

Why Oregano

  • I run wikis straight from Git
  • Easy to apply local patches
  • Updating is easy: git pull --rebase
  • Easy to automate: wikiupdate

Why Oregano

Issues happen when:

  • Broken code from master
  • Missing composer update
  • Missing database update
  • Merge conflict

→ Wiki usually completely down

Oregano

Design goal: Avoid unnecessary downtime while keeping the flexibility of working with git.

Assumption: single-server setup.

Announcement in my blog.

Oregano features

  • Almost atomic updates
  • Environments
  • Quick rollback
  • Hooks

Oregano usage

oregano tag [name]
creates new tag

oregano deploy [tag] [target]
deploys tag (default latest) to the target (default production)

oregano rollback [target]
rollbacks to previously deployed tag in the target (default production)

Oregano setup 1/2

“download oregano”

mkdir -p /srv/mediawiki/workdir

cd /srv/mediawiki/workdir && do stuff

cd /srv/mediawiki && oregano tag && oregano deploy

Oregano setup 2/2

mkdir -p /www/{test.,}example.com/{docroot,logs}

ln -s /www/example.com/docroot/w /srv/mediawiki/targets/production

ln -s /www/test.example.com/docroot/w /srv/mediawiki/workdir

Oregano hooks

/srv/mediawiki/hooks/tag (example)

#!/bin/sh

udpcast created tag $1

Oregano hooks

/srv/mediawiki/hooks/deploy (example)

#!/bin/sh

udpcast deployed tag $1 to $2

sudo /usr/sbin/service mw-jobrunner restart

Oregano gotchas

  • Localisation cache (LC) can be inside or outside workdir or in the database
  • Takes about 2 GB without LC (keeps 4 tags)
  • Do not put uploads inside workdir
  • Hard to find ;)

Running jobs

Jobqueue alternatives

Jobqueue handles background tasks

  • Default (during web requests)
    • Used to slow down requests, slow
  • Cron
    • Hard to control concurrency
    • Startup overhead (esp. with hhvm)
  • Service
    • Good?

mw-jobrunner

[Service]

ExecStart=/usr/bin/php /srv/mediawiki/targets/production/maintenance/runJobs.php --wait --maxjobs=1000 --procs=1

Restart=always

RestartSec=15

https://github.com/wikimedia/translatewiki/blob/master/puppet/modules/wiki/files/mw-jobrunner.service (full example)

mw-jobrunner

  • --wait makes the script poll for new changes instead of exiting
  • --maxjobs avoid memory leaks (optional)
  • You can point it to the “production” target
    • Remember to restart in oregano deploy hook
  • I also use cron+udpcast to alert if it is not running

Final polish

Pretty main page URLs

https://example.com/https://example.com/wiki/MainPage

  • This redirect adds delay
  • The plain domain url is nicer to share

Pretty main page URLs

$wgHooks['GetLocalURL'][] = function ( &$title, &$url, $query ) {

if ( $title->isExternal() || $query != '' && $title->isMainPage() ) {

$url = '/';

}

};

// Tell MediaWiki that "/" should not be redirected

$wgHooks['TestCanonicalRedirect'][] = function ( $request ) {

return $request->getRequestURL() !== '/';

};

Full example for nginx in my blog.

Oregano - Wien - Google Slides