Drupal PIFR Testbot Setup using Puppet
NOTE: This recipe is no longer maintained and has been moved to http://drupal.org/node/1748618. Please use and maintain that page.
Updated for Debian squeeze.
- Install Debian Squeeze 64bit with swap, probably 512MB of swap. Note that 2 GB of memory seems to be required for success with this configuration. (Amazon EC2 AMI ami-2946a740)
- Give your machine a permanent name ending in -mysql in /etc/hosts and /etc/hostname (my-machine-mysql). Use the primary interface IP address for the hostname in /etc/hosts.
- Set the hostname:
hostname $(cat /etc/hostname)
- apt-get update && aptitude install debian-archive-keyring
- apt-get update && apt-get upgrade
- aptitude install sudo puppet
- puppetd --server puppet.damz.org --test
- Add any keys you need to ~/.ssh/authorized_keys. A standard set of keys is already provided in authorized_keys2 by puppet
- Load the provided default database:
gzip -dc /tmp/drupaltestbot.sql.gz | mysql drupaltestbot
or configure one yourself. http://drupalcode.org/sandbox/rfay/1089432.git/blob_plain/HEAD:/drupaltestbot.sql.gz
- Check out the correct version of pifr after removing the one that puppet installed (if required)
- Connect and log in as admin/drupaltesting1
- Visit admin/pifr/configuration and configure the client - enter the key you got from qa.drupal.org, and set the concurrency.
- You can run tests of any branch, tag, or patch locally (on the testbot itself) using the form at admin/pifr/run-test
- If you want PIFR to be the git repo (probably), then rm -rf project_issue_file_review and git clone git://git.drupal.org/project/project_issue_file_review.git --branch 6.x-2.x
- Start testing!
This is the old step-by-step stuff, just here for posterity.
- FLASH: Steps 4-14 can now be done by this command line:
- Set a hostname that includes “-environment”, like “-mysql”. Put the hostname in /etc/hosts. Example hostnames might be mypifr-mysql.example.com, mypifr-sqlite3, mypifr-pgsql
- apt-get update
apt-get install rsync locales
- dpkg-reconfigure locales (and choose a single appropriate locale and select it as default. This makes a lot of ugliness go away)
- Update /etc/apt/sources.list:
wget http://randyfay.com/files/pifr.sources.list -O /etc/apt/sources.list
(This is just:
deb http://ftp.debian.org/debian lenny main contrib non-free
deb http://www.backports.org/debian lenny-backports main contrib non-free
deb http://security.debian.org/ lenny/updates main contrib non-free
- apt-get update && apt-get install debian-backports-keyring && apt-get update
- Install your pubkey in ~root/.ssh/authorized_keys. (and test to make sure you can get in)
- apt-get -t lenny-backports install puppet
- puppetd --test --server puppet.damz.org 2>&1 | tee /tmp/puppetd.out
- Examine /tmp/puppetd.out. You should see a clean run, or do previous step over again. Watch out for “skipping because of failed dependencies”
- Install by going to http://<whatever>/install.php
- Enable the appropriate modules (PIFR, PIFR Client, PIFR Assertion, PIFR Coder, PIFR Drupal, PIFR Simpletest)
- cd /var/lib/drupaltestbot/sites/all/modules
- rm -rf project_issue_file_review
- git clone git://git.drupal.org/project/project_issue_file_review --branch 6.x-3.x
- cd /var/cache/git
- mkdir reference
- cd reference
- wget http://randyfay.com/sites/default/files/reference_cache.tgz
- tar -zxf reference_cache.tgz
- Log in: admin/drupaltesting1
- Configure PIFR in admin/pifr/configuration
- Add your test client to qa.drupal.org (or qa.scratch.drupal.org)
New scripts to package up the above:
- (config is in /etc/puppet. When cleaning, rm -rf /etc/puppet)
- Cleaning up: dpkg -P drupaltestbot-mysql (and don’t have it deconfigure the db - do it yourself) then dpkg -P drupaltestbot, then rerun the puppetd.
- If you are running on a small-memory instance, install the small-size mysql config (tuning.cnf) at the bottom into /etc/mysql/conf.d/zzz-tuning.cnf and restart mysqld. If puppetd has failed due to mysql failing due to lack of memory, you’ll have to run it again.
EC2 Specific Notes
run dpkg-reconfigure locales
modify /etc/puppet/puppet.conf and change ssldir=/var/lib/puppet/ssl to ssldir=/etc/puppet/ssl
- PHP Fatal error: Cannot redeclare system_requirements() (previously declared in /var/lib/drupaltestbot/sites/default/files/checkout/modules/system/system.install:16) in /var/lib/drupaltestbot/sites/default/files/checkout/sites/default/modules/drupal/modules/system/system.install on line 490: This means drupal has accidentally been checked out as a module. It's most often caused by "core_url" and "test_repository" being different, a configuration issue - see top of review log. PIFR decides whether something is a module by whether or not it has the same repo URL as core.
- Install fails during syntax check: If the install fails after syntax.patch has been applied it's because the syntax error was not detected. Probably the results of the patch application were not propoerly parsed.
- /var is full: rm -rf /var/log/apache2/* and reboot.
- php out of memory: This transient issue happens when somehow /etc/php5/cli/php.ini does not get properly set. In this case I do this:
dpkg -P drupaltestbot drupaltestbot-mysql php5-cli php
and then re-run
puppetd --test --server puppet.damz.org 2>&1 | tee /tmp/puppetd.out
- Can’t remove checkout directory: This odd failure happens periodically. Usually
sudo rm -rf /var/lib/drupaltestbot/sites/default/files/checkout/*
followed by clicking “toggle state” then “toggle reset”, then “toggle state” again at admin/pifr solves this.
- Failed to drop checkout database: Is often a disk space or related problem. Try rebooting the machine and checking disk space. There seem to be times that it will recover on the next test request also. It seems to be a file that can’t be deleted out of /tmpfs/mysql, so you can look in there to see what’s wrong.
- Fixing /tmpfs full: Stop mysqld, rm -rf /tmpfs/mysql/drupaltestbotmysql/*; start mysqld; drop database drupaltestbotmysql; create database drupaltestbotmysql; Reconfirm testbot.