Working with Long-term Projects

For long-term projects one of the engineering challenges is to cope with technical evolution. The languages, libraries, and third party software like database engines and web servers we are using are under continuous development; new features are only added to the current development branches and security updates are only applied to currently supported releases. Using the example of one of our projects I will point out specific challenges and possible ways to deal with them in a series of posts.

On https://premium.zeit.de the German weekly DIE ZEIT offers subscriptions to digital formats of their newspaper in epub, mobi, pdf and mp3. Since 2011 Drupal 6 with Ubercart was used to take orders and deliver the content. Fulfillment is handled by a third party also providing the service to the company's print publications. User accounts and customer information are stored by a dedicated service running behind the company’s firewall.

In 2013 several challenges slowed down development: The web stack was still running on Ubuntu 8 with PHP 5.2 which had reached end of life. Lack of a comprehensive test suite meant upgrading would be a risky operation or require extensive end-user testing. Also it had turned out Ubercart was not a good fit for the project. Just maintaining a catalog of products and taking orders to be sent to the fulfillment provider does not require a full e-commerce solution.

The first step of reviving the project and upgrading to Drupal 7 was to set up a virtual machine for development, resembling the intended production environment (Debian 7 with Apache, PHP and MySQL). Together with the client and our partners Spry Group, we decided to use Vagrant to create a virtual machine from configuration and provisioning scripts which were committed to the repository. Before writing the provisioning scripts, tests were written to ensure the configuration of the development environment matched expectations.

  /**
   * Tests if a Drupal web site is served on port 80, and there are no errors.
   */
  public function testDrupalWorks() {
    $data = file_get_contents('http://127.0.0.1/');
    $pos = stripos($data, 'name="Generator" content="Drupal 7"');
    $this->assertTrue($pos !== FALSE);
  }

  /**
   * Tests that drush is properly configured.
   */
  public function testDrushWorks() {
    $output = shell_exec('drush st --pipe');
    $this->assertRegExp('#drush_configuration=/etc/drush/drushrc.php#', $output);
  }

Above code sample shows two of our infrastructure tests written in PHPUnit. They are executed inside the virtual machine which is supposed to serve a fully working Drupal installation. The second test asserts drush is available and is using the appropriate configuration file. We chose PHPUnit as our testing framework, and we also used it for functional testing of the Drupal web site later. In one of the upcoming posts I am going to explain the approach we developed with Spry Group. These and other infrastructure test suites proved very useful during our transition to provisioning with Chef.

This combination—of a virtual machine to simulate the production environment with corresponding tests validating its functionality—enabled us to adapt to technological evolution in a well-controlled environment. For example, trying out a new stable release of the operating system or even a different OS is a simple matter of creating a new virtual machine and running the test suite against it. The more code we would cover with tests the more likely it would be to detect any issues introduced by a newer version of PHP for example.

Over the coming months I will share further insights that we have learned from this long-term project.