Web Application Architect

Jeff Rigby is a core Drupal Developer at DPCI, having architected solutions for organizations such as Allied Advertising (www.gofobo.com), IPI (www.memberlink.net), and PMG (https://warl.securedonation.org/warl/form). Jeff was also a core contributor to projects at NYSHA (www.nysha.org), Hachette Filipacchi Media US (www.pointclickhome.com), and many others.

Blogs

Recent Webinars

> Integrating K4 and Drupal to Support Multi-channel Workflows

White Papers

> Best Practice Design Tools for Content Management Templating

> Widgets, Wizzbangs and Whoozits: Do They Support Your Business Model?

> The Challenges with ECM With or Without a Down Economy

October 05, 2009

Automating Drupal Site Rebuilds With Drush

Del.icio.us

Technorati

Digg

StumbleUpon

Testing against an exact copy of a live site is essential to successful Drupal deployments. This, unfortunately, can be both time consuming and mind-numbingly tedious. If, for example, something goes wrong with an update script that changes the database schema you need to fix the issue, rebuild the database, and test again. A potentially laborious task.

The process involves some variation of the following:

Replace the staging database with the production database
Update all the code to the latest development build using SVN
Copy all the files not under version control (images, PDFs, etc.) from the production build to the staging build
Run the Drupal update scripts at update.php
Configure the site for debugging (turn on the devel module, send emails to the log, etc)

After going through the process of rebuilding sites manually hundreds of times, I finally wised up and wrote a script to do everything in one swoop. I can run it, get a coffee, and come back to a completely rebuilt staging environment with no further action required.

An approximation of this script can be used for any Drupal build with some minor tweaks. Here's how to set it up:

Before running the script you'll need to install Drush (Drupal + Shell) on your server. Drush allows you to trigger many common Drupal actions from the command line like enabling/disabling modules, clearing the cache, running cron, etc. It's a powerful time saver and used extensively in the script.

For this script the following is assumed:

OS: Linux/Unix based server

Production Site
Database Name: prod_db
Path: /var/www/html/prod

Staging Site
Database Name: staging_db
path: /var/www/html/staging

The Script

This particular database has three very large tables that cache zip code information. These tables rarely change and ignoring them speeds the import significantly. In line 2 I specify that MySQL should not include these tables in the database dump and line 4 I drop all the tables on the staging site except the ignored tables.

echo "Dumping Production Build"

mysqldump -u root -ppassword --ignore-table=prod_db.zipcodes --ignore-table=prod_db.zipcodes_distances_theaters --ignore-table=prod_db.zipcodes_distances_theaters_city database > /tmp/prod.sql

echo "Deleting all replaceable tables"

mysql -u root -ppassword database_staging -e "show tables" -s | egrep -v "^zipcode" | xargs -I "@@" mysql -u root -ppassword database_staging -e "DROP TABLE @@"

If you want to export the entire database you can use this:

 echo "Dumping Production Build"

 drush -r /var/www/html/prod -y --result-file=/tmp/prod.sql

 echo "Deleting all replaceable tables"

 mysql -u root -ppassword database_staging -e "show tables" -s | "@@" mysql -u root -ppassword database_staging -e "DROP TABLE @@"

Lines 5-8 load the dumped production database into the staging database.

 echo "Reloading DB"

 mysql -u root -ppassword staging_db < /tmp/prod.sql

 echo "Removing db dump"

 rm -rf /tmp/prod.sql

Lines 9-17 copy all of the production files to the staging files. It also clears out the cached CSS files and resets the permissions on all the copied files.

 echo "Removing Old files"

rm -rf /var/www/html/staging/files/*

echo "Copying files from production"

cp -R /var/www/html/prod/files/* /var/www/html/staging/files/

cp -R /var/www/html/prod/files/.htaccess /var/www/html/staging/files/

echo "Removing cache files"

rm -rf /var/www/html/staging/files/css/*

echo "Fixing permissions on files"

chown -R apache /var/www/html/staging/files

Line 19 updates staging to the latest code from the subversion repository. This script assumes that staging is already pointed to the correct path in the repository.

echo "Updating code from repository"

svn up /var/www/html/staging

I like to clear the cache and watchdog table when recreating a staging site. I find that it's much easier to track down issues specific to the new build if you start with a clean slate.

echo "Clearing Cache and watchdog"

drush -r /var/www/html/staging -y watchdog delete all

drush -r /var/www/html/staging -y cache clear

Lines 24-27 is the equivalent of going to update.php in a browser and running all of the update scripts. I run it three times to ensure all updates are triggered. I've seen cases where updates are only available after running the update once.

echo "Running DB Updates (3 times to be safe)"

drush -r /var/www/html/staging -y updatedb

drush -r /var/www/html/staging -y updatedb

drush -r /var/www/html/staging -y updatedb

drush -r /var/www/html/staging -y cache clear

In these final lines I set some variables to do the following:

Enable the devel module
Turn on error reporting to the screen
Turn off CSS aggregation
Turn off the Boost page cache
Turn off Javascript aggregation
Reload the block cache to pick up any blocks defined in hook_block

echo "Disabling SMTP Server and enabling error reporting"

drush -r /var/www/html/staging -y enable drush

drush -r /var/www/html/staging eval "variable_set('smtp_library', 'sites/all/modules/devel/devel.module');"

drush -r /var/www/html/staging eval "variable_set('error_level', 1);"

drush -r /var/www/html/staging eval "variable_set('preprocess_css', 0);"

drush -r /var/www/html/staging eval "variable_set('boost', 0);"

drush -r /var/www/html/staging eval "variable_set('javascript_aggregator_aggregate_js', 0);"

drush -r /var/www/html/staging eval "variable_set('javascript_aggregator_jsmin', 0);"

drush -r /var/www/html/staging cache clear

echo "Reseting Block Cache"

drush -r /var/www/html/staging eval "block_admin_display('My Theme');"

echo "Done."

You can use this last section to set anything you commonly need in a staging site. This section save me a lot of mouse clicks and form submissions.

Once the script is written and saved remember to grant execute privileges to the file. The following command will do it:

# chmod u+x path/to/script

This script saves me an enormous amount of time and it enables us to easily test the deployment as many times as necessary. This leads to smoother deployments and happier clients.

Posted at 12:38 pm by Jeff Rigby