Staging/Live Setup with Exorcist

1. Exorcist vs. Repligard
2. Setting up Staging/Live
1. Database setup
2. Exorcist installation
3. Staging2Live scripts installation
4. Configuration
3. Networked Staging/Live
4. Clustered Staging/Live
5. Preventing html links from live to staging

NOTE: The Exorcist-based setup has been replaced for Midgard 1.8 and MidCOM 2.8 with Staging to Live Setup with MidCOM. Exorcist is only recommended for Midgard 1.7 installations.

Staging/Live is a concept where content is produced and site changes tested on a separate CMS installation called Staging. When the changes have been approved, they get copied to the production site, called Live.

With Midgard CMS, the Staging/Live concept is handled by running Midgard on two separate databases. The Staging and Live databases can run either on the same server or on different servers, depending on your security requirements.

Content is copied from the Staging server by using the Exorcist replication utility. Exorcist first dumps the content from Staging database into a generic XML format, then filters out unapproved content from it, and finally imports it to the Live database.

Exorcist vs. Repligard

Exorcist replaces the older Repligard-based Staging/Live system that originated from Hong Kong Linux Center's Nadmin Studio distribution. Exorcist provides faster replication and better reliability than Repligard. Additionally, it is able to support all MgdSchema types.

Since October 2005, Exorcist is the preferred way of replicating content with Midgard CMS.

Setting up Staging/Live

Database setup

• Install your Midgard database as midgardstaging
  • This will be the database used for content production and testing
  • You can also use an existing database with a different name, just remember to change the example configuration accordingly
• After creating sitegroup (and preferably hosts), create an identical copy of the staging database as midgardlive.
  • It is essential that the same sitegroup object exists in both databases
  • Remember to have separate staging and live hosts in the staging database, if you just change one host on the live database it will be overwritten during next replication.
  • This will be the database used by normal users when browsing the site

Exorcist installation

• Download Exorcist from http://sourceforge.net/projects/exorcist/

• Or compile Exorcist yourself

  • Install Maven 1.0.2 from http://maven.apache.org/maven-1.x/
  • Install Midgard Core (recent CVS HEAD or 1.8 version needed)
  • Check out Exorcist from CVS at sourceforge.net/projects/exorcist

  • Compile it via Maven

    $ maven dist

  • Maven will compile Exorcist to target/distributions/exorcist-1.0.tar.gz

• Unpack the Exorcist tarball to /opt/exorcist

• Compile the DateFunctions.java

  $ javac DateFunctions.java

• Make sure the DateFunctions.class is in /opt/exorcist/classes directory

Staging2Live scripts installation

• Download the staging2live scripts from http://midgard.tigris.org/source/browse/midgard/src/tools/staging2live/
• Copy the files to /opt/exorcist

Configuration

• Get staging/live configuration scripts for Exorcist from Midgard
  CVS directory src/tools/staging2live
• Exorcist configuration uses Spring Framework's Bean configuration file format
• Edit the staging2live.xml configuration file to use the correct
  database settings for both exporter and importer
  • In this case exporter will be midgardstaging and importer
    will be midgardlive
  • The staging2live.xslt will be used as the converter
  • Make sure the staging2live.xml importer bean has property 'delete' set to 'true' if you want 'deletes' to be imported also
• Try the replication manually first by running staging2live.sh

• If you have memory problems, you can try to give more memory by editing /opt/exorcist/bin/exorcist.sh

  java -Xmx512m -cp "$CP" net.sf.exorcist.core.Exorcist $*

• Once all settings are right, copy the contents of staging2live.crontab to your cron.

NOTE: If you use two databases on same server make sure you use different MidCOM cache directory on each, set $GLOBALS['midcom_config_local']['cache_base_directory'] in snippet /sitegroup-config/midcom-template/config by whatever you use to distinguish between staging and live sites (usually port)

NOTE: If the elements don't seem to replicate properly, you have to empty the Midgard cache directory of the live site. You can accomplish this by adding something like 'rm /var/cache/midgard/midgardlive/*' to the staging2live.sh script.

Networked Staging/Live

It is possible to set up the staging/live process also to happen over a network. In this case you can keep the staging database in secure internal network, and run the live server(s) on DMZ. To do this you need to modify the staging/live scripts a bit:

NODE=... # Name of the live server
cd /opt/exorcist
sh bin/exorcist.sh staging2live-export.xml
scp content.zip $NODE:/opt/exorcist
ssh $NODE 'cd /opt/exorcist; sh bin/exorcist.sh staging2live-import.xml'

This configuration assumes that you have configured SSH public key authentication or some other mechanism that allows the script to connect to the live server without entering a password.

Clustered Staging/Live

A similar script can be used to replicate content to a clustered live server that consists of two or more nodes.

NODE1=... # Name of the first live node
NODE2=... # Name of the second live node
...
cd /opt/exorcist
sh bin/exorcist.sh staging2live-export.xml
for NODE in $NODE1 $NODE2 ...; do
    scp content.zip $NODE:/opt/exorcist
    ssh $NODE 'cd /opt/exorcist; sh bin/exorcist.sh staging2live-import.xml'
done

Preventing html links from live to staging

How to prevent html links from live to staging