Deeden

• Articles
• Reading
• Podcast
• Feeds

Migrating ExpressionEngine from development to production

25th of June, 2008 | 17 comments

Moving an ExpressionEngine site from one server to another, or even one domain to another domain, is a complicated process. Whenever I’ve attempted to do this I’ve found that the hardest part is making sure that the database is correct for the new location. This involves copying the database across and then making sure I change every path and domain through the control panel to the new values.

I’ve never succeeded in migrating successfully using the manual method. I always miss something and have to deal with some small problem that inevitably crops up. Finally, I decided to make it easier on myself, so I wrote a script to perform the update for me. This will not be for everyone, however if you’re comfortable with command line scripts and ruby this may do the job for you.

The script (the impatient among you can find it at the end of this article) does one job and one job only. It takes a MySQL dump from an EE database and makes the updates required for any domain name and path changes, writing these out again in the format of another MySQL dump. For example, if I tell it that my development site lives at /www/deeden.tld and my production site at /www/deeden.co.uk it updates the database to reflect this, dealing with all of the complications that arise, such as correctly dealing with the serialisation which some of the tables use.

An example of performing the above update would be…

ee_migrator --current_domain=deeden.tld --new_domain=deeden.co.uk --current_path=/Users/steve/Sites/deeden.tld --new_path=/www/deeden.co.uk development_sql.txt > live_sql.txt

This runs the script (assuming it is named ee_migrator) and changes all occurrences of the domain deeden.tld to deeden.co.uk as well as changing the path to the web site from /Users/steve/Sites/deeden.tld to /www/deeden.co.uk. Rather useful when I’ve been developing a site on my iBook and want to move it to the live server.

I’ll put together a page with full instructions on the use of the script in the next few days. For now I’ll give a brief explanation of the parameters…

-h, --help

Display usage instructions.

-t, --table_prefix

Specify a different table prefix from the default, which is exp.

-a, --all_data

Keep all data from the initial dump. By default the script ignores a number of tables which should not be copied from a development, such as control panel logs and email caches. The --all_data allows all data to be copied, unsurprisingly.

-c, --current_domain <OLD DOMAIN>

The domain being moved from. In the example above it would be deeden.tld. In the case where the domain isn’t changing simply leave this option out.

-n, --new_domain <NEW DOMAIN>

The domain being moved to. In the example above it would be deeden.co.uk. In the case where the domain isn’t changing simply leave this option out as well.

-s, --current_path <CURRENT SITE PATH>

The path to where the old domain lives. If the path isn’t being changed then leave the option out.

-p, --new_path <NEW SITE PATH>

Specify the new path to the web directory. Again, if this isn’t being changed simply leave it out.

The input file is a mysqldump file, and the output is the same file with the appropriate changes made. Generally I do a dump of my development database, do a quick scan of the output to see that everything looks alright and them import the new database file into my a fresh database for the live site. Your mileage might vary.

There are a few things to bear in mind if you do plan to use this script, such as…

• Backup your database.
• I accept no responsibility for any damage this script may do. Use it at your own risk.
• If you find a bug please do report it.
• If you make an enhancement please send it to me, it may be useful to other people.
• Seriously, backup your database.

The most recent version of the script is version 1.4. You can also find the script on github.

17 comments

Kris Khaira / 24th of October, 2008 / #1

Just to let everyone know, I’ve been using this method for several projects already without any problems. Thanks to Stephen Rushe for making my work easier!

Stephen / 8th of December, 2008 / #2

Kris, I’m glad it has been of use :)

Matthew Spiel / 30th of January, 2009 / #3

I just used the script on my site after I moved to my new server… it is awesome!

but the script didn’t work (because my url was a little funky [sub domain and a /js on the end of it)... what I found that most places don’t say (EE wiki, etc) is that you need to go into your Cache folder (system/cache) and nuc (nook!) everything that is in there… this will allow the DB and EE to rewrite them with the proper permissions…

Stephen / 30th of January, 2009 / #4

The sub-domain shouldn’t be an issue Matthew. I’ve used it on sub-domains before without any problems. Feel free to tell me the paths you used, I’ll have a look and see if they should cause any trouble.

As for the cache thing, that’s outside the remit of the script. I deliberately wrote it to do the one job of updating the database :)

Wes Baker / 19th of March, 2009 / #5

I appreciate the work you’ve done here, but I ran the script and nothing seemed to have changed.

You can see the original file here: old.sql
And the new file here: new.sql

Wes Baker / 19th of March, 2009 / #6

I suppose it would help to see the command:

ruby ee_migrator.rb -c eeorig -n ee -s /Users/wes/Sites/ee -p /Users/wes/Sites/ee2 expressionengine.sql > newsql.sql

Stephen / 19th of March, 2009 / #7

Thanks for the report Wes. I’ve fixed the issue (case-sensitivity) and you can get a new copy of the script from the same place as before.

Wes Baker / 19th of March, 2009 / #8

Thanks for the quick turn around, after trying it out I’m getting this error:

The option—current_path must have a leading slash
ee_migrator.rb:93: undefined method `<<’ for nil:NilClass (NoMethodError)
  from ee_migrator.rb:88:in `each’
  from ee_migrator.rb:88


Here’s the command I’m running:

ruby ee_migrator.rb -c http://eeorig/ -n http://ee/ expressionengine.sql < newsite.sql


Stephen / 19th of March, 2009 / #9

What will I do with you? ;)

I’ll have a look at it now. The football is on and I can sit and happily look at this while listening to that.

Stephen / 19th of March, 2009 / #10

Wes, I’ve put a new version here for you to try out. It should fix the issue you reported. I’ve also extended it to allow http:// in the domain parameters.

Let me know how it goes. If it’s fine I’ll replace the existing version with it.

Wes Baker / 20th of March, 2009 / #11

Stephen,

It mostly worked. Using the same command and files, it replaced all instances of eeorig with ee. However, it seemed to delete some of the inserts. You can see what happened in the files I posted earlier—I updated them.

Stephen / 20th of March, 2009 / #12

By default the script ignores entries from some tables, such as the security_hashes, sessions and cp_log as the information they contain is of no great use when moving from a development to a production server.

You can tell the script not to ignore those lines if you provide the —all_data or -a option.

Hopefully that clarifies the issue.

Wes Baker / 20th of March, 2009 / #13

AHA! Works perfectly now. Thanks for all your hard work.

Stephen / 20th of March, 2009 / #14

Glad to help. Thanks for your feedback.

Wes Baker / 20th of April, 2009 / #15

Having another problem with the script, it seems to be generating a huge regex that seems redundant. Here’s the command and the result.

Command:
root@ubuntuserver:/var/www/brp75# ruby /usr/bin/ee-migrate/ee_migrator.rb -a—current_domain=brp75—new_domain=brp75.newcitymedia.com—current_path=/Users/wes/Sites/brp75—new_path=/var/www/brp75 brp75.sql > brp75.new.sql


Result

Wes Baker / 20th of April, 2009 / #16

I realize the command I pasted doesn’t have a space after the -a but I tried it with a space there and got the same result.

Avi Block / 19th of May, 2009 / #17

I would like to add that I find it helpful to put as many configuration options outsite the database and into config.php. It becomes easier to edit them in a file, and works better with version control also. Any setting which is accessed using the $PREF->ini() method can be placed in config.php. This includes tmpl_file_basepath, show_queries/template_debugging, cookie_domain and others.