Linux: Automatically Updating a Remote Site

Monday, 18 Aug 2008
NB: Don't let the cute kitty fool you!  This is an advanced tutorial.  Don't bother reading it if you're not familiar with Linux / web development.

The Sleeping Geek Kitten - Angers -So, I'm a webdeveloper, right?  And I often find myself in a position where I've worked on an existing site and need to upload my changes to that site, right?  And, being a coder-type-person, I want to do it the easiest way possible right?

Well, I've got just the solution if you're like me.  First off, set up passwordless ssh into the remote server.

Next, use this command to automatically copy everything across...

find . -mtime -30 -exec scp -r '{}' user@example.com:www/ ';'

Let's break this up...

find
This is a great command that allows you to find almost any file anywhere on your server.
find .
Here, we're telling find to only find files from the current directory ('.') and below.
-mtime -30
The -mtime switch basically lets us find files that have been modified within a specified amount of time.  The -30 says we're looking for files that have been modified in under 30 days since now.  To find files that were modified OVER 30 days ago, you'd just type:  -mtime 30
-exec
This switch tells find to execute the command that follows with EVERY file it finds.  Yup, this is where it starts getting juicy (for us geeks, that is ;))
scp
This is the command used to transfer files over an SSH connection.  It's much more secure than FTP because your login credentials (and the whole session!) is encrypted before being sent to the other server.  With FTP, anyone on either computer (or, in some cases, on either network!!) can simply grab your details and use them to log in as well!
-r
This stands for 'recursive' and it will create directories for you.  So if your command above finds a file inside a directory, it will first create that directory on the remote server (if it doesn't yet exist) before copying the file across.
'{}'
This is a placeholder for the results that the find command returns.  In other words, it will be replace with each filename that you want copied across.  Pretty important!
user
Replace this with your username on the remote server.
example.com
This should be replaced with the hostname of the remote server.
www/
And this is the remote directory you want all the files copied into.  This is usually 'www/'  or  'public_html/'
';'
And this is just how you tell the find command that this is the end of the command that the -exec switch should execute.

That's it!  Provided you have everything set up right with your passwordless ssh, this will be a pretty damn easy way to update any site you work on.  Of course, you can tweak the find command to your heart's content.  It's really one of the most powerful commands you have in your arsenal.  Use it wisely.

NB! NB!  Before screwing up a live server, test this out on some mickey-mouse account of yours that it's OK to mess up a little.  ANY time you automate something, you face the risk of screwing EVERYTHING up en-masse.  Been there, done that, didn't have too much fun!  Backups are a good thing.  Use them!
<!-- technorati tags begin -->

Tags: , , ,

<!-- technorati tags end -->

Litestep - Monday, 18 Aug 2008

Have you ever heard about rsync? :P

Norio - Monday, 18 Aug 2008

Yup but never tried it. Does it require any special services listening on the target machine?

Inflames (Lean) - Tuesday, 9 Feb 2010

Hi Norio,

2 years later, so you probably know this.

If you have setup server certificates on both servers then you can use rsync without passwords.

You don't need any special services listening on a target machine as long as sshd is running.

So:

rsync -xarv "-e ssh" /home/lean/ norio@10.1.1.10:/home/lean/

This will send all my data on my server (/home/lean/) to your server (10.1.1.10 /home/lean/). If I run the same command again it will only update the newer files.

Cheers,
Inflames

Norio - Tuesday, 9 Feb 2010

Thanks :) Yep I've used passwordless SSH before. Makes things go much faster but haven't really bothered setting it up lately. I guess I'm just used to doing things the long way around as I don't have to copy files around too much.

Norio - Tuesday, 9 Feb 2010

Ahh, I see your main point was using passwordless ssh with rsync as opposed to find + scp. Sorry - didn't quite realise in the beginning. But, yep, ended up doing that for a few projects. Much better than my find solution :P

Post new comment

Image CAPTCHA
Enter the characters shown in the image.