How to mirror/sync a remote ftp folder with a local folder
lftp is one of the best command line utilities currently available for working with ftp servers, One of its useful features is mirroring which is suitable for backup purposes and very useful for slow connections. In this article i'm going to show you how to use this utility to backup and mirror your files a ftp server.
The following script mirros your local folder to a remote ftp server, it only updates the changes you've made since the last execution so only the first time requires lots of bandwidth :
#!/bin/bash HOST='mysite.com' USER='myuser' PASS='myuser' TARGETFOLDER='/new' SOURCEFOLDER='/home/myuser/backups' lftp -f " open $HOST user $USER $PASS lcd $SOURCEFOLDER mirror --reverse --delete --verbose $SOURCEFOLDER $TARGETFOLDER bye "
You can save it as upload.sh and execute it using the following command :
lftp offers many parameters, here i'll explain some of the most useful ones :
- --reverse : Indicates that you want to update your remote folder. if you want to update your local folder then you should remove this option and also swap values of TARGETFOLDER and SOURCEFOLDER variables
- --delete : It tells lftp that you want to remove all the files/folders that no longer exist on source folder
- --use-cache : If you're about to sync very large number of files, this option can speed up things a lot. It does not have any effect the first time but in the next sessions it uses local cache instead of scanning the remote server
- --exclude : Sometimes there are folder/files on both source and target folders that have nothing to do with the mirroring, using this option we can tell lftp to ignore this sort of files
FTP servers' version and configuration is not always the same , so the above scripts might not always work,
The usual issue is Passive Mode, and occurs when the client (you) is behind proxy or firewall.
It's possible to use this utility under windows by downloading this files from Cygwin Information and Installation :
I've attached executable in case you had trouble downloading them from cygwin server. BUT CHECK THEM FILES WITH YOUR ANTIVIRUS FIRST
- How do you use LFTP to mirror MOVEit DMZ Folders?
- How to Use wget for Updating a Mirror via FTP ?