Monday 24 December 2012

rsync


login to destination host and moved to the destination directory and then issue the following command
rsync -avrc you@dsthost:/<Dest Dir>
$ rsync options source destination
$ rsync -zvr /var/opt/installation/inventory/ /root/temp
    -z is to enable compression
    -v verbose
    -r indicates recursive

rsync option -a indicates archive mode. -a option does the following,
    Recursive mode
    Preserves symbolic links
    Preserves permissions
    Preserves timestamp
    Preserves owner and group

$ rsync -azv /var/opt/installation/inventory/ /root/temp/

I would prefer to use -arvz for the best :)

at ubuntu you can use -n for a dryrun:
 
When you want to synchronize files from remote to local:
$ rsync -avz thegeekstuff@192.168.200.10:/var/lib/rpm /root/temp
 
NOTE: using you can do the following [ not sync from remote_system to remote_system ] Example 7. Do Not Overwrite the Modified Files at the Destination
In a typical sync situation, if a file is modified at the destination, we might not want to overwrite the file with the old file from the source. Use rsync -u option to do exactly that. (i.e do not overwrite a file at the destination, if it is modified). In the following example, the file called Basenames is already modified at the destination. So, it will not be overwritten with rsync -u.
 
NOTE: now how about the option: -avrzu
Example 8. Synchronize only the Directory Tree Structure (not the files)
Use rsync -d option to synchronize only directory tree from source to the destination. The below example, synchronize only directory tree in recursive manner, not the files in the directories.
 
Note: --progress  [ view the rsync process during transfer. ]
Example 10. Delete the Files Created at the Target
If a file is not present at the source, but present at the target, you might want to delete the file at the target during rsync.
In that case, use –delete option as shown below. rsync delete option deletes files that are not there in source directory.
 
Example 11. Do not Create New File at the Target
If you like, you can update (Sync) only the existing files at the target. In case source has new files, which is not there at the target, you can avoid creating these new files at the target. If you want this feature, use –existing option with rsync command.

Next, execute the rsync from the target.
$ rsync -avz --existing root@192.168.1.2:/var/lib/rpm/ .
root@192.168.1.2's password:
receiving file list ... done
./
sent 26 bytes  received 419 bytes  46.84 bytes/sec
total size is 88551424  speedup is 198991.96

Example 12. View the Changes Between Source and Destination
This option is useful to view the difference in the files or directories between source and destination.

Example 13. Include and Exclude Pattern during File Transfer
rsync allows you to give the pattern you want to include and exclude files or directories while doing synchronization.

Example 14. Do Not Transfer Large Files
You can tell rsync not to transfer files that are greater than a specific size using rsync –max-size option.

Example 15. Transfer the Whole File
One of the main feature of rsync is that it transfers only the changed block to the destination, instead of sending the whole file.

If network bandwidth is not an issue for you (but CPU is), you can transfer the whole file, using rsync -W option. This will speed-up the rsync process, as it doesn’t have to perform the checksum at the source and destination.




1. Test rsync over ssh (with password):
rsync -avz -e ssh /home/user1/ user1@remoteBackupServer:/backup/user1

for without password:
create public-key setup at the remote system, and then you can try doing the same.

################ rsync with exclude file(s) or/and dir(s) #######################

$ cd ~
$ mkdir -p source/dir1/dir2
$ mkdir -p source/dir3
$ touch source/file1.txt
$ touch source/file2.txt
$ touch source/dir1/dir2/file3.txt
$ touch source/dir3/file4.txt

If you don’t want to sync the dir1 (including all it’s subdirectories)
from the source to the destination folder, use the rsync –exclude option
 as shown below.
$ rsync -avz --exclude 'dir1' source/ destination/

The following example will exclude any directory (or subdirectories) under source/ that matches the pattern “dir*”
$ rsync -avz --exclude 'dir*' source/ destination/
$ find destination
destination
destination/file2.txt
destination/file1.txt

3. Exclude a specific file
To exclude a specific file, use the relative path of the file in the exclude option as shown below.
$ rm -rf destination $ rsync -avz --exclude 'dir1/dir2/file3.txt' source/ destination/ building file list ... done created directory destination ./ file1.txt file2.txt dir1/ dir1/dir2/ dir3/ dir3/file4.txt

Verify the destination directory to make sure it didn’t copy the specific file ( dir1/dir2/file3.txt in this example).
$ find destination destination destination/file2.txt destination/file1.txt destination/dir1 destination/dir1/dir2 destination/dir3 destination/dir3/file4.txt

4. Exclude path is always relative
If you are not careful, you might make this mistake.

In the following example, the exclude option seems to have a full path (i.e /dir1/dir2/file3.txt). But, from rsync point of view, exclude path is always relative, and it will be treated as dir1/dir2/file3.txt. In the example below, rsync will look for dir1 under source directory (and not under / root directory).
$ rsync -avz --exclude '/dir1/dir2/file3.txt' source/ destination/
So, the above command is exactly same as the following. Just to avoid confusion (and to make it easy to read), don’t give / in front of the exclude path.
$ rsync -avz --exclude 'dir1/dir2/file3.txt' source/ destination/
 
5. Exclude a specific file type
To exclude a specific file type that has a specific extension, use the appropriate pattern. For example, to exclude all the files that contains .txt as extension, do the following.
$ rsync -avz --exclude '*.txt' source/ destination/ building file list ... done created directory destination ./ dir1/ dir1/dir2/ dir3/

Verify the destination directory to make sure it didn’t copy the *.txt files.
$ find destination destination destination/dir1 destination/dir1/dir2 destination/dir3

Note: The above is very helpful, when you want to backup your home directory, but exclude all those huge image and video files that has a specific file extension.
 
6. Exclude multiple files and directories at the same time
When you want to exclude multiple files and directories, you can always specify multiple rsync exclude options in the command line as shown below.
$ rsync -avz --exclude file1.txt --exclude dir3/file4.txt source/ destination/

Wait. What if I had tons of files that I want to exclude from rsync?

I can’t keep adding them in the command line using multiple –exclude, which is hard to read, and hard to re-use the rsync command for later.

So, the better way is to use rsync –exclude-from option as shown below, where you can list all the files (and directories) you want to exclude in a file.

First, create a text file with a list of all the files and directories you don’t want to backup. This is the list of files and directories you want to exclude from the rsync.
$ vim exclude-list.txt file1.txt dir3/file4.txt

Next, execute the rsync using –exclude-from option with the exclude-list.txt as shown below.
$ rm -rf destination $ rsync -avz --exclude-from 'exclude-list.txt' source/ destination/ building file list ... done created directory destination ./ file2.txt dir1/ dir1/dir2/ dir1/dir2/file3.txt dir3/

Verify the desitination directory to make sure the files and directories listed in the exclude-list.txt file is not backed-up.
$ find destination destination destination/file2.txt destination/dir1 destination/dir1/dir2 destination/dir1/dir2/file3.txt destination/dir3





No comments:

Post a Comment