RSync
rsync's speciality lies in its ability to analyse files and only copy the changes made to files rather than all files.
This can lead to enormous improvements when copying a directory tree a second time.
Trailing slashes
-
/home/user/dir: without the trailing slash, it will copy the directory contents in its entirety -
/home/user/dir/: with the trailing slash, will copy the contents of the directory but won't recreate the directory
If you're trying to replicate a directory structure with rsync, you should omit the trailing slash, for instance, if you're mirroring /var/www on another machine or something like that.
Hidden files
rsync will move hidden files without any special options.
If you want to exclude hidden files, you can use the option --exclude=".*/"
You can also use the --exclude option to prevent copying things like Vim's swap files (.swp) and automatic backups (.bak) created by some programs.
Delete files
If you want to make sure that local files you've deleted since the last time you ran rsync are deleted from the external system as well, you'll want to add the --deleted option, like so:
rsync -avh --delete /home/user/dir/ /media/disk/backup
Dry runs
While you're getting used to rsync, it's probably a good idea to use the --dry-run option with your commands to run through the transfer first, without actually copying or synching files.
Examples
rsync [options] source destination
export SRC=/media/igaray/1373c3ad-4dc6-4e88-aac3-2c29e0f94d68/igarai@gmail.com/backup/applications/
export DST=/media/igaray/DATA-4T/igarai@gmail.com/applications/
rsync --dry-run --delete --verbose --human-readable --stats --itemize-changes --progress --recursive --links --no-perms --no-times --group --owner --devices --specials --size-only --exclude='.DS_Store' --exclude='Thumbs.db' --exclude='\._*' $SRC $DST
Options Summary
Here is a short summary of the options available in rsync.
-v, --verbose increase verbosity
--info=FLAGS fine-grained informational verbosity
--debug=FLAGS fine-grained debug verbosity
--msgs2stderr special output handling for debugging
-q, --quiet suppress non-error messages
--no-motd suppress daemon-mode MOTD (see caveat)
-c, --checksum skip based on checksum, not mod-time & size
-a, --archive archive mode; equals -rlptgoD (no -H,-A,-X)
--no-OPTION turn off an implied OPTION (e.g. --no-D)
-r, --recursive recurse into directories
-R, --relative use relative path names
--no-implied-dirs don't send implied dirs with --relative
-b, --backup make backups (see --suffix & --backup-dir)
--backup-dir=DIR make backups into hierarchy based in DIR
--suffix=SUFFIX backup suffix (default ~ w/o --backup-dir)
-u, --update skip files that are newer on the receiver
--inplace update destination files in-place
--append append data onto shorter files
--append-verify --append w/old data in file checksum
-d, --dirs transfer directories without recursing
-l, --links copy symlinks as symlinks
-L, --copy-links transform symlink into referent file/dir
--copy-unsafe-links only "unsafe" symlinks are transformed
--safe-links ignore symlinks that point outside the tree
--munge-links munge symlinks to make them safer
-k, --copy-dirlinks transform symlink to dir into referent dir
-K, --keep-dirlinks treat symlinked dir on receiver as dir
-H, --hard-links preserve hard links
-p, --perms preserve permissions
-E, --executability preserve executability
--chmod=CHMOD affect file and/or directory permissions
-A, --acls preserve ACLs (implies -p)
-X, --xattrs preserve extended attributes
-o, --owner preserve owner (super-user only)
-g, --group preserve group
--devices preserve device files (super-user only)
--specials preserve special files
-D same as --devices --specials
-t, --times preserve modification times
-O, --omit-dir-times omit directories from --times
-J, --omit-link-times omit symlinks from --times
--super receiver attempts super-user activities
--fake-super store/recover privileged attrs using xattrs
-S, --sparse turn sequences of nulls into sparse blocks
--preallocate allocate dest files before writing
-n, --dry-run perform a trial run with no changes made
-W, --whole-file copy files whole (w/o delta-xfer algorithm)
--checksum-choice=STR choose the checksum algorithms
-x, --one-file-system don't cross filesystem boundaries
-B, --block-size=SIZE force a fixed checksum block-size
-e, --rsh=COMMAND specify the remote shell to use
--rsync-path=PROGRAM specify the rsync to run on remote machine
--existing skip creating new files on receiver
--ignore-existing skip updating files that exist on receiver
--remove-source-files sender removes synchronized files (non-dir)
--del an alias for --delete-during
--delete delete extraneous files from dest dirs
--delete-before receiver deletes before xfer, not during
--delete-during receiver deletes during the transfer
--delete-delay find deletions during, delete after
--delete-after receiver deletes after transfer, not during
--delete-excluded also delete excluded files from dest dirs
--ignore-missing-args ignore missing source args without error
--delete-missing-args delete missing source args from destination
--ignore-errors delete even if there are I/O errors
--force force deletion of dirs even if not empty
--max-delete=NUM don't delete more than NUM files
--max-size=SIZE don't transfer any file larger than SIZE
--min-size=SIZE don't transfer any file smaller than SIZE
--partial keep partially transferred files
--partial-dir=DIR put a partially transferred file into DIR
--delay-updates put all updated files into place at end
-m, --prune-empty-dirs prune empty directory chains from file-list
--numeric-ids don't map uid/gid values by user/group name
--usermap=STRING custom username mapping
--groupmap=STRING custom groupname mapping
--chown=USER:GROUP simple username/groupname mapping
--timeout=SECONDS set I/O timeout in seconds
--contimeout=SECONDS set daemon connection timeout in seconds
-I, --ignore-times don't skip files that match size and time
--size-only skip files that match in size
-@, --modify-window=NUM set the accuracy for mod-time comparisons
-T, --temp-dir=DIR create temporary files in directory DIR
-y, --fuzzy find similar file for basis if no dest file
--compare-dest=DIR also compare received files relative to DIR
--copy-dest=DIR ... and include copies of unchanged files
--link-dest=DIR hardlink to files in DIR when unchanged
-z, --compress compress file data during the transfer
--compress-level=NUM explicitly set compression level
--skip-compress=LIST skip compressing files with suffix in LIST
-C, --cvs-exclude auto-ignore files in the same way CVS does
-f, --filter=RULE add a file-filtering RULE
-F same as --filter='dir-merge /.rsync-filter'
repeated: --filter='- .rsync-filter'
--exclude=PATTERN exclude files matching PATTERN
--exclude-from=FILE read exclude patterns from FILE
--include=PATTERN don't exclude files matching PATTERN
--include-from=FILE read include patterns from FILE
--files-from=FILE read list of source-file names from FILE
-0, --from0 all *from/filter files are delimited by 0s
-s, --protect-args no space-splitting; wildcard chars only
--address=ADDRESS bind address for outgoing socket to daemon
--port=PORT specify double-colon alternate port number
--sockopts=OPTIONS specify custom TCP options
--blocking-io use blocking I/O for the remote shell
--outbuf=N|L|B set out buffering to None, Line, or Block
--stats give some file-transfer stats
-8, --8-bit-output leave high-bit chars unescaped in output
-h, --human-readable output numbers in a human-readable format
--progress show progress during transfer
-P same as --partial --progress
-i, --itemize-changes output a change-summary for all updates
-M, --remote-option=OPTION send OPTION to the remote side only
--out-format=FORMAT output updates using the specified FORMAT
--log-file=FILE log what we're doing to the specified FILE
--log-file-format=FMT log updates using the specified FMT
--password-file=FILE read daemon-access password from FILE
--list-only list the files instead of copying them
--bwlimit=RATE limit socket I/O bandwidth
--stop-at=y-m-dTh:m Stop rsync at year-month-dayThour:minute
--time-limit=MINS Stop rsync after MINS minutes have elapsed
--write-batch=FILE write a batched update to FILE
--only-write-batch=FILE like --write-batch but w/o updating dest
--read-batch=FILE read a batched update from FILE
--protocol=NUM force an older protocol version to be used
--iconv=CONVERT_SPEC request charset conversion of filenames
--checksum-seed=NUM set block/file checksum seed (advanced)
--noatime do not alter atime when opening source files
-4, --ipv4 prefer IPv4
-6, --ipv6 prefer IPv6
--version print version number
(-h) --help show this help (see below for -h comment)
-v, --verbose
This option increases the amount of information you are given during the transfer. By default, rsync works silently. A single -v will give you information about what files are being
transferred and a brief summary at the end. Two -v options will give you information on what files are being skipped and slightly more information at the end. More than two -v options
should only be used if you are debugging rsync.
In a modern rsync, the -v option is equivalent to the setting of groups of --info and --debug options. You can choose to use these newer options in addition to, or in place of using
--verbose, as any fine-grained settings override the implied settings of -v. Both --info and --debug have a way to ask for help that tells you exactly what flags are set for each in‐
crease in verbosity.
However, do keep in mind that a daemon’s "max verbosity" setting will limit how high of a level the various individual flags can be set on the daemon side. For instance, if the max is 2,
then any info and/or debug flag that is set to a higher value than what would be set by -vv will be downgraded to the -vv level in the daemon’s logging.
--info=FLAGS
This option lets you have fine-grained control over the information output you want to see. An individual flag name may be followed by a level number, with 0 meaning to silence that out‐
put, 1 being the default output level, and higher numbers increasing the output of that flag (for those that support higher levels). Use --info=help to see all the available flag names,
what they output, and what flag names are added for each increase in the verbose level. Some examples:
rsync -a --info=progress2 src/ dest/
rsync -avv --info=stats2,misc1,flist0 src/ dest/
Note that --info=name’s output is affected by the --out-format and --itemize-changes (-i) options. See those options for more information on what is output and when.
This option was added to 3.1.0, so an older rsync on the server side might reject your attempts at fine-grained control (if one or more flags needed to be send to the server and the
server was too old to understand them). See also the "max verbosity" caveat above when dealing with a daemon.
FLAGS:
Use OPT or OPT1 for level 1 output, OPT2 for level 2, etc.; OPT0 silences.
BACKUP Mention files backed up
COPY Mention files copied locally on the receiving side
DEL Mention deletions on the receiving side
FLIST Mention file-list receiving/sending (levels 1-2)
MISC Mention miscellaneous information (levels 1-2)
MOUNT Mention mounts that were found or skipped
NAME Mention 1) updated file/dir names, 2) unchanged names
PROGRESS Mention 1) per-file progress or 2) total transfer progress
REMOVE Mention files removed on the sending side
SKIP Mention files that are skipped due to options used
STATS Mention statistics at end of run (levels 1-3)
SYMSAFE Mention symlinks that are unsafe
ALL Set all --info options (e.g. all4)
NONE Silence all --info options (same as all0)
HELP Output this help message
Options added for each increase in verbose level:
1) COPY,DEL,FLIST,MISC,NAME,STATS,SYMSAFE
2) BACKUP,MISC2,MOUNT,NAME2,REMOVE,SKIP
--debug=FLAGS
This option lets you have fine-grained control over the debug output you want to see. An individual flag name may be followed by a level number, with 0 meaning to silence that output, 1
being the default output level, and higher numbers increasing the output of that flag (for those that support higher levels). Use --debug=help to see all the available flag names, what
they output, and what flag names are added for each increase in the verbose level. Some examples:
rsync -avvv --debug=none src/ dest/
rsync -avA --del --debug=del2,acl src/ dest/
Note that some debug messages will only be output when --msgs2stderr is specified, especially those pertaining to I/O and buffer debugging.
This option was added to 3.1.0, so an older rsync on the server side might reject your attempts at fine-grained control (if one or more flags needed to be send to the server and the
server was too old to understand them). See also the "max verbosity" caveat above when dealing with a daemon.
FLAGS:
Use OPT or OPT1 for level 1 output, OPT2 for level 2, etc.; OPT0 silences.
ACL Debug extra ACL info
BACKUP Debug backup actions (levels 1-2)
BIND Debug socket bind actions
CHDIR Debug when the current directory changes
CONNECT Debug connection events (levels 1-2)
CMD Debug commands+options that are issued (levels 1-2)
DEL Debug delete actions (levels 1-3)
DELTASUM Debug delta-transfer checksumming (levels 1-4)
DUP Debug weeding of duplicate names
EXIT Debug exit events (levels 1-3)
FILTER Debug filter actions (levels 1-2)
FLIST Debug file-list operations (levels 1-4)
FUZZY Debug fuzzy scoring (levels 1-2)
GENR Debug generator functions
HASH Debug hashtable code
HLINK Debug hard-link actions (levels 1-3)
ICONV Debug iconv character conversions (levels 1-2)
IO Debug I/O routines (levels 1-4)
OWN Debug ownership changes in users & groups (levels 1-2)
PROTO Debug protocol information
RECV Debug receiver functions
SEND Debug sender functions
TIME Debug setting of modified times (levels 1-2)
ALL Set all --debug options (e.g. all4)
NONE Silence all --debug options (same as all0)
HELP Output this help message
Options added for each increase in verbose level:
2) BIND,CONNECT,CMD,DEL,DELTASUM,DUP,FILTER,FLIST,ICONV
3) ACL,BACKUP,CONNECT2,DEL2,DELTASUM2,EXIT,FILTER2,FLIST2,FUZZY,GENR,OWN,RECV,SEND,TIME
4) CMD2,DEL3,DELTASUM3,EXIT2,FLIST3,ICONV2,OWN2,PROTO,TIME2
5) CHDIR,DELTASUM4,FLIST4,FUZZY2,HASH,HLINK
--size-only
This modifies rsync’s "quick check" algorithm for finding files that need to be transferred, changing it from the default of transferring files with either a changed size or a changed
last-modified time to just looking for files that have changed in size. This is useful when starting to use rsync after using another mirroring system which may not preserve timestamps
exactly.
-c, --checksum
This changes the way rsync checks if the files have been changed and are in need of a transfer. Without this option, rsync uses a "quick check" that (by default) checks if each file’s
size and time of last modification match between the sender and receiver. This option changes this to compare a 128-bit checksum for each file that has a matching size. Generating the
checksums means that both sides will expend a lot of disk I/O reading all the data in the files in the transfer (and this is prior to any reading that will be done to transfer changed
files), so this can slow things down significantly.
The sending side generates its checksums while it is doing the file-system scan that builds the list of the available files. The receiver generates its checksums when it is scanning for
changed files, and will checksum any file that has the same size as the corresponding sender’s file: files with either a changed size or a changed checksum are selected for transfer.
Note that rsync always verifies that each transferred file was correctly reconstructed on the receiving side by checking a whole-file checksum that is generated as the file is trans‐
ferred, but that automatic after-the-transfer verification has nothing to do with this option’s before-the-transfer "Does this file need to be updated?" check.
For protocol 30 and beyond (first supported in 3.0.0), the checksum used is MD5. For older protocols, the checksum used is MD4.
--no-OPTION
You may turn off one or more implied options by prefixing the option name with "no-". Not all options may be prefixed with a "no-": only options that are implied by other options (e.g.
--no-D, --no-perms) or have different defaults in various circumstances (e.g. --no-whole-file, --no-blocking-io, --no-dirs). You may specify either the short or the long option name af‐
ter the "no-" prefix (e.g. --no-R is the same as --no-relative).
For example: if you want to use -a (--archive) but don’t want -o (--owner), instead of converting -a into -rlptgD, you could specify -a --no-o (or -a --no-owner).
The order of the options is important: if you specify --no-r -a, the -r option would end up being turned on, the opposite of -a --no-r. Note also that the side-effects of the
--files-from option are NOT positional, as it affects the default state of several options and slightly changes the meaning of -a (see the --files-from option for more details).
-r, --recursive
This tells rsync to copy directories recursively. See also --dirs (-d).
Beginning with rsync 3.0.0, the recursive algorithm used is now an incremental scan that uses much less memory than before and begins the transfer after the scanning of the first few di‐
rectories have been completed. This incremental scan only affects our recursion algorithm, and does not change a non-recursive transfer. It is also only possible when both ends of the
transfer are at least version 3.0.0.
Some options require rsync to know the full file list, so these options disable the incremental recursion mode. These include: --delete-before, --delete-after, --prune-empty-dirs, and
--delay-updates. Because of this, the default delete mode when you specify --delete is now --delete-during when both ends of the connection are at least 3.0.0 (use --del or --delete-dur‐
ing to request this improved deletion mode explicitly). See also the --delete-delay option that is a better choice than using --delete-after.
Incremental recursion can be disabled using the --no-inc-recursive option or its shorter --no-i-r alias.
-p, --perms
This option causes the receiving rsync to set the destination permissions to be the same as the source permissions. (See also the --chmod option for a way to modify what rsync considers
to be the source permissions.)
When this option is off, permissions are set as follows:
o Existing files (including updated files) retain their existing permissions, though the --executability option might change just the execute permission for the file.
o New files get their "normal" permission bits set to the source file’s permissions masked with the receiving directory’s default permissions (either the receiving process’s umask,
or the permissions specified via the destination directory’s default ACL), and their special permission bits disabled except in the case where a new directory inherits a setgid bit
from its parent directory.
Thus, when --perms and --executability are both disabled, rsync’s behavior is the same as that of other file-copy utilities, such as cp(1) and tar(1).
In summary: to give destination files (both old and new) the source permissions, use --perms. To give new files the destination-default permissions (while leaving existing files un‐
changed), make sure that the --perms option is off and use --chmod=ugo=rwX (which ensures that all non-masked bits get enabled). If you’d care to make this latter behavior easier to
type, you could define a popt alias for it, such as putting this line in the file ~/.popt (the following defines the -Z option, and includes --no-g to use the default group of the desti‐
nation dir):
rsync alias -Z --no-p --no-g --chmod=ugo=rwX
You could then use this new option in a command such as this one:
rsync -avZ src/ dest/
(Caveat: make sure that -a does not follow -Z, or it will re-enable the two "--no-*" options mentioned above.)
The preservation of the destination’s setgid bit on newly-created directories when --perms is off was added in rsync 2.6.7. Older rsync versions erroneously preserved the three special
permission bits for newly-created files when --perms was off, while overriding the destination’s setgid bit setting on a newly-created directory. Default ACL observance was added to the
ACL patch for rsync 2.6.7, so older (or non-ACL-enabled) rsyncs use the umask even if default ACLs are present. (Keep in mind that it is the version of the receiving rsync that affects
these behaviors.)
-E, --executability
This option causes rsync to preserve the executability (or non-executability) of regular files when --perms is not enabled. A regular file is considered to be executable if at least one
’x’ is turned on in its permissions. When an existing destination file’s executability differs from that of the corresponding source file, rsync modifies the destination file’s permis‐
sions as follows:
o To make a file non-executable, rsync turns off all its ’x’ permissions.
o To make a file executable, rsync turns on each ’x’ permission that has a corresponding ’r’ permission enabled.
If --perms is enabled, this option is ignored.
-o, --owner
This option causes rsync to set the owner of the destination file to be the same as the source file, but only if the receiving rsync is being run as the super-user (see also the --super
and --fake-super options). Without this option, the owner of new and/or transferred files are set to the invoking user on the receiving side.
The preservation of ownership will associate matching names by default, but may fall back to using the ID number in some circumstances (see also the --numeric-ids option for a full dis‐
cussion).
-g, --group
This option causes rsync to set the group of the destination file to be the same as the source file. If the receiving program is not running as the super-user (or if --no-super was spec‐
ified), only groups that the invoking user on the receiving side is a member of will be preserved. Without this option, the group is set to the default group of the invoking user on the
receiving side.
The preservation of group information will associate matching names by default, but may fall back to using the ID number in some circumstances (see also the --numeric-ids option for a
full discussion).
--devices
This option causes rsync to transfer character and block device files to the remote system to recreate these devices. This option has no effect if the receiving rsync is not run as the
super-user (see also the --super and --fake-super options).
--specials
This option causes rsync to transfer special files such as named sockets and fifos.
-t, --times
This tells rsync to transfer modification times along with the files and update them on the remote system. Note that if this option is not used, the optimization that excludes files that
have not been modified cannot be effective; in other words, a missing -t or -a will cause the next transfer to behave as if it used -I, causing all files to be updated (though rsync’s
delta-transfer algorithm will make the update fairly efficient if the files haven’t actually changed, you’re much better off using -t).
-O, --omit-dir-times
This tells rsync to omit directories when it is preserving modification times (see --times). If NFS is sharing the directories on the receiving side, it is a good idea to use -O. This
option is inferred if you use --backup without --backup-dir.
This option also has the side-effect of avoiding early creation of directories in incremental recursion copies. The default --inc-recursive copying normally does an early-create pass of
all the sub-directories in a parent directory in order for it to be able to then set the modify time of the parent directory right away (without having to delay that until a bunch of re‐
cursive copying has finished). This early-create idiom is not necessary if directory modify times are not being preserved, so it is skipped. Since early-create directories don’t have
accurate mode, mtime, or ownership, the use of this option can help when someone wants to avoid these partially-finished directories.