File Transfer

The file transfer task moves a set of files or directories from a single computer to a set of destination computers.

Source Asset

The source asset is the computer you want to transfer files from.

Concurrency

Concurrency controls how many transfer operations may be ongoing at any one time. By default the "Limit Concurrency to:" checkbox is not checked, meaning send to all destinations at the same time. If this checkbox is checked, then the number of simultaneous transfers is limited to the concurrency value.

Destination Assets

The destination assets table lists the destinations of the file transfer. Because some file systems are not compatible, each asset listed must be of the same kind (UNIX computers or Windows computers).

Files

File transfer can operate two different ways. Rsync mode copies only files which have modified. Non-Rsync mode copies all the files specified using the transaction model described below.

Transaction Model (Non-Rsync Mode)

Each individual entry in the "files" table is a transaction. When executed, Situate will move all the data for each transaction into temporary locations within the same file system as the destination directory. Once all the data for each transaction has been copied, Situate moves the data into its intended destination.

Adding, Removing or Editing Components

The standard set of "+", "-" or "pencil" icons are used to add, remove or edit the list of file components.

The File Transfer Component Dialog

The file transfer component dialog allows you to setup a component of the overall file transfer transaction.

Source

The file or directory to move.

Destination

The destination must be a directory into which the source will be moved.

Copy Changes Only (Rsync Mode)

When this option is used, only new files or files that have changed are moved. If the "Remove files on destination that do not exist on source" is checked, then files existing on the destination that are not found on the source are removed.

Since this kind of operation is designed to keep two directories in sync, this kind of operation does not use transactions (as described above).

Rsync Local Indexes

When Rsync mode is enabled, Situate must index the destination in order to determine what has changed since the last file transfer. The constant indexing can create problems or speed bottlenecks in a workflow because Situate must check each time, no matter the number of changes. Local Indexing can rectify thissue. Local Indexing allows Situate to "remember" it's previous indexing, allowing Situate to skip the indexing process. When "Use Index" is checked, Situate reads the index from a file already on the source, eliminating the need to perform the remote indexing operation. When "Use Index" is checked the following will occur... The first file transfer with "Use Index" will index as normal, creating an index file at the destination. For every File Transfer after that, Situate will read the index file at the destination.

The Dangers of Local Indexes

When Rsync Local Indexes are used, Situate does not index the destination, so any other changes since the index file was created will not be accounted for. This can be effectively combatted by having the index file deleted periodically, allowing Situate to only have to index, say, once a week, while keeping the index file relatively up to date.

Source Mode

The checkbox "When source is a directory, omit source directory name" controls Source Mode copying.

Let's assume you are copying "/x/a" to "/y/b", and that "/x/a" is a directory. Let's also assume that f1, f2, and d1 are files (f1, f2) and a directory (d1) contained within "/x/a".

With source mode off (unchecked), the "a" will be copied into /y/b so after the copy, you will have /y/b/a containing f1, f2, and d1.

With source mode on (checked), the source directory is omitted on the destination. The result will be /y/b containing f1, f2, and d1.

Rename

The rename operation may be used to rename the source file or directory as it is being moved. For example, you can move a file named backup.zip to backup.xyzserver.zip or product-installer.exe to product-installer-2.2.0.exe.

Preserving Permissions

If checked, the "Preserve Permissions" checkbox will cause the ownership, group ownership, security information and modification times to be preserved as the file is copied. This can only be done when the calling user has Administrator or root privileges on both the source and the destination asset.

Create Destination Directory

When selected, this option cause the destination directory to be created if it does not already exist. On UNIX systems, the security on new component directories is set to 755. On windows, the system default is used.

Template Files

When "process file as template", is checked, Situate treats the file as a template, expanding variables and macros as it's copied. This useful when copying a control or configuration file that needs to be edited to add environment-specific data. For more information on templates, see Expanding With Templates.

Move Files

When the "Remove the source file after a successful copy (Move)" checkbox is selected, File Transfer will remove the original file or directory copied, essentially moving the file or directory

Backups During File Transfer

By default, Situate will create a backup of the original files and directories during each copy. For example, if your source is /a/b/c and your destination is /x/y, then assuming /x/y/c already exists on the destination, Situate will create a file (or directory) called /x/y/.c.old where the old "c" is moved. This feature allows quick and easy restoration of files after a file transfer.

Keeping a second copy can waste disk space. If you uncheck the "Keep a backup copy on the destination", Situate will not keep a backup copy.

Ignoring Errors

When the "Ignore errors that might be caused by files/directories changing during the copy operation" checkbox is selected, Situate will ignore errors caused by files being modified, added, or deleted during the copy operation. This may be useful when backing up files in Rsync mode.

Special Directories

The drop down boxes to the left of the source and destination path allow you to select a "special" directory.

For UNIX computers, you may select ${HOME}, meaning the home directory of the user the transfer runs as.

For Windows, there is a list of well-known windows locations that are specific to the user the transfer runs as.

Link Options

Copy Links

When the "Copy Links" checkbox is unchecked, links are ignored during the copy operation.

Links Pointing Outside the Tree

When the "Ignore links that point outside the "Tree" being copied" box is checked, links that point to files or directories outside the set of files and directories being copied are ignored.

Links that Point Outside the File System

When the "Ignore links that point to files on other file systems" box is checked, links that point to files or directories outside the current file system or volume are ignored.

Network Options

Compression

Compression is useful when sending files over a slow network. Compression levels 1 through 9 are supported. These correspond to the same levels as used by gzip.

By default, a compression level of 3 is used. This level is optimal for local area file transfer. Additionally, Situate is able to detect files that are already compressed (such as .zip, .mp3, .png, etc.). Situate will disable compression for these files as they are already in an optimal transmission format.

Bandwidth Limit

When the limit bandwidth box is checked, the bandwidth used by the file transfer operation will be limited to the value specified.

Filters

Filters allow you to control how certain files and/or directories are handled by the file transfer. Each filter has a "type" and "pattern". The type controls who the filter behaves and the pattern is used to describe certain files and/or directories.

Filters can be applied to items by first clicking the "add or +" button to add a filter. Then, you may apply filter patterns. Items can be rearranged by selecting the item you would like to move and pressing the up and down arrows opposite the "add or + "minus" and "edit" buttons.

Include and Exclude

Filters are processed in order. Each file or directory transferred is tested against the filters. If the particular file first matches an include filter, the file/directory is transferred. If a file/directory first matches an "exclude" filter, the file/directory is not transferred. If the end of the filter list is reached and there are no matches, the file is transferred.

Protect

When using the rsync mode copy, it is possible to check the "Remove files from destination not found on source. Files/directories matching the protect filters are NOT removed from the destination.

Patterns

Patterns follow the following rules:

- If a pattern starts with a leading "/", the pattern is matched from the *beginning* of the *path*.  If there is no leading slash, then the pattern is matched against the *end* of the *path*.

The path being copied is the full pathname minus the "source" of the file transfer. For example, if you specify a source of /home/mike, then the file transfer might copy "/home/mike/situate/packages". In this case, the path is "/situate/packages".

Therefore, if you want to exclude the "packages" directory, the pattern would be "/situate/packages/". Note: the leading slash causes the path to be matched from the beginning.

If you want to exclude all files ending in .shar from any directory, you would write ".*.shar". Note: the lack of a leading slash causes the end of the path to be matched.

- If a pattern ends in a trailing "/", then it only matches directories.  If a pattern does not end in a "/", then only files are matched.   If a pattern ends in "/***", then both files and directories are matched.

For example, if you want to match the situate/packages directory, then your pattern must end in a trailing "/". If you want to match all text files, you would use ".**.txt" with no trailing slash.

- Two asterisks between slashes ("/**/") means one or more sub-directories.  For example, "/situate/**/.*\.class" would match situate/a/foo.class, situate/a/b/foo.class but not situate/foo.class".  Note: ** must match at least ONE sub-directory.

- Except for the rules above, regular expressions are expected between "/" characters.  (Note: rsync does not use regular expressions).  So, for example, to match files ending in ".so", one would write ".*\.so".

Situate uses standard GNU regular expressions. For information on GNU regular expressions, see https://www.gnu.org/software/sed/manual/html_node/Regular-Expressions.html

Variables as Assets

Variables can be used (for example ${srcAsset}) instead of actual asset names for both the source and destination of a file transfer operation. When this done, the Situate UI cannot automatically know if the source files or directories and/or the destination directories should use UNIX or Windows file name semantics.

In this case, the UI will present a drop-down box to the right of the source and/or destination allowing you to specify the semantics that should be used to input the desired files and or directories.