Signiant Support

13.3 Agility Integration User's Guide Print



Agility Integration

The Agility component enables a Signiant workflow to submit files to an Agility system for transcoding. The component then monitors the transcode operation, reporting the results. The Signiant agent on which the Agility component is executed does not have to be installed on the Transcode Manager, but must be able to communicate with it.

The Agility sample workflow allows users to employ Signiant transfer technology with the Agility Transcode Manager to transfer files from a Signiant agent to a second Signiant agent that is located close to an Agility Transcode Manager. The Agility component allows the user to specify an Agility job to be executed when the file transfer has completed. The resulting transcoded files will then be transferred from the residing Signiant agent to another user-specified Signiant agent.


Agility Configuration

In order to integrate with the Signiant-Agility integration system, ensure that Agility is installed, licensed, and in working order for integration with Signiant.

Apply the following configuration:

  • Ensure that the Agility SOAP Server is installed on the Telestream Agility Server. The SOAP server installation is a totally separate install/product that is available only from Anystream.
  • Edit the AgilitySoap Web site's web site properties, and ensure that the ASP.net setting lists ASP.net version 2.0.50727 (or above).
  • Currently only profiles that were installed by the Telestream Agility software (during its setup) are visible to the SOAP API. In order to "publish" user-created job profiles, you must ensure that the Profile location specifies the use of the Agility Webinterface folder ("webinterfaces\defaultproperties"). This is set in Agility Preferences>Profiles Location field.


Agility Components

The AgilityDropBox component specifies the parameters relating to scheduling the workflow, such as when and how the job should run. The SigniantAggregate, Agility and SigniantDistribute components are grouped together as part of the execution sequence that transfers files from one or more Signiant agents to the Agility Transcode Manager, then transfers the transcoded files to one or more Signiant Agents.

The DropBoxJobReport component specifies information relating to a job report about the Agility job, including recipients, email subject line, number of report versions to maintain, and so on. An exception hierarchy allows you to specify conditions under which the report is sent (for example, if one of the components fails, the report may go to different recipients than if the entire job succeeds).

The following describe the input and output properties of the Agility component:

Inputs

  • Target Agent

    The Signiant agent on which the component is to be executed.

  • Source Data

    List of specific files to be transcoded.

  • Server

    Server

  • UNC Path Mapping

    The Agility Agent can be on the Agility server or it can reside elsewhere. If located elsewhere, it may be on a non-Windows agent and you must specify a UNC path map. The UNC path map should be the Windows equivalent of the Agility agent's "Input Directory". For example, if you have a Mac Agility agent with an input dir = /mnt/silon/agility/Media, the UNC path map should be something like \\silon\ifs\agility\Media These two paths point to the same location. Since the Agility server is Windows-based, it needs the Windows UNC path to access the files.

  • Job Profile

    Specifies a predefined job profile that will be used to determine how to transcode the specified media files. The Agility Job Profile MUST follow the format of <job profile name>.job.awp. For example, myAgilityjob.job.awp.

  • Title

    Specifies a title for the transcoded file(s). This metadata is applied to every file the job transcodes.

  • Author

    Specifies an author for the transcoded file(s). This metadata is applied to every file the job transcodes.

  • Keywords

    Specifies keywords to associate with the transcoded file(s). This metadata is applied to every file the job transcodes.

  • Description

    Specifies a description for the transcoded file(s). This metadata is applied to every file the job transcodes.

  • Copyright

    Specifies a copyright notice for the transcoded file(s). This metadata is applied to every file the job transcodes.
  • Rating

    Specifies a viewer classification for the transcoded file(s). Choose the following from the drop-down list:

    • None
    • General - All Ages
    • Parental Guidance Recommended
    • Adult Supervision Required
    • Adults Only

    This metadata is applied to every file the job transcodes.

  • Exit Transcode on First Failure

    If the transcode fails and this option is set to 0, the output list of will be empty.

  • Fail on No Files to Transcode

    If the input source file list is empty and this option is set to 0, the return code for the job will be 0 (success).

  • Timeout

    Time in seconds after which the component will exit in error.

Outputs

  • Transferred File List

    The list of files which were successfully transcoded.

  • Return Code

    Set to zero if the component completes successfully. Set to non-zero if any errors occurred.

  • Errored File List

    The list of files which that failed transcoding.

  • Summary Stats

    Summary of statistics from the transfer in an internal format. Not used by any other component.

  • Detail Stats

    Detailed statistics from the transfer in an internal format. Not used by any other component.

  • Return Message

    A short message indicating the overall result of the component.


Agility Prompts

The location of the transcoded files must be accessible to the Agility Agent as well as the Destination agent (if a distinct Destination agent is used) in order for the encoded files to be transferred to their ultimate destination. To run the Agility sample workflow, either select the published menu item or select and create a job from the job template library.

The following table describes the prompts associated when creating an Agility job. Mandatory prompts are indicated with an asterisk.

Prompt Description
Job Name

A unique name for this job. This must start with a letter and can only contain underscores to separate words. This name should help operators to recognize the functions of this job (e.g., Send_Media_to_Site2).

Transfer 1: To Agility Server
Source Agent/Drop Box Agent

The agent(s) from which you are transferring the data.

Source Directory

Folder from which to transfer files. Type the folder in the prompt, or click the computer icon to browse and select an agent folder. The Source folder specified may be in the following format:

  • Windows root drives, e.g., C:\ApplicationData
  • Unix root drives, e.g., /home
  • Windows UNC names, e.g., \\machine1\applicationdata

If these exact paths are not visible to the Agility Server, use the UNC Path Mapping input. Make sure that the usual permissions and sharing set up exist with UNC paths.

Include Patterns

Allows users to specify which files are transferred by filtering on the names to include. Use a comma to separate multiple filters. By default, an asterisk appears in the prompt, to include all file names/types. For example, if you type *.doc, *.ppt, in the prompt, the transfer wills include files with these extensions.

Exclude File Patterns

Allows users to specify which files are excluded from transfer. Use a comma to separate multiple filters. If the source directory for this jobs is also being used as the target directory for another job, you should add #work_file#* and #chkpt_file#*.

Exclude Subdirectories

To exclude sub-folders, choose from the following options:

  • None (all sub-folders will be transferred - none will be excluded)
  • All (no sub-folders will be transferred - all of them will be excluded)
  • Specified (a list of sub-folders to exclude - type sub-folders in the Subdirectories to Exclude prompt, described below)
Subdirectories to Exclude

Sub-folders may be excluded by specifying them in this prompt. Multiple entries must be separated with a comma. When the job runs, all folders that match those specified in the Subdirectories to exclude prompt will be excluded. Note that normal behavior is to exclude sub-folders that match regardless of where they appear in the folder path. Using the anchoring expression (@) changes this behavior to anchor the exclude folder path.

For example, if you specify a source folder of C:\data\docs, and an exclude folder of temp, any sub-folders called temp will be excluded, even those nested within another sub-folder. (For example the sub-folder C:\data\docs\publish\release\temp would be excluded as well.)

If you who want to exclude a folder only at a certain level can use the @ symbol to anchor the exclude folder path at the starting source folder level. For example, specifying @temp in the above example would mean that the C:\data\docs\temp folder would be excluded, but the C:\data\docs\publish\release\temp folder would be included.

If you instead wanted to make sure just the C:\data\docs\publish\release\temp folder was excluded, but wanted to have a source folder path of c:\data\docs, the user would need to type C:\data\docs in the source folder prompt and @publish\release\temp in the exclude prompt. Special characters allow you to make use of pattern matching on the folder path. You must escape special characters with a backslash to be matched literally. Characters include the following:

  • * (matches zero or more characters)
  • ? (matches any single character)
  • [...] (matches any one of the enclosed characters - for example, [ch] would match the characters "c" or "h")
  • A pair of digit, lowercase or uppercase characters separated by a hyphen '-' denotes a range set and matches any character sorted in the range. If the first character following the '[' is '^' or '!', then any character not enclosed is matched.
  • Use commas to specify multiple distinct patterns.

 

File Readiness Check Type

Specify the kind of check type an agent uses to determine if a file is ready to be transferred (i.e., the file is not in use). For some configurations, you may or may not wish to select the file readiness check option:

  1. For Windows agents and local Windows file systems, there is no need to use the file readiness check, as the core file in use - behavior is sufficient.
  2. For Windows agent and NAS Windows file system, as long as the NAS file system properly supports file locking for in-use files, the readiness check is unnecessary. However, it is still available in case the NAS file system does not properly support file locking.
  3. For non-Windows agents of any kind (Linux, UNIX, MacOS). If the local or network file system supports file locking, the readiness check is not required. Determine this on a case-by-case basis.

Choose one of the following:

  • None - Files that are in use are skipped by default.
  • Consecutive Seek (recommended) - Reads the last few bytes at the end of the file, waits for delay period number of seconds and checks again. If the last few bytes match in both cases the file is sent. Very useful on large files where the MD5 option would simply take too long. 
  • MD5 Hash - Use an MD5 hash to compute a hash on all files under the drop-box folder. After the wait time specified in the Readiness Check Delay prompt, the hashes are recomputed. Only files with identical hashes are transferred. Note that job initialization may take a great deal of time if large files are present.
  • Date/Time & Size - Use a difference in date/time or size on the file to compare its readiness. Only files that have identical date/time & size are transferred.
Readiness Check Delay

The amount of time (in seconds) to pause between the two phases of the specified file readiness check type ("MD5 Hash", "Consecutive Seek", or "Date/Time & Size") for an open file. The file is transferred only if the results of the two checks are identical. The default value is 10 seconds. If a large number of files are being checked, a value of 0 seconds may be sufficient, since there will likely be sufficient time between the first and second checks of a given file to determine if its content has changed without inserting an additional delay.

Delivery Mode

Specifies the mode to transfer files:

  • Fast - Do not create temporary work files. This mode is optimized for speed.
  • Normal - Use temporary work files during transfer. Temporary files allow an interrupted transfer to resume from where the interruption occurred. The creation of temporary files adds time to the transfer, but provides resiliency.
  • Log File Name - Log file names transferred. Logs are stored on the Manager.
  • Certify File Delivery - Create agent certified delivery log of files transferred.
Source Deletions After Successful Transfer

Specifies removal of files or folders from the dropbox folder after a successful transfer: Choose from one of the following:

  • None (do not delete any source files or empty folders)
  • Transferred Files (remove the files)
  • Empty folders (remove folders that are left over after the source files have been removed)
  • Transferred Files and Empty Directories (remove both transferred files and empty folders)
  • Transferred Files and Directories ( remove both transferred files and all folders immediately after transfer)

There are certain requirements with the empty folders options: the folders must contain no files (they may contain other empty folders that will be removed as well). The folder cannot have been modified within the previous 5 minutes; if it has, it will not be removed.

Source Directories to Exclude from Deletion Scan

Allows you to specify a comma-separated list of directories that you do not want to delete.

Agility Transcode Specification
Agility Agent

Defines the Signiant agent associated with the Agility server. The agent will receive the files to be transcoded. The Agility agent can be on the Agility server or it can reside elsewhere. If elsewhere, the specified remote agent must be a Windows agent, and the default user for that agent must be someone other than NTAuthority\System. The Agility manager requires UNC paths for its input/output directories, and the NT Authority\System cannot access UNC paths (see below).

Server

Specifies the server where the Agility manager is running. This may be different from the agent specified in the Agility Agent field.

*Input Directory

The Agility agent (Signiant agent associated with the Agility server) uses this directory for delivering the input files to a location accessible to the Agility server. This directory must be specified, and accessible by Signiant Agents. If the Agility Agent and Agility server are co-located on the same machine, you can specify just the input directory as an absolute path. If the Agility Agent and Agility server are not co-located, you must specify the UNC Path Map (see below).

UNC Path Map 

Use when the agent that is being used is not co-resident with the Agility Server, and the agent is a non-Windows Agent. The Agility Server SOAP API is not able to translate Linux mount points (or even non-local Windows mount points) to a viable path that the Agility Server can understand. The end result is that the Agility server does not have access to the source file paths (e.g., target of the files from the previous component was an NFS path). If the Agility server can see this directory, but from a UNC path, then this is where the UNC Path Mapping input is used. The base file names from the Source Data input are used with a prefix of the UNC Path Mapping input to give the Agility server an absolute path where it can find each file for transcoding.

*Job Profile

Specifies a predefined job profile that will be used to determine how to transcode the specified media files. The Agility Job Profile MUST follow the format of <job profile name>.job.awp. For example, myAgilityjob.job.awp.

Title

Specifies a title for the transcoded file(s). This metadata is applied to every file the job transcodes.

Author

Specifies an author for the transcoded file(s). This metadata is applied to every file the job transcodes.

Keywords

Specifies keywords to associate with the transcoded file(s). This metadata is applied to every file the job transcodes.

Description

Specifies a description for the transcoded file(s). This metadata is applied to every file the job transcodes.

Copyright

Specifies a copyright notice for the transcoded file(s). This metadata is applied to every file the job transcodes.

Rating

Specifies a rating for the transcoded file(s). Choose the following from the drop-down list:

  • None
  • General - All Ages
  • Parental Guidance Recommended
  • Adult Supervision Required
  • Adults Only

This metadata is applied to every file the job transcodes.

Transfer 2: From Agility Server
Destination Agent

The agent to which you want to transfer the files.

Destination Directory 

Folder to which to transfer files. Type the folder in the prompt, or click the computer icon to browse and select an agent. Source folder specified may be in the following format:

  • Windows root drives, e.g., C:\ApplicationData
  • Unix root drives, e.g., /home
  • Windows UNC names, e.g., \\machine1\applicationdata folder.

Make sure that the usual permissions and sharing set up exist with UNC paths. Note that the source of these files is specified on the Anystream Agility Encoding Platform. For more information, see the note at the start of this procedure.

Include File Patterns

Allows users to specify which files are transferred by filtering on the names to include. Use a comma to separate multiple filters. By default, an asterisk appears in the prompt, to include all file names/types. For example, if you type *.doc, *.ppt, in the prompt, the transfer wills include files with these extensions.

Exclude File Patterns

Allows users to specify which files are excluded from transfer. Use a comma to separate multiple filters. If the source directory for this jobs is also being used as the target directory for another job, you should add #work_file#* and #chkpt_file#*.

Exclude Subdirectories

To exclude sub-folders, choose from the following options:

  • None (all sub-folders will be transferred - none will be excluded)
  • All (no sub-folders will be transferred - all of them will be excluded)
  • Specified (a list of sub-folders to exclude - type sub-folders in the Subdirectories to Exclude prompt, described below)
Subdirectories to Exclude

Sub-folders may be excluded by specifying them in this prompt. Multiple entries must be separated with a comma. When the job runs, all folders that match those specified in the Subdirectories to exclude prompt will be excluded. Note that normal behavior is to exclude sub-folders that match regardless of where they appear in the folder path. Using the anchoring expression (@) changes this behavior to anchor the exclude folder path.

For example, if you specify a source folder of C:\data\docs, and an exclude folder of temp, any sub-folders called temp will be excluded, even those nested within another sub-folder. (For example the sub-folder C:\data\docs\publish\release\temp would be excluded as well.)

If you who want to exclude a folder only at a certain level can use the @ symbol to anchor the exclude folder path at the starting source folder level. For example, specifying @temp in the above example would mean that the C:\data\docs\temp folder would be excluded, but the C:\data\docs\publish\release\temp folder would be included.

If you instead wanted to make sure just the C:\data\docs\publish\release\temp folder was excluded, but wanted to have a source folder path of c:\data\docs, the user would need to type C:\data\docs in the source folder prompt and @publish\release\temp in the exclude prompt. Special characters allow you to make use of pattern matching on the folder path. You must escape special characters with a backslash to be matched literally. Characters include the following:

  • * (matches zero or more characters)
  • ? (matches any single character)
  • [...] (matches any one of the enclosed characters - for example, [ch] would match the characters "c" or "h")
  • A pair of digit, lowercase or uppercase characters separated by a hyphen '-' denotes a range set and matches any character sorted in the range. If the first character following the '[' is '^' or '!', then any character not enclosed is matched.
  • Use commas to specify multiple distinct patterns.
File Readiness Check Type

Specify the kind of check type an agent uses to determine if a file is ready to be transferred (i.e., the file is not in use). For some configurations, you may or may not wish to select the file readiness check option:

  1. For Windows agents and local Windows file systems, there is no need to use the file readiness check, as the core file in use - behavior is sufficient.
  2. For Windows agent and NAS Windows file system, as long as the NAS file system properly supports file locking for in-use files, the readiness check is unnecessary. However, it is still available in case the NAS file system does not properly support file locking.
  3. For non-Windows agents of any kind (Linux, UNIX, MacOS). If the local or network file system supports file locking, the readiness check is not required. Determine this on a case-by-case basis.

Choose one of the following:

  • None - Files that are in use are skipped by default.
  • Consecutive Seek (recommended) - Reads the last few bytes at the end of the file, waits for delay period number of seconds and checks again. If the last few bytes match in both cases the file is sent. Very useful on large files where the MD5 option would simply take too long. 
  • MD5 Hash - Use an MD5 hash to compute a hash on all files under the drop-box folder. After the wait time specified in the Readiness Check Delay prompt, the hashes are recomputed. Only files with identical hashes are transferred. Note that job initialization may take a great deal of time if large files are present.
  • Date/Time & Size - Use a difference in date/time or size on the file to compare its readiness. Only files that have identical date/time & size are transferred.
Readiness Check Delay

The amount of time (in seconds) to pause between the two phases of the specified file readiness check type ("MD5 Hash", "Consecutive Seek", or "Date/Time & Size") for an open file. The file is transferred only if the results of the two checks are identical. The default value is 10 seconds. If a large number of files are being checked, a value of 0 seconds may be sufficient, since there will likely be sufficient time between the first and second checks of a given file to determine if its content has changed without inserting an additional delay.

Delivery Mode

Specifies the mode to transfer files:

  • Fast - Do not create temporary work files. This mode is optimized for speed.
  • Normal - Use temporary work files during transfer. Temporary files allow an interrupted transfer to resume from where the interruption occurred. The creation of temporary files adds time to the transfer, but provides resiliency.
  • Log File Name - Log file names transferred. Logs are stored on the Manager.
  • Certify File Delivery - Create agent certified delivery log of files transferred.
Source Deletions After Successful Transfer

Specifies removal of files or folders from the dropbox folder after a successful transfer. Choose from one of the following:

  • None (do not delete any source files or empty folders)
  • Transferred Files (remove the files)
  • Empty folders (remove folders that are left over after the source files have been removed)
  • Transferred Files and Empty Directories (remove both transferred files and empty folders)
  • Transferred Files and Directories ( remove both transferred files and all folders immediately after transfer)

There are certain requirements with the empty folders options: the folders must contain no files (they may contain other empty folders that will be removed as well). The folder cannot have been modified within the previous 5 minutes; if it has, it will not be removed.

Source Directories to Exclude from Deletion Scan

Allows you to specify a comma-separated list of directories that you do not want to delete.

Job Schedule
Start Date/Time

The date and time at which you want the job to run.

Note that editing the start date/time of a job that is currently in a queued state within a queued job group will have no effect on the job until the next time the job is run.

Frequency

Specifies how often the job runs. Users can choose from the following options:

  • None - Use for jobs that run only at irregular, user-defined times.
  • Hourly, Daily, Weekly, Monthly, Yearly - Run the job once every selected interval.
  • Monthend - Run the job once every last day of the month.
  • Once - Run the job only once, when first created.

Note that editing the frequency of a job that is currently in a queued state within a queued job group will have no effect on the job until the next time the job is run.

Time Zone

Specifies the time zone in which the displayed times are set. For example, if an Eastern Time zone specifies a start time of 9:00 am and a time zone of PST, the job runs at 9:00 am Pacific Standard Time or noon Eastern Standard Time.

Job Options
Log Detail Level

The type of logging information for this job. Choose from Error, Warn (warning), Info (Information) or Debug. Debug provides the greatest level of detail while Error provides the least.

Notification
Email Job Report

Specifies when a job report is emailed to users specified in the Email Job Report To prompt. Choose from the following:

  • Always - Users receive a job report after every job run.
  • On Error - Users receive a job report after a job run that reports an error exit code.
  • Never - Users never receive job reports.
Email Job report To

The email address of the person who you want to receive the logging report. Separate multiple addresses with a comma.

Email Subject

Text that appears in the subject prompt of the report email message.