add-job

Add a new job to an existing workflow.

Usage: add-job -workflow W -project P -track T [options] [command [args...]]
----------------------------------------------------------------------------
Add a job to workflow W, specifying that <command [args...]> should be run on
track T of the batch system, with time accounted to project <P>.  If your
process cannot be launched via a simple parameterized command-line you can
use the -script option to specify a script file to be executed by your shell.
Note that, as with commands like ssh, the <command [args...]> that you specify
will be executed by your shell.  To prevent special characters from being
interpreted while invoking add-job, you must escape them.  For example, use
single-quotes in bash.

Create: -create
---------------
If the named workflow does not already exit, create it.

Name: -name
-----------
Specify a name for the job.  Default is "wf-n", where wf is the workflow
name and n is the ordinal number of the job.

CPU: -cores core-count
----------------------
How many CPU cores the process will use.

Disk: -disk integer[units]
--------------------------
Maximum bytes or kilo-, mega-, giga-, terabytes your process will write
to local disk.  You can append any portion of the byte units to the end of
the integer, otherwise 'bytes' is assumed.  Examples 2g, 1000kilobyes, 3TB.

Input: -input local-file remote-uri
-----------------------------------
Process will access the data at <remote-uri> via <local-file>.  The local
file name is always relative to the job's execution directory.  To copy a file
from the tape library, specify your remote-uri as mss:<stub-path>.  Or to
copy from an accessible filesystem specify file:<path>.  At present these are
the only two scchemes supported.  Future versions will likely add more.

Output: -output local-file remote-uri
-------------------------------------
Like -input, but copying from local-file to remote-uri.

Phase: -phase phase-number
--------------------------
Every job may be assigned an integer phase number that indicates what segment
of the workflow it belongs to.  No jobs belonging to a higher phase will be
released to the batch system until all jobs belonging to a lower phase have
completed or optionally canceled.  Once all jobs in a phase have completed,
the workflow will advance to the next phase number specified by constituent
jobs.

This can allow graduated introduction of a jobs into the system, or can
provide a crude way to implement a form of dependency between stages of a
workflow.

If unspecified, the job is assumed to be runnable at any time.

Project: -project project-name
------------------------------
Project to which the time should be accounted. For a list of allowed projects,
consult https://scicomp2015.jlab.org/scicomp/#/projectName.

RAM: -ram integer[units]
------------------------
The maximum amount of memory that your process will use.  Values specified as
described for -disk, above.

Script: -script file-name
--------------------------
If you do not specify a command as trailing arguments, you must specify a file
that contains a script to be interpreted by the shell spawned to run your job.
Note that the scipt itself will not be referenced by the job.  Rather, the
file will be uploaded to the swif database, and will be supplied to the job
when it runs.  If you want the script to be run via an explicit path, use the
trailing command and arguments option.  Note that this script will be supplied
as standard input to a non-interactive login shell.  See -shell below.

Shell: -shell shell-path
------------------------
The shell that will be spawned to execute the command you specify either via
trailing arguments or via -script, as described above.  Note that <shell-path>
must indicate the absolute path to your shell on the batch machine.  Most
likely this will be once of /bin/bash or /bin/tcsh.  Note that this shell will
be executed as a login shell by the batch system.  Consult the shell manual
for details about which dot-files will be sourced.

Standard Error: -stderr uri
---------------------------
Send anything generated on your shell's standard error to the specified URI.
At present only the file: scheme is supported.

Standard Output -stdout uri
---------------------------
Redirect standard output as described for -stderr, above.

Time: -time integer[units]
--------------------------
Maximum number of seconds the job will run.  You may append any portion of the
units: seconds, minutes, hours, days to the integer value, otherwise seconds
is assumed.  Example: 5min, 24h, 3days, 1000s.

Track: -track track-name
------------------------
The track to which the process will be assigned in the batch system.  For more
information, consult https://scicomp.jlab.org/docs/batch_job_tracks.

Tags: -tag tag-name tag-value
-----------------------------
You can associate a key/value pair with the job.  The tag-name must consist of
at most 100 alphanumeric characters, including '_', and must start with an
alphabetic character.  The value can contain regular ascii characters and is
limited to 200 characters in length.  You can specify multiple tags by using
this option more than once.

Antecedents: -antecedent job-name
---------------------------------
Specify a job that must complete successfully before this job is eligible to
run.  The named job must already exist within the same workflow.  To specify
multiple antecedents, repeat the flag for each.

Conditions: -condition uri
--------------------------
Specify a condition that must exist before the job is eligible to run.  A
condition is a URI that must resolve without error to be considered valid.
Supported URI schemes are 'http' and 'file'.  To specify multiple conditions,
repeat the flag for each.  Note that conditions are checked periodically and,
once found to be true, will no longer be checked.