Output Management

Standard behavior (files outputs)

Each application must define its outputs in the boutiques file, in the "outputs" section. For each output, the path-template property defines the filename of the output, this must be the exact filename the application will produce. In most case it will be a single filename (no "/"), but sub-directories are supported (see dedicated section). The value-key allows to define a variable internally in the boutiques file to mention the output filename in the command-line (for instance to give the target filename as a parameter of the application).

Output name good practice

When an app is suited to run multiple jobs within a single VIP execution workflow, for instance through a combination of input parameters with multiple values, special attention should be paid to the naming of output files. Assuming each job produces one output file, then the output file name must be constructed in "path-template" in a such way that:

• each job produces a unique output filename for a given combination of inputs. If two different jobs within a single execution workflow have the same output filename, then the second job that runs will overwrite the result of the first job, which is generally not the desired behavior.
• the total number of jobs that get executed must exactly match the number of output filename combinations. It typically means that it's not possible to give full control of the output filename to end users : settings such as "path-template": "[OUTNAME]" should then be avoided, in favor of some more "mechanical" naming.

One way to achieve this is to build "path-template" as a combination of all input parameters that can take multiple independent values within a single execution workflow. The base name of an input file can be reused as a part of the output name, with some extra prefix and/or extensions stripping. For instance:

    "path-template": "output_[INPUT_FILE]_[PARAM1]_[PARAM2].out",
    "path-template-stripped-extensions": [".in"],

Results location

Each execution takes a special parameter that determines where the results will be written. It will be required in the portal ("results directory" parameter) or through the API ("resultsLocation" field).

By default, VIP will create a timestamp-based subdirectory where outputs will be stored. This can be customized through the "vip:resultsDirectorySuffix" custom boutiques property (see Advanced boutiques descriptors).

In most cases, an execution will take a single "results location" value and all the results produced will be copied in it. However, if an application has a "dot" combination (see Advanced boutiques descriptors), it is possible to give several "results location" values, exactly the same number as the number of tuples of the dot inputs.

Files in sub-directory support

The "path-template" parameter also supports uploading outputs from (and to) sub-directories :

    "path-template":"subdir/res_[NAME].txt"

This will also put the file in a sub-directory with the same name on the desired results location selected. The target sub-directory is created if is does not exist.

Directory support

Using "path-template", it is also possible to select folders as "output-files", in which case all of the files under this folder (including sub-folders) will be uploaded. In order to do this, the "path-template" parameter must be specified with the name of the folder to upload as value :

    "path-template":"folderName"

As of now this functionality only works for local and girder executions. Be aware that if the folder or files with the same name already exist at the chosen path, VIP will fail and return an error with local execution.