Ep3: Extremely Pluggable Pipeline Processor

This is a workflow engine for the Common Workflow Language which aims to have a pluggable architecture.

Conformance tests for CWL v1.0 for the latest release

Classes

Required features

Optional features

Notes:

• ep3 will not pass the test #61 with required and command_line_tool tags due to common-workflow-language#761.
• The conformance badge for SoftwareRequirement is not available because there are no conformance tests for this feature.
• Currently ScatterFeatureRequirement (scatter tag) and StepInputExpressionRequirement (step_input tag) are not supported.
  • It affects the result of the tests of Workflow (workflow tag), InlineJavascriptRequirement (inline_javascript tag), MultipleInputFeatureRequirement (multiple_input tag) and SubworkflowFeatureRequirement (subworkflow tag).

Requirements

• medal
• bash
• ruby 2.5.1 or later
• jq
• nodejs for InlineJavascriptRequirement
• docker for DockerRequirement

How to install

• Install medal, bash, ruby, jq, nodejs and docker
• Execute the following commands and add /path/to/ep3 to $PATH.

$ git clone --recursive https://github.com/tom-tan/ep3.git

Usage

See ep3-runner --help for details.

$ ep3-runner <cwl> [job]

It prints the log and debug outputs to stderr and prints the output object to stdout. Both types of outputs are printed in JSON Lines format.

Here is an example:

$ ep3-runner --quiet /path/to/ep3/examples/workflow.cwl /path/to/ep3/examples/inputs.yml | jq .
{
  "output": {
    "class": "File",
    "location": "file:///current/directory/output",
    "path": "/current/directory/output",
    "basename": "output",
    "dirname": "/current/directory",
    "nameroot": "output",
    "nameext": "",
    "checksum": "sha1$c28e458d4e943c743b9b3c46fdab10688a6d68b6",
    "size": 687
  }
}

For developers

How to test

$ git clone --recursive https://github.com/tom-tan/ep3.git
$ cd ep3
$ cwltest --tool $PWD/ep3-runner --test test.yml
Test [1/1] Workflow example
All tests passed

ep3 internals

The ep3-runner command consists of the following internal commands:

• ep3 init
  • Generates Petri nets for medal that represents a given CWL workflow (including internal states in workflow engines such as staging processes)
• ep3 run
  • Executes a medal to run a workflow
• ep3 list
  • Shows the output object for execution result
• ep3 resume (Unimplemented)
• ep3 stop (Unimplemented)