Skip to content

WfCommons Project
EMPOWERING SCIENTIFIC WORKFLOW RESEARCH

        

WfCommons is a collaborative framework empowering scientific workflow research and development. Our tools analyze workflow execution instances, generate synthetic yet realistic workflows, define benchmark specifications, and pioneer new techniques to optimize execution on complex distributed infrastructures.

  • WfFormat: A universal JSON format, ensuring seamless integration across simulators and frameworks supported by WfCommons

  • WfInstances: Curated collection of open-access production workflow executions in the standardized WfFormat

  • WfChef: Automated creation of synthetic workflow generators by analyzing real workflows to uncover task patterns and performance characteristics

  • WfGen: Automated generation of realistic synthetic workflows using recipes to create diverse instances for simulation or benchmark generation

  • WfBench: Generate realistic workflow benchmarks with diverse performance characteristics and task dependencies for current workflow systems

  • WfSim: Facilitate simulation for developing and evaluating scheduling algorithms and computing platforms, overcoming real-world limitations

Citing WfCommons

When citing WfCommons, please use the following paper. You should also actually read that paper, as it provides a recent and general overview on the framework.

@article{wfcommons,
    title = {{WfCommons: A Framework for Enabling Scientific Workflow Research and Development}},
    author = {Coleman, Taina and Casanova, Henri and Pottier, Loic and Kaushik, Manav and Deelman, Ewa and Ferreira da Silva, Rafael},
    journal = {Future Generation Computer Systems},
    volume = {128},
    number = {},
    pages = {16--27},
    doi = {10.1016/j.future.2021.09.043},
    year = {2022},
}

Pinned Loading

  1. WfCommons Public

    WfCommons: A Framework for Enabling Scientific Workflow Research and Development

    Python 32 13

  2. WfFormat Public

    The WfCommons JSON Schema for workflow instances and generators

    Python 5 4

  3. WfInstances Public

    Collection and curation of open access production workflow executions from various scientific applications using WfFormat

    Python 3 1

Repositories

Showing 10 of 10 repositories
  • WfInstances Public

    Collection and curation of open access production workflow executions from various scientific applications using WfFormat

    Python 3 LGPL-3.0 1 0 0 Updated Apr 22, 2025
  • WfCommons Public

    WfCommons: A Framework for Enabling Scientific Workflow Research and Development

    Python 32 LGPL-3.0 13 3 1 Updated Apr 21, 2025
  • TypeScript 0 3 0 0 Updated Apr 18, 2025
  • wfcommons.github.io Public

    WfCommons website

    CSS 0 GPL-3.0 0 0 0 Updated Apr 15, 2025
  • WfFormat Public

    The WfCommons JSON Schema for workflow instances and generators

    Python 5 LGPL-3.0 4 0 0 Updated Oct 18, 2024
  • makeflow-instances Public archive

    Collection of workflow execution instances for the Makeflow workflow management system.

    2 GPL-3.0 0 0 0 Updated Jun 23, 2024
  • pegasus-instances Public archive

    Collection of workflow execution instances for the Pegasus workflow management system.

    18 4 0 0 Updated Jun 23, 2024
  • nextflow-instances Public archive

    Collection of workflow execution instances for the Nextflow workflow system.

    0 GPL-3.0 1 0 0 Updated Jun 23, 2024
  • .github Public
    0 0 0 0 Updated Jun 23, 2024
  • nextflow_workflow_tracer Public

    A tool to trace nextflow workflow executions and produce WfFormat workflow instances

    Python 0 0 0 0 Updated May 23, 2023