-
Notifications
You must be signed in to change notification settings - Fork 689
Update plugins documentation #6033
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
christopher-hakkaart
wants to merge
24
commits into
nextflow-io:master
Choose a base branch
from
christopher-hakkaart:docs-plugins
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 6 commits
Commits
Show all changes
24 commits
Select commit
Hold shift + click to select a range
c487aaa
Update plugins docs
christopher-hakkaart a046331
Add run command example
christopher-hakkaart 989af21
Update links
christopher-hakkaart 5471abc
Add repo link
christopher-hakkaart 3dfb945
Add Trace observers note
christopher-hakkaart e254b63
Revise defined registry
christopher-hakkaart 5a6e4d9
Add plugin create documentation
christopher-hakkaart 0a5f013
Merge branch 'master' into pr/christopher-hakkaart/6033
bentsherman 9053ca9
Fix plugin config scope example
bentsherman ce87361
Only support registry
christopher-hakkaart d68a8d7
Merge branch 'docs-plugins' of https://github.com/christopher-hakkaar…
christopher-hakkaart 2bb3960
cleanup
bentsherman e153d96
Remove registry URL
bentsherman 1be0336
cleanup migration guide
bentsherman 7bfc78d
cleanup gradle plugin guide
bentsherman 93691b8
Merge branch 'master' into docs-plugins
christopher-hakkaart fa2dc0f
Unlist h3 headings
christopher-hakkaart ea585de
Merge branch 'master' into pr/christopher-hakkaart/6033
bentsherman aa87a22
Replace nf-hello example with plugin template
bentsherman 758b818
Cleanup private beta notes
bentsherman e0866bc
Add transition plan for legacy index -> registry
bentsherman 5ab28f6
Merge branch 'master' into pr/christopher-hakkaart/6033
bentsherman de7aaa1
Apply suggestions from code review
bentsherman d011970
Unlist timeline subheadings
bentsherman File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file was deleted.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,138 @@ | ||
(gradle-plugin-page)= | ||
|
||
# Using the Gradle plugin | ||
|
||
The [Gradle plugin for Nextflow plugins](https://github.com/nextflow-io/nextflow-plugin-gradle) simplifies plugin development by configuring default dependencies needed for Nextflow integration and incorporates custom Gradle tasks that streamline building, testing, and publishing Nextflow plugins. This guide describes how to use the Gradle plugin for plugin development. | ||
|
||
:::{note} | ||
Nextflow Plugins can be developed without the Gradle plugin. However, this approach is only suggested if you are an advanced developer and your project is incompatible with the Gradle plugin. | ||
::: | ||
|
||
(gradle-plugin-create)= | ||
|
||
## Creating a plugin | ||
|
||
The [nf-hello](https://github.com/nextflow-io/nf-hello/tree/gradle-plugin-example) plugin uses the Gradle plugin and is a valuable starting point for developers. | ||
bentsherman marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
To create a Nextflow plugin with the Gradle plugin: | ||
|
||
1. Fork the [nf-hello](https://github.com/nextflow-io/nf-hello/tree/gradle-plugin-example) plugin. See {ref}`nf-hello-page` for more information. | ||
2. Rename the forked `nf-hello` directory with your plugin name. | ||
3. Replace the contents of `settings.gradle` with the following: | ||
|
||
```groovy | ||
rootProject.name = '<PLUGIN_NAME>' | ||
``` | ||
|
||
Replace `PLUGIN_NAME` with your plugin name. | ||
|
||
4. Replace the contents of `build.gradle` with the following: | ||
|
||
```groovy | ||
// Plugins | ||
plugins { | ||
id 'io.nextflow.nextflow-plugin' version '0.0.1-alpha' | ||
} | ||
|
||
// Dependencies (optional) | ||
dependencies { | ||
<DEPENDENCY> | ||
} | ||
|
||
// Plugin version | ||
version = '<PLUGIN_VERSION>' | ||
|
||
nextflowPlugin { | ||
// Minimum Nextflow version | ||
nextflowVersion = '<MINIMUM_NEXTFLOW_VERSION>' | ||
|
||
// Plugin metadata | ||
provider = '<PROVIDER>' | ||
className = '<CLASS_NAME>' | ||
extensionPoints = [ | ||
'<EXTENSION_POINT>' | ||
] | ||
|
||
publishing { | ||
github { | ||
repository = '<GITHUB_REPOSITORY>' | ||
userName = project.findProperty('github_username') | ||
authToken = project.findProperty('github_access_token') | ||
email = project.findProperty('github_commit_email') | ||
indexUrl = '<GITHUB_INDEX_URL>' | ||
} | ||
} | ||
bentsherman marked this conversation as resolved.
Show resolved
Hide resolved
|
||
} | ||
``` | ||
|
||
Replace the following: | ||
|
||
- `DEPENDENCY`: (optional) your plugins dependency libraries—for example, `commons-io:commons-io:2.18.0`. | ||
- `PLUGIN_VERSION:` your plugin version—for example, `0.5.0`. | ||
- `MINIMUM_NEXTFLOW_VERSION`: the minimum Nextflow version required to run your plugin—for example, `24.11.0-edge`. | ||
- `PROVIDER`: your name or organization—for example, `nextflow`. | ||
- `CLASS_NAME`: your plugin class name—for example, `nextflow.hello.HelloPlugin`. | ||
- `EXTENSION_POINT`: your extension point identifiers that the plugin will implement or expose—for example, `nextflow.hello.HelloFactory`. | ||
- `GITHUB_REPOSITORY`: your GitHub plugin repository name—for example, `nextflow-io/nf-hello`. | ||
- `GITHUB_INDEX_URL`: the URL of your fork of the plugins index repository—for example, [`plugins.json`](https://github.com/username/plugins/blob/main/plugins.json)</code>. | ||
5. Develop your plugin extension points. See {ref}`dev-plugins-extension` for descriptions and examples. | ||
6. In the plugin root directory, run `make assemble`. | ||
|
||
(gradle-plugin-install)= | ||
|
||
## Installing a plugin | ||
|
||
Plugins can be installed locally without being packaged, uploaded, and published. | ||
|
||
To install a plugin locally: | ||
|
||
1. In the plugin root directory, run `make install`. | ||
|
||
:::{note} | ||
Running `make install` will add your plugin to your `$HOME/.nextflow/plugins` directory. | ||
::: | ||
|
||
2. Configure your plugin. See {ref}`using-plugins-page` for more information. | ||
3. Run your pipeline: | ||
|
||
```bash | ||
nextflow run main.nf | ||
``` | ||
|
||
(gradle-plugin-unit-test)= | ||
|
||
## Unit testing a plugin | ||
|
||
Unit tests are small, focused tests designed to verify the behavior of individual plugin components and are an important part of software development. | ||
|
||
To run unit tests: | ||
|
||
1. Develop your unit tests. See [HelloDslTest.groovy](https://github.com/nextflow-io/nf-hello/blob/gradle-plugin-example/src/test/groovy/nextflow/hello/HelloDslTest.groovy) in the [nf-hello](https://github.com/nextflow-io/nf-hello/tree/gradle-plugin-example) plugin for unit test examples. | ||
2. In the plugin root directory, run `make test`. | ||
|
||
(gradle-plugin-package)= | ||
|
||
## Packaging, uploading, and publishing a plugin | ||
|
||
The Gradle plugin for Nextflow plugins simplifies publishing your plugin. | ||
|
||
To package, upload, and publish your plugin: | ||
|
||
1. Fork the [Nextfow plugins index repository](https://github.com/nextflow-io/plugins). | ||
2. In the plugin root directory, open `build.gradle` and ensure that: | ||
* `github.repository` matches the plugin repository. | ||
* `github.indexUrl` matches your fork of the plugins index repository. | ||
3. Create a file named `$HOME/.gradle/gradle.properties` and add the following: | ||
|
||
```bash | ||
github_username=<GITHUB_USERNAME> | ||
github_access_token=<GITHUB_ACCESS_TOKEN> | ||
github_commit_email=<GITHUB_EMAIL> | ||
``` | ||
|
||
Replace the following: | ||
* `GITHUB_USERNAME`: your GitHub username granting access to the plugin repository. | ||
* `GITHUB_ACCESS_TOKEN`: your GitHub access token with permission to upload and commit changes to the plugin repository. | ||
* `GITHUB_EMAIL`: your email address associated with your GitHub account. | ||
4. Run `make release`. | ||
5. Create a pull request against the [Nextfow plugins index repository](https://github.com/nextflow-io/plugins) from your fork. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,132 @@ | ||
# Migrating to the Gradle plugin for Nextflow plugins | ||
|
||
This page introduces the Gradle plugin for Nextflow plugins, the Nextflow plugin registry, and how to migrate to the new plugin framework. | ||
|
||
|
||
## Improvements to the plugin framework | ||
|
||
The Nextflow plugin ecosystem is evolving to support a more robust and user-friendly experience by simplifying plugin development, streamlining publishing and discovery, and improving how plugins are loaded into workflows. These improvements make plugins more accessible, maintainable, and interoperable with Nextflow. | ||
|
||
### Gradle plugin for Nextflow plugins | ||
|
||
The Gradle plugin for Nextflow plugins simplifies and standardizes the development of Nextflow plugins. It configures default dependencies required for Nextflow integration and introduces custom Gradle tasks to streamline building, testing, packaging, and publishing plugins. | ||
|
||
The Gradle plugin is versioned and published to the [Gradle Plugin Portal](https://plugins.gradle.org/), allowing developers to manage it like any other dependency. As the plugin ecosystem evolves, this Gradle plugin will enable easier maintenance and adoption of ongoing improvements to the Nextflow plugin framework. | ||
|
||
### Nextflow plugin registry | ||
|
||
The Nextflow plugin registry is a centralized repository of assembled plugins. It hosts an index of plugin metadata that supports plugin discovery, accessibility, and version tracking. The registry is integrated with the Nextflow runtime. Nextflow will automatically locate and download configured plugins. | ||
|
||
## Impact on users and developers | ||
|
||
The impact of the Gradle plugin for Nextflow plugins and Nextflow plugin registry differs for plugin users and developers. | ||
|
||
### Plugin Users | ||
|
||
If you are a plugin user, no immediate actions are required. The plugin configuration has not changed. | ||
|
||
### Plugin developers | ||
|
||
Developers are encouraged to migrate to the Gradle plugin for Nextflow plugins and benefit from features that simplify plugin development and integration with the wider plugin ecosystem. | ||
|
||
To migrate an existing Nextflow plugin: | ||
|
||
1. Remove the following files and folders: | ||
- `buildSrc/` | ||
- `nextflow.config` | ||
- `launch.sh` | ||
- `plugins/build.gradle` | ||
2. If your plugin uses a `plugins` directory, move the `src` directory to the project root. \ | ||
|
||
:::{note} | ||
Plugin sources should be in `src/main/groovy` or `src/main/java`. | ||
::: | ||
|
||
3. Replace the contents of `settings.gradle` with the following: | ||
|
||
```groovy | ||
rootProject.name = '<PLUGIN_NAME>' | ||
``` | ||
|
||
Replace `PLUGIN_NAME` with your plugin name. | ||
|
||
4. In the project root, create a new `build.gradle` file with the following configuration: | ||
|
||
```groovy | ||
// Plugins | ||
plugins { | ||
id 'io.nextflow.nextflow-plugin' version '0.0.1-alpha' | ||
} | ||
|
||
// Dependencies (optional) | ||
dependencies { | ||
<DEPENDENCY> | ||
} | ||
|
||
// Plugin version | ||
version = '<PLUGIN_VERSION>' | ||
|
||
nextflowPlugin { | ||
// Minimum Nextflow version | ||
nextflowVersion = '<MINIMUM_NEXTFLOW_VERSION>' | ||
|
||
// Plugin metadata | ||
provider = '<PROVIDER>' | ||
className = '<CLASS_NAME>' | ||
extensionPoints = [ | ||
'<EXTENSION_POINT>' | ||
] | ||
|
||
publishing { | ||
github { | ||
repository = '<GITHUB_REPOSITORY>' | ||
userName = project.findProperty('github_username') | ||
authToken = project.findProperty('github_access_token') | ||
email = project.findProperty('github_commit_email') | ||
indexUrl = '<GITHUB_INDEX_URL>' | ||
} | ||
} | ||
} | ||
``` | ||
|
||
Replace the following: | ||
|
||
- `DEPENDENCY`: (Optional) Your plugins dependency libraries—for example, `commons-io:commons-io:2.18.0`. | ||
- `PLUGIN_VERSION:` Your plugin version—for example, `0.5.0`. | ||
- `MINIMUM_NEXTFLOW_VERSION`: The minimum Nextflow version required to run your plugin—for example, `24.11.0-edge`. | ||
- `PROVIDER`: Your name or organization—for example, `nextflow`. | ||
- `CLASS_NAME`: Your plugin class name—for example, `nextflow.hello.HelloPlugin`. | ||
- `EXTENSION_POINT`: Your extension point identifiers that the plugin will implement or expose—for example, `nextflow.hello.HelloFactory`. | ||
- `GITHUB_REPOSITORY`: Your GitHub plugin repository name—for example, `nextflow-io/nf-hello`. | ||
- `GITHUB_INDEX_URL`: The URL of your fork of the plugins index repository—for example, [`plugins.json`](https://github.com/username/plugins/blob/main/plugins.json). | ||
|
||
5. Replace the contents of `Makefile` with the following: | ||
|
||
``` | ||
# Build the plugin | ||
assemble: | ||
./gradlew assemble | ||
|
||
clean: | ||
rm -rf .nextflow* | ||
rm -rf work | ||
rm -rf build | ||
./gradlew clean | ||
|
||
# Run plugin unit tests | ||
test: | ||
./gradlew test | ||
|
||
# Install the plugin into local nextflow plugins dir | ||
install: | ||
./gradlew install | ||
|
||
# Publish the plugin | ||
release: | ||
./gradlew releasePlugin | ||
``` | ||
|
||
6. Update `README.md` with information about the structure of your plugin. | ||
7. In the plugin root directory, run `make assemble`. | ||
|
||
The Gradle plugin for Nextflow plugins also supports publishing plugins. See {ref}`gradle-plugin-package` for more information. |
This file was deleted.
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.