Skip to content

Commit

Permalink
Apply Suggestions
Browse files Browse the repository at this point in the history
  • Loading branch information
RahulGautamSingh committed Jan 19, 2025
1 parent 3788661 commit cd68238
Show file tree
Hide file tree
Showing 6 changed files with 74 additions and 46 deletions.
32 changes: 17 additions & 15 deletions docs/usage/configuration-options.md
Original file line number Diff line number Diff line change
Expand Up @@ -706,9 +706,11 @@ You can define custom managers to handle:
- Proprietary file formats or conventions
- Popular file formats not yet supported as a manager by Renovate

Currently we have two custom managers.
The `regex` manager which is based on using Regular Expression named capture groups.
The `jsonata` manager which is based on using JSONata queries.
Renovate has two custom managers:
| Custom manager | Matching engine |

Check failure on line 710 in docs/usage/configuration-options.md

View workflow job for this annotation

GitHub Actions / lint-docs

Tables should be surrounded by blank lines

docs/usage/configuration-options.md:710 MD058/blanks-around-tables Tables should be surrounded by blank lines [Context: "| Custom manager | Matching en..."] https://github.com/DavidAnson/markdownlint/blob/v0.37.2/doc/md058.md
| -------------- | ---------------------------------------------- |
| `regex` | Regular Expression, with named capture groups. |
| `jsonata` | JSONata query. |

You must capture/extract the following three fields _or_ configure its corresponding template (e.g. `depNameTemplate`) for these fields:

Expand All @@ -718,10 +720,10 @@ You must capture/extract the following three fields _or_ configure its correspon

We recommend you use only _one_ of these methods, or you'll get confused.

We recommend that you also tell Renovate what `versioning` to use.
We also recommend that you also tell Renovate what `versioning` to use.
If the `versioning` field is missing, then Renovate defaults to using `semver` versioning.

For more details and examples about it, see our documentation for the [`regex` manager](modules/manager/regex/index.md) and the [`JSONata` manager](modules/manager/jsonata/index.md).
For more details and examples regarding custom managers, see our documentation for the [`regex` manager](modules/manager/regex/index.md) and the [`JSONata` manager](modules/manager/jsonata/index.md).
For template fields, use the triple brace `{{{ }}}` notation to avoid Handlebars escaping any special characters.

<!-- prettier-ignore -->
Expand Down Expand Up @@ -765,7 +767,7 @@ image: my.new.registry/aRepository/andImage:1.21-alpine
<!-- prettier-ignore -->
!!! note
Can only be used with the custom regex maanger.
Can only be used with the custom regex manager.
### currentValueTemplate
Expand Down Expand Up @@ -807,7 +809,7 @@ Example:

### datasourceTemplate

If the `datasource` for a dependency is not captured with a named group then it can be defined in config using this field.
If the `datasource` for a dependency is not captured with a named group, then it can be defined in config using this field.
It will be compiled using Handlebars and the regex `groups` result.

### depNameTemplate
Expand All @@ -822,15 +824,15 @@ It will be compiled using Handlebars and the regex `groups` result.

### extractVersionTemplate

If `extractVersion` cannot be captured with a named capture group in `matchString` then it can be defined manually using this field.
If `extractVersion` cannot be captured with a named capture group in `matchString`, then it can be defined manually using this field.
It will be compiled using Handlebars and the regex `groups` result.

### fileFormat

It specifies the syntax of the package file being managed by the custom JSONata manager.
`fileFormat` specifies the syntax of the package file that's managed by the custom JSONata manager.
This setting helps the system correctly parse and interpret the configuration file's contents.

Currently, only the `json` format is supported.
Only the `json` format is supported.

```json title="Parsing a JSON file with a custom manager"
{
Expand All @@ -849,12 +851,12 @@ Currently, only the `json` format is supported.

### matchStrings

Each `matchStrings` must be one of the two:
Each `matchStrings` must be one of the following:

1. a valid regular expression, optionally with named capture groups (if using `customType=regex`)
2. a valid, escaped [JSONata](https://docs.jsonata.org/overview.html) query (if using `customType=json`)
1. A valid regular expression, which may optionally include named capture groups (if using `customType=regex`)
2. Or, a valid, escaped [JSONata](https://docs.jsonata.org/overview.html) query (if using `customType=json`)

See [`customType`](#customtype) docs, to know more them.
Read the [`customType`](#customtype) docs, to learn more.

Example:

Expand Down Expand Up @@ -885,7 +887,7 @@ Three options are available:

<!--prettier-ignore-->
!!! note
Only to be used with custom regex manager.
`matchStringsStrategy` can only be used in a custom regex manager config!

#### any

Expand Down
3 changes: 2 additions & 1 deletion lib/config/validation-helpers/utils.ts
Original file line number Diff line number Diff line change
Expand Up @@ -112,7 +112,8 @@ export function validateRegexManagerFields(
} else {
errors.push({
topic: 'Configuration Error',
message: `Each Custom Manager must contain a non-empty matchStrings array`,
message:
'Each Custom Manager `matchStrings` array must have at least one item.',
});
}

Expand Down
8 changes: 4 additions & 4 deletions lib/modules/manager/custom/jsonata/index.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ describe('modules/manager/custom/jsonata/index', () => {
fileFormat: 'json',
} as JsonataExtractConfig);
expect(res).toBeNull();
expect(logger.logger.warn).toHaveBeenCalledWith(
expect(logger.logger.debug).toHaveBeenCalledWith(
expect.anything(),
'Error while parsing file',
);
Expand Down Expand Up @@ -209,7 +209,7 @@ describe('modules/manager/custom/jsonata/index', () => {
],
};
const res = await extractPackageFile('{}', 'unused', config);
expect(logger.logger.warn).toHaveBeenCalledWith(
expect(logger.logger.debug).toHaveBeenCalledWith(
{
packageFile: 'unused',
jsonataQuery:
Expand All @@ -230,7 +230,7 @@ describe('modules/manager/custom/jsonata/index', () => {
};
const res = await extractPackageFile('{}', 'unused', config);
expect(res).toBeNull();
expect(logger.logger.warn).toHaveBeenCalledWith(
expect(logger.logger.debug).toHaveBeenCalledWith(
expect.anything(),
'Error compiling template for JSONata manager',
);
Expand All @@ -246,7 +246,7 @@ describe('modules/manager/custom/jsonata/index', () => {
};
const res = await extractPackageFile('{}', 'unused', config);
expect(res).not.toBeNull();
expect(logger.logger.warn).toHaveBeenCalledWith(
expect(logger.logger.debug).toHaveBeenCalledWith(
{ url: 'this-is-not-a-valid-url-foo' },
'Invalid JSONata manager registryUrl',
);
Expand Down
2 changes: 1 addition & 1 deletion lib/modules/manager/custom/jsonata/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ export async function extractPackageFile(
break;
}
} catch (err) {
logger.warn(
logger.debug(
{ err, fileName: packageFile, fileFormat: config.fileFormat },
'Error while parsing file',
);
Expand Down
61 changes: 41 additions & 20 deletions lib/modules/manager/custom/jsonata/readme.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,14 @@
With `customManagers` using `JSONata` queries you can configure Renovate so it finds dependencies in JSON files, that are not detected by its other built-in package managers.

Renovate uses the `jsonata` package to process the `json` file content. Read about the [jsonata query language](https://docs.jsonata.org/overview.html) in their readme.
Renovate uses the `jsonata` package to process the `json` file content using the queries.

The JSONata manager is unique in Renovate in because:
For more on the jsonata query language, read the [jsonata query language site](https://docs.jsonata.org/overview.html).

- It is configurable via [JSONata](https://jsonata.org/) queries
- It can extract any `datasource`
- By using the `customManagers` config, you can create multiple "JSONata managers" the same repository
The JSONata manager is unique in Renovate, because:

- It can be used with any `datasource`
- It can be configured via [JSONata](https://jsonata.org/) queries
- you can create multiple "JSONata managers" the same repository, using the `customManagers` config

### Required Fields

Expand All @@ -29,24 +31,24 @@ Before Renovate can look up a dependency and decide about updates, it must have

You must:

- Capture the `currentValue` of the dependency
- Capture the `depName` or `packageName`. Or use a template field: `depNameTemplate` and `packageNameTemplate`
- Capture the `datasource`, or a use `datasourceTemplate` config field
- Capture the `currentValue` of the dependency _or_ use the `currentValueTemplate` template field
- Capture the `depName` or `packageName`. _Or_ use a template field: `depNameTemplate` and `packageNameTemplate`
- Capture the `datasource`, _or_ use the `datasourceTemplate` template field

#### Optional fields you can include in the resulting structure

You may use any of these items:

- `depType`, or a use `depTypeTemplate` config field
- `versioning`, or a use `versioningTemplate` config field. If neither are present, Renovate defaults to `semver-coerced`
- `extractVersion`, or use an `extractVersionTemplate` config field
- `depType`, _or_ use the `depTypeTemplate` template field
- `versioning`, _or_ the use `versioningTemplate` template field. If neither are present, Renovate defaults to `semver-coerced`
- `extractVersion`, _or_ use the `extractVersionTemplate` template field
- `currentDigest`
- `registryUrl`, or a use `registryUrlTemplate` config field. If it's a valid URL, it will be converted to the `registryUrls` field as a single-length array
- `indentation`. It must be either empty, or whitespace only (otherwise `indentation` will be reset to an empty string)
- `registryUrl`, _or_ use the `registryUrlTemplate` template field. If it's a valid URL, it will be converted to the `registryUrls` field as a single-length array
- `indentation`. Must be empty, _or_ whitespace. Else Renovate restes only `indentation` to an empty string

### Usage

To configure it, use the following syntax:
When you configure a JSONata manager, use the following syntax:

```javascript
{
Expand All @@ -62,9 +64,28 @@ To configure it, use the following syntax:
}
```

Where `<query>` is a [JSONata](https://docs.jsonata.org/overview.html) query that transform the contents into a JSON object with the following schema:
Overwrite the `<query>` placeholder text with your [JSONata](https://docs.jsonata.org/overview.html) query.
The JSONata query transforms the content to a JSON object, similar to the this:

```javascript dependencies information extracted usig jsonata query
[
{
depName: 'some_dep',
currentValue: '1.0.0',
datasource: 'docker',
versioning: 'semver',
},
];
```

Creating your Renovate JSONata manager config is easier if you understand JSONata queries.
We recommend you follow these steps:

1. Read the official JSONata query language docs
2. Check our example queries below
3. You're ready to make your own config

To be effective with the JSONata manager, you should understand jsonata queries. But enough examples may compensate for lack of experience.
Alternatively you can "try and error" to a working config, by adjusting our examples.

#### Example queries

Expand Down Expand Up @@ -117,7 +138,7 @@ Query:
*.{ "depName": package, "currentValue": version }
```

```json title="The dependency name is in a JSON node name and the version is in a child leaf to that node"
```json title="The dependency name is in a JSON node name, and the version is in a child leaf to that node"
{
"foo": {
"version": "1.2.3"
Expand All @@ -134,7 +155,7 @@ Query:
$each(function($v, $n) { { "depName": $n, "currentValue": $v.version } })
```

```json title="The name of the dependency and the version are both value nodes of the same parent node"
```json title="The dependency name and its version are both value nodes of the same parent node"
{
"packages": [
{
Expand All @@ -155,7 +176,7 @@ Query:
packages.{ "depName": package, "currentValue": version }
```

```json title="The name of the dependency and the version are in the same string"
```json title="The dependency name and version are part of the same string"
{
"packages": ["[email protected]", "[email protected]"]
}
Expand All @@ -167,7 +188,7 @@ Query:
$map($map(packages, function ($v) { $split($v, "@") }), function ($v) { { "depName": $v[0], "currentVersion": $v[1] } })
```

```json title="JSONata manager config to extract deps from package.json file in the renovate repository"
```json title="JSONata manager config to extract deps from a package.json file in the Renovate repository"
{
"customType": "jsonata",
"fileMatch": ["package.json"],
Expand Down
14 changes: 9 additions & 5 deletions lib/modules/manager/custom/jsonata/utils.ts
Original file line number Diff line number Diff line change
Expand Up @@ -20,10 +20,14 @@ export async function handleMatching(
// won't fail as this is verified during config validation
const jsonataExpression = jsonata(query);
// this does not throw error, just returns undefined if no matches
let queryResult = (await jsonataExpression.evaluate(json)) ?? [];
let queryResult = await jsonataExpression.evaluate(json);

if (is.emptyObject(queryResult) || is.emptyArray(queryResult)) {
logger.warn(
if (
!queryResult ||
is.emptyObject(queryResult) ||
is.emptyArray(queryResult)
) {
logger.debug(
{
jsonataQuery: query,
packageFile,
Expand Down Expand Up @@ -67,7 +71,7 @@ export function createDependency(
const compiled = template.compile(tmpl, queryResult, false);
updateDependency(field, compiled, dependency);
} catch {
logger.warn(
logger.debug(
{ template: tmpl },
'Error compiling template for JSONata manager',
);
Expand All @@ -89,7 +93,7 @@ function updateDependency(
case 'registryUrl': {
const url = parseUrl(value)?.toString();
if (!url) {
logger.warn({ url: value }, 'Invalid JSONata manager registryUrl');
logger.debug({ url: value }, 'Invalid JSONata manager registryUrl');
break;
}
dependency.registryUrls = [url];
Expand Down

0 comments on commit cd68238

Please sign in to comment.