Skip to content

Commit bee160a

Browse files
djaglowskidjaglowski
authored
Updated docs to reflect operator terminology (#50)
* Updated docs to reflect operator terminology
1 parent b4c08e2 commit bee160a

32 files changed

+207
-206
lines changed

‎README.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -91,7 +91,7 @@ pipeline:
9191
record: This is a sample log generated by Carbon
9292

9393
# An example output that sends captured logs to elasticsearch.
94-
# For more info: https://github.com/observIQ/carbon/blob/master/docs/plugins/elastic_output.md
94+
# For more info: https://github.com/observIQ/carbon/blob/master/docs/operators/elastic_output.md
9595
- type: elastic_output
9696
addresses:
9797
- http://my_node_address:9200
@@ -105,14 +105,14 @@ Once you've confirmed you're able to send a log entry, you'll want to connect ca
105105
pipeline:
106106
...
107107
# An example input that monitors the contents of a file.
108-
# For more info: https://github.com/observIQ/carbon/blob/master/docs/plugins/file_input.md
108+
# For more info: https://github.com/observIQ/carbon/blob/master/docs/operators/file_input.md
109109
- type: file_input
110110
include:
111111
- /sample/file/path
112112
...
113113

114114
# An example output that sends captured logs to elasticsearch.
115-
# For more info: https://github.com/observIQ/carbon/blob/master/docs/plugins/elastic_output.md
115+
# For more info: https://github.com/observIQ/carbon/blob/master/docs/operators/elastic_output.md
116116
- type: elastic_output
117117
addresses:
118118
- http://my_node_address:9200

‎docs/README.md

Lines changed: 21 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
This repo contains documentation for the Carbon Log Agent.
44

55
## How do I configure the agent?
6-
The agent is configured using a YAML config file that is passed in using the `--config` flag. This file defines a collection of plugins beneath a top-level `plugins` key. Each plugin possesses a `type` and `id` field.
6+
The agent is configured using a YAML config file that is passed in using the `--config` flag. This file defines a collection of operators beneath a top-level `pipeline` key. Each operator possesses a `type` and `id` field.
77

88
```yaml
99
plugins:
@@ -18,36 +18,36 @@ plugins:
1818
```
1919
2020
## What is an operator?
21-
An operator is the most basic unit of log processing. Each operator fulfills only a single responsibility, such as reading lines from a file, or parsing JSON from a field. These plugins are then chained together in a pipeline to achieve a desired result.
21+
An operator is the most basic unit of log processing. Each operator fulfills only a single responsibility, such as reading lines from a file, or parsing JSON from a field. These operators are then chained together in a pipeline to achieve a desired result.
2222
23-
For instance, a user may read lines from a file using the `file_input` plugin. From there, the results of this operation may be sent to a `regex_parser` plugin that creates fields based on a regex pattern. And then finally, these results may be sent to a `elastic_output` plugin that writes each line to Elasticsearch.
23+
For instance, a user may read lines from a file using the `file_input` operator. From there, the results of this operation may be sent to a `regex_parser` operator that creates fields based on a regex pattern. And then finally, these results may be sent to a `elastic_output` operator that writes each line to Elasticsearch.
2424

2525
## What operators are available?
2626

2727
Inputs:
28-
- [File input](/docs/plugins/file_input.md)
29-
- [TCP input](/docs/plugins/tcp_input.md)
30-
- [UDP input](/docs/plugins/udp_input.md)
31-
- [Journald input](/docs/plugins/journald_input.md)
32-
- [Generate input](/docs/plugins/generate_input.md)
28+
- [File input](/docs/operators/file_input.md)
29+
- [TCP input](/docs/operators/tcp_input.md)
30+
- [UDP input](/docs/operators/udp_input.md)
31+
- [Journald input](/docs/operators/journald_input.md)
32+
- [Generate input](/docs/operators/generate_input.md)
3333

3434
Parsers:
35-
- [JSON parser](/docs/plugins/json_parser.md)
36-
- [Regex parser](/docs/plugins/regex_parser.md)
37-
- [Syslog parser](/docs/plugins/syslog_parser.md)
38-
- [Severity parser](/docs/plugins/severity_parser.md)
39-
- [Time parser](/docs/plugins/time_parser.md)
35+
- [JSON parser](/docs/operators/json_parser.md)
36+
- [Regex parser](/docs/operators/regex_parser.md)
37+
- [Syslog parser](/docs/operators/syslog_parser.md)
38+
- [Severity parser](/docs/operators/severity_parser.md)
39+
- [Time parser](/docs/operators/time_parser.md)
4040

4141
Outputs:
42-
- [Google Cloud Logging](/docs/plugins/google_cloud_output.md)
43-
- [Elasticsearch](/docs/plugins/elastic_output.md)
44-
- [Stdout](/docs/plugins/stdout.md)
42+
- [Google Cloud Logging](/docs/operators/google_cloud_output.md)
43+
- [Elasticsearch](/docs/operators/elastic_output.md)
44+
- [Stdout](/docs/operators/stdout.md)
4545

4646
General purpose:
47-
- [Metadata](/docs/plugins/metadata.md)
48-
- [Restructure records](/docs/plugins/restructure.md)
49-
- [Router](/docs/plugins/router.md)
50-
- [Kubernetes Metadata Decorator](/docs/plugins/k8s_metadata_decorator.md)
51-
- [Rate limit](/docs/plugins/rate_limit.md)
47+
- [Metadata](/docs/operators/metadata.md)
48+
- [Restructure records](/docs/operators/restructure.md)
49+
- [Router](/docs/operators/router.md)
50+
- [Kubernetes Metadata Decorator](/docs/operators/k8s_metadata_decorator.md)
51+
- [Rate limit](/docs/operators/rate_limit.md)
5252

5353
Or create your own [plugins](/docs/plugins.md) for a technology-specific use case.

‎docs/plugins/elastic_output.md renamed to ‎docs/operators/elastic_output.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,12 @@
1-
## `elastic_output` plugin
1+
## `elastic_output` operator
22

3-
The `elastic_output` plugin will send entries to an Elasticsearch instance
3+
The `elastic_output` operator will send entries to an Elasticsearch instance
44

55
### Configuration Fields
66

77
| Field | Default | Description |
88
| --- | --- | --- |
9-
| `id` | `elastic_output` | A unique identifier for the plugin |
9+
| `id` | `elastic_output` | A unique identifier for the operator |
1010
| `addresses` | required | A list of addresses to send entries to |
1111
| `username` | | Username for HTTP basic authentication |
1212
| `password` | | Password for HTTP basic authentication |

‎docs/plugins/file_input.md renamed to ‎docs/operators/file_input.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,13 @@
1-
## `file_input` plugin
1+
## `file_input` operator
22

3-
The `file_input` plugin reads logs from files. It will place the lines read into the `message` field of the new entry.
3+
The `file_input` operator reads logs from files. It will place the lines read into the `message` field of the new entry.
44

55
### Configuration Fields
66

77
| Field | Default | Description |
88
| --- | --- | --- |
9-
| `id` | `file_input` | A unique identifier for the plugin |
10-
| `output` | Next in pipeline | The connected plugin(s) that will receive all outbound entries |
9+
| `id` | `file_input` | A unique identifier for the operator |
10+
| `output` | Next in pipeline | The connected operator(s) that will receive all outbound entries |
1111
| `include` | required | A list of file glob patterns that match the file paths to be read |
1212
| `exclude` | [] | A list of file glob patterns to exclude from reading |
1313
| `poll_interval` | 200ms | The duration between filesystem polls |
@@ -23,7 +23,7 @@ Note that by default, no logs will be read unless the monitored file is actively
2323

2424
#### `multiline` configuration
2525

26-
If set, the `multiline` configuration block instructs the `file_input` plugin to split log entries on a pattern other than newlines.
26+
If set, the `multiline` configuration block instructs the `file_input` operator to split log entries on a pattern other than newlines.
2727

2828
The `multiline` configuration block must contain exactly one of `line_start_pattern` or `line_end_pattern`. These are regex patterns that
2929
match either the beginning of a new log entry, or the end of a log entry.

‎docs/plugins/generate_input.md renamed to ‎docs/operators/generate_input.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,14 @@
1-
## `generate_input` plugin
1+
## `generate_input` operator
22

3-
The `generate_input` plugin generates log entries with a static record. This is useful for testing pipelines, especially when
4-
coupled with the [`rate_limit`](/docs/plugins/rate_limit.md) plugin.
3+
The `generate_input` operator generates log entries with a static record. This is useful for testing pipelines, especially when
4+
coupled with the [`rate_limit`](/docs/operators/rate_limit.md) operator.
55

66
### Configuration Fields
77

88
| Field | Default | Description |
99
| --- | --- | --- |
10-
| `id` | `generate_input` | A unique identifier for the plugin |
11-
| `output` | Next in pipeline | The connected plugin(s) that will receive all outbound entries |
10+
| `id` | `generate_input` | A unique identifier for the operator |
11+
| `output` | Next in pipeline | The connected operator(s) that will receive all outbound entries |
1212
| `write_to` | $ | A [field](/docs/types/field.md) that will be set to the path of the file the entry was read from |
1313
| `entry` | | A [entry](/docs/types/entry.md) log entry to repeatedly generate |
1414
| `count` | 0 | The number of entries to generate before stopping. A value of 0 indicates unlimited |

‎docs/plugins/google_cloud_output.md renamed to ‎docs/operators/google_cloud_output.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,12 @@
1-
## `google_cloud_output` plugin
1+
## `google_cloud_output` operator
22

3-
The `google_cloud_output` plugin will send entries to Google Cloud Logging.
3+
The `google_cloud_output` operator will send entries to Google Cloud Logging.
44

55
### Configuration Fields
66

77
| Field | Default | Description |
88
| --- | --- | --- |
9-
| `id` | `google_cloud_output` | A unique identifier for the plugin |
9+
| `id` | `google_cloud_output` | A unique identifier for the operator |
1010
| `credentials` | | The JSON-formatted credentials for the logs writer service account |
1111
| `credentials_file` | | A path to a file containing the JSON-formatted credentials |
1212
| `project_id` | | The Google Cloud project ID the logs should be sent to. Defaults to project_id found in credentials |

‎docs/plugins/journald_input.md renamed to ‎docs/operators/journald_input.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,17 +1,17 @@
1-
## `journald_input` plugin
1+
## `journald_input` operator
22

3-
The `journald_input` plugin reads logs from the systemd journal using the `journalctl` binary, which must be in the `$PATH` of the agentt.
3+
The `journald_input` operator reads logs from the systemd journal using the `journalctl` binary, which must be in the `$PATH` of the agentt.
44

55
By default, `journalctl` will read from `/run/journal` or `/var/log/journal`. If either `directory` or `files` are set, `journalctl` will instead read from those.
66

7-
The `journald_input` plugin will use the `__REALTIME_TIMESTAMP` field of the journald entry as the parsed entry's timestamp. All other fields are added to the entry's record as returned by `journalctl`.
7+
The `journald_input` operator will use the `__REALTIME_TIMESTAMP` field of the journald entry as the parsed entry's timestamp. All other fields are added to the entry's record as returned by `journalctl`.
88

99
### Configuration Fields
1010

1111
| Field | Default | Description |
1212
| --- | --- | --- |
13-
| `id` | `journald_input` | A unique identifier for the plugin |
14-
| `output` | Next in pipeline | The connected plugin(s) that will receive all outbound entries |
13+
| `id` | `journald_input` | A unique identifier for the operator |
14+
| `output` | Next in pipeline | The connected operator(s) that will receive all outbound entries |
1515
| `directory` | | A directory containing journal files to read entries from |
1616
| `files` | | A list of journal files to read entries from |
1717
| `write_to` | $ | A [field](/docs/types/field.md) that will be set to the path of the file the entry was read from |

‎docs/plugins/json_parser.md renamed to ‎docs/operators/json_parser.md

Lines changed: 11 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -1,18 +1,18 @@
1-
## `json_parser` plugin
1+
## `json_parser` operator
22

3-
The `json_parser` plugin parses the string-type field selected by `parse_from` as JSON.
3+
The `json_parser` operator parses the string-type field selected by `parse_from` as JSON.
44

55
### Configuration Fields
66

7-
| Field | Default | Description |
8-
| --- | --- | --- |
9-
| `id` | `json_parser` | A unique identifier for the plugin |
10-
| `output` | Next in pipeline | The connected plugin(s) that will receive all outbound entries |
11-
| `parse_from` | $ | A [field](/docs/types/field.md) that indicates the field to be parsed as JSON |
12-
| `parse_to` | $ | A [field](/docs/types/field.md) that indicates the field to be parsed as JSON |
13-
| `preserve` | false | Preserve the unparsed value on the record |
14-
| `on_error` | `send` | The behavior of the plugin if it encounters an error. See [on_error](/docs/types/on_error.md) |
15-
| `timestamp` | `nil` | An optional [timestamp](/docs/types/timestamp.md) block which will parse a timestamp field before passing the entry to the output plugin |
7+
| Field | Default | Description |
8+
| --- | --- | --- |
9+
| `id` | `json_parser` | A unique identifier for the operator |
10+
| `output` | Next in pipeline | The connected operator(s) that will receive all outbound entries |
11+
| `parse_from` | $ | A [field](/docs/types/field.md) that indicates the field to be parsed as JSON |
12+
| `parse_to` | $ | A [field](/docs/types/field.md) that indicates the field to be parsed as JSON |
13+
| `preserve` | false | Preserve the unparsed value on the record |
14+
| `on_error` | `send` | The behavior of the operator if it encounters an error. See [on_error](/docs/types/on_error.md) |
15+
| `timestamp` | `nil` | An optional [timestamp](/docs/types/timestamp.md) block which will parse a timestamp field before passing the entry to the output operator |
1616

1717

1818
### Example Configurations

‎docs/plugins/k8s_metadata_decorator.md renamed to ‎docs/operators/k8s_metadata_decorator.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,13 @@
1-
## `k8s_metadata_decorator` plugin
1+
## `k8s_metadata_decorator` operator
22

3-
The `k8s_metadata_decorator` plugin adds labels and annotations to the entry using data from the Kubernetes metadata API.
3+
The `k8s_metadata_decorator` operator adds labels and annotations to the entry using data from the Kubernetes metadata API.
44

55
### Configuration Fields
66

77
| Field | Default | Description |
88
| --- | --- | --- |
9-
| `id` | `k8s_metadata_decorator` | A unique identifier for the plugin |
10-
| `output` | Next in pipeline | The connected plugin(s) that will receive all outbound entries |
9+
| `id` | `k8s_metadata_decorator` | A unique identifier for the operator |
10+
| `output` | Next in pipeline | The connected operator(s) that will receive all outbound entries |
1111
| `namespace_field` | `namespace` | A [field](/docs/types/field.md) that contains the k8s namespace associated with the log entry |
1212
| `pod_name_field` | `pod_name` | A [field](/docs/types/field.md) that contains the k8s pod name associated with the log entry |
1313
| `cache_ttl` | 10m | A [duration](/docs/types/duration.md) indicating the time it takes for a cached entry to expire |

0 commit comments

Comments
 (0)