diff --git a/docs/developers/how-to-build-plugins.md b/docs/developers/how-to-build-plugins.md index 8ff0d89..9bcee17 100644 --- a/docs/developers/how-to-build-plugins.md +++ b/docs/developers/how-to-build-plugins.md @@ -5,9 +5,9 @@ sidebar-position: 1 # How to build plugins The IF is designed to be as composable as possible. This means you can develop your own plugins and use them in a pipeline. -To help developers write Typescript plugins to integrate easily into IF, we provide the `ExecutePlugin` interface. Here's an overview of the stages you need to follow to integrate your plugin: +To help developers write Typescript plugins to integrate easily into IF, we provide the `PluginFactory` interface. Here's an overview of the stages you need to follow to integrate your plugin: -- create a Typescript file that implements the `ExecutePlugin` +- create a Typescript file that implements the `PluginFactory` from [`if-core`](https://github.com/Green-Software-Foundation/if-core) - install the plugin - initialize and invoke the plugin in your manifest file @@ -25,66 +25,64 @@ Now your project is setup, you can focus on your plugin logic. The entry point f The following sections describe the rules your plugin code should conform to. We also have an [appendix](#appendix-walk-through-of-the-sum-plugin) that deep dives a real plugin. -### The plugin interface +### Plugin interface -The `ExecutePlugin` is structured as follows: +Your plugin must implement the `PluginFactory` interface, which is a higher-order function that takes a `params` object of type `PluginFactoryParams`. This factory function returns another function (referred to as the "inner function") that manages the plugin’s `config`, `parametersMetadata`, and `mapping`. + +The `PluginFactory` is structured as follows: ```ts -export type ExecutePlugin = { - execute: ( - inputs: PluginParams[], - config?: Record - ) => PluginParams[]; - metadata: { - kind: string; - inputs?: ParameterMetadata; - outputs?: ParameterMetadata; - }; - [key: string]: any; -}; +export const PluginFactory = + (params: PluginFactoryParams) => + ( + config: C = {} as C, + parametersMetadata: PluginParametersMetadata, + mapping: MappingParams + ) => ({ + metadata: { + inputs: {...params.metadata.inputs, ...parametersMetadata?.inputs}, + outputs: parametersMetadata?.outputs || params.metadata.outputs, + }, + execute: async (inputs: PluginParams[]) => { + // Generic plugin functionality goes here + // E.g., mapping, arithmetic operations, validation + // Process inputs and mapping logic + }); + }); ``` -The interface requires an execute function where your plugin logic is implemented. It should also return metadata. This can include any relevant metadata you want to include, with a minimum requirement being `kind: execute`. +The inner function returned by the `PluginFactory` handles the following parameters: -### Global config +- **`config`**: An object of type `ConfigParams`. This parameter holds the configuration settings for the plugin and defaults to an empty object (`{}`). +- **`parametersMetadata`**: An object of type `PluginParametersMetadata` that contains metadata describing the plugin’s parameters. +- **`mapping`**: A `MappingParams` object that outlines how plugin parameters are mapped. -Global config is passed as an argument to the plugin. In your plugin code you can handle it as follows: +### Config -```ts -// Here's the function definition - notice that global config is passed in here! -export const Plugin = ( - globalConfig: YourConfig, - parametersMetadata: PluginParametersMetadata, - mapping: MappingParams -): ExecutePlugin => { - // in here you have access to globalConfig[your-params] -}; -``` - -The parameters available to you in `globalConfig` depends upon the parameters you pass in the manifest file. For example, the `Sum` plugin has access to `input-parameters` and `output-parameter` in its global config, and it is defined in the `Initialize` block in the manifest file as follows: +The `config` object is passed as an argument to your plugin and can be handled as shown in the example above. The structure of the config depends on what is defined in the manifest file. For example, the `Sci` plugin has access to `input-parameters` and `output-parameter` fields in its global configuration, as defined in the `Initialize` block of the manifest file: ```yaml initialize: plugins: sum: - method: Sum + method: Sci path: 'builtin' - global-config: + config: input-parameters: ['cpu/energy', 'network/energy'] output-parameter: 'energy' ``` ### Parameter metadata -The `parameter-metadata` is passed as an argument to the plugin as the global config. It contains information about the `description`, `unit` and `aggregation-method` of the parameters of the inputs and outputs that defined in the manifest. +The `parameter-metadata` is passed as an argument to the plugin as the config. It contains information about the `description`, `unit` and `aggregation-method` of the parameters of the inputs and outputs that defined in the manifest. ```yaml initialize: plugins: sum: - method: Sum + method: Sci path: 'builtin' - global-config: + config: input-parameters: ['cpu/energy', 'network/energy'] output-parameter: 'energy-sum' parameter-metadata: @@ -92,21 +90,27 @@ initialize: cpu/energy: description: energy consumed by the cpu unit: kWh - aggregation-method: sum + aggregation-method: + time: sum + component: sum network/energy: description: energy consumed by data ingress and egress unit: kWh - aggregation-method: sum + aggregation-method: + time: sum + component: sum outputs: energy-sum: description: sum of energy components unit: kWh - aggregation-method: sum + aggregation-method: + time: sum + component: sum ``` ### Mapping -The `mapping` is an optional argument passed to the plugin. Its purpose is to rename the arguments expected or returned from the plugin as part of the plugin's execution, avoiding the need to use additional plugins to rename parameters. +The `mapping` is an optional argument passed to the plugin. Its purpose is to rename the arguments expected or returned from the plugin as part of the plugin's execution, avoiding the need to use additional plugins to rename parameters. For example, your plugin might expect `cpu/energy` and your input data has the parameter `cpu-energy` returned from another plugin. Instead of using an additional plugin to rename the parameter and add a new one, you can use `mapping` to: @@ -114,7 +118,6 @@ a) rename the output from the first plugin so that `cpu/energy` is returned inst b) instruct the second plugin to accept `cpu-energy` instead of the default `cpu/energy` - The `mapping` config is an object with key-value pairs, where the `key` is the 'original' parameter name that the plugin uses, and the `value` is the 'new' name that you want to use instead. The `mapping` block is an optional and allows mapping the input and output parameters of the plugin. The structure of the `mapping` block is: @@ -150,37 +153,61 @@ tree: In the `outputs`, the `sci` value returned by the `Sci` plugin will be named `if-sci`. -### Methods - -#### execute - -`execute()` is where the main calculation logic of the plugin is implemented. It always takes `inputs` (an array of `PluginParams`) as an argument and returns an updated set of `inputs`. - -#### Params - -| Param | Type | Purpose | -| -------- | ---------------- | ------------------------------------------------------------------------------ | -| `inputs` | `PluginParams[]` | Array of data provided in the `inputs` field of a component in a manifest file | - -#### Returns +### Plugin example -| Return value | Type | Purpose | -| ------------ | ------------------------- | ----------------------------------------------------------- | -| `outputs` | `Promise` | `Promise` resolving to an array of updated `PluginParams[]` | +Here’s a minimal example of a plugin that sums inputs based on the configuration: -### What are `PluginParams`? +```ts +export const Plugin = PluginFactory({ + metadata: { + inputs: { + // Define your input parameters here + }, + outputs: { + // Define your output parameters here + }, + }, + configValidation: (config: ConfigParams) => { + // Implement validation logic for config here + }, + inputValidation: (input: PluginParams, config: ConfigParams) => { + // Implement validation logic for inputs here + }, + implementation: async (inputs: PluginParams[], config: ConfigParams) => { + // Implement plugin logic here + // e.g., summing input parameters + }, + allowArithmeticExpressions: [], +}); + +const plugin = Plugin(config, parametersMetadata, mapping); +const result = await plugin.execute(inputs); +``` -## What are `PluginParams`? +### PluginFactoryParams -`PluginParams` are a fundamental data type in the Impact Framework. The type is defined as follows: +The `PluginFactory` interface requires the mandatory parameters defined in the `PluginFactoryParams` interface: ```ts -export type PluginParams = { - [key: string]: any; -}; +export interface PluginFactoryParams { + implementation: ( + inputs: PluginParams[], + config: C, + mapping?: MappingParams + ) => Promise; + metadata?: PluginParametersMetadata; + configValidation?: z.ZodSchema | ConfigValidatorFunction; + inputValidation?: z.ZodSchema | InputValidatorFunction; + allowArithmeticExpressions?: string[]; +} ``` -The `PluginParams` type therefore defines an array of key-value pairs. +Additional Notes + +- `Implement`: You should implement `implementation` function. It should contains the primary logic to generate outputs. +- `Validation`: You should define appropriate `zod` schemas or validation functions for both config and inputs. This ensures that invalid data is caught early and handled appropriately. +- `Arithmetic Expressions`: By including configuration, input, and output parameters of the plugin in the `allowArithmeticExpressions` array, you enable dynamic evaluation of mathematical expressions within parameter values. This eliminates the need for manual pre-calculation and allows basic mathematical operations to be embedded directly within parameter values in manifest files. More details [here.](../reference/features.md) +- `Mapping`: Ensure your plugin correctly handles the mapping of parameters. This is essential when working with dynamic input and output configurations. ## Step 3: Install your plugin @@ -214,7 +241,7 @@ initialize: new-plugin: method: YourFunctionName path: 'new-plugin' - global-config: + config: something: true ``` @@ -242,7 +269,7 @@ For example, for a plugin saved in `github.com/my-repo/new-plugin` you can do th npm install https://github.com/my-repo/new-plugin ``` -Then, in your manifest file, provide the path in the plugin instantiation. You also need to specify which function the plugin instantiates. Let's say you are using the `Sum` plugin from the example above: +Then, in your manifest file, provide the path in the plugin instantiation. You also need to specify which function the plugin instantiates. Let's say you are using the `Sci` plugin from the example above: ```yaml name: plugin-demo @@ -250,14 +277,12 @@ description: loads plugin tags: null initialize: plugins: - - name: new-plugin - kind: plugin + new-plugin: method: FunctionName path: https://github.com/my-repo/new-plugin tree: children: child: - config: inputs: ``` @@ -284,67 +309,55 @@ You should also create unit tests for your plugin to demonstrate correct executi You can read our more advanced guide on [how to refine your plugins](./how-to-refine-plugins.md). -## Appendix: Walk-through of the Sum plugin +## Appendix: Walk-through of the Sci plugin -To demonstrate how to build a plugin that conforms to the `ExecutePlugin`, let's examine the `sum` plugin. +To demonstrate how to build a plugin that conforms to the `PluginFactory`, let's examine the `Sum` plugin. The `sum` plugin implements the following logic: -- sum whatever is provided in the `input-parameters` field from `globalConfig`. -- append the result to each element in the output array with the name provided as `output-parameter` in `globalConfig`. +- sum whatever is provided in the `input-parameters` field from `config`. +- append the result to each element in the output array with the name provided as `output-parameter` in `config`. Let's look at how you would implement this from scratch: -The plugin must be a function conforming to `ExecutePlugin`. You can call the function `Sum`, and inside the body you can add the signature for the `execute` method: - -```typescript -export const Sum = ( - globalConfig: SumConfig, - parametersMetadata: PluginParametersMetadata -): ExecutePlugin => { - const errorBuilder = buildErrorMessage(Sum.name); - const metadata = { - kind: 'execute', - inputs: parametersMetadata?.inputs, - outputs: parametersMetadata?.outputs, - }; - - /** - * Calculate the sum of each input. - */ - const execute = async (inputs: PluginParams[]): Promise => {}; +The plugin must be a function conforming to `PluginFactory`. - return { - metadata, - execute, - }; -}; +```ts +export const Sum = PluginFactory({ + configValidation: z.object({ + 'input-parameters': z.array(z.string()), + 'output-parameter': z.string().min(1), + }), + inputValidation: (input: PluginParams, config: ConfigParams) => { + return validate(validationSchema, inputData); + }, + implementation: async (inputs: PluginParams[], config: ConfigParams) => {}, + allowArithmeticExpressions: [], +}); ``` -Your plugin now has the basic structure required for IF integration. Your next task is to add code to the body of `execute` to enable the actual plugin logic to be implemented. +Your plugin now has the basic structure required for IF integration. Your next task is to add code to the body of `implementation` to enable the actual plugin logic to be implemented. -The `execute` function should grab the `input-parameters` (the values to sum) from `globalConfig`. it should then iterate over the `inputs` array, get the values for each of the `input-parameters` and append them to the `inputs` array, using the name from the `output-parameter` value in `globalConfig`. Here's what this can look like, with the actual calculation pushed to a separate function, `calculateSum`. +The `implementation` function should grab the `input-parameters` (the values to sum) from `config`. It should then iterate over the `inputs` array, get the values for each of the `input-parameters` and append them to the `inputs` array, using the name from the `output-parameter` value in `config`. Here's what this can look like, with the actual calculation pushed to a separate function, `calculateSum`. ```ts -/** - * Calculate the sum of each input. - */ -const execute = async (inputs: PluginParams[]): Promise => { - const inputParameters = globalConfig['input-parameters']; - const outputParameter = globalConfig['output-parameter']; - - return inputs.map((input) => { - return { - ...input, - [outputParameter]: calculateSum(input, inputParameters), - }; - }); - - return { - metadata, - execute, +{ + implementation: async (inputs: PluginParams[], config: ConfigParams) => { + const { + 'input-parameters': inputParameters, + 'output-parameter': outputParameter, + } = config; + + return inputs.map((input) => { + const calculatedResult = calculateSum(input, inputParameters); + + return { + ...input, + [outputParameter]: calculatedResult, + }; + }); }; -}; +} ``` Now we just need to define what happens in `calculateSum` - this can be a simple `reduce`: @@ -364,7 +377,7 @@ Note that this example did not include any validation or error handling - you wi ## Managing errors -If framework provides it's own set of error classes which will make user's live much more easier! -[If Core](https://github.com/Green-Software-Foundation/if-core) plugin has a set of error classes which can be used for having full integration with the IF framework. More details about each error class can be found at [Errors Reference](../reference//errors.md) +The IF framework provides its own set of error classes, making your task as a plugin builder much simpler! These are available to you in the `if-core` package that comes bundled with IF. You can import the appropriate error classes and add custom messages. +The [If Core](https://github.com/Green-Software-Foundation/if-core) repository contains the `PluginFactory` interface, utility functions, and a set of error classes that can be fully integrated with the IF framework. Detailed information on each error class can be found in the [Errors Reference](../reference/errors.md). Now you are ready to run your plugin using the `if-run` CLI tool! diff --git a/docs/developers/how-to-refine-plugins.md b/docs/developers/how-to-refine-plugins.md index 474ac40..426316c 100644 --- a/docs/developers/how-to-refine-plugins.md +++ b/docs/developers/how-to-refine-plugins.md @@ -1,6 +1,7 @@ --- sidebar-position: 2 --- + # How to make plugins production ready Our [How to build plugins](./how-to-build-plugins.md) guide covered the basics for how to construct an Impact Framework plugin. This guide will help you to refine your plugin to make it production-ready. These are best practice guidelines - if you intend to contribute your plugin to one of our repositories, following these guidelines will help your PR to get merged. Even if you are not aiming to have a plugin merged into one of our repositories, consistency with our norms is useful for debugging and maintaining and for making your plugin as useful as possible for other Impact Framework developers. @@ -34,27 +35,27 @@ We prefer the following ordering of imports in your plugin code: 1. Node built-in modules (e.g. `import fs from 'fs';`) 2. External modules (e.g. `import {z} from 'zod';`) 3. Internal modules (e.g. `import config from 'src/config';`) -4. Interfaces (e.g. `import type {PluginInterface} from 'interfaces'`) -5. Types (e.g. `import {PluginParams} from '../../types/common'`;) +4. Interfaces (e.g. `import {PluginInterface} from '@grnsft/if-core/types';`) +5. Types (e.g. `import {PluginParams} from '@grnsft/if-core/types';`) ### Comments Each logical unit in the code should be preceded by an appropriate explanatory comment. Sometimes it is useful to include short comments inside a function that clarifies the purpose of a particular statement. Here's an example from our codebase: ```ts - /** - * Calculates the energy consumption for a single input. - */ - const calculateEnergy = (input: PluginParams) => { - const { - 'memory/capacity': totalMemory, - 'memory/utilization': memoryUtil, - 'energy-per-gb': energyPerGB, - } = input; - - // GB * kWh/GB == kWh - return totalMemory * (memoryUtil / 100) * energyPerGB; - }; +/** + * Calculates the energy consumption for a single input. + */ +const calculateEnergy = (input: PluginParams) => { + const { + 'memory/capacity': totalMemory, + 'memory/utilization': memoryUtil, + 'energy-per-gb': energyPerGB, + } = input; + + // GB * kWh/GB == kWh + return totalMemory * (memoryUtil / 100) * energyPerGB; +}; ``` ### Error handling @@ -71,33 +72,46 @@ import {ERRORS} from '@grnsft/if-core/util'; const {MissingInputDataError} = ERRORS; -... +... throw new MissingInputDataError("my-plugin is missing my-parameter from inputs[0]"); ``` - ### Validation -We recommend using validation techniques to ensure the integrity of input data. Validate input parameters against expected types, ranges, or constraints to prevent runtime errors and ensure data consistency. +We recommend using `inputValidation` property from `PluginFactory` for validation to ensure the integrity of input data. Validate input parameters against expected types, ranges, or constraints to prevent runtime errors and ensure data consistency. -We use `zod` to validate data. Here's an example from our codebase: +You need to use `zod` schema or `InputValidatorFunction`. Here's an example from our codebase: + +- When using function with `InputValidatorFunction` type. ```ts -/** - * Checks for required fields in input. - */ -const validateInput = (input: PluginParams) => { - const schema = z - .object({ - 'cpu/name': z.string(), - }) - .refine(allDefined, { - message: '`cpu/name` should be present.', - }); +// `inputValidation` from plugin definition +inputValidation: (input: PluginParams, config: ConfigParams) => { + const inputData = { + 'input-parameter': input[config['input-parameter']], + }; + const validationSchema = z.record(z.string(), z.number()); + validate(validationSchema, inputData); - return validate>(schema, input); -} + return input; +}; +``` + +- When using `zod` schema + +```ts +// `inputValidation` from plugin definition +inputValidation: z.object({ + duration: z.number().gt(0), + vCPUs: z.number().gt(0).default(1), + memory: z.number().gt(0).default(16), + ssd: z.number().gte(0).default(0), + hdd: z.number().gte(0).default(0), + gpu: z.number().gte(0).default(0), + 'usage-ratio': z.number().gt(0).default(1), + time: z.number().gt(0).optional(), +}); ``` ### Code Modularity @@ -105,7 +119,6 @@ const validateInput = (input: PluginParams) => { Break down complex functionality into smaller, manageable methods with well-defined responsibilities. Encapsulate related functionality into private methods to promote code reusability and maintainability. - ## 3. Unit tests Your plugin should have unit tests with 100% coverage. We use `jest` to handle unit testing. We strive to have one `describe` per function. Each possible outcome from each function is separated using `it` with a precise and descriptive message. @@ -113,19 +126,20 @@ Your plugin should have unit tests with 100% coverage. We use `jest` to handle u Here's an example that covers plugin initialization and the happy path for the `execute()` function. ```ts -import {Sum} from '../../../../lib'; +import { ERRORS } from '@grnsft/if-core/utils'; -import {ERRORS} from '@grnsft/if-core/util/'; +import { Sum } from '../../../if-run/builtins/sum'; -const {InputValidationError} = ERRORS; +const { InputValidationError, WrongArithmeticExpressionError } = ERRORS; -describe('lib/sum: ', () => { +describe('builtins/sum: ', () => { describe('Sum: ', () => { - const globalConfig = { + const config = { 'input-parameters': ['cpu/energy', 'network/energy', 'memory/energy'], 'output-parameter': 'energy', }; - const sum = Sum(globalConfig); + const parametersMetadata = {}; + const sum = Sum(config, parametersMetadata, {}); describe('init: ', () => { it('successfully initalized.', () => { @@ -161,9 +175,9 @@ describe('lib/sum: ', () => { expect(result).toStrictEqual(expectedResult); }); - } - }) -}) + }); + }); +}); ``` We have a [dedicated page](./how-to-write-unit-tests.md) explaining in more detail how to write great unit tests for Impact Framework plugins. diff --git a/docs/developers/how-to-write-unit-tests.md b/docs/developers/how-to-write-unit-tests.md index 0b73b87..60635af 100644 --- a/docs/developers/how-to-write-unit-tests.md +++ b/docs/developers/how-to-write-unit-tests.md @@ -2,67 +2,65 @@ sidebar-position: 3 --- -# How to write unit tests +# How to write unit tests Impact Framework unit tests follow a standard format. We use the `jest` testing library. You can run all our existing tests by opening the project directory and running `npm test`. This page explains how you can add new unit tests for your plugins (or add some for our plugins if you notice a gap). ## Test files -Both the IF and the project repositories include a `__test__` directory. Inside, you will find subdirectory `unit/lib` containing directories for each plugin. Your plugin repository should also follow this structure. Inside the plugin directory you can add `index.test.ts`. This is where you write your unit tests. For example, here's the directory tree for our `teads-curve` test file: - +The IF includes a `__test__` directory. Inside, you will find subdirectory `if-run/builtins` containing test files for each plugin. Your plugin repository should also follow this structure. Inside the `builtins` you can add `plugin.test.ts`. This is where you write your unit tests. For example, here's the directory tree for our `sum` test file: ```sh -if-unofficial-plugins +if | |- src | |-__tests__ | - |-unit + |-if-run | - |-lib + |-builtins | - teads-curve - | - |- index.test.ts + sum.test.ts ``` - ## Setting up your test file You will need to import your plugin so that it can be instantiated and tested. You will also need some elements from `jest/globals`: For example, these are the imports for our `Sum` plugin. ```ts -import {Sum} from '../../../../lib'; -import {ERRORS} from '../../../../util/errors'; -const {InputValidationError} = ERRORS; +import { ERRORS } from '@grnsft/if-core/utils'; + +import { Sum } from '../../../if-run/builtins/sum'; + +const { InputValidationError } = ERRORS; ``` You may require other imports for your specific set of tests. ## Describe -Each method should have its own dedicated `describe` block. +Each method should have its own dedicated `describe` block. -Your unit tests should have *at least* two `describe` blocks, one to test the plugin initialization and one for `execute`. +Your unit tests should have _at least_ two `describe` blocks, one to test the plugin initialization and one for `execute`. ```ts -describe("init", ()=> {}) -describe("execute", ()=> {}) +describe('init', () => {}); +describe('execute', () => {}); ``` For example, here is a describe block checking that the `Sum` plugin initializes correctly: ```typescript -describe('lib/sum: ', () => { +describe('builtins/sum: ', () => { describe('Sum: ', () => { - const globalConfig = { + const config = { 'input-parameters': ['cpu/energy', 'network/energy', 'memory/energy'], 'output-parameter': 'energy', }; - const sum = Sum(globalConfig); + const sum = Sum(config); describe('init: ', () => { it('successfully initalized.', () => { @@ -70,8 +68,8 @@ describe('lib/sum: ', () => { expect(sum).toHaveProperty('execute'); }); }); - }) -}) + }); +}); ``` ## It @@ -81,57 +79,53 @@ Within each `describe` block, each effect to be tested should have a dedicated ` Here's an example of a new `describe` block for the `execute()` method on the `Sum` plugin. The `describe` block indicates that we are testing effects of the `execute()` method. `it` is specific to a single outcome - in this case there are two `it` blocks that test that the plugin returns a specific result in the happy path and throws an exception if the user has provided invalid config data, specifically that the user-provided `cpu/energy` parameter is missing: ```typescript - describe('execute(): ', () => { - it('successfully applies Sum strategy to given input.', async () => { - expect.assertions(1); - - const expectedResult = [ - { - duration: 3600, - 'cpu/energy': 1, - 'network/energy': 1, - 'memory/energy': 1, - energy: 3, - timestamp: '2021-01-01T00:00:00Z', - }, - ]; - - const result = await sum.execute([ - { - duration: 3600, - 'cpu/energy': 1, - 'network/energy': 1, - 'memory/energy': 1, - timestamp: '2021-01-01T00:00:00Z', - }, - ]); - - expect(result).toStrictEqual(expectedResult); - }); - - it('throws an error on missing params in input.', async () => { - const expectedMessage = - 'Sum: cpu/energy is missing from the input array.'; - - expect.assertions(1); - - try { - await sum.execute([ - { - duration: 3600, - timestamp: '2021-01-01T00:00:00Z', - }, - ]); - } catch (error) { - expect(error).toStrictEqual( - new InputValidationError(expectedMessage) - ); - } - }); - }) +describe('execute(): ', () => { + it('successfully applies Sum strategy to given input.', async () => { + expect.assertions(1); + + const expectedResult = [ + { + duration: 3600, + 'cpu/energy': 1, + 'network/energy': 1, + 'memory/energy': 1, + energy: 3, + timestamp: '2021-01-01T00:00:00Z', + }, + ]; + + const result = await sum.execute([ + { + duration: 3600, + 'cpu/energy': 1, + 'network/energy': 1, + 'memory/energy': 1, + timestamp: '2021-01-01T00:00:00Z', + }, + ]); + + expect(result).toStrictEqual(expectedResult); + }); + + it('throws an error on missing params in input.', async () => { + const expectedMessage = 'Sum: cpu/energy is missing from the input array.'; + + expect.assertions(1); + + try { + await sum.execute([ + { + duration: 3600, + timestamp: '2021-01-01T00:00:00Z', + }, + ]); + } catch (error) { + expect(error).toStrictEqual(new InputValidationError(expectedMessage)); + } + }); +}); ``` - ## Errors We prefer to use `expect` to check the errors returned from a test. We do this by writing `expect` in a `catch` block. Here's an example from our `sci` plugin tests: @@ -157,7 +151,7 @@ it('throws an exception on missing functional unit data.', async () => { }); ``` -It is also necessary to include `expect.assertions(n)` for testing asynchronous code, where `n` is the number of assertiosn that should be tested before the test completes. +It is also necessary to include `expect.assertions(n)` for testing asynchronous code, where `n` is the number of assertiosn that should be tested before the test completes. ## Mocks @@ -170,16 +164,15 @@ We do have mock backends in several of our tests, and we also have a mock data g Please use `jest --coverage` to see a coverage report for your plugin - your unit tests should yield 100% coverage. The snippet below shows what to expect from the coverage report: ```sh --------------------------------|---------|----------|---------|---------|------------------- -| File | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s | -| --------------------------- | ------- | -------- | ------- | ------- | ----------------- | -| All files | 100 | 100 | 100 | 100 | -| lib | 100 | 100 | 100 | 100 | -| index.ts | 100 | 100 | 100 | 100 | -| lib/cloud-metadata | 100 | 100 | 100 | 100 | -| index.ts | 100 | 100 | 100 | 100 | -| lib/e-mem | 100 | 100 | 100 | 100 | -| index.ts | 100 | 100 | 100 | 100 | -| lib/e-net | 100 | 100 | 100 | 100 | -| index.ts | 100 | 100 | 100 | 100 | +------------------------------------|---------|----------|---------|---------|------------------- +| File | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s | +| --------------------------------- | ------- | -------- | ------- | ------- | ----------------- | +| All files | 100| 100 | 100 | 100 | +| if-run/builtins/coefficient | 100 | 100 | 100 | 100 | +| index.ts | 100 | 100 | 100 | 100 | +| if-run/builtins/copy-param | 100 | 100 | 100 | 100 | +| index.ts | 100 | 100 | 100 | 100 | +| if-run/builtins/csv-lookup | 100 | 100 | 100 | 100 | +| index.ts | 100 | 100 | 100 | 100 | +| if-run/builtins/divide | 100 | 94.11 | 100 | 100 | ``` diff --git a/docs/major-concepts/aggregation.md b/docs/major-concepts/aggregation.md index f263d90..9457ee6 100644 --- a/docs/major-concepts/aggregation.md +++ b/docs/major-concepts/aggregation.md @@ -38,7 +38,7 @@ There are two fields: `metrics` and `type`. `metrics` is an array of metrics that you want to aggregate. You can provide any value here, but they must match a key that exists in your output data (i.e. if you tell IF to aggregate `carbon` but `carbon` is not in your outputs you will receive an error message and aggregation will fail). You can provide any number of metrics. In the example above, the aggregation feature will operate on the `carbon` and `energy` values. -`type` determines which kind of aggregation you want to perform. The choices are `horizontal` (time-series aggregation only), `vertical` (tree aggregation only) or `both` (both kinds of aggregation will be performed). In the example above, both types of aggregation will be performed over the two selected metrics. +`type` determines which kind of aggregation you want to perform. The choices are `time` (previously `horizontal`: time-series aggregation only), `component` (previously `vertical`: tree aggregation only) or `both` (both kinds of aggregation will be performed). In the example above, both types of aggregation will be performed over the two selected metrics. ## Aggregation methods diff --git a/docs/major-concepts/exhaust-script.md b/docs/major-concepts/exhaust-script.md index 8331faa..063526f 100644 --- a/docs/major-concepts/exhaust-script.md +++ b/docs/major-concepts/exhaust-script.md @@ -25,7 +25,7 @@ initialize: sum: method: Sum path: 'builtin' - global-config: + config: input-parameters: ['cpu/energy', 'network/energy'] output-parameter: 'energy' tree: @@ -36,8 +36,6 @@ tree: regroup: compute: - sum - config: - sum: inputs: - timestamp: 2023-08-06T00:00 duration: 3600 diff --git a/docs/major-concepts/if.md b/docs/major-concepts/if.md index b9d530e..8ee125a 100644 --- a/docs/major-concepts/if.md +++ b/docs/major-concepts/if.md @@ -16,6 +16,7 @@ The available options and their shortcuts are: - `--no-output` or `-n` (optional): suppress the output to console - `--help` or `-h`: prints out help instruction - `--debug`: enables IF execution logs +- `--append`: allows you to rerun an already-computed manifest and append new values to the existing data. The only required command is `--manifest`. Without a valid path to a manifest file, `if-run` has nothing to execute. diff --git a/docs/major-concepts/manifest-file.md b/docs/major-concepts/manifest-file.md index e3b7e9b..e90359f 100644 --- a/docs/major-concepts/manifest-file.md +++ b/docs/major-concepts/manifest-file.md @@ -43,7 +43,6 @@ tree: observe: regroup: compute: - config: defaults: inputs: - timestamp: 2023-08-06T00:00 @@ -60,7 +59,7 @@ The global metadata includes the `name`, `description`, and `tags` that can be u #### Initialize -The initialize section is where you define which plugins will be used in your manifest file and provide the global configuration for them. Below is sample for initialization: +The initialize section is where you define which plugins will be used in your manifest file and provide the configuration for them. Below is sample for initialization: ```yaml initialize: @@ -74,7 +73,7 @@ Where required values are: - `method`: the name of the function exported by the plugin. - `path`: the path to the plugin code. For example, for a plugin from our standard library, this value would be `builtin` -There is also an optional `global-config` field that can be used to set _global_ configuration that is common to a plugin wherever it is invoked across the entire manifest file. +There is also an optional `config` field that can be used to set _config_ that is common to a plugin wherever it is invoked across the entire manifest file. Impact Framework uses the `initialize` section to instantiate each plugin. A plugin cannot be invoked elsewhere in the manifest file unless it is included in this section. @@ -107,24 +106,28 @@ You can also add information to the plugin's initialize section about parameter ```yaml plugins: - "sum-carbon": - path: "builtin" - method: Sum - global-config: + sum-carbon: + path: 'builtin' + method: Sum + config: input-parameters: - - carbon-operational - - embodied-carbon + - carbon-operational + - embodied-carbon output-parameter: carbon - parameter-metadata: + parameter-metadata: inputs: - carbon-operational: + carbon-operational: description: "carbon emitted due to an application's execution" - unit: "gCO2eq" - aggregation-method: 'sum', - embodied-carbon: + unit: 'gCO2eq' + aggregation-method: + time: sum + component: sum + embodied-carbon: description: "carbon emitted during the production, distribution and disposal of a hardware component, scaled by the fraction of the component's lifespan being allocated to the application under investigation" - unit: "gCO2eq" - aggregation-method: 'sum' + unit: 'gCO2eq' + aggregation-method: + time: sum + component: sum ``` #### Execution (auto-generated) @@ -161,25 +164,31 @@ explain: carbon: plugins: - sci - unit: gCO2eq - description: >- - total carbon emissions attributed to an application's usage as the sum - of embodied and operational carbon - aggregation-method: 'sum' + unit: gCO2eq + description: >- + total carbon emissions attributed to an application's usage as the sum + of embodied and operational carbon + aggregation-method: + time: sum + component: sum requests: plugins: - sci - unit: requests - description: number of requests made to application in the given timestep - aggregation-method: 'sum' + unit: requests + description: number of requests made to application in the given timestep + aggregation-method: + time: sum + component: sum sci: plugins: - sci - unit: gCO2eq/request - description: >- - software carbon intensity expressed as a rate of carbon emission per - request - aggregation-method: 'sum' + unit: gCO2eq/request + description: >- + software carbon intensity expressed as a rate of carbon emission per + request + aggregation-method: + time: sum + component: sum ``` ### Tree @@ -234,7 +243,6 @@ tree: regroup: compute: - sum - config: null defaults: null inputs: - timestamp: 2023-07-06T00:00 @@ -255,7 +263,6 @@ tree: regroup: compute: - sum - config: null defaults: null inputs: - timestamp: 2023-07-06T00:00 @@ -506,7 +513,7 @@ initialize: sum: path: builtin method: Sum - global-config: + config: input-parameters: - cpu/energy - network/energy @@ -557,8 +564,6 @@ tree: regroup: compute: - sum - config: - sum: null inputs: - timestamp: 2023-08-06T00:00 duration: 3600 diff --git a/docs/major-concepts/time.md b/docs/major-concepts/time.md index 8b1b83f..2475d00 100644 --- a/docs/major-concepts/time.md +++ b/docs/major-concepts/time.md @@ -8,9 +8,9 @@ Every `observation` in an array of `inputs` represents a snapshot with a known s ```yml inputs: - - timestamp: 2024-01-15T00:00:00.000Z - duration: 10 - cpu-util: 20 + - timestamp: 2024-01-15T00:00:00.000Z + duration: 10 + cpu-util: 20 ``` The total time covered by an inputs array is determined by the timestamp of the first observation in the array and the timestamp and duration in the last observation in the array. Since every observation needs both a timestamp and a duration, an inputs array is always a time series. @@ -19,7 +19,7 @@ The total time covered by an inputs array is determined by the timestamp of the The time series for each component is defined by its inputs array. However, a manifest file can contain many separate components, each with their own time series. There is no guarantee that an individual time series is continuous, or that all the components in a manifest file have the same start time, end time and resolution. This makes it difficult to aggregate, visualize or do like-for-like comparisons between components. -To solve this problem, we provide a built-in `time-sync` feature that synchronizes the time series' across all the components in a tree. The time-sync feature takes a global start time, end time and interval, then forces every individual time series to conform to this global configuration. +To solve this problem, we provide a built-in `time-sync` feature that synchronizes the time series' across all the components in a tree. The time-sync feature takes a global start time, end time and interval, then forces every individual time series to conform to this configuration. - This works by first upsampling each time series to a common base resolution (typically 1s). - Any gaps in the time series are filled in with "zero objects", which have an identical structure to the real observations but with usage metrics set to zero (we assume that when there is no data, there is no usage). @@ -27,14 +27,12 @@ To solve this problem, we provide a built-in `time-sync` feature that synchroniz - If a component's time series starts after the global start time, we pad the start of the time series with "zero objects" so that the start times are identical. - If the component's time series starts *before* the global start time, we trim the time series down, discarding observations from before the global start time. The same trimming logic is applied to the end times. - After synchronizing the start and end times and padding any discontinuities, we have a set of continuous time series' of identical length. -- Next, we batch observations together into time bins whose size is define by the global `interval` value. This means that the resolution of the final time series' are identical and equal to `interval`. +- Next, we batch observations together into time bins whose size is define by the global `interval` value. This means that the resolution of the final time series' are identical and equal to `interval`. This process yields synchronized time series for all components across a tree, enabling easy visualization and intercomparison. This synchronization is also a prerequisite for our aggregation function. - ![](../../static/img/time-sync-schematic.png) - ## Toggling off time sync Some applications will not want to pad with zero values, and may be strict about continuous time series' being provided in the raw manifest file. In these cases, simply toggle the padding off in the manifest file. diff --git a/docs/pipelines/cpu-to-carbon.md b/docs/pipelines/cpu-to-carbon.md index c78b5a6..f5ad824 100644 --- a/docs/pipelines/cpu-to-carbon.md +++ b/docs/pipelines/cpu-to-carbon.md @@ -25,11 +25,11 @@ This pipeline takes the observations described above, and generates carbon emiss ## Scope -This pipeline takes into account the operational carbon of the server running our application. This includes the energy used to run the application, calculated from CPU and memory utilization. It does not account for any embodied carbon, nor networking energy, nor anything related to the end user. In real applications, the pipeline described here will be part of a much larger manifest that considers other parts of the system. +This pipeline takes into account the operational carbon of the server running our application. This includes the energy used to run the application, calculated from CPU and memory utilization. It does not account for any embodied carbon, nor networking energy, nor anything related to the end user. In real applications, the pipeline described here will be part of a much larger manifest that considers other parts of the system. ## Description -The Teads CPU power curve CPU utilization (as a percentage) against a scaling factor that can be applied to the CPUs thermal design power to estimate the power drawn by the CPU in Watts. +The Teads CPU power curve CPU utilization (as a percentage) against a scaling factor that can be applied to the CPUs thermal design power to estimate the power drawn by the CPU in Watts. The research underpinning the curve was summarized in a pair of blog posts: @@ -40,12 +40,12 @@ The curve has become very widely used as a general purpose utilization-to-wattag The wattage can be transformed into energy by doing the following: -1) Measure your CPU utilization -2) Determine the thermal design power of your processor -3) Determine the scaling factor for your CPU utilization by interpolating the Teads curve -4) Determine the power drawn by your CPU by multiplying your scaling factor by the CPU's thermal design power -5) Perform a unit conversion to convert power in Watts to energy in kwH -6) Scale the energy estimated for the entire chip to the portion of the chip that is actually in use. +1. Measure your CPU utilization +2. Determine the thermal design power of your processor +3. Determine the scaling factor for your CPU utilization by interpolating the Teads curve +4. Determine the power drawn by your CPU by multiplying your scaling factor by the CPU's thermal design power +5. Perform a unit conversion to convert power in Watts to energy in kwH +6. Scale the energy estimated for the entire chip to the portion of the chip that is actually in use. These steps can be executed in IF using just three plugins: @@ -53,20 +53,17 @@ These steps can be executed in IF using just three plugins: - `Multiply` - `Divide` - ## Common patterns The logical flow from CPU utilization to carbon via a power-curve and thermal design power is a common pattern that is likely to be re-used elsewhere. - ## Constants and coefficients: | parameter | description | value | unit | source | -| ----------------------- | ------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------- | +| ----------------------- | ------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------- | ------------- | ------------------------------------------------------------------------------------------------------------ | | `x`, `y` | Points on power curve relating CPU utilization to a coefficient used to scale the processor's thermal design power | `x: [0, 10, 50, 100], y: [0.12, 0.32, 0.75, 1.02]` | dimensionless | [Davy, 2021](https://medium.com/teads-engineering/building-an-aws-ec2-carbon-emissions-dataset-3f0fd76c98ac) | | `grid-carbon-intensity` | the carbon emitted per unit energy from the electrical grid | 750 | gCO2e/kWh | approximates global average | - ## Assumptions and limitations The following are assumed to be true in this manifest: @@ -75,7 +72,6 @@ The following are assumed to be true in this manifest: - the temporal granularity of the observations are sufficient to accurately capture the behaviour of our application - the grid carbon intensity is sufficiently accurate for the location where the computational work is done - ## Components There is only one component in this example. It represents the entire application. The component pipeline looks as follows: @@ -92,7 +88,6 @@ pipeline: - energy-to-carbon ``` - ## Plugins ### Interpolate @@ -124,28 +119,28 @@ cpu-factor-to-wattage: input-parameters: - cpu-factor - cpu/thermal-design-power -output-parameter: +output-parameter: - cpu-wattage wattage-times-duration: input-parameters: - cpu-wattage - duration -output-parameter: +output-parameter: - cpu-wattage-times-duration energy-to-carbon: input-parameters: - grid-carbon-intensity - energy-cpu-kwh -output-parameter: +output-parameter: - carbon ``` ### Divide -The `Divide` plugin is used several times in this manifest. The instances are: +The `Divide` plugin is used several times in this manifest. The instances are: - `wattage-to-energy-kwh`. used to convert energy in W/duration to kWh. - `calculate-vcpu-ratio`: used to calculate the ratio of allocated vCPUs to total vCPUS @@ -171,10 +166,8 @@ output: cpu/energy ``` - ## Manifest - ```yaml name: teads curve demo description: null @@ -239,24 +232,26 @@ initialize: path: builtin method: Multiply config: - input-parameters: ['grid-carbon-intensity', 'cpu-energy-kwh'] - output-parameter: 'carbon' + input-parameters: + - grid-carbon-intensity + - cpu-energy-kwh + output-parameter: carbon execution: command: >- /home/user/.npm/_npx/1bf7c3c15bf47d04/node_modules/.bin/ts-node /home/user/if/src/index.ts -m manifests/examples/teads-curve.yml environment: - if-version: 0.3.3-beta.0 - os: linux - os-version: 5.15.0-107-generic - node-version: 21.4.0 - date-time: 2024-06-06T14:33:25.188Z (UTC) + if-version: 0.6.0 + os: macOS + os-version: 14.6.1 + node-version: 18.20.4 + date-time: 2024-10-03T15:11:48.498Z (UTC) dependencies: - '@babel/core@7.22.10' - '@babel/preset-typescript@7.23.3' - '@commitlint/cli@18.6.0' - '@commitlint/config-conventional@18.6.0' - - '@grnsft/if-unofficial-plugins@v0.3.1' + - '@grnsft/if-core@0.0.25' - '@jest/globals@29.7.0' - '@types/jest@29.5.8' - '@types/js-yaml@4.0.9' @@ -264,6 +259,7 @@ execution: - '@types/node@20.9.0' - axios-mock-adapter@1.22.0 - axios@1.7.2 + - cross-env@7.0.3 - csv-parse@5.5.6 - csv-stringify@6.4.6 - fixpack@4.0.0 @@ -280,7 +276,7 @@ execution: - typescript-cubic-spline@1.0.1 - typescript@5.2.2 - winston@3.11.0 - - zod@3.22.4 + - zod@3.23.8 status: success tree: children: @@ -326,6 +322,7 @@ tree: thermal-design-power: 100 vcpus-total: 8 vcpus-allocated: 2 + grid-carbon-intensity: 750 cpu-factor: 0.13999999999999999 cpu-wattage: 13.999999999999998 cpu-wattage-times-duration: 5039.999999999999 @@ -339,6 +336,7 @@ tree: thermal-design-power: 100 vcpus-total: 8 vcpus-allocated: 2 + grid-carbon-intensity: 750 cpu-factor: 0.32 cpu-wattage: 32 cpu-wattage-times-duration: 11520 @@ -352,6 +350,7 @@ tree: thermal-design-power: 100 vcpus-total: 8 vcpus-allocated: 2 + grid-carbon-intensity: 750 cpu-factor: 0.75 cpu-wattage: 75 cpu-wattage-times-duration: 27000 @@ -365,6 +364,7 @@ tree: thermal-design-power: 100 vcpus-total: 8 vcpus-allocated: 2 + grid-carbon-intensity: 750 cpu-factor: 1.02 cpu-wattage: 102 cpu-wattage-times-duration: 36720 diff --git a/docs/pipelines/instance-metadata.md b/docs/pipelines/instance-metadata.md index 37ce8e0..d2729e1 100644 --- a/docs/pipelines/instance-metadata.md +++ b/docs/pipelines/instance-metadata.md @@ -18,24 +18,20 @@ This pipeline looks up metadata associated with the given cloud instance. It doe This pipeline is likely to be used as part of a larger pipeline. All we are doing here is retrieving metadata from an external file. Typicaly, this metadata will be used to feed further plugind to support impactestimates. - ## Description The instance metadata pipeline simply looks up a metadata for a given virtual machine instance name using the `csv-lookup` plugin from the IF standard library. However, the target dataset can return multiple processor names for a given VM instance where there are multiple possibilitiers. This means we need to create a pipeline that includes the `regex` plugin so parse out just one of the possible values. For this demo we'll just extract the first value if there are multiple available for the `processor-name`. - ## Tags csv, instance-metadata, regex - ## Common Patterns The lookup process described on this page will likely be a common pattern used in other pipelines. - ## Assumptions and limitations The following are assumed to be true in this manifest: @@ -43,7 +39,6 @@ The following are assumed to be true in this manifest: - the target dataset is up to date - where there are multiple possible processors associated with an instance name, it is appropriate to select the first in the list. - ## Components There is only one component in this example. It represents the entire application. The component pipeline looks as follows: @@ -55,37 +50,37 @@ pipeline: - extract-processor-name ``` - ## Plugins ### csv-lookup -The `csv-lookup` plugin is used once. The instance is named `cloud-instance-metadata`. It targets a csv file in our `if-data` repository. +The `csv-lookup` plugin is used once. The instance is named `cloud-instance-metadata`. It targets a csv file in our `if-data` repository. #### config -``` +```yaml cloud-instance-metadata: filepath: https://raw.githubusercontent.com/Green-Software-Foundation/if-data/main/cloud-metdata-azure-instances.csv query: instance-class: "cloud/instance-type" output: "*" ``` - ### regex The `regex` plugin is used once. The instance is named `extract-processor-name`. It parses the response from the csv lookup plugin and extracts the first entry from the returned list. #### config -``` +```yaml extract-processor-name: -parameter: cpu-model-name -match: /^([^,])+/g -output: cpu/name + method: Regex + path: 'builtin' + config: + parameter: cpu-model-name + match: /^([^,])+/g + output: cpu/name ``` - ## Manifest ```yaml @@ -96,16 +91,16 @@ initialize: plugins: cloud-instance-metadata: method: CSVLookup - path: "builtin" - global-config: + path: 'builtin' + config: filepath: https://raw.githubusercontent.com/Green-Software-Foundation/if-data/main/cloud-metdata-azure-instances.csv query: - instance-class: "cloud/instance-type" - output: "*" + instance-class: 'cloud/instance-type' + output: '*' extract-processor-name: method: Regex - path: "builtin" - global-config: + path: 'builtin' + config: parameter: cpu-model-name match: /^([^,])+/g output: cpu/name @@ -135,7 +130,6 @@ if-run -m instance-metadata.yml -o output.yml Your new `output.yml` file will contain the following: - ```yaml name: csv-demo description: null @@ -145,7 +139,7 @@ initialize: cloud-instance-metadata: path: builtin method: CSVLookup - global-config: + config: filepath: >- https://raw.githubusercontent.com/Green-Software-Foundation/if-data/main/cloud-metdata-azure-instances.csv query: @@ -154,7 +148,7 @@ initialize: extract-processor-name: path: builtin method: Regex - global-config: + config: parameter: cpu-model-name match: /^([^,])+/g output: cpu/name @@ -163,17 +157,17 @@ execution: /home/user/.npm/_npx/1bf7c3c15bf47d04/node_modules/.bin/ts-node /home/user/Code/if/src/index.ts -m manifests/examples/instance-metadata.yml environment: - if-version: 0.3.3-beta.0 - os: linux - os-version: 5.15.0-107-generic - node-version: 21.4.0 - date-time: 2024-06-06T15:21:50.108Z (UTC) + if-version: 0.6.0 + os: macOS + os-version: 14.6.1 + node-version: 18.20.4 + date-time: 2024-10-03T15:15:36.328Z (UTC) dependencies: - '@babel/core@7.22.10' - '@babel/preset-typescript@7.23.3' - '@commitlint/cli@18.6.0' - '@commitlint/config-conventional@18.6.0' - - '@grnsft/if-unofficial-plugins@v0.3.1' + - '@grnsft/if-core@0.0.25' - '@jest/globals@29.7.0' - '@types/jest@29.5.8' - '@types/js-yaml@4.0.9' @@ -181,6 +175,7 @@ execution: - '@types/node@20.9.0' - axios-mock-adapter@1.22.0 - axios@1.7.2 + - cross-env@7.0.3 - csv-parse@5.5.6 - csv-stringify@6.4.6 - fixpack@4.0.0 @@ -197,7 +192,7 @@ execution: - typescript-cubic-spline@1.0.1 - typescript@5.2.2 - winston@3.11.0 - - zod@3.22.4 + - zod@3.23.8 status: success tree: children: diff --git a/docs/pipelines/sci.md b/docs/pipelines/sci.md index 15afc4d..0d38905 100644 --- a/docs/pipelines/sci.md +++ b/docs/pipelines/sci.md @@ -6,16 +6,14 @@ sidebar-position: 2 ## Description -The [software carbon intensity (SCI)](https://greensoftware.foundation/articles/software-carbon-intensity-sci-specification-project) score is perhaps the most important value that can be generated using Impact Framework. +The [software carbon intensity (SCI)](https://greensoftware.foundation/articles/software-carbon-intensity-sci-specification-project) score is perhaps the most important value that can be generated using Impact Framework. SCI is an ISO-recognized standard for reporting the carbon costs of running software. This tutorial demonstrates how to organize a pipeline of Impact framework plugins to calculate SCI scores from some simple observations that are commonly available for software applications running in the cloud. - ## Tags SCI, cloud, cpu, memory, power-curve - ## Prerequisites This tutorial builds on top of the [Teads curve](./teads.md) pipeline tutorial. That tutorial demonstrates how to organize a pipeline that converts CPU utilization observations into CPU energy. This tutorial uses the same pipeline but goes several steps further, including converting the CPU energy estimates into carbon, adding the embodied carbon associated with the hardware being used and calculating the SCI score. @@ -30,7 +28,6 @@ We employ the well known power curve from [Davy, 2021](https://medium.com/teads- We also use the networking energy and embodied carbon estimation methods from [Cloud Carbon Footprint](https://www.cloudcarbonfootprint.org/docs/methodology). This includes using the networking energy coefficient they suggest and implementing their method for calculating embodied emissions in an [Impact Framework plugin](https://github.com/Green-Software-Foundation/if/tree/main/src/if-run/builtins/sci-embodied). - ## Observations This manifest requires the following observations: @@ -44,18 +41,16 @@ This manifest requires the following observations: - data transferred in/out of the application - users per timestep - ## Constants and coefficients: | parameter | description | value | unit | source | -| ---------------------------- | ------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------- | +| ---------------------------- | ------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------- | ------------- | ------------------------------------------------------------------------------------------------------------ | | `network-energy-coefficient` | Coefficient relating data sent over network to energy | 0.001 | kWh/GB | [CCF](https://www.cloudcarbonfootprint.org/docs/methodology/#networking) | | `x`, `y` | Points on power curve relating CPU utilization to a coefficient used to scale the processor's thermal design power | `x: [0, 10, 50, 100], y: [0.12, 0.32, 0.75, 1.02]` | dimensionless | [Davy, 2021](https://medium.com/teads-engineering/building-an-aws-ec2-carbon-emissions-dataset-3f0fd76c98ac) | | `baseline-emissions` | embodied emissions for a "baseline" server with 1 CPU, 16GB RAM | 1000000 | gCO2e | [CCF](https://www.cloudcarbonfootprint.org/docs/methodology/#embodied-emissions) | | `lifespan` | lifespan for the server running our application | 126144000 | seconds | none, assumed 4 years is typical | | `usage-ratio` | scaling factor for adjusting total embodied carbon down tot he portion our application is responsible for | 1 | dimensionless | no usage scaling is done here as we assume dedicated hardware, we only scale by time | - ## Assumptions and limitations The following are assumed to be true in this manifest: @@ -68,7 +63,6 @@ The following are assumed to be true in this manifest: - it is appropriate to consider end user embodied carbon, end user operational carbon and the operationl and embodied emissions of the data center to be out of scope. - the temporal granularity of the observations are sufficient to accurately capture the behaviour of our application - ## Components There is only one component in this example. It represents the entire application. The component pipeline looks as follows: @@ -117,25 +111,25 @@ The `Multiply` plugin is used several times. The instances are: ``` cpu-factor-to-wattage: -input-parameters: - - cpu-factor - - cpu/thermal-design-power -output-parameter: - - cpu-wattage + input-parameters: + - cpu-factor + - cpu/thermal-design-power + output-parameter: + - cpu-wattage wattage-times-duration: -input-parameters: - - cpu-wattage - - duration -output-parameter: - - cpu-wattage-times-duration + input-parameters: + - cpu-wattage + - duration + output-parameter: + - cpu-wattage-times-duration operational-carbon: -input-parameters: - - energy - - grid/carbon-intensity -output-parameter: - - carbon-operational + input-parameters: + - energy + - grid/carbon-intensity + output-parameter: + - carbon-operational ``` @@ -147,9 +141,9 @@ The `Divide` plugin is used once in this manifest. The instance is named `wattag ``` wattage-to-energy-kwh: -numerator: cpu-wattage-times-duration -denominator: 3600000 -output: cpu-energy-raw + numerator: cpu-wattage-times-duration + denominator: 3600000 + output: cpu-energy-raw ``` ### Sum @@ -159,23 +153,22 @@ The `Sum` plugin is used several times in this manifest. The instances are: - `sum-energy-components`: used to sum all the various components of energy into a single value, called `energy`. - `sum-carbon`: used to sum the various components of carbon into a single value, named `carbon`. - #### config -``` +```yaml sum-energy-components: -input-parameters: - - cpu/energy - - network/energy -output-parameter: - - energy + input-parameters: + - cpu/energy + - network/energy + output-parameter: + - energy sum-carbon: -input-parameters: - - carbon-operational - - carbon-embodied -output-parameter: - - carbon + input-parameters: + - carbon-operational + - carbon-embodied + output-parameter: + - carbon ``` ### SciEmbodied @@ -186,7 +179,6 @@ The `SciEmbodied` plugin is used once. Its purpose is to calculate the embodied We use the plugin defaults for all the `SciEmbodied` config. This means we assume the total embodied emissions to be 1000000 gCO2e and the server to be a simple rack server with 1 CPU and 16GB RAM and no other components. - ### SCI The `SCI` plugin is used once. It is used to calculate the software carbon intensity by dividing `carbon` by a functional unit, that has to be available in the manifest `inputs` array at the time the plugin is executed. The functional unit in this example is users in each timestep. @@ -198,7 +190,6 @@ sci: functional-unit: users ``` - ## Manifest ```yaml @@ -210,7 +201,7 @@ aggregation: - carbon - sci type: both - + initialize: plugins: interpolate: @@ -279,7 +270,7 @@ initialize: path: builtin method: SciEmbodied config: - output-parameter: carbon-embodied + output-parameter: embodied-carbon operational-carbon: path: builtin method: Multiply @@ -326,172 +317,171 @@ tree: grid/carbon-intensity: 130 inputs: - timestamp: '2024-07-22T00:00:00' - duration: 3600 + duration: 3600 site-visits: 228 cpu/utilization: 45 component: 1 users: 1100 - timestamp: '2024-07-23T00:00:00' - duration: 3600 + duration: 3600 site-visits: 216 cpu/utilization: 30 component: 1 users: 1050 - timestamp: '2024-07-24T00:00:00' - duration: 3600 + duration: 3600 site-visits: 203 cpu/utilization: 50 component: 1 users: 1055 - timestamp: '2024-07-25T00:00:00' - duration: 3600 + duration: 3600 site-visits: 203 cpu/utilization: 33 component: 1 users: 996 - timestamp: '2024-07-26T00:00:00' - duration: 3600 + duration: 3600 site-visits: 172 cpu/utilization: 29 component: 1 users: 899 - timestamp: '2024-07-27T00:00:00' - duration: 3600 + duration: 3600 site-visits: 38 cpu/utilization: 68 component: 1 users: 1080 - timestamp: '2024-07-28T00:00:00' - duration: 3600 + duration: 3600 site-visits: 63 cpu/utilization: 49 component: 1 users: 1099 - timestamp: '2024-07-29T00:00:00' - duration: 3600 + duration: 3600 site-visits: 621 cpu/utilization: 77 component: 1 users: 1120 - timestamp: '2024-07-30T00:00:00' - duration: 3600 + duration: 3600 site-visits: 181 cpu/utilization: 31 component: 1 users: 1125 - timestamp: '2024-07-31T00:00:00' - duration: 3600 + duration: 3600 site-visits: 213 cpu/utilization: 29 component: 1 users: 1113 - timestamp: '2024-08-01T00:00:00' - duration: 3600 + duration: 3600 site-visits: 167 cpu/utilization: 29 component: 1 users: 1111 - timestamp: '2024-08-02T00:00:00' - duration: 3600 + duration: 3600 site-visits: 428 cpu/utilization: 29 component: 1 users: 1230 - timestamp: '2024-08-03T00:00:00' - duration: 3600 + duration: 3600 site-visits: 58 cpu/utilization: 64 component: 1 users: 1223 - timestamp: '2024-08-04T00:00:00' - duration: 3600 + duration: 3600 site-visits: 66 cpu/utilization: 59 component: 1 users: 1210 - timestamp: '2024-08-05T00:00:00' - duration: 3600 + duration: 3600 site-visits: 301 cpu/utilization: 60 component: 1 users: 1011 - timestamp: '2024-08-06T00:00:00' - duration: 3600 + duration: 3600 site-visits: 193 cpu/utilization: 35 component: 1 users: 999 - timestamp: '2024-08-07T00:00:00' - duration: 3600 + duration: 3600 site-visits: 220 cpu/utilization: 37 component: 1 users: 1010 - timestamp: '2024-08-08T00:00:00' - duration: 3600 + duration: 3600 site-visits: 215 cpu/utilization: 43 component: 1 users: 1008 - timestamp: '2024-08-09T00:00:00' - duration: 3600 + duration: 3600 site-visits: 516 cpu/utilization: 28 component: 1 users: 992 - timestamp: '2024-08-10T00:00:00' - duration: 3600 + duration: 3600 site-visits: 42 cpu/utilization: 39 component: 1 users: 1101 - timestamp: '2024-08-11T00:00:00' - duration: 3600 + duration: 3600 cpu/utilization: 40 site-visits: 76 component: 1 users: 1000 - timestamp: '2024-08-12T00:00:00' - duration: 3600 + duration: 3600 site-visits: 226 cpu/utilization: 55 component: 1 users: 845 - timestamp: '2024-08-13T00:00:00' - duration: 3600 + duration: 3600 site-visits: 180 cpu/utilization: 62 component: 1 users: 1006 - timestamp: '2024-08-14T00:00:00' - duration: 3600 + duration: 3600 site-visits: 232 cpu/utilization: 71 component: 1 users: 1076 - timestamp: '2024-08-15T00:00:00' - duration: 3600 + duration: 3600 site-visits: 175 cpu/utilization: 75 component: 1 users: 1050 - timestamp: '2024-08-16T00:00:00' - duration: 3600 + duration: 3600 site-visits: 235 cpu/utilization: 77 component: 1 users: 1047 - timestamp: '2024-08-17T00:00:00' - duration: 3600 + duration: 3600 site-visits: 44 cpu/utilization: 80 component: 1 users: 1020 - timestamp: '2024-08-18T00:00:00' - duration: 3600 + duration: 3600 site-visits: 31 cpu/utilization: 84 component: 1 users: 1038 - ``` diff --git a/docs/pipelines/teads.md b/docs/pipelines/teads.md new file mode 100644 index 0000000..3e4d10f --- /dev/null +++ b/docs/pipelines/teads.md @@ -0,0 +1,444 @@ +--- +sidebar_position: 2 +--- + +# Teads CPU pipeline + +The Teads CPU power curve CPU utilization (as a percentage) against a scaling factor that can be applied to the CPUs thermal design power to estimate the power drawn by the CPU in Watts. + +The research underpinning the curve was summarized in a pair of blog posts: + +[TEADS Engineering: Buildiong an AWS EC2 Carbon Emissions Dataset](https://medium.com/teads-engineering/building-an-aws-ec2-carbon-emissions-dataset-3f0fd76c98ac) +[Teads Engineering: Estimating AWS EC2 Instances Power Consumption](https://medium.com/teads-engineering/estimating-aws-ec2-instances-power-consumption-c9745e347959) + +The curve has become very widely used as a general purpose utilization-to-wattage converter for CPUs, despite the fact that it does not geenralize well. + +The wattage can be transformed into energy by doing the following: + +1. Measure your CPU utilization +2. Determine the thermal design power of your processor +3. Determine the scaling factor for your CPU utilization by interpolating the Teads curve +4. Determine the power drawn by your CPU by multiplying your scaling factor by the CPU's thermal design power +5. Perform a unit conversion to convert power in Watts to energy in kwH +6. Scale the energy estimated for the entire chip to the portion of the chip that is actually in use. + +These steps can be executed in IF using just three plugins: + +- `Interpolate` +- `Multiply` +- `Divide` + +We'll go through each step in the energy estimate and examine how to implement it in a manifest file using IF's standard library of `builtin`s. + +## Impact Framework implementation + +First, create a manifest file and add this following boilerplate code: + +```yaml +name: carbon-intensity plugin demo +description: +tags: +initialize: + plugins: +tree: + children: + child: + pipeline: + observe: + regroup: + compute: + defaults: + inputs: +``` + +If this structure looks unfamiliar to you, you can go back to our [manifests page](../major-concepts/manifest-file.md). + +### Step 1: measure CPU utilization + +The first step was to measure your CPU utilization. In real use cases you would typoically do this using an importer plugin that grabs data from a monitor API or similar. However, for this example we will just manually create some dummy data. Add some timestamps, durations and cpu/utilization data to your `inputs` array, as follows: + +```yaml +name: teads demo +description: +tags: +initialize: + plugins: +tree: + children: + child: + pipeline: + observe: + regroup: + compute: + defaults: + inputs: + - timestamp: 2023-08-06T00:00 + duration: 360 + cpu/utilization: 1 + carbon: 30 + - timestamp: 2023-09-06T00:00 + duration: 360 + carbon: 30 + cpu/utilization: 10 + - timestamp: 2023-10-06T00:00 + duration: 360 + carbon: 30 + cpu/utilization: 50 + - timestamp: 2023-10-06T00:00 + duration: 360 + carbon: 30 + cpu/utilization: 100 +``` + +### Step 2: Determine the thermal design power of your processor + +Typically determinign the TDP of your processor would be done using a CSV lookup. We have a pipeline example for [tdp-finder](./tdp-finder.md) in these docs - combining this pipeline with the `tdp-finder` pipeline would eb a great follow on exercise after you have finished this tutorial. Foir now, we will just hartd code some TDP data into your manifest so we can focus on the CPU utilization to energy calculations. Add `thermal-design-power` to `defaults` - this is a shortcut to providing it in every timestep in your `inputs` array. + +```yaml +default: + thermal-design-power: 100 +``` + +### Step 3: Interpolate the Teads curve + +The Teads curve has CPU utilization ont he `x` axis and a scaling factor on the `y` axis. There are only four points on the published curve. Your task is to get the scaling factor for your specific CPU utilization values by interpolating between the known points. Luckily, we have a `builtin` for that purpose! + +Add the `Interpolation` plugin to your list of plugins in the `initialize` block. + +```yaml +initialize: + plugins: + interpolate: + method Interpolation + path: builtin +``` + +The details about the interpolation you want to do and the values to return are configured in the `config` whoch is also added int he `initialize block`. Specifically, you have to provide the known points of the curve you want to interpolate, the `input-parameter` (which is the `x` value whose correspondiong `y` value you want to find out, i.e. your CPU utilization value) and the `output-parameter` (the name you want to give to your retrieved `y` value). + +You want to interpolate the Teads curve, so you can provide the `x` and `y` values obtained from the articles linked in the introduction section above: + +``` +x: [0, 10, 50, 100] +y: [0.12, 0.32, 0.75, 1.02] +``` + +Your `input-parameter` is your `cpu/utilization` and we'll name give the `output-parameter` the name `cpu-factor`. + +Your compelted `initialize` block for `interpolate` should look as follows: + +```yaml +interpolate: + method: Interpolation + path: 'builtin' + config: + method: linear + x: [0, 10, 50, 100] + y: [0.12, 0.32, 0.75, 1.02] + input-parameter: 'cpu/utilization' + output-parameter: 'cpu-factor' +``` + +### Step 4: Convert CPU factor to power + +The interpoaltion only gave use the scaling factor; we need to apply that scaling factor to the processor's TDP to get the power drawn by the CPU at your specific CPU utilization. + +To do this, we can use the `Multiply` plugin in the IF standard library. We'll give the instance of `Multiply` the name `cpu-factor-to-wattage` and int he `config` we'll define `cpu-factor` and `thermal-design-power` as the two elements in our `inputs` array that we want to multiply together. Then we'll name the result `cpu-wattage`: + +```yaml +cpu-factor-to-wattage: + method: Multiply + path: builtin + config: + input-parameters: ['cpu-factor', 'thermal-design-power'] + output-parameter: 'cpu-wattage' +``` + +Add this to your `initialize` block. + +### Step 5: Convert wattage to energy + +Next we have to perform some unit conversions. Wattage is a measure of power (energy over time). To convert to energy, we can first multiply by the number of seconds our observation covers (`duration`) to yield energy in joules. Then, convert to kWh by applying a scaling factor that takes seconds to hours and watts to kilowatts. + +You can do this in two steps: the first uses another instance of `Multiply` an the second uses `Divide`: + +To do the initial multiplication of the CPU wattage and the observation duration, add the following config to your `initialize` block: + +```yaml +wattage-times-duration: + method: Multiply + path: builtin + config: + input-parameters: ['cpu-wattage', 'duration'] + output-parameter: 'cpu-wattage-times-duration' +``` + +next, use the `Divide` plugin to do the unit conversion: + +```yaml +wattage-to-energy-kwh: + method: Divide + path: 'builtin' + config: + numerator: cpu-wattage-times-duration + denominator: 3600000 + output: cpu-energy-raw +``` + +### Step 6: Scale the energy by the allocated CPUs + +The `cpu-energy-raw` value you just configured is for the entire chip. But your application probably doesn't use the entire chip. Chances are you have some number of VCPUs allocated to you that is less than the total available. So you can scale your energy estimate by the ratio of VCPUs allocated to VCPUS available. + +Let's assume you know the number of VCPUs allocated and available in advance and that they are the same in every timestep. In this case, you can just add the values to `defaults` so they become available in every timestep, just as you did with `thermal-design-power`. + +```yaml +defaults: + thermal-design-power: 100 + vcpus-total: 8 + vcpus-allocated: 2 +``` + +You need one instance of `Divide` to calculate the `vcpu-ratio` and another to apply that `vcpu-ratio` to your `cpu-energy-raw` value and yield your final result: `cpu-energy-kwh`. Add the following to your `initialize` block to achieve those steps: + +```yaml +calculate-vcpu-ratio: + method: Divide + path: 'builtin' + config: + numerator: vcpus-total + denominator: vcpus-allocated + output: vcpu-ratio +correct-cpu-energy-for-vcpu-ratio: + method: Divide + path: 'builtin' + config: + numerator: cpu-energy-raw + denominator: vcpu-ratio + output: cpu-energy-kwh +``` + +### Step 7: Define your pipeline + +Now you have configured all your plugins, covering all the stages of the calculation, you can simple define them in order in the `pipeline` section of your manifest, as follows: + +```yaml +tree: + children: + child: + pipeline: + observe: + regroup: + compute: + - interpolate + - cpu-factor-to-wattage + - wattage-times-duration + - wattage-to-energy-kwh + - calculate-vcpu-ratio + - correct-cpu-energy-for-vcpu-ratio +``` + +You also need to add some input data that your pipeline can operate over. + +You can see the full manifest in the [IF repository](https://github.com/Green-Software-Foundation/if/blob/main/manifests/examples/teads-curve.yml). + +That's it! Your manifest is ready to run! + +## Running the manifest + +Having saved your manifest as `teads-curve.yaml` you can run it using IF: + +```sh +if-run -m teads-curve.yml -o teads-output.yml +``` + +This will yield the following output file: + +```yaml +name: teads curve demo +description: null +tags: null +initialize: + plugins: + interpolate: + path: builtin + method: Interpolation + config: + method: linear + x: + - 0 + - 10 + - 50 + - 100 + 'y': + - 0.12 + - 0.32 + - 0.75 + - 1.02 + input-parameter: cpu/utilization + output-parameter: cpu-factor + cpu-factor-to-wattage: + path: builtin + method: Multiply + config: + input-parameters: + - cpu-factor + - thermal-design-power + output-parameter: cpu-wattage + wattage-times-duration: + path: builtin + method: Multiply + config: + input-parameters: + - cpu-wattage + - duration + output-parameter: cpu-wattage-times-duration + wattage-to-energy-kwh: + path: builtin + method: Divide + config: + numerator: cpu-wattage-times-duration + denominator: 3600000 + output: cpu-energy-raw + calculate-vcpu-ratio: + path: builtin + method: Divide + config: + numerator: vcpus-total + denominator: vcpus-allocated + output: vcpu-ratio + correct-cpu-energy-for-vcpu-ratio: + path: builtin + method: Divide + config: + numerator: cpu-energy-raw + denominator: vcpu-ratio + output: cpu-energy-kwh +execution: + command: >- + /home/user/.npm/_npx/1bf7c3c15bf47d04/node_modules/.bin/ts-node + /home/user/if/src/index.ts -m manifests/examples/teads-curve.yml + environment: + if-version: 0.6.0 + os: macOS + os-version: 14.6.1 + node-version: 18.20.4 + date-time: 2024-10-03T15:05:11.948Z (UTC) + dependencies: + - '@babel/core@7.22.10' + - '@babel/preset-typescript@7.23.3' + - '@commitlint/cli@18.6.0' + - '@commitlint/config-conventional@18.6.0' + - '@grnsft/if-core@0.0.25' + - '@jest/globals@29.7.0' + - '@types/jest@29.5.8' + - '@types/js-yaml@4.0.9' + - '@types/luxon@3.4.2' + - '@types/node@20.9.0' + - axios-mock-adapter@1.22.0 + - axios@1.7.2 + - cross-env@7.0.3 + - csv-parse@5.5.6 + - csv-stringify@6.4.6 + - fixpack@4.0.0 + - gts@5.2.0 + - husky@8.0.3 + - jest@29.7.0 + - js-yaml@4.1.0 + - lint-staged@15.2.2 + - luxon@3.4.4 + - release-it@16.3.0 + - rimraf@5.0.5 + - ts-command-line-args@2.5.1 + - ts-jest@29.1.1 + - typescript-cubic-spline@1.0.1 + - typescript@5.2.2 + - winston@3.11.0 + - zod@3.23.8 + status: success +tree: + children: + child: + pipeline: + observe: + regroup: + compute: + - interpolate + - cpu-factor-to-wattage + - wattage-times-duration + - wattage-to-energy-kwh + - calculate-vcpu-ratio + - correct-cpu-energy-for-vcpu-ratio + defaults: + thermal-design-power: 100 + vcpus-total: 8 + vcpus-allocated: 2 + inputs: + - timestamp: 2023-08-06T00:00 + duration: 360 + cpu/utilization: 1 + carbon: 30 + - timestamp: 2023-09-06T00:00 + duration: 360 + carbon: 30 + cpu/utilization: 10 + - timestamp: 2023-10-06T00:00 + duration: 360 + carbon: 30 + cpu/utilization: 50 + - timestamp: 2023-10-06T00:00 + duration: 360 + carbon: 30 + cpu/utilization: 100 + outputs: + - timestamp: 2023-08-06T00:00 + duration: 360 + cpu/utilization: 1 + carbon: 30 + thermal-design-power: 100 + vcpus-total: 8 + vcpus-allocated: 2 + cpu-factor: 0.13999999999999999 + cpu-wattage: 13.999999999999998 + cpu-wattage-times-duration: 5039.999999999999 + cpu-energy-raw: 0.0013999999999999998 + vcpu-ratio: 4 + cpu-energy-kwh: 0.00034999999999999994 + - timestamp: 2023-09-06T00:00 + duration: 360 + carbon: 30 + cpu/utilization: 10 + thermal-design-power: 100 + vcpus-total: 8 + vcpus-allocated: 2 + cpu-factor: 0.32 + cpu-wattage: 32 + cpu-wattage-times-duration: 11520 + cpu-energy-raw: 0.0032 + vcpu-ratio: 4 + cpu-energy-kwh: 0.0008 + - timestamp: 2023-10-06T00:00 + duration: 360 + carbon: 30 + cpu/utilization: 50 + thermal-design-power: 100 + vcpus-total: 8 + vcpus-allocated: 2 + cpu-factor: 0.75 + cpu-wattage: 75 + cpu-wattage-times-duration: 27000 + cpu-energy-raw: 0.0075 + vcpu-ratio: 4 + cpu-energy-kwh: 0.001875 + - timestamp: 2023-10-06T00:00 + duration: 360 + carbon: 30 + cpu/utilization: 100 + thermal-design-power: 100 + vcpus-total: 8 + vcpus-allocated: 2 + cpu-factor: 1.02 + cpu-wattage: 102 + cpu-wattage-times-duration: 36720 + cpu-energy-raw: 0.0102 + vcpu-ratio: 4 + cpu-energy-kwh: 0.00255 +``` diff --git a/docs/reference/cli.md b/docs/reference/cli.md index 114f236..39a18bd 100644 --- a/docs/reference/cli.md +++ b/docs/reference/cli.md @@ -341,3 +341,340 @@ if-csv -m ./my-manifest.yml -p carbon ```sh if-run -m ./my-manifest.yml | if-csv -p carbon -o ./my-outdata ``` + + +## `--append` + +You can re-use a manifest file to make multiple batches of observations, appending the results to the existing outputs. The command that makes this possible is `--append`. To use `--append` you have to pass a manifest files that has **already been computed** - i.e.it already has outputs. If you do, then the newly generated outputs will be appended to the existing output data. + +The use case for this is when you want to repeatedly monitor the same resource or set of resources without changign the manifest config - you just want to grab new observations. The `--append` command allows you to do this without havign to generate lots of individual manifest files. + +### example + +With a computed manifest: + +```yaml +name: append +description: >- + a complete pipeline that starts with mocked CPU utilization data and outputs + operational carbon in gCO2eq +initialize: + plugins: + mock-observations: + path: builtin + method: MockObservations + config: + timestamp-from: '2024-03-05T00:00:04.000Z' + timestamp-to: '2024-03-05T00:00:07.000Z' + duration: 1 + components: + - name: server-1 + cloud/instance-type: Standard_E64_v3 + cloud/region: westus3 + generators: + common: + cloud/vendor: azure + randint: + cpu/energy: + min: 1 + max: 99 + mem/energy: + min: 1 + max: 99 + sum: + path: builtin + method: Sum + config: + input-parameters: + - cpu/energy + - mem/energy + output-parameter: energy +execution: + command: >- + /home/user/.npm/_npx/1bf7c3c15bf47d04/node_modules/.bin/ts-node + /home/user/Code/if/src/index.ts -m + manifests/examples/mock-cpu-util-to-carbon.yml -s + environment: + if-version: 0.4.0 + os: linux + os-version: 5.15.0-107-generic + node-version: 21.4.0 + date-time: 2024-06-18T14:18:44.864Z (UTC) + dependencies: + - '@babel/core@7.22.10' + - '@babel/preset-typescript@7.23.3' + - '@commitlint/cli@18.6.0' + - '@commitlint/config-conventional@18.6.0' + - '@grnsft/if-core@0.0.3' + - '@jest/globals@29.7.0' + - '@types/jest@29.5.8' + - '@types/js-yaml@4.0.9' + - '@types/luxon@3.4.2' + - '@types/node@20.9.0' + - axios-mock-adapter@1.22.0 + - axios@1.7.2 + - cross-env@7.0.3 + - csv-parse@5.5.6 + - csv-stringify@6.4.6 + - fixpack@4.0.0 + - gts@5.2.0 + - husky@8.0.3 + - jest@29.7.0 + - js-yaml@4.1.0 + - lint-staged@15.2.2 + - luxon@3.4.4 + - release-it@16.3.0 + - rimraf@5.0.5 + - ts-command-line-args@2.5.1 + - ts-jest@29.1.1 + - typescript-cubic-spline@1.0.1 + - typescript@5.2.2 + - winston@3.11.0 + - zod@3.22.4 + status: success +tree: + pipeline: + compute: + - mock-observations + - sum + regroup: + - cloud/region + - name + defaults: null + inputs: + - timestamp: '2024-03-05T00:00:00.000Z' + duration: 1 + name: server-1 + cloud/instance-type: Standard_E64_v3 + cloud/region: westus3 + cloud/vendor: azure + cpu/energy: 5 + mem/energy: 10 + - timestamp: '2024-03-05T00:00:01.000Z' + duration: 1 + name: server-1 + cloud/instance-type: Standard_E64_v3 + cloud/region: westus3 + cloud/vendor: azure + cpu/energy: 71 + mem/energy: 5 + - timestamp: '2024-03-05T00:00:02.000Z' + duration: 1 + name: server-1 + cloud/instance-type: Standard_E64_v3 + cloud/region: westus3 + cloud/vendor: azure + cpu/energy: 36 + mem/energy: 74 + outputs: + - timestamp: '2024-03-05T00:00:00.000Z' + duration: 1 + name: server-1 + cloud/instance-type: Standard_E64_v3 + cloud/region: westus3 + cloud/vendor: azure + cpu/energy: 5 + mem/energy: 10 + energy: 15 + - timestamp: '2024-03-05T00:00:01.000Z' + duration: 1 + name: server-1 + cloud/instance-type: Standard_E64_v3 + cloud/region: westus3 + cloud/vendor: azure + cpu/energy: 71 + mem/energy: 5 + energy: 76 + - timestamp: '2024-03-05T00:00:02.000Z' + duration: 1 + name: server-1 + cloud/instance-type: Standard_E64_v3 + cloud/region: westus3 + cloud/vendor: azure + cpu/energy: 36 + mem/energy: 74 + energy: 110 +``` + +run + +```sh +npm run if-run -- -m manifests/outputs/features/append.yaml -o manifests/outputs/features/re-append --append +``` + +And see the following output (with new observations appended to old observations): + +```yaml +name: append +description: >- + a complete pipeline that starts with mocked CPU utilization data and outputs + operational carbon in gCO2eq +initialize: + plugins: + mock-observations: + path: builtin + method: MockObservations + config: + timestamp-from: '2024-03-05T00:00:04.000Z' + timestamp-to: '2024-03-05T00:00:07.000Z' + duration: 1 + components: + - name: server-1 + cloud/instance-type: Standard_E64_v3 + cloud/region: westus3 + generators: + common: + cloud/vendor: azure + randint: + cpu/energy: + min: 1 + max: 99 + mem/energy: + min: 1 + max: 99 + sum: + path: builtin + method: Sum + config: + input-parameters: + - cpu/energy + - mem/energy + output-parameter: energy +execution: + command: >- + /Users/jcrowley/.npm/_npx/1bf7c3c15bf47d04/node_modules/.bin/ts-node + /Users/jcrowley/Development/gsf/if/src/if-run/index.ts -m + manifests/outputs/features/append.yaml -o + manifests/outputs/features/re-append --append + environment: + if-version: 0.6.0 + os: macOS + os-version: 14.6.1 + node-version: 20.16.0 + date-time: 2024-09-04T01:05:58.758Z (UTC) + dependencies: + - '@babel/core@7.22.10' + - '@babel/preset-typescript@7.23.3' + - '@commitlint/cli@18.6.0' + - '@commitlint/config-conventional@18.6.0' + - '@grnsft/if-core@0.0.16' + - '@jest/globals@29.7.0' + - '@types/jest@29.5.8' + - '@types/js-yaml@4.0.9' + - '@types/luxon@3.4.2' + - '@types/node@20.9.0' + - axios-mock-adapter@1.22.0 + - axios@1.7.2 + - cross-env@7.0.3 + - csv-parse@5.5.6 + - csv-stringify@6.4.6 + - fixpack@4.0.0 + - gts@5.2.0 + - husky@8.0.3 + - jest@29.7.0 + - js-yaml@4.1.0 + - lint-staged@15.2.2 + - luxon@3.4.4 + - release-it@16.3.0 + - rimraf@5.0.5 + - ts-command-line-args@2.5.1 + - ts-jest@29.1.1 + - typescript-cubic-spline@1.0.1 + - typescript@5.2.2 + - winston@3.11.0 + - zod@3.23.8 + status: success +tree: + pipeline: + compute: + - mock-observations + - sum + regroup: + - cloud/region + - name + defaults: null + children: + westus3: + children: + server-1: + inputs: + - timestamp: '2024-03-05T00:00:00.000Z' + duration: 1 + name: server-1 + cloud/instance-type: Standard_E64_v3 + cloud/region: westus3 + cloud/vendor: azure + cpu/energy: 5 + mem/energy: 10 + - timestamp: '2024-03-05T00:00:01.000Z' + duration: 1 + name: server-1 + cloud/instance-type: Standard_E64_v3 + cloud/region: westus3 + cloud/vendor: azure + cpu/energy: 71 + mem/energy: 5 + - timestamp: '2024-03-05T00:00:02.000Z' + duration: 1 + name: server-1 + cloud/instance-type: Standard_E64_v3 + cloud/region: westus3 + cloud/vendor: azure + cpu/energy: 36 + mem/energy: 74 + outputs: + - timestamp: '2024-03-05T00:00:00.000Z' + duration: 1 + name: server-1 + cloud/instance-type: Standard_E64_v3 + cloud/region: westus3 + cloud/vendor: azure + cpu/energy: 5 + mem/energy: 10 + energy: 15 + - timestamp: '2024-03-05T00:00:01.000Z' + duration: 1 + name: server-1 + cloud/instance-type: Standard_E64_v3 + cloud/region: westus3 + cloud/vendor: azure + cpu/energy: 71 + mem/energy: 5 + energy: 76 + - timestamp: '2024-03-05T00:00:02.000Z' + duration: 1 + name: server-1 + cloud/instance-type: Standard_E64_v3 + cloud/region: westus3 + cloud/vendor: azure + cpu/energy: 36 + mem/energy: 74 + energy: 110 + - timestamp: '2024-03-05T00:00:04.000Z' + duration: 1 + name: server-1 + cloud/instance-type: Standard_E64_v3 + cloud/region: westus3 + cloud/vendor: azure + cpu/energy: 2 + mem/energy: 26 + energy: 28 + - timestamp: '2024-03-05T00:00:05.000Z' + duration: 1 + name: server-1 + cloud/instance-type: Standard_E64_v3 + cloud/region: westus3 + cloud/vendor: azure + cpu/energy: 67 + mem/energy: 27 + energy: 94 + - timestamp: '2024-03-05T00:00:06.000Z' + duration: 1 + name: server-1 + cloud/instance-type: Standard_E64_v3 + cloud/region: westus3 + cloud/vendor: azure + cpu/energy: 88 + mem/energy: 6 + energy: 94 +``` diff --git a/docs/reference/errors.md b/docs/reference/errors.md index 023837e..82b4e1e 100644 --- a/docs/reference/errors.md +++ b/docs/reference/errors.md @@ -31,11 +31,11 @@ The remedy for this issue is to add an `initialize` block into the manifest. ### `InvalidGroupingError` -Errors of the `InvalidGroupingError` are only emitted by the `group-by` plugin. There is only one associated message; it is emitted when the requested groups do not exist in the tree. +Errors of the `InvalidGroupingError` are only emitted by the `regroup` feature. There is only one associated message; it is emitted when the requested groups do not exist in the tree. -| message | cause | remedy | -| ------------------------ | ------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------- | -| `Invalid group ${type}.` | you are requested groupby to regroup the tree based on fields that do not exist | Check the spelling of the values passed to `groupby` and ensure the values exist in the tree | +| message | cause | remedy | +| ------------------------ | ----------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------- | +| `Invalid group ${type}.` | you are requested the feature to regroup the tree based on fields that do not exist | Check the spelling of the values passed to `regroup` and ensure the values exist in the tree | ### `WriteFileError` @@ -122,22 +122,22 @@ Errors of the `InvalidExhaustPluginError` class are caused by using unsupported Plugins can emit their own custom error messages, but we still prefer those messages to be attached to one of a finite set of predefined error classes. Those classes are listed in this section. -### `GlobalConfigError` +### `ConfigError` -Errors of the `GlobalConfigError` are used when part of the config data provided to a plugin is invalid or missing. +Errors of the `ConfigError` are used when part of the config data provided to a plugin is invalid or missing. -For example the `Divide` plugin throws a `GlobalConfigError` when it receives a denominator equal to zero. +For example the `Divide` plugin throws a `ConfigError` when it receives a denominator equal to zero. The message should name the config element that was invalid and describe the reason why. For example: -`GlobalConfigError: "denominator" parameter is number must be greater than 0. Error code: too_small.` +`ConfigError: "denominator" parameter is number must be greater than 0. Error code: too_small.` ### `MissingInputDataError` -Errors of the `MissingInputDataError` class arise because your plugin is not receiving the data it expects in `input` data, global config or node-level config. +Errors of the `MissingInputDataError` class arise because your plugin is not receiving the data it expects in `input` data or config. The specific messages depend on the plugin. It is expected that the messages emitted by each plugin are listed in their own documentation. -The example below is a message emitted by the `interpolation` plugin when the `method` given in global config is _not_ one of the expected enum variants: +The example below is a message emitted by the `interpolation` plugin when the `method` given in config is _not_ one of the expected enum variants: `MissingInputDataError: "interpolation" parameter is invalid enum value. expected 'spline' | 'linear', received 'dummy'. Error code: invalid_enum_value.` @@ -228,29 +228,43 @@ When you run a [manifest](../major-concepts/manifest-file.md), IF generates outp For example, the following is an output file generated by running a manifest whose `input` data omitted the required `duration` field: ```yaml -name: basic-error-demo -description: +name: input-error-missing-duration +description: >- + a negative test case that fails due to the required `duration` field being + omitted from input data tags: initialize: plugins: - teads-curve: - path: '@grnsft/if-unofficial-plugins' - method: TeadsCurve - global-config: - interpolation: spline + interpolate: + method: Interpolation + path: builtin + config: + method: linear + x: + - 0 + - 10 + - 50 + - 100 + 'y': + - 0.12 + - 0.32 + - 0.75 + - 1.02 + input-parameter: cpu/utilization + output-parameter: cpu-factor execution: status: fail - error: 'InputValidationError: "duration" parameter is required. Error code: invalid_type'. + error: >- + InputValidationError: "duration" parameter is required at index 0. Error + code: invalid_type. tree: children: child-0: defaults: cpu/thermal-design-power: 100 pipeline: - observe: - regroup: compute: - - teads-curve + - interpolate inputs: - timestamp: 2023-07-06T00:00 cpu/utilization: 20 diff --git a/docs/reference/features.md b/docs/reference/features.md index 70fac30..5dde001 100644 --- a/docs/reference/features.md +++ b/docs/reference/features.md @@ -38,24 +38,145 @@ You can override the parameter metadata provided in a plugin's source code by ad ```yaml explainer: true plugins: - "sum-carbon": - path: "builtin" - method: Sum - global-config: + 'sum-carbon': + path: 'builtin' + method: Sum + config: input-parameters: - - carbon-operational - - carbon-embodied + - carbon-operational + - carbon-embodied output-parameter: carbon - parameter-metadata: + parameter-metadata: inputs: - carbon-operational: + carbon-operational: description: "carbon emitted due to an application's execution" - unit: "gCO2eq" - aggregation-method: 'sum', - carbon-embodied: + unit: 'gCO2eq' + aggregation-method: + time: sum + component: sum, + carbon-embodied: description: "carbon emitted during the production, distribution and disposal of a hardware component, scaled by the fraction of the component's lifespan being allocated to the application under investigation" - unit: "gCO2eq" - aggregation-method: 'sum' + unit: 'gCO2eq' + aggregation-method: + time: sum + component: sum ``` Read more on [explainer](../users/how-to-use-the-explain-feature.md) + +## Inline Arithmetic Expressions + +Inline arithmetic expressions allow basic mathematical operations to be embedded directly within `config` parameters and `inputs` values in manifest files. This enables dynamic calculations using constants or input variables, eliminating the need for manual pre-calculation of parameters. + +### Supported Symbols and Operations: + +- `=`: Indicates the start of an arithmetic expression. +- Supported operators: `*` (multiplication), `+` (addition), `-` (subtraction), `/` (division). + +### Syntax: + +- To define an inline arithmetic expression, the string must start with an equal sign (`=`). For example: + ```yaml + 'input-parameter': '= 2 * carbon' + ``` + This expression evaluates the multiplication of `2` by the value of the `carbon` parameter from the input. +- Arithmetic operations between two constants can also be defined without using the equal sign (`=`): + ```yaml + coefficient: 2 * 2 + ``` + This expression evaluates the multiplication of `2` by `2` directly. +- If the parameter name contains symbols, it should be placed in the quotes. The expresion should look like: + ```yaml + output-parameter: '= 2 * "carbon-product"' + ``` + +### Example: + +```yaml +config: + 'input-parameter': '= 2 * carbon' + coefficient: 2 * 2 + 'output-parameter': '= 2 * "carbon-product"' +--- +inputs: + - timestamp: 2023-08-06T00:00 + duration: 3600 * 60 + carbon: = 10 * "other-param + other-param: 3 +``` + +### Plugin support + +To enable inline arithmetic expressions in your plugin, specify it in your plugin’s definition function like this: + +```ts +allowArithmeticExpressions: ['input-parameter']; +``` + +In the `allowArithmeticExpressions` array, list all parameters (whether in config, inputs, or outputs) that can contain arithmetic expressions. The calculations are handled internally (in the PluginFactory interface). + +If your plugin doesn’t have specified parameters but has dynamic output parameters that should support evaluation, you can enable `arithmeticExpressions` with an empty array: + +```ts +allowArithmeticExpressions: []; +``` + +To design your plugin with support for arithmetic expressions, you can use various utility functions. + +- If your plugin's config parameters must be of type `number`, you can use the `validateArithmeticExpression` function from `@grnsft/if-core/utils`: + +```ts +import {validateArithmeticExpression} from '@grnsft/if-core/utils'; + +// Plugin definition + +configValidation: (config: ConfigParams) => { + const configSchema = z.object({ + coefficient: z.preprocess( + value => validateArithmeticExpression('coefficient', value, 'number'), + z.number() + ), + 'input-parameter': z.string().min(1), + 'output-parameter': z.string().min(1), + }); + + return validate>( + configSchema as ZodType, + config + ); + }, +``` + +- If your config parameters contain arithmetic expressions like the following: + +```yaml +config: + keep-existing: false + from: = 4 * "if-size" + to: 'if-repo-size' +``` + +But during implementation, you need to extract the pure parameter name (e.g., `if-size`), you can use the `getParameterFromArithmeticExpression` function: + +```ts +import { getParameterFromArithmeticExpression } from '@grnsft/if-core/utils'; + +// Plugin definition + +configValidation: (config: ConfigParams) => { + const configSchema = z.object({ + 'keep-existing': z.boolean(), + from: z.string().min(1), + to: z.string().min(1), + }); + + const extractedFrom = getParameterFromArithmeticExpression(config.from); + const updatedConfig = config['keep-existing'] + ? config + : { ...config, 'pure-from': extractedFrom }; + + validate>(configSchema, updatedConfig); + + return updatedConfig; +}; +``` diff --git a/docs/reference/plugins.md b/docs/reference/plugins.md index 5bbac94..788086e 100644 --- a/docs/reference/plugins.md +++ b/docs/reference/plugins.md @@ -18,8 +18,6 @@ IF builtins all come bundled with IF. Below you will find a list of each builtin - [Time Sync](https://github.com/Green-Software-Foundation/if/tree/main/src/builtins#readme): Takes a heterogeneous set of time series data that might be offset, discontinuous or irregularly spaces and returns time series conforming to a user defined time grid. E.g. a user can define that all sets of observations should start at some global start time, end at some global end time and have a specific temporal resolution. -- [Groupby](https://github.com/Green-Software-Foundation/if/tree/main/src/builtins#readme): Allows a user to regroup their output data according to given keys. - - [SCI-embodied](https://github.com/Green-Software-Foundation/if/tree/main/src/builtins/sci-embodied) - Calculates the embodied carbon for a component. - [SCI](https://github.com/Green-Software-Foundation/if/tree/main/src/builtins/sci): Calculates the software carbon intensity. diff --git a/docs/users/how-to-compare-files-with-if-diff.md b/docs/users/how-to-compare-files-with-if-diff.md index 7019820..a25de63 100644 --- a/docs/users/how-to-compare-files-with-if-diff.md +++ b/docs/users/how-to-compare-files-with-if-diff.md @@ -25,7 +25,7 @@ initialize: sum: method: Sum path: 'builtin' - global-config: + config: input-parameters: ['cpu/energy', 'network/energy'] output-parameter: 'energy' tree: @@ -36,8 +36,6 @@ tree: regroup: compute: - sum - config: - sum: inputs: - timestamp: 2023-08-06T00:00 duration: 3600 @@ -61,8 +59,8 @@ initialize: plugins: sum: method: Sum - path: "builtin" - global-config: + path: 'builtin' + config: input-parameters: ['cpu/energy', 'network/energy'] output-parameter: 'energy' tree: @@ -73,8 +71,6 @@ tree: regroup: compute: - sum - config: - sum: inputs: - timestamp: 2023-08-06T00:00 duration: 3600 diff --git a/docs/users/how-to-export-csv-file-with-if-csv.md b/docs/users/how-to-export-csv-file-with-if-csv.md index a40a4b3..900729f 100644 --- a/docs/users/how-to-export-csv-file-with-if-csv.md +++ b/docs/users/how-to-export-csv-file-with-if-csv.md @@ -19,34 +19,28 @@ initialize: sum: path: builtin method: Sum - global-config: + config: input-parameters: - cpu/energy - network/energy output-parameter: energy - outputs: - - yaml execution: command: >- /Users/manushak/.npm/_npx/1bf7c3c15bf47d04/node_modules/.bin/ts-node /Users/manushak/Documents/Projects/Green-Software/if/src/if-run/index.ts -m - ./manifests/test.yaml -o ./manifests/re-test + manifests/examples/test.yaml environment: - if-version: 0.5.0 + if-version: 0.6.0 os: macOS - os-version: 13.6.7 - node-version: 18.20.0 - date-time: 2024-07-09T16:00:58.218Z (UTC) + os-version: 14.6.1 + node-version: 18.20.4 + date-time: 2024-10-03T15:23:26.460Z (UTC) dependencies: - '@babel/core@7.22.10' - '@babel/preset-typescript@7.23.3' - '@commitlint/cli@18.6.0' - '@commitlint/config-conventional@18.6.0' - - '@grnsft/if-core@0.0.10' - - '@grnsft/if-plugins@v0.3.2 extraneous -> file:../../../if-models' - - >- - @grnsft/if-unofficial-plugins@v0.3.0 extraneous -> - file:../../../if-unofficial-models + - '@grnsft/if-core@0.0.25' - '@jest/globals@29.7.0' - '@types/jest@29.5.8' - '@types/js-yaml@4.0.9' @@ -71,7 +65,7 @@ execution: - typescript-cubic-spline@1.0.1 - typescript@5.2.2 - winston@3.11.0 - - zod@3.22.4 + - zod@3.23.8 status: success tree: children: @@ -81,8 +75,6 @@ tree: regroup: compute: - sum - config: - sum: null inputs: - timestamp: 2023-08-06T00:00 duration: 3600 diff --git a/docs/users/how-to-import-plugins.md b/docs/users/how-to-import-plugins.md index 6be1e0b..fccc75f 100644 --- a/docs/users/how-to-import-plugins.md +++ b/docs/users/how-to-import-plugins.md @@ -4,7 +4,7 @@ sidebar_position: 3 # How to load plugins -Plugins are developed separately to the Impact Framework core. However, the IF core developers maintain a standard library of plugins come bundled with IF. These are known as `builtins`. +Plugins are developed separately to the Impact Framework core. However, the IF core developers maintain a standard library of plugins come bundled with IF. These are known as `builtins`. Builtins have to be initialized in a manifest file using the path `builtin`. Then they can be invoked in pipelines. @@ -14,17 +14,16 @@ description: demo pipeline tags: initialize: plugins: - "sum": - path: "builtin" + 'sum': + path: 'builtin' method: Sum - global-config: + config: input-parameters: - cpu/energy - network/energy output-parameter: energy-sum ``` - Other plugins are hosted externally to the IF. Anyone can build a plugin and provide it as an npm package or a public code repository (such as Github) and share it using our [Explorer](https://explorer.if.greensoftware.foundation). These external plugins are loaded into IF by installing locally and initializing in a manifest. @@ -41,7 +40,7 @@ Then, in the manifest's `initialize` section, you'll need to provide the followi - `method`: the function name exported by your plugin, e.g. `AzureImporter` - `path`: the path to the plugin -And, if your plugin requires it, add its `global-config` too. +And, if your plugin requires it, add its `config` too. ```yaml name: plugin-demo diff --git a/docs/users/how-to-use-the-explain-feature.md b/docs/users/how-to-use-the-explain-feature.md index cc13465..f71640f 100644 --- a/docs/users/how-to-use-the-explain-feature.md +++ b/docs/users/how-to-use-the-explain-feature.md @@ -8,19 +8,15 @@ Manifest files can get complicated, especially when there are many plugin instan `explainer` adds a block to the manifest that simply lists the parameter metadata used be the plugin's instance in the manifest. The metadata contains: -- **method:** the function name being executed by the plugin -- **path**: the import path for the plugin -- **inputs**: a list of each input parameter -- **outputs**: a list of each output parameter - -Each element in `inputs` and `outputs` contains the following information about each specific parameter: - +- **plugins:** the list of plugins where the parameter is used +- **unit**: the unit in which the parameter is expressed - **description:** a plain-language summary of the parameter -- **unit:**: The unit the parameter is expressed in - **aggregation-method:**: The appropriate method to use when aggregating the parameter across time or components (e.g. should it be summed, averaged, or held constant) This information allows you to check that the units output by one plugin are consistent with those expected as inputs to another, in one clear itemized list in your output manifest. +Note that when the parameter has different units across instances, an error will occur. + ## Toggling `explainer` on or off To enable the `explainer` feature, add the following line to your manifest, somewhere in the manifest context (e.g. above the `plugins` block): @@ -35,64 +31,81 @@ If you set `explainer` to `false` or omit the line altogether, the `explainer` f Plugins are expected to ship with default values for their parameter metadata in their source code. For example, our plugin for calculating embodied carbon, `SciEmbodied`, includes the following metadata definition: -```Typescript -export const SciEmbodied = ( - config: ConfigParams = {}, - parametersMetadata: PluginParametersMetadata, - mapping: MappingParams -): ExecutePlugin => { - const metadata = { - kind: 'execute', +```ts +export const SciEmbodied = PluginFactory({ + metadata: { inputs: { - ...({ - vCPUs: { - description: 'number of CPUs allocated to an application', - unit: 'CPUs', - 'aggregation-method': 'copy', + vCPUs: { + description: 'number of CPUs allocated to an application', + unit: 'CPUs', + 'aggregation-method': { + time: 'copy', + component: 'copy', }, - memory: { - description: 'RAM available for a resource, in GB', - unit: 'GB', - 'aggregation-method': 'copy', + }, + memory: { + description: 'RAM available for a resource, in GB', + unit: 'GB', + 'aggregation-method': { + time: 'copy', + component: 'copy', }, - ssd: { - description: 'number of SSDs available for a resource', - unit: 'SSDs', - 'aggregation-method': 'copy', + }, + ssd: { + description: 'number of SSDs available for a resource', + unit: 'SSDs', + 'aggregation-method': { + time: 'copy', + component: 'copy', }, - hdd: { - description: 'number of HDDs available for a resource', - unit: 'HDDs', - 'aggregation-method': 'copy', + }, + hdd: { + description: 'number of HDDs available for a resource', + unit: 'HDDs', + 'aggregation-method': { + time: 'copy', + component: 'copy', }, - gpu: { - description: 'number of GPUs available for a resource', - unit: 'GPUs', - 'aggregation-method': 'copy', + }, + gpu: { + description: 'number of GPUs available for a resource', + unit: 'GPUs', + 'aggregation-method': { + time: 'copy', + component: 'copy', }, - 'usage-ratio': { - description: - 'a scaling factor that can be used to describe the ratio of actual resource usage comapred to real device usage, e.g. 0.25 if you are using 2 out of 8 vCPUs, 0.1 if you are responsible for 1 out of 10 GB of storage, etc', - unit: 'dimensionless', - 'aggregation-method': 'copy', + }, + 'usage-ratio': { + description: + 'a scaling factor that can be used to describe the ratio of actual resource usage comapred to real device usage, e.g. 0.25 if you are using 2 out of 8 vCPUs, 0.1 if you are responsible for 1 out of 10 GB of storage, etc', + unit: 'dimensionless', + 'aggregation-method': { + time: 'copy', + component: 'copy', }, - time: { - description: - 'a time unit to scale the embodied carbon by, in seconds. If not provided,time defaults to the value of the timestep duration.', - unit: 'seconds', - 'aggregation-method': 'copy', + }, + time: { + description: + 'a time unit to scale the embodied carbon by, in seconds. If not provided,time defaults to the value of the timestep duration.', + unit: 'seconds', + 'aggregation-method': { + time: 'copy', + component: 'copy', }, - } as ParameterMetadata), - ...parametersMetadata?.inputs, + }, }, - outputs: parametersMetadata?.outputs || { + outputs: { 'embodied-carbon': { description: 'embodied carbon for a resource, scaled by usage', unit: 'gCO2e', - 'aggregation-method': 'sum', + 'aggregation-method': { + time: 'sum', + component: 'sum', + }, }, }, - }; + }, +}); ``` However, there are cases where a plugin might not have parameter metadata in its source code, either because it was omitted, it was not knowable in advance, or the plugin was built before we shipped the `explain` feature. Sometimes, you might want to override the hard-coded defaults and use alternative metadata. In these cases, you can define new plugin metadata in the manifest file. It is considered best-practice to ensure all plugin instances have a complete set of plugin metadata. @@ -102,10 +115,10 @@ Setting parameter metadata from the manifest file is done in the plugin instance ```yaml initialize: plugins: - 'interpolate': + interpolate: method: Interpolation path: 'builtin' - global-config: + config: method: linear x: [0, 10, 50, 100] y: [0.12, 0.32, 0.75, 1.02] @@ -116,12 +129,16 @@ initialize: cpu/utilization: description: 'portion of the total CPU capacity being used by an application' unit: 'percentage' - aggregation-method: 'avg' + aggregation-method: + time: avg + component: avg outputs: cpu-factor: description: "a dimensionless intermediate used to scale a processor's thermal design power by CPU usage" unit: 'dimensionless' - aggregation-method: 'avg' + aggregation-method: + time: avg + component: avg ``` ## Example manifest @@ -153,16 +170,22 @@ initialize: carbon-operational: description: "carbon emitted due to an application's execution" unit: 'gCO2eq' - aggregation-method: 'sum' + aggregation-method: + time: sum + component: sum embodied-carbon: description: "carbon emitted during the production, distribution and disposal of a hardware component, scaled by the fraction of the component's lifespan being allocated to the application under investigation" unit: 'gCO2eq' - aggregation-method: 'sum' + aggregation-method: + time: sum + component: sum outputs: carbon: description: "total carbon emissions attributed to an application's usage as the sum of embodied and operational carbon" unit: 'gCO2eq' - aggregation-method: 'sum' + aggregation-method: + time: sum + component: sum sci: kind: plugin method: Sci @@ -174,16 +197,22 @@ initialize: carbon: description: "total carbon emissions attributed to an application's usage as the sum of embodied and operational carbon" unit: 'gCO2eq' - aggregation-method: 'sum' + aggregation-method: + time: sum + component: sum requests: description: 'number of requests made to application in the given timestep' unit: 'requests' - aggregation-method: 'sum' + aggregation-method: + time: sum + component: sum outputs: sci: description: 'software carbon intensity expressed as a rate of carbon emission per request' unit: 'gCO2eq/request' - aggregation-method: 'sum' + aggregation-method: + time: sum + component: sum tree: children: child: @@ -210,98 +239,116 @@ When we execute this manifest, the following `explain` block is added to the out ```yaml explain: - sci-embodied: - method: SciEmbodied - path: builtin - inputs: - vCPUs: - description: number of CPUs allocated to an application - unit: CPUs - aggregation-method: copy - memory: - description: RAM available for a resource, in GB - unit: GB - aggregation-method: copy - ssd: - description: number of SSDs available for a resource - unit: SSDs - aggregation-method: copy - hdd: - description: number of HDDs available for a resource - unit: HDDs - aggregation-method: copy - gpu: - description: number of GPUs available for a resource - unit: GPUs - aggregation-method: copy - usage-ratio: - description: >- - a scaling factor that can be used to describe the ratio of actual - resource usage comapred to real device usage, e.g. 0.25 if you are - using 2 out of 8 vCPUs, 0.1 if you are responsible for 1 out of 10 GB - of storage, etc - unit: dimensionless - aggregation-method: copy - time: - description: >- - a time unit to scale the embodied carbon by, in seconds. If not - provided,time defaults to the value of the timestep duration. - unit: seconds - aggregation-method: copy - outputs: - embodied-carbon: - description: embodied carbon for a resource, scaled by usage - unit: gCO2e - aggregation-method: sum - sum-carbon: - method: Sum - path: builtin - inputs: - carbon-operational: - unit: gCO2eq - description: carbon emitted due to an application's execution - aggregation-method: sum - embodied-carbon: - unit: gCO2eq - description: >- - carbon emitted during the production, distribution and disposal of a - hardware component, scaled by the fraction of the component's lifespan - being allocated to the application under investigation - aggregation-method: sum - outputs: - carbon: - unit: gCO2eq - description: >- - total carbon emissions attributed to an application's usage as the sum - of embodied and operational carbon - aggregation-method: sum + vCPUs: + plugins: + - sci-embodied + unit: CPUs + description: number of CPUs allocated to an application + aggregation-method: + time: copy + component: copy + memory: + plugins: + - sci-embodied + unit: GB + description: RAM available for a resource, in GB + aggregation-method: + time: copy + component: copy + ssd: + plugins: + - sci-embodied + unit: SSDs + description: number of SSDs available for a resource + aggregation-method: + time: copy + component: copy + hdd: + plugins: + - sci-embodied + unit: HDDs + description: number of HDDs available for a resource + aggregation-method: + time: copy + component: copy + gpu: + plugins: + - sci-embodied + unit: GPUs + description: number of GPUs available for a resource + aggregation-method: + time: copy + component: copy + usage-ratio: + plugins: + - sci-embodied + unit: dimensionless + description: >- + a scaling factor that can be used to describe the ratio of actual resource + usage comapred to real device usage, e.g. 0.25 if you are using 2 out of 8 + vCPUs, 0.1 if you are responsible for 1 out of 10 GB of storage, etc + aggregation-method: + time: copy + component: copy + time: + plugins: + - sci-embodied + unit: seconds + description: >- + a time unit to scale the embodied carbon by, in seconds. If not + provided,time defaults to the value of the timestep duration. + aggregation-method: + time: copy + component: copy + embodied-carbon: + plugins: + - sci-embodied + - sum-carbon + unit: gCO2eq + description: >- + carbon emitted during the production, distribution and disposal of a + hardware component, scaled by the fraction of the component's lifespan + being allocated to the application under investigation + aggregation-method: + time: sum + component: sum + carbon-operational: + plugins: + - sum-carbon + unit: gCO2eq + description: carbon emitted due to an application's execution + aggregation-method: + time: sum + component: sum + carbon: + plugins: + - sum-carbon + - sci + unit: gCO2eq + description: >- + total carbon emissions attributed to an application's usage as the sum of + embodied and operational carbon + aggregation-method: + time: sum + component: sum + requests: + plugins: + - sci + unit: requests + description: number of requests made to application in the given timestep + aggregation-method: + time: sum + component: sum sci: - method: Sci - path: builtin - inputs: - carbon: - unit: gCO2eq - description: >- - total carbon emissions attributed to an application's usage as the sum - of embodied and operational carbon - aggregation-method: sum - functional-unit: - description: >- - the name of the functional unit in which the final SCI value should be - expressed, e.g. requests, users - unit: none - aggregation-method: sum - requests: - unit: requests - description: number of requests made to application in the given timestep - aggregation-method: sum - outputs: - sci: - unit: gCO2eq/request - description: >- - software carbon intensity expressed as a rate of carbon emission per - request - aggregation-method: sum + plugins: + - sci + unit: gCO2eq/request + description: >- + software carbon intensity expressed as a rate of carbon emission per + request + aggregation-method: + time: sum + component: sum ``` ## When _not_ to use `explainer` diff --git a/docs/users/how-to-verify-files-with-if-check.md b/docs/users/how-to-verify-files-with-if-check.md index 316ad51..f89cbf0 100644 --- a/docs/users/how-to-verify-files-with-if-check.md +++ b/docs/users/how-to-verify-files-with-if-check.md @@ -2,7 +2,6 @@ sidebar_position: 6 --- - # Verifying IF outputs with `if-check` IF includes a command line tool called `if-check` that can be used to verify the results in a manifest file. @@ -30,7 +29,7 @@ initialize: sci: path: builtin method: Sci - global-config: + config: functional-unit: requests execution: command: >- @@ -124,7 +123,6 @@ tree: sci: 0.0802 ``` - Alice runs : ``` @@ -137,10 +135,8 @@ And receives the response: if-check: successfully verified bobs-manifest ``` - Charlie also has a copy of Bob's manifest. He wants to trick Alice into thinking his SCI score is lower, so he overwrites the values in the manifest file, making them lower. Charlie's manifest looks like this: - ```yaml # start name: charlies-manifest @@ -151,7 +147,7 @@ initialize: sci: path: builtin method: Sci - global-config: + config: functional-unit: requests execution: command: >- @@ -268,7 +264,6 @@ if-check -d /my-folder-of-manifests Each manifest will be run through `if-check` in sequence. - ## `if-check` limitations -`if-check` can verify that a manifest is correctly calculated. However, if someone really wanted to use a fraudulent manifest, they could provide fraudulent *input* data not *output* data. There's little we can really do about this - if someone provides fake input data it is out of IF's remit. This means that although the examples above are good for demonstrating how `if-check` works, it's more likely to be used to check for bugs and configuration errors than it is to be used to detect fraud. +`if-check` can verify that a manifest is correctly calculated. However, if someone really wanted to use a fraudulent manifest, they could provide fraudulent _input_ data not _output_ data. There's little we can really do about this - if someone provides fake input data it is out of IF's remit. This means that although the examples above are good for demonstrating how `if-check` works, it's more likely to be used to check for bugs and configuration errors than it is to be used to detect fraud. diff --git a/docs/users/how-to-write-manifests.md b/docs/users/how-to-write-manifests.md index 1916c59..4379e9b 100644 --- a/docs/users/how-to-write-manifests.md +++ b/docs/users/how-to-write-manifests.md @@ -42,7 +42,7 @@ tags: ### Initialize -The `initialize` fields are where you specify each individual plugin that will be initialized in your pipeline. The plugins can be initialized in any order, but can only be invoked elsewhere in the manifest if they have been initialized first here. In each case, you will need to provide the `name`, `path` and `method` (and `global-config` if your plugin requires it): +The `initialize` fields are where you specify each individual plugin that will be initialized in your pipeline. The plugins can be initialized in any order, but can only be invoked elsewhere in the manifest if they have been initialized first here. In each case, you will need to provide the `name`, `path` and `method` (and `config` if your plugin requires it): ```yaml initialize: @@ -124,7 +124,7 @@ initialize: 'interpolate': method: Interpolation path: 'builtin' - global-config: + config: method: linear x: [0, 10, 50, 100] y: [0.12, 0.32, 0.75, 1.02] @@ -135,14 +135,20 @@ initialize: cpu/utilization: description: refers to CPU utilization unit: percentage + aggregation-method: + time: avg + component: avg outputs: cpu-factor: description: the factor of cpu unit: kWh + aggregation-method: + time: avg + component: avg 'cpu-factor-to-wattage': method: Multiply path: builtin - global-config: + config: input-parameters: ['cpu-factor', 'cpu/thermal-design-power'] output-parameter: 'cpu-wattage' parameter-metadata: @@ -150,37 +156,46 @@ initialize: cpu-factor: description: the factor of cpu unit: kWh + aggregation-method: + time: avg + component: avg cpu/thermal-design-power: description: thermal design power for a processor unit: kwh + aggregation-method: + time: avg + component: avg outputs: cpu-wattage: description: cpu in Wattage unit: wattage + aggregation-method: + time: sum + component: sum 'wattage-times-duration': method: Multiply path: builtin - global-config: + config: input-parameters: ['cpu-wattage', 'duration'] output-parameter: 'cpu-wattage-times-duration' 'wattage-to-energy-kwh': method: Divide path: 'builtin' - global-config: + config: numerator: cpu-wattage-times-duration denominator: 3600000 output: cpu-energy-raw 'calculate-vcpu-ratio': method: Divide path: 'builtin' - global-config: + config: numerator: vcpus-total denominator: vcpus-allocated output: vcpu-ratio 'correct-cpu-energy-for-vcpu-ratio': method: Divide path: 'builtin' - global-config: + config: numerator: cpu-energy-raw denominator: vcpu-ratio output: cpu-energy-kwh @@ -190,13 +205,13 @@ initialize: 'operational-carbon': method: Multiply path: builtin - global-config: + config: input-parameters: ['cpu-energy-kwh', 'grid/carbon-intensity'] output-parameter: 'carbon-operational' 'sci': path: 'builtin' method: Sci - global-config: + config: functional-unit-time: 1 sec functional-unit: requests # factor to convert per time to per f.unit parameter-metadata: @@ -204,17 +219,26 @@ initialize: carbon: description: an amount of carbon emitted into the atmosphere unit: gCO2e + aggregation-method: + time: sum + component: sum requests: description: factor to convert per time to per f.unit unit: number + aggregation-method: + time: sum + component: sum outputs: sci: description: carbon expressed in terms of the given functional unit unit: gCO2e + aggregation-method: + time: avg + component: sum 'sum-carbon': path: 'builtin' method: Sum - global-config: + config: input-parameters: - carbon-operational - embodied-carbon @@ -222,7 +246,7 @@ initialize: 'time-sync': method: TimeSync path: 'builtin' - global-config: + config: start-time: '2023-12-12T00:00:00.000Z' end-time: '2023-12-12T00:01:00.000Z' interval: 5 @@ -233,6 +257,8 @@ tree: pipeline: observe: regroup: + - cloud/region + - cloud/instance-type compute: - interpolate - cpu-factor-to-wattage @@ -245,11 +271,6 @@ tree: - sum-carbon - time-sync # - sci - config: - group-by: - group: - - cloud/region - - cloud/instance-type defaults: cpu/thermal-design-power: 100 grid/carbon-intensity: 800 @@ -309,10 +330,11 @@ The recommended method for integrating data is to use the plugin system of the I There are already some community plugins available, including plugins for fetching data from Kubernetes, GCP, and third-party data aggregators like Datadog. -If there is no fitting plugin available yet, we encourage you to write and add one for your specific use case. See [developer documentation](./developers/) for more information on how to build a plugin. There is a [Azure-Importer](https://github.com/Green-Software-Foundation/if-unofficial-plugins/blob/main/src/lib/azure-importer/README.md) you can as a prototype and starting point for your own development. -If you already have external scripts you might have a look at the [shell plugin](https://github.com/Green-Software-Foundation/if-plugins/blob/main/src/lib/shell/README.md) to integrate them with the Impact Framework. +If there is no fitting plugin available yet, we encourage you to write and add one for your specific use case. See [developer documentation](./developers/) for more information on how to build a plugin. + +If you already have external scripts you might have a look at the [shell plugin](https://github.com/Green-Software-Foundation/if/blob/main/src/if-run/builtins/shell/README.md) to integrate them with the Impact Framework. -If you just need data for testing purposes, you can use the [mock-observation](https://github.com/Green-Software-Foundation/if-plugins/blob/main/src/lib/mock-observations/README.md) plugin. +If you just need data for testing purposes, you can use the [mock-observation](https://github.com/Green-Software-Foundation/if/blob/main/src/if-run/builtins/mock-observations/README.md) plugin. ## Running a manifest diff --git a/docs/users/quick-start.md b/docs/users/quick-start.md index 99af968..4ab4d3e 100644 --- a/docs/users/quick-start.md +++ b/docs/users/quick-start.md @@ -19,7 +19,7 @@ Read our detailed guide to [installing IF](./how-to-install-if.md). ## 2: Create a manifest file -A manifest file contains all the configuration and input data required to measure your application's energy and carbon impacts and should have a `.yml` extension. +A manifest file contains all the configuration and input data required to measure your application's energy and carbon impacts and should have a `.yml` extension. Open the file, add your data and save the file. The minimal example below runs two snapshot observations through a single plugin - all it does is multiply a value in each element of the `input` data by 2. @@ -36,6 +36,7 @@ initialize: input-parameter: "cpu-utilization" coefficient: 2 output-parameter: "cpu-utilization-doubled" + tree: children: child-0: @@ -69,12 +70,15 @@ The output will be printed to the console. :tada:**Congratulations** :tada:! You have just used the Impact Framework to compute a manifest file! Your challenge now is to use these principles to construct manifest files for real applications. Our docs will help! + ## Next steps Now you know how to use the `if-run` you can start building more complex pipelines of plugins and more complicated manifest files. Your overall aim is to create a manifest file that accurately represents a real software application, and a plugin pipeline that yields an environmental metric that's important to you (e.g. `carbon`). + Experiment by adding more plugins to the pipeline and observe how each plugin enriches each element in the `inputs` array with new values. + You can also configure `if` to save your output data to another `yaml` file. To do this, add the `--output` flag and the path to the output file where the results are saved. The command is then as follows: