Skip to content

Latest commit

 

History

History
242 lines (188 loc) · 6.57 KB

grep-0021-YAML driven block implementation.md

File metadata and controls

242 lines (188 loc) · 6.57 KB
Raw

GREP [0021] -- YAML Driven Block Implementation

History:

  • 29-Mar-2021: Initial Draft

Abstract

In order to encourage more developers to create and upstream blocks, this GREP proposes to streamline the process of creating and maintaining blocks, to where in most cases, the developer inputs the minimal amount of information into a conf file such as yaml, and only has to maintain a work function

Copyright / License

CC-BY-ND license

Motivation

One of the complaints I've heard over the years with developing and maintaining GNU Radio blocks is the need to update tons of boilerplate code when small changes such as adding a parameter to the constructor are added. This includes the parameter list in 3 files (.h, _impl.h, _impl.cc), python bindings, grc file

Also, when navigating the source tree, files are scattered all over the place, so working on a single block leads to going in and out of all the directories in the module.

The goal of the GREP is to reformulate the GR block design workflow, and automate as much as possible so that the developers only have to update things in one place where possible

Description

Design

  1. Reorganize the block tree by block folders and keep all pertinent files per block folder
gr-blocks
├── copy
│   ├── copy_impl.h
│   ├── copy_impl.cc
│   ├── copy.yml
|   ├── copy.h
│   ├── copy_python.cc
│   └── CMakeLists.txt
  1. For most blocks, the public header, impl header, grc, python bindings can be automatically generated, so the folder structure might looks something like:
gr-blocks
├── copy
│   ├── copy_impl.cc
│   ├── copy.yml
│   └── CMakeLists.txt

Code Generation

Code generation would be accomplished with something like Jinja2 or Mako, with generic templates that take the yml as input, and output the appropriate files into the build directory (or generated code dir), but not stored in-tree

Workflow

By having all files in one place, this would simplify the task of modtool and even allow modtool to pretty easily be used to create in-tree blocks.

On a make, the template-driven source files, such as public header and _impl.h, would be generated prior to compiling the _impl.cc

If the block implementation is primarily contained in the yml file, it would be reasonable to use a graphical tool to generate the yaml

There are several options for workflows that could be supported

Simple block workflow

In this case, the only thing the user messes with is the YAML file + the _impl.cc file to implement the work function

  1. Create the YAML file (generated from a template or modtool)
  2. Add private variables with initializations to the YAML file
  3. Implement work in blockname_impl.cc

Custom constructor

  1. Create the YAML file
  2. Implement constructor in blockname_impl.cc - macro available for parameter list
  3. Implement work in blockname_impl.cc

Custom everything

  1. Create the YAML file - will not be used for code generation, just metadata
  2. Implement all the files (public header, _impl.h, _impl.cc)

YAML in this case still ties everything together and could do some autogeneration

Another option is to allow any appropriately named files in the block directory to override the auto-generation

Python block workflow

For python blocks, autogeneration of boilerplate is less necessary but could still simplify things. With python we can split the autogenerated part from the pieces that are manually maintained.

  1. Create YAML (indicate python block in there somewhere)
  2. Implement work function in _work.py which will define work()
  3. Optionally, implement __init__ in _init.py
  4. Autogenerated python will create blockname.py and tie in work and init files
  5. Build process automatically ties it into module __init__.py

YAML File Format

Should look similar to the current GRC file, perhaps more information about templating and types

Top level properties

Describe the block as a whole

module: blocks
block: annotator
label: Annotator
blocktype: sync

Parameters

These are things that become constructor arguments, and potentially things that can be changed via tags, callbacks, RPC, etc.

parameters:
-   id: when
    label: When
    dtype: uint64_t
    settable: false
-   id: itemsize
    label: Item Size
    dtype: size_t
    settable: false
-   id: num_inputs
    label: Num Inputs
    dtype: size_t
    settable: false
-   id: num_outputs
    label: Num Outputs
    dtype: size_t
    settable: false
-   id: tpp
    label: Tag Propagation Policy
    dtype: tag_propagation_policy_t
    settable: false

Ports

Describe the ports as would be done in GRC

ports:
-   domain: stream
    id: in
    direction: input
    type: untyped
    size: itemsize
    multiplicity: num_inputs

-   domain: stream
    id: out
    direction: output
    type: untyped
    size: itemsize
    multiplicity: num_outputs

Private Variables

TBD if this is a good idea - the goal is to keep the boilerplate things like the constructor and the make function with the repeated parameter lists hidden away but that then keeps the class definition from the user. One option is to put some of the things that would go in the class definition directly in the YAML

variables:
-   name:  tag_counter
    dtype: uint64_t
    init:  0
-   name: stored_tags
    dtype: std::vector<tag_t>
    init: '{}'

Callbacks

Public methods - need to standardize these so they work over RPC and other mechanisms as well

callbacks:
-   name: data
    return: std::vector<tag_t>
    const: true

Init

Lines of code that go in the constructor if it is hidden away

init:
-   set_tag_propagation_policy(tpp)

Implementations

For blocks that have more than one 'implementation' - such as cpu and cuda, list those here so boilerplate can be done consistently and automatically

implementations:
-   id: cpu

Templating

YAML should make it clear if a block is to be expanded over different types via templating - similar to how grc does it, but this would get propagated to the c++/python code

properties:
-   id: blocktype
    value: sync
-   id: type
    label: IO Type
    dtype: enum
    options: 
        - dtype: int16_t
          suffix: ss 
        - dtype: int32_t
          suffix: ii
        - dtype: float
          suffix: ff
        - dtype: gr_complex
          suffix: cc

Timeline

Definitely not prior to GR 4.0