feat: add support for multiple astyle versions; add 3.4.7

Author: igrr

.gitignore

@@ -1,7 +1,7 @@
 tmp
 dist
-build
 *.egg-info
 __pycache__
 .venv
 .idea
+.pytest_cache

CONTRIBUTING.md

@@ -6,16 +6,23 @@ Contributions in the form of pull requests, issue reports, and feature requests
 
 Please do the following before submitting a PR:
 
-- [ ] Install pre-commit hooks: `pip install pre-commit && pre-commit install`
 - [ ] Create a virtual environment: `python3 -m venv .venv` and activate it: `. .venv/bin/activate`
 - [ ] Install testing prerequisites: `pip install -e ".[dev]"`
+- [ ] Install pre-commit hooks: `pre-commit install`
 - [ ] Run the tests: `pytest`
 
 ## Building Astyle
 
-This Python package uses Astyle compiled into WebAssembly ([astyle_py/libastyle.wasm](astyle_py/libastyle.wasm)). To rebuild libastyle.wasm, run `build.sh` in the `builder/` directory.
+This Python package uses Astyle compiled into WebAssembly (astyle_py/lib/<VERSION>/libastyle.wasm). To build libastyle.wasm for a new Astyle version, run `build.sh <VERSION>` in the `build-scripts/3.3+` directory. For example, to build the library for Astyle 3.4.7:
 
-Updating Astyle to a newer version might require tweaking the patch applied in [builder/build_inner.sh](builder/build_inner.sh).
+```bash
+cd build-scripts/3.3+
+./build.sh 3.4.7
+```
+
+You need Docker installed to do the build.
+
+A build script for the older Astyle 3.1 is also provided in `build-scripts/3.1`.
 
 ## Tagging a new release
 

README.md

@@ -19,7 +19,7 @@ The main reason to use this Python wrapper, rather than native `astyle` binaries
        rev: v1.0.0
        hooks:
        -   id: astyle_py
-           args: [--style=linux]
+           args: [--astyle-version 3.4.7 --style=linux]
    ```
 
 Place the required astyle formatting options to the `args` array. See the next section for details.
@@ -48,6 +48,7 @@ astyle_py [options] <files to format>
 ### Common options
 
 * `--version` — print the version and exit.
+* `--astyle-version <VER>` — choose the version of Astyle to use.
 * `--quiet` — don't print diagnostic messages; by default, the list of files which are formatted is printed to `stderr`.
 * `--dry-run` — don't format the files, only check the formatting. Returns non-zero exit code if any file would change after formatting.
 
@@ -86,6 +87,8 @@ Here is an example of a rules file:
 ```yml
 
 DEFAULT:
+    # Version of Astyle to use
+    version: "3.4.7"
     # These formatting options will be used by default
     options: "--style=otbs --indent=spaces=4 --convert-tabs"
 
@@ -104,14 +107,27 @@ code_to_ignore_for_now:
         - "tests/"     # matches a subdirectory 'tests' anywhere in the source tree
 ```
 
-## Implementation notes
+## Supported Astyle versions
+
+This python wrapper bundles multiple copies of Astyle, you can choose which one to use:
+- In the CLI: via `--astyle-version` argument
+- If you are using a rules file: using `version` key
+- When using astyle_py as a library: by passing the version to `Astyle()` constructor
+
+The following versions are supported:
 
+- 3.1 — used by default, unless a different version is specified
+- 3.4.7
+
+## Implementation notes
 
 To simplify distribution of astyle, it is compiled to WebAssembly ([astyle_py/libastyle.wasm](astyle_py/libastyle.wasm)) and executed using [wasmtime runtime](https://github.com/bytecodealliance/wasmtime) via its [Python bindings](https://github.com/bytecodealliance/wasmtime-py). This package should work on all operating systems supported by wasmtime — at the time of writing these are:
 - x86_64 (amd64) Windows, Linux, macOS
 - aarch64 (arm64) Linux and macOS
 
-There is another project which wraps astyle into a Python package, without using WebAssembly: https://github.com/timonwong/pyastyle. At the time of writing, it is unmaintained.
+Other project which wraps astyle into a Python package include:
+- https://github.com/timonwong/pyastyle — unmaintained at the time of writing, uses native Astyle binaries
+- https://github.com/Freed-Wu/astyle-wheel/ — actively maintained, uses native Astyle binaries
 
 ## Contributing
 
@@ -120,4 +136,6 @@ Please see [CONTRIBUTING.md](CONTRIBUTING.md) for details.
 ## Copyright and License
 
 * The source code in this repository is Copyright (c) 2020-2022 Ivan Grokhotkov and licensed under the [MIT license](LICENSE).
-* `libastyle.wasm` binary bundled herein is built from Artistic Style project, Copyright (c) 2018 by Jim Pattee <[email protected]>, also licensed under the MIT license. See http://astyle.sourceforge.net/ for details.
+* `libastyle.wasm` binaries bundled under [astyle_py/lib](astyle_py/lib) directory are built from [Artistic Style project](https://gitlab.com/saalen/astyle), Copyright (c) 2018 by Jim Pattee <[email protected]>, also licensed under the MIT license. See http://astyle.sourceforge.net/ for details.
+
+Thanks to André Simon for maintaining Astyle project!

astyle_py/__main__.py

@@ -4,22 +4,27 @@
 
 from . import __version__
 from .args import parse_args
-from .astyle_wrapper import Astyle
+from .astyle_wrapper import ASTYLE_COMPAT_VERSION, Astyle
 from .files_iter import iterate_files
 
 
 def main():
-    if '--version' in sys.argv:
-        print('astyle_py v{} with astyle v{}'.format(__version__, Astyle().version()))
-        raise SystemExit(0)
-
     try:
         args = parse_args(sys.argv[1:])
     except ValueError as e:
         print(str(e), file=sys.stderr)
         raise SystemExit(1)
 
-    astyle = Astyle()
+    if args.astyle_version:
+        astyle_version = args.astyle_version
+    else:
+        astyle_version = ASTYLE_COMPAT_VERSION
+
+    astyle = Astyle(version=astyle_version)
+
+    if args.version:
+        print('astyle-py {} with Astyle v{}'.format(__version__, astyle.version()))
+        raise SystemExit(0)
 
     def diag(*args_):
         if not args.quiet:

astyle_py/args.py

@@ -6,7 +6,16 @@
 
 AstyleArgs = namedtuple(
     'AstyleArgs',
-    ['rules', 'options', 'files', 'exclude_list', 'fix_formatting', 'quiet'],
+    [
+        'rules',
+        'options',
+        'files',
+        'exclude_list',
+        'fix_formatting',
+        'quiet',
+        'version',
+        'astyle_version',
+    ],
 )
 
 
@@ -25,6 +34,8 @@ def parse_args(args) -> AstyleArgs:
     quiet = False
     rules = None
     options_to_remove = []
+    version = False
+    astyle_version = None
 
     for o in options:
         o_trimmed = o[2:] if o.startswith('--') else o
@@ -64,6 +75,15 @@ def ensure_value():
             ensure_value()
             rules = value
 
+        elif opt == 'version':
+            options_to_remove.append(o)
+            version = True
+
+        elif opt == 'astyle-version':
+            options_to_remove.append(o)
+            ensure_value()
+            astyle_version = value
+
     for o in options_to_remove:
         options.remove(o)
 
@@ -81,4 +101,6 @@ def ensure_value():
         exclude_list=exclude_list,
         fix_formatting=fix_formatting,
         quiet=quiet,
+        version=version,
+        astyle_version=astyle_version,
     )

astyle_py/astyle_wrapper.py

@@ -78,14 +78,27 @@ def from_addr(context: WasmContext, addr: int) -> 'WasmString':
         return WasmString(context, addr, n_bytes, string)
 
 
+ASTYLE_COMPAT_VERSION = '3.1'
+ASTYLE_SUPPORTED_VERSIONS = os.listdir(os.path.join(os.path.dirname(__file__), 'lib'))
+
+
 class Astyle:
-    def __init__(self):
+    def __init__(self, version: str = ASTYLE_COMPAT_VERSION):
+        if version not in ASTYLE_SUPPORTED_VERSIONS:
+            raise ValueError(
+                'Unsupported astyle version: {}. Available versions: {}'.format(
+                    version, ', '.join(ASTYLE_SUPPORTED_VERSIONS)
+                )
+            )
+
         self.context = WasmContext()
         err_handler_type = FuncType([ValType.i32(), ValType.i32()], [])
         err_handler_func = Func(self.context.store, err_handler_type, self._err_handler)
         self.context.linker.define('env', 'AStyleErrorHandler', err_handler_func)
 
-        wasm_file = os.path.join(os.path.dirname(__file__), 'libastyle.wasm')
+        wasm_file = os.path.join(
+            os.path.dirname(__file__), 'lib', version, 'libastyle.wasm'
+        )
         module = Module.from_file(self.context.store.engine, wasm_file)
         self.context.inst = self.context.linker.instantiate(self.context.store, module)
         self.context.call_func('_initialize')

astyle_py/files_iter.py

@@ -11,7 +11,7 @@
 from .utils import pattern_to_regex
 
 FileItem = namedtuple('FileItem', ['filename', 'astyle_options'])
-Rule = namedtuple('Rule', ['check', 'include', 'options'])
+Rule = namedtuple('Rule', ['check', 'include', 'options', 'version'])
 
 
 def file_matches_patterns(fname: str, patterns: List[Pattern]) -> bool:
@@ -52,7 +52,9 @@ def iterate_files_rules(
         rules_dict = yaml.safe_load(rf)
 
     # set the default rule
-    default_rule = Rule(check=True, include=[pattern_to_regex('*')], options=[])
+    default_rule = Rule(
+        check=True, include=[pattern_to_regex('*')], options=[], version=None
+    )
 
     # if 'DEFAULT' key is in the YAML file, use it to update the default rule
     default_rule_dict = rules_dict.get('DEFAULT')
@@ -77,6 +79,7 @@ def get_rule_from_dict(rule_name: str, rule_dict, defaults: Rule) -> Rule:
     options = defaults.options
     check = defaults.check
     include = defaults.include
+    version = defaults.version
 
     for k, v in rule_dict.items():
         if k == 'options':
@@ -97,12 +100,18 @@ def get_rule_from_dict(rule_name: str, rule_dict, defaults: Rule) -> Rule:
                     f'Unexpected value of \'include\' in rule {rule_name}, expected a list of strings, found {v}'
                 )
             include = [re.compile(pattern_to_regex(x)) for x in v]
+        elif k == 'version':
+            if not isinstance(v, str):
+                raise ValueError(
+                    f'Unexpected value of \'version\' in rule {rule_name}, expected a string, found {v}'
+                )
+            version = v
         else:
             raise ValueError(
                 f'Unexpected key \'{k}\' in rule {rule_name}, expected one of: options, check, include'
             )
 
-    return Rule(check, include, options)
+    return Rule(check, include, options, version)
 
 
 def process_rules_for_file(
@@ -113,4 +122,8 @@ def process_rules_for_file(
         if file_matches_patterns(fname, rule.include):
             selected_rule = rule
     if selected_rule.check:
-        yield FileItem(filename=fname, astyle_options=selected_rule.options)
+        options = []
+        if selected_rule.version:
+            options.append(f'--astyle-version={selected_rule.version}')
+        options += selected_rule.options
+        yield FileItem(filename=fname, astyle_options=options)

astyle_py/lib/3.1/libastyle.wasm

@@ -0,0 +1,9 @@
+Script to build WASM library for Astyle 3.1.
+
+Run:
+
+```bash
+./build.sh
+```
+
+This should build the docker image with Emscripten SDK, download Astyle and build it with Emscripten. The resulting library will be copied to `astyle_py/lib/3.1/libastyle.wasm`.

astyle_py/lib/3.4.7/libastyle.wasm

@@ -2,9 +2,10 @@
 set -euo pipefail
 
 TMP_DIR=$PWD/../tmp
-DST_DIR=${PWD}/../astyle_py
+DST_DIR=${PWD}/../../astyle_py/lib/3.1
+DOCKER_DIR=${PWD}/../docker
 
-docker build -t emsdk .
+docker build -t emsdk ${DOCKER_DIR}
 mkdir -p ${TMP_DIR}
 cp build_inner.sh ${TMP_DIR}
 docker run --rm -v ${TMP_DIR}:/work -w /work emsdk ./build_inner.sh

astyle_py/libastyle.wasm

@@ -47,6 +47,7 @@ emmake make static
 emcc -o bin/libastyle.wasm \
     -s EXPORTED_FUNCTIONS=["_AStyleGetVersion","_AStyleWrapper","_malloc","_free"] \
     -s STANDALONE_WASM=1 \
+    -s ERROR_ON_UNDEFINED_SYMBOLS=0 \
     --no-entry \
     bin/libastyle.a
 

build-scripts/3.1/README.md

@@ -0,0 +1,34 @@
+cmake_minimum_required(VERSION 3.15)
+
+if(NOT DEFINED ENV{EMSDK_PATH})
+    message(FATAL_ERROR "EMSDK_PATH environment variable not set")
+endif()
+
+if(NOT DEFINED ASTYLE_VER)
+    message(FATAL_ERROR "ASTYLE_VER cache variable not set")
+endif()
+
+set(CMAKE_TOOLCHAIN_FILE $ENV{EMSDK_PATH}/upstream/emscripten/cmake/Modules/Platform/Emscripten.cmake)
+
+set(BUILD_STATIC_LIBS 1 CACHE BOOL "Build static libraries")
+
+project(astyle-wasm)
+
+include(FetchContent)
+FetchContent_Declare(
+    astyle
+    GIT_REPOSITORY https://gitlab.com/saalen/astyle.git
+    GIT_TAG ${ASTYLE_VER}
+)
+FetchContent_MakeAvailable(astyle)
+
+
+add_executable(astyle-wasm wrapper.c)
+target_link_libraries(astyle-wasm PRIVATE astyle)
+
+target_link_libraries(astyle-wasm PRIVATE "-s EXPORTED_FUNCTIONS=[\"_AStyleGetVersion\",\"_AStyleWrapper\",\"_malloc\",\"_free\",\"_AStyleErrorHandler\"]")
+target_link_libraries(astyle-wasm PRIVATE "-s STANDALONE_WASM=1")
+target_link_libraries(astyle-wasm PRIVATE "-s ERROR_ON_UNDEFINED_SYMBOLS=0")
+target_link_libraries(astyle-wasm PRIVATE "-s WARN_ON_UNDEFINED_SYMBOLS=1")
+target_link_libraries(astyle-wasm PRIVATE --no-entry)
+target_link_libraries(astyle-wasm PRIVATE -Wl,--import-undefined)

builder/build.sh renamed to build-scripts/3.1/build.sh

@@ -0,0 +1,13 @@
+Build script for Astyle 3.3 and later.
+
+Run:
+
+```
+./build.sh <version>
+```
+
+where `<version>` is the version of Astyle you want to build. For example, `3.4.7`.
+
+The script will build a Docker image with Emscripten SDK, download the required version of Astyle and build it with Emscripten. Since Astyle >=3.3 has a CMake build system, everything is conveniently handled in CMakeLists.txt.
+
+The resulting library will be copied to `astyle_py/lib/<version>/libastyle.wasm`.

builder/build_inner.sh renamed to build-scripts/3.1/build_inner.sh

@@ -0,0 +1,21 @@
+#!/bin/bash
+set -euo pipefail
+
+# argument is the version number
+if [ $# -ne 1 ]; then
+    echo "Usage: $0 <astyle_version>"
+    exit 1
+fi
+
+ASTYLE_VERSION=$1
+
+DOCKER_DIR=${PWD}/../docker
+docker build -t emsdk ${DOCKER_DIR}
+
+TMP_DIR=${PWD}/../tmp/${ASTYLE_VERSION}
+mkdir -p ${TMP_DIR}
+DST_DIR=${PWD}/../../astyle_py/lib/${ASTYLE_VERSION}
+mkdir -p ${DST_DIR}
+
+docker run --rm -v ${PWD}:/src -v ${TMP_DIR}:/build emsdk bash -c "cmake -S /src -B /build -DASTYLE_VER=${ASTYLE_VERSION} && cmake --build /build"
+cp ${TMP_DIR}/astyle-wasm.wasm ${DST_DIR}/libastyle.wasm

build-scripts/3.3+/CMakeLists.txt

@@ -0,0 +1,8 @@
+#include <stdlib.h>
+void AStyleErrorHandler(int errorNumber, const char* errorMessage);
+char* AStyleMain(const char* pSourceIn, const char* pOptions, void (*fpError)(int, const char*), void* (*fpAlloc)(unsigned long));
+
+char* AStyleWrapper(const char* pSourceIn, const char* pOptions)
+{
+    return AStyleMain(pSourceIn, pOptions, &AStyleErrorHandler, malloc);
+}