Skip to content

Add Switches to no color when running for CI so that console logs can be processed by JUnit test reports #123

@marcellodesales

Description

@marcellodesales

Problem

  • Tests reports for Continuous Integration usually use JUnit test reports
    • Jenkins, Gitlab, Github
  • When converting an output of test cases to JUNit, while preserving the formatted output of test reports, we get errors
  • Since the output is in XML, the PCDATA embedded as texts must be escaped as it will fail any XML lint

NOTE: This was identified in a CI build for Mobile apps using a junit dart formatter https://gitlab.com/gitlab-org/gitlab/-/issues/268035#note_801222659

Possible Solution

  • Use an alternative output without color formatting for the test reports
  • Switch the use a logger formatting that does NOT show the colors, special characters, etc

Question: How can we use flutter test and format the output of the tests to be without colors?

JUnit converted test report with output with colors

flutter test --coverage --machine | tojunit --output build/app/reports/junit-tests-report.xml
cat output build/app/reports/junit-tests-report.xml
  • The output of the junit test reports include the raw output from logger
....
....
┌───────────────────────────────────────────────────────────────────────────────
│ #0   UserRepository.saveUserAddress (package:parking_web_app_maicero_shop/infrastructure/user/user_repository.dart:573:17)
│ #1   <asynchronous suspension>
│ #2   StackZoneSpecification._registerUnaryCallback.<anonymous closure> (package:stack_trace/src/stack_zone_specification.dart)
├┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄
│ 12:51:49.751 (+0:00:03.910572)
├┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄
│ ⛔ Critical server error: Http status error [500] in the function: saveUserAddress() on user_repository
└──────────────────────────────

XML Linter errors with special characters from colored output

  • Since the output is in XML, the PCDATA embedded as texts must be escaped...
$ docker run -i --rm -v $PWD:/data mribeiro/xmllint build/app/reports/junit-tests-report.xml
Unable to find image 'mribeiro/xmllint:latest' locally
latest: Pulling from mribeiro/xmllint
Image docker.io/mribeiro/xmllint:latest uses outdated schema1 manifest format. Please upgrade to a schema2 image for better future compatibility. More information at https://docs.docker.com/registry/spec/deprecated-schema-v1/
a3ed95caeb02: Pull complete
5f6cc4b8eedc: Pull complete
7ebc7bd59630: Pull complete
Digest: sha256:e8293a7906daf6bba7cc2812e9cc54c0e0932afe6a162987cf999bcfd08c93a2
Status: Downloaded newer image for mribeiro/xmllint:latest
build/app/reports/junit-tests-report.xml:63: parser error : PCDATA invalid Char value 27
┌──────────────────────
^
build/app/reports/junit-tests-report.xml:63: parser error : PCDATA invalid Char value 27
��──────────────────────────
┌──────────────────────
^
build/app/reports/junit-tests-report.xml:63: parser error : PCDATA invalid Char value 27
��──────────────────────────
                                                                               ^
build/app/reports/junit-tests-report.xml:64: parser error : PCDATA invalid Char value 27
│ #0   UserRepository.signUp (package:parking_web_app_maicero_shop/
^
build/app/reports/junit-tests-report.xml:64: parser error : PCDATA invalid Char value 27
ge:parking_web_app_maicero_shop/infrastructure/user/user_repository.dart:178:17)
                                                                               ^
build/app/reports/junit-tests-report.xml:65: parser error : PCDATA invalid Char value 27
│ #1   <asynchronous suspension>
^
build/app/reports/junit-tests-report.xml:65: parser error : PCDATA invalid Char value 27
│ #1   <asynchronous suspension>
                                                ^
build/app/reports/junit-tests-report.xml:66: parser error : PCDATA invalid Char value 27
│ #2   StackZoneSpecification._registerUnaryCallback.<anonymous
^

Requirement

  • We should be able to switch the formatting via command-line
  • We should be able to switch the formatting via configuration

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions