Skip to content

Commit

Permalink
add patch files for milestone 9
Browse files Browse the repository at this point in the history
  • Loading branch information
@ninetteadhikari
ninetteadhikari committed Apr 18, 2024
1 parent 525100b commit 2e26415
Show file tree
Hide file tree
Showing 4 changed files with 526 additions and 0 deletions.
52 changes: 52 additions & 0 deletions 0000-cover-letter.patch
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
From 0ff5f57f1f5669db2a0bebb09a7500e95a7ace7f Mon Sep 17 00:00:00 2001
From: Ninette Adhikari <[email protected]>
Date: Fri, 12 Apr 2024 16:17:52 +0200
Subject: [PATCH 0/3] Improvements for performance test report view

This work is done according to "Milestone 9: Build performance test report view" as stated in the Scope of Work with Sovereign Tech Fund (STF) (https://www.sovereigntechfund.de/).
The current report can be accessed here:
Performance test report HTML (https://autobuilder.yocto.io/pub/non-release/20240117-15/testresults/buildperf-alma8/perf-alma8_master_20240117090048_663f180574.html)
The report is created using the `oe-build-perf-report` script in the poky repository. This script generates a performance test report in HTML format using the data from the yocto-buildstats (https://git.yoctoproject.org/yocto-buildstats/) repository.
The report displays the performance test results in line chart format. The chart x-axis represents the commit numbers, and the y-axis represents the test duration in minutes.
The report also includes a table that displays the measurement statistics data for each test. The report is interactive and allows users to zoom in on specific sections of the line chart.

The current report format required some updates to make it more interactive and user-friendly. And this patch addresses such improvements:

- Add [Apache echart](https://echarts.apache.org/en/index.html) library to create oe build performance report charts and make them interactive.
- Restructure data to time and value array format to be used by echarts. It also converts test duration to minutes and adds zoom to the line charts.
- Update measurement statistics data to include `start_time` so that time can be displayed instead of commit numbers on the chart. It also updates default commit history length to 300.
- Add styling updates including page margin, labels for x and y axis, tooltip, and section descriptions.

Updated report screenshots:
https://github.com/neighbourhoodie/poky/assets/13760198/65a1890c-fd2a-40d4-ac90-f13055735e53
https://github.com/neighbourhoodie/poky/assets/13760198/1ed43876-73a9-487e-aed3-ca0edf97514c

For local setup, you can do the following:

1. Clone the yocto-buildstats (https://git.yoctoproject.org/yocto-buildstats/) and the poky repository (https://git.yoctoproject.org/poky/)

2. In the poky repository run the following to build the report HTML:
```bash
./scripts/oe-build-perf-report -r "LOCAL_PATH_TO_YOCTO_BUILDSTATS" --branch "master" --commit "663f1805742ff6fb6955719d0ab7846a425debcf" --branch2 "master" --html > test.html
```
Note:
- Add your local path to the yocto-buildstats repo
- The above command builds the report in a file called `test.html`. You can access it in the root directory in poky.
- This exmaple report uses the commit `663f1805742ff6fb6955719d0ab7846a425debcf` from `master` branch.


Ninette Adhikari (3):
oe-build-perf-report: Add apache echarts to make report interactive
oe-build-perf-report: Display more than 300 commits and date instead
of commit number
oe-build-perf-report: Improve report styling and add descriptions

.../build_perf/html/measurement_chart.html | 116 +++++++++++-------
scripts/lib/build_perf/html/report.html | 96 ++++++++++-----
scripts/lib/build_perf/report.py | 4 +-
scripts/oe-build-perf-report | 6 +-
4 files changed, 143 insertions(+), 79 deletions(-)

--
2.44.0

163 changes: 163 additions & 0 deletions 0001-oe-build-perf-report-Add-apache-echarts-to-make-repo.patch
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,163 @@
From 1bebc64d0ac5b4a844bcda9673e0f4a89627e8b5 Mon Sep 17 00:00:00 2001
From: Ninette Adhikari <[email protected]>
Date: Tue, 19 Mar 2024 16:18:21 +0100
Subject: [PATCH 1/3] oe-build-perf-report: Add apache echarts to make report
interactive

- Add Apache echarts (https://echarts.apache.org/en/index.html) library to create build performance charts.
- Restructure data to time and value array format so that it can be used by echarts.
- This commit also converts test duration to minutes to map against the values axis.
- Zoom is added to the line charts.

Signed-off-by: Ninette Adhikari <[email protected]>
---
.../build_perf/html/measurement_chart.html | 116 +++++++++++-------
scripts/lib/build_perf/html/report.html | 6 +-
2 files changed, 72 insertions(+), 50 deletions(-)

diff --git a/scripts/lib/build_perf/html/measurement_chart.html b/scripts/lib/build_perf/html/measurement_chart.html
index 65f1a227ad..ffec3d09db 100644
--- a/scripts/lib/build_perf/html/measurement_chart.html
+++ b/scripts/lib/build_perf/html/measurement_chart.html
@@ -1,50 +1,76 @@
-<script type="text/javascript">
- chartsDrawing += 1;
- google.charts.setOnLoadCallback(drawChart_{{ chart_elem_id }});
- function drawChart_{{ chart_elem_id }}() {
- var data = new google.visualization.DataTable();
+<script type="module">
+ // Get raw data
+ const rawData = [
+ {% for sample in measurement.samples %}
+ [{{ sample.commit_num }}, {{ sample.mean.gv_value() }}, {{ sample.start_time }}],
+ {% endfor %}
+ ];

- // Chart options
- var options = {
- theme : 'material',
- legend: 'none',
- hAxis: { format: '', title: 'Commit number',
- minValue: {{ chart_opts.haxis.min }},
- maxValue: {{ chart_opts.haxis.max }} },
- {% if measurement.type == 'time' %}
- vAxis: { format: 'h:mm:ss' },
- {% else %}
- vAxis: { format: '' },
- {% endif %}
- pointSize: 5,
- chartArea: { left: 80, right: 15 },
- };
+ const convertToMinute = (time) => {
+ return time[0]*60 + time[1] + time[2]/60 + time[3]/3600;
+ }

- // Define data columns
- data.addColumn('number', 'Commit');
- data.addColumn('{{ measurement.value_type.gv_data_type }}',
- '{{ measurement.value_type.quantity }}');
- // Add data rows
- data.addRows([
- {% for sample in measurement.samples %}
- [{{ sample.commit_num }}, {{ sample.mean.gv_value() }}],
- {% endfor %}
- ]);
+ // Convert raw data to the format: [time, value]
+ const data = rawData.map(([commit, value, time]) => {
+ return [
+ new Date(time * 1000).getTime(), // The Date object takes values in milliseconds rather than seconds. So to use a Unix timestamp we have to multiply it by 1000.
+ Array.isArray(value) ? convertToMinute(value) : value // Assuming the array values are duration in the format [hours, minutes, seconds, milliseconds]
+ ]
+ });

- // Finally, draw the chart
- chart_div = document.getElementById('{{ chart_elem_id }}');
- var chart = new google.visualization.LineChart(chart_div);
- google.visualization.events.addListener(chart, 'ready', function () {
- //chart_div = document.getElementById('{{ chart_elem_id }}');
- //chart_div.innerHTML = '<img src="' + chart.getImageURI() + '">';
- png_div = document.getElementById('{{ chart_elem_id }}_png');
- png_div.outerHTML = '<a id="{{ chart_elem_id }}_png" href="' + chart.getImageURI() + '">PNG</a>';
- console.log("CHART READY: {{ chart_elem_id }}");
- chartsDrawing -= 1;
- if (chartsDrawing == 0)
- console.log("ALL CHARTS READY");
- });
- chart.draw(data, options);
-}
+ // Set chart options
+ const option = {
+ tooltip: {
+ trigger: 'axis',
+ position: function (pt) {
+ return [pt[0], '10%'];
+ },
+ valueFormatter: (value) => value.toFixed(2)
+ },
+ xAxis: {
+ type: 'time',
+ },
+ yAxis: {
+ name: '{{ measurement.value_type.quantity }}' == 'time' ? 'Duration (minutes)' : 'Disk size (MB)',
+ type: 'value',
+ min: function(value) {
+ return Math.round(value.min - 0.5);
+ },
+ max: function(value) {
+ return Math.round(value.max + 0.5);
+ }
+ },
+ dataZoom: [
+ {
+ type: 'inside',
+ start: 0,
+ end: 100
+ },
+ {
+ start: 0,
+ end: 100
+ }
+ ],
+ series: [
+ {
+ name: '{{ measurement.value_type.quantity }}',
+ type: 'line',
+ smooth: true,
+ symbol: 'none',
+ data: data
+ }
+ ]
+ };
+
+ // Draw chart
+ const chart_div = document.getElementById('{{ chart_elem_id }}');
+ const measurement_chart= echarts.init(chart_div, null, {
+ height: 320
+ });
+ // Change chart size with browser resize
+ window.addEventListener('resize', function() {
+ measurement_chart.resize();
+ });
+ measurement_chart.setOption(option);
</script>

diff --git a/scripts/lib/build_perf/html/report.html b/scripts/lib/build_perf/html/report.html
index d1ba6f2578..653fd985bc 100644
--- a/scripts/lib/build_perf/html/report.html
+++ b/scripts/lib/build_perf/html/report.html
@@ -3,11 +3,7 @@
<head>
{# Scripts, for visualization#}
<!--START-OF-SCRIPTS-->
-<script type="text/javascript" src="https://www.gstatic.com/charts/loader.js"></script>
-<script type="text/javascript">
-google.charts.load('current', {'packages':['corechart']});
-var chartsDrawing = 0;
-</script>
+<script src=" https://cdn.jsdelivr.net/npm/[email protected]/dist/echarts.min.js "></script>

{# Render measurement result charts #}
{% for test in test_data %}
--
2.44.0

71 changes: 71 additions & 0 deletions 0002-oe-build-perf-report-Display-more-than-300-commits-a.patch
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
From 8704001521719bec22d572dda1ea30dad26823cc Mon Sep 17 00:00:00 2001
From: Ninette Adhikari <[email protected]>
Date: Tue, 19 Mar 2024 16:26:54 +0100
Subject: [PATCH 2/3] oe-build-perf-report: Display more than 300 commits and
date instead of commit number

- This commit updates measurement statistics data to include start_time so that time can be displayed instead of commit numbers on the chart.
- It also updates default commit history length to 300.

Signed-off-by: Ninette Adhikari <[email protected]>
---
scripts/lib/build_perf/report.py | 4 +++-
scripts/oe-build-perf-report | 6 ++++--
2 files changed, 7 insertions(+), 3 deletions(-)

diff --git a/scripts/lib/build_perf/report.py b/scripts/lib/build_perf/report.py
index ab77424cc7..82c56830d7 100644
--- a/scripts/lib/build_perf/report.py
+++ b/scripts/lib/build_perf/report.py
@@ -294,7 +294,7 @@ class SizeVal(MeasurementVal):
return "null"
return self / 1024

-def measurement_stats(meas, prefix=''):
+def measurement_stats(meas, prefix='', time=0):
"""Get statistics of a measurement"""
if not meas:
return {prefix + 'sample_cnt': 0,
@@ -319,6 +319,7 @@ def measurement_stats(meas, prefix=''):
stats['quantity'] = val_cls.quantity
stats[prefix + 'sample_cnt'] = len(values)

+ start_time = time # Add start time for both type sysres and disk usage
mean_val = val_cls(mean(values))
min_val = val_cls(min(values))
max_val = val_cls(max(values))
@@ -334,6 +335,7 @@ def measurement_stats(meas, prefix=''):
stats[prefix + 'max'] = max_val
stats[prefix + 'minus'] = val_cls(mean_val - min_val)
stats[prefix + 'plus'] = val_cls(max_val - mean_val)
+ stats[prefix + 'start_time'] = start_time

return stats

diff --git a/scripts/oe-build-perf-report b/scripts/oe-build-perf-report
index 7812ea4540..266700d294 100755
--- a/scripts/oe-build-perf-report
+++ b/scripts/oe-build-perf-report
@@ -336,7 +336,9 @@ def print_html_report(data, id_comp, buildstats):
test_i = test_data['tests'][test]
meas_i = test_i['measurements'][meas]
commit_num = get_data_item(meta, 'layers.meta.commit_count')
- samples.append(measurement_stats(meas_i))
+ # Add start_time for both test measurement types of sysres and disk usage
+ start_time = test_i['start_time'][0]
+ samples.append(measurement_stats(meas_i, '', start_time))
samples[-1]['commit_num'] = commit_num

absdiff = samples[-1]['val_cls'](samples[-1]['mean'] - samples[id_comp]['mean'])
@@ -473,7 +475,7 @@ Examine build performance test results from a Git repository"""
group.add_argument('--branch', '-B', default='master', help="Branch to find commit in")
group.add_argument('--branch2', help="Branch to find comparision revisions in")
group.add_argument('--machine', default='qemux86')
- group.add_argument('--history-length', default=25, type=int,
+ group.add_argument('--history-length', default=300, type=int,
help="Number of tested revisions to plot in html report")
group.add_argument('--commit',
help="Revision to search for")
--
2.44.0

Loading

0 comments on commit 2e26415

Please sign in to comment.