Skip to content

Commit abd64f0

Browse files
rubenszinhorubenszinho
committed
feat: initial mvp implementation of ENEMETER Data Processing Tool (#1)
* feat: initial mvp implementation of ENEMETER Data Processing Tool * fix: handle file.Close() error in all three places by using defer with a function that checks the close error, include MaxRecords check within Parse function loop and remove the unused calculateAverage function
1 parent f53f7a8 commit abd64f0

File tree

12 files changed

+1687
-39
lines changed

12 files changed

+1687
-39
lines changed

‎.github/workflows/deploy.yml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -16,8 +16,8 @@ jobs:
1616
id: app-token
1717
uses: actions/create-github-app-token@v1
1818
with:
19-
app-id: \${{ vars.RUBRION_APP_ID }}
20-
private-key: \${{ secrets.RUBRION_APP_SECRET }}
19+
app-id: ${{ vars.RUBRION_APP_ID }}
20+
private-key: ${{ secrets.RUBRION_APP_SECRET }}
2121

2222
- name: Set Git User Identity
2323
run: |

‎.gitignore‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
# Exclude build files
22
dist/
3+
data/
34
vendor/
4-
.env
5+
.env

‎Makefile‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
11
build:
2-
go build -o dist/rubrion-cli ./cmd/rubrion-cli/main.go
2+
go build -o dist/enemeter-data-processing ./cmd/enemeter-data-processing/main.go
33

44
.PHONY: build

‎README.md‎

Lines changed: 227 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -1,22 +1,234 @@
1-
# Rubrion CLI Template
1+
# ENEMETER Data Processing Tool
22

3-
This repository is a template for building Go CLI tools for Rubrion.
3+
A powerful command-line tool for efficiently processing and analyzing ENEMETER energy measurement data.
44

5-
## Getting Started
5+
## Overview
66

7-
To run the CLI locally:
8-
\`\`\`
9-
go run ./cmd/rubrion-cli/main.go
10-
\`\`\`
7+
This tool processes data from ENEMETER devices, which measure various electrical parameters such as voltage, current, and temperature. The tool is designed to handle large CSV files efficiently and provide meaningful energy metrics for analysis.
118

12-
## Build
9+
## Features
1310

14-
You can build the CLI using:
15-
\`\`\`
16-
make build
17-
\`\`\`
11+
- Process large CSV files efficiently with streaming mode
12+
- Filter data by time range, temperature, voltage, or current
13+
- Sample data to reduce processing requirements
14+
- Extract specific metrics for targeted analysis
15+
- Export results in multiple formats (text, JSON, CSV)
16+
- Calculate energy consumption metrics and statistics
17+
- Analyze battery charging/discharging patterns
18+
- Evaluate solar panel contribution
1819

19-
## Release Process
20+
## Installation
2021

21-
This template includes GitHub Actions workflows for QA checks and production releases.
22-
Refer to the files under \`.github/workflows/\` for details.
22+
### Prerequisites
23+
24+
- Go 1.23 or later
25+
26+
### Building from source
27+
28+
```bash
29+
git clone https://github.com/yourusername/enemeter-data-processing.git
30+
cd enemeter-data-processing
31+
go build
32+
```
33+
34+
## Data Format
35+
36+
ENEMETER data is provided in CSV format with four columns:
37+
38+
1. **Time Delta (milliseconds)** - Time elapsed since the last measurement
39+
2. **Temperature (millicelsius)** - System temperature
40+
3. **Voltage (microvolts)** - Battery voltage
41+
4. **Current (nanoamperes)** - System current (positive when charging, negative when discharging)
42+
43+
Example:
44+
```
45+
0000052051,0004815430,-1275169536,0000023192
46+
0000000047,0004815430,-1275169536,0000023192
47+
0000000047,0004815790,-1276443264,0000023192
48+
```
49+
50+
## Basic Usage
51+
52+
Process a CSV file with default settings:
53+
54+
```bash
55+
./enemeter-data-processing --input=data/data.csv
56+
```
57+
58+
Save results to a text file:
59+
60+
```bash
61+
./enemeter-data-processing --input=data/data.csv --output=report.txt
62+
```
63+
64+
## Command-line Options
65+
66+
### Input/Output Options
67+
68+
- `--input=<path>`: Path to the input CSV file (required)
69+
- `--output=<path>`: Path to save the output report (optional)
70+
- `--format=<text|json|csv>`: Output format (default: text)
71+
72+
### Processing Options
73+
74+
- `--stream`: Use memory-efficient streaming mode for large files
75+
- `--sample=<N>`: Process every Nth record (default: 1, process all records)
76+
- `--max=<N>`: Maximum number of records to process (default: 0, no limit)
77+
78+
### Time Filtering Options
79+
80+
- `--start=<time>`: Start time for filtering (format: YYYY-MM-DD[THH:MM:SS])
81+
- `--end=<time>`: End time for filtering (format: YYYY-MM-DD[THH:MM:SS])
82+
- `--window=<duration>`: Time window to process (e.g., 1h, 30m, 24h)
83+
84+
### Data Filtering Options
85+
86+
- `--min-temp=<value>`: Minimum temperature threshold in millicelsius
87+
- `--volt-min=<value>`: Minimum voltage threshold in microvolts
88+
- `--volt-max=<value>`: Maximum voltage threshold in microvolts
89+
- `--curr-min=<value>`: Minimum current threshold in nanoamperes
90+
- `--curr-max=<value>`: Maximum current threshold in nanoamperes
91+
92+
### Metric Extraction
93+
94+
- `--metric=<name>`: Extract a specific metric (see below)
95+
96+
## Available Metrics
97+
98+
- `total_energy`: Total energy consumption in joules
99+
- `average_power`: Average power in watts
100+
- `peak_power`: Peak power in watts
101+
- `temperature`: Temperature statistics
102+
- `energy_by_hour`: Energy consumption by hour
103+
- `voltage_stats`: Voltage statistics
104+
- `current_stats`: Current statistics
105+
- `battery_discharge`: Battery discharge statistics
106+
- `solar_contribution`: Solar panel contribution
107+
108+
## Examples
109+
110+
### Basic Processing
111+
112+
Process a CSV file and display all metrics:
113+
114+
```bash
115+
./enemeter-data-processing --input=data/data.csv
116+
```
117+
118+
### Memory-Efficient Processing for Large Files
119+
120+
Use streaming mode with sampling for very large files:
121+
122+
```bash
123+
./enemeter-data-processing --input=big-data.csv --stream --sample=10
124+
```
125+
126+
This processes only every 10th record, reducing memory requirements.
127+
128+
### Filtering by Time
129+
130+
Process data from a specific time range:
131+
132+
```bash
133+
./enemeter-data-processing --input=data.csv --start="2025-04-01" --end="2025-04-02"
134+
```
135+
136+
Process the last 24 hours of data:
137+
138+
```bash
139+
./enemeter-data-processing --input=data.csv --window=24h
140+
```
141+
142+
### Extracting Specific Metrics
143+
144+
Get only temperature statistics in JSON format:
145+
146+
```bash
147+
./enemeter-data-processing --input=data.csv --metric=temperature --format=json
148+
```
149+
150+
Extract hourly energy consumption in CSV format:
151+
152+
```bash
153+
./enemeter-data-processing --input=data.csv --metric=energy_by_hour --format=csv
154+
```
155+
156+
### Filtering by Data Values
157+
158+
Process only records with voltage between specified values:
159+
160+
```bash
161+
./enemeter-data-processing --input=data.csv --volt-min=3500000 --volt-max=4200000
162+
```
163+
164+
Process only records where temperature is above a certain threshold:
165+
166+
```bash
167+
./enemeter-data-processing --input=data.csv --min-temp=25000
168+
```
169+
170+
## Output Examples
171+
172+
### Text Output (Default)
173+
174+
```
175+
========== ENEMETER DATA PROCESSING REPORT ==========
176+
Input File: data.csv
177+
Date: 2025-04-09 10:15:32
178+
Data Points: 1253
179+
Time Range: 2025-04-08 08:30:15 to 2025-04-09 08:30:10
180+
181+
ENERGY METRICS
182+
-------------
183+
Total Energy Consumed: 156.4578 joules
184+
Average Power: 0.0548 watts
185+
Peak Power: 0.1245 watts
186+
Estimated Energy per Day: 157.9825 joules
187+
Measurement Duration: 86395.00 seconds
188+
189+
TEMPERATURE STATISTICS
190+
---------------------
191+
Minimum Temperature: 22.45 °C
192+
Maximum Temperature: 28.95 °C
193+
Average Temperature: 24.75 °C
194+
195+
... additional sections ...
196+
```
197+
198+
### JSON Output
199+
200+
```json
201+
{
202+
"TotalJoules": 156.4578,
203+
"AveragePowerWatts": 0.0548,
204+
"PeakPowerWatts": 0.1245,
205+
"JoulesPerDay": 157.9825,
206+
"DurationSeconds": 86395.0,
207+
"TemperatureStats": {
208+
"MinTempCelsius": 22.45,
209+
"MaxTempCelsius": 28.95,
210+
"AvgTempCelsius": 24.75
211+
},
212+
...
213+
}
214+
```
215+
216+
### CSV Output
217+
218+
```
219+
Metric,Value
220+
TotalJoules,156.457800
221+
AveragePowerWatts,0.054800
222+
PeakPowerWatts,0.124500
223+
JoulesPerDay,157.982500
224+
DurationSeconds,86395.00
225+
...
226+
```
227+
228+
## Contributing
229+
230+
Contributions are welcome! Please feel free to submit a Pull Request.
231+
232+
## License
233+
234+
This project is licensed under the MIT License - see the LICENSE file for details.

‎cmd/enemeter-data-processing/main.go‎

Lines changed: 60 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,60 @@
1+
package main
2+
3+
import (
4+
"enemeter-data-processing/internal/commands"
5+
"fmt"
6+
"os"
7+
)
8+
9+
func main() {
10+
// Display version and app name
11+
fmt.Printf("%s version: %s\n", commands.AppName, commands.CurrentVersion)
12+
13+
if len(os.Args) < 2 {
14+
printUsage()
15+
os.Exit(1)
16+
}
17+
18+
// Check the command
19+
switch os.Args[1] {
20+
case "process":
21+
// Create and configure the process command
22+
processCmd := commands.SetupProcessCommand()
23+
if err := processCmd.Parse(os.Args[2:]); err != nil {
24+
fmt.Fprintf(os.Stderr, "Error parsing arguments: %v\n", err)
25+
processCmd.Usage()
26+
os.Exit(1)
27+
}
28+
29+
// Parse CLI options and execute the process command
30+
options := commands.ParseCommandLineOptions(processCmd)
31+
if err := commands.ProcessCommand(options); err != nil {
32+
fmt.Fprintf(os.Stderr, "Error: %v\n", err)
33+
os.Exit(1)
34+
}
35+
36+
case "version":
37+
fmt.Printf("%s\n", commands.CurrentVersion)
38+
39+
case "help":
40+
printUsage()
41+
42+
default:
43+
fmt.Fprintf(os.Stderr, "Unknown command: %s\n", os.Args[1])
44+
printUsage()
45+
os.Exit(1)
46+
}
47+
}
48+
49+
// printUsage displays help information for the CLI tool
50+
func printUsage() {
51+
fmt.Printf("%s - A tool for processing and analyzing ENEMETER data\n\n", commands.AppName)
52+
fmt.Println("Usage:")
53+
fmt.Println(" enemeter-cli <command> [options]")
54+
fmt.Println("\nAvailable Commands:")
55+
fmt.Println(" process Process ENEMETER data files")
56+
fmt.Println(" version Show version information")
57+
fmt.Println(" help Show help information")
58+
fmt.Println("\nFor command-specific help:")
59+
fmt.Println(" enemeter-cli <command> --help")
60+
}

‎cmd/rubrion-cli/main.go‎

Lines changed: 0 additions & 18 deletions
This file was deleted.

‎go.mod‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,3 @@
1-
module rubrion-cli
1+
module enemeter-data-processing
22

33
go 1.23

‎go.sum‎

Whitespace-only changes.

0 commit comments

Comments
 (0)