Solar battery sim date intervals not exact number of days?

[pcjc2]

Hi,

I have been looking at adding an option to the app to allow looking at previous years solar data. (To avoid date confusion, just subtracting a years worth of milliseconds from the start and end timestamps).

I noticed that sometimes I would be out by one on the length of feed data returned, and that the difference in ms between start and end was not a round number of days. (Divide (end-start) by (1000 * 60 * 60 * 24).)

[pcjc2]

commit 2c3f4557c905de4175e2f5a3f811862ddb13067f (HEAD -> master)
Author: Peter Clifton <pcjc@xev-power.com>
Date:   Thu Sep 29 20:19:23 2022 +0100

    Add year offset function

diff --git a/apps/OpenEnergyMonitor/solarbatterysim/solarbatterysim.php b/apps/OpenEnergyMonitor/solarbatterysim/solarbatterysim.php
index 8aba5d7..c896918 100644
--- a/apps/OpenEnergyMonitor/solarbatterysim/solarbatterysim.php
+++ b/apps/OpenEnergyMonitor/solarbatterysim/solarbatterysim.php
@@ -69,6 +69,11 @@ textarea {
         </div><br>


+        <div class="input-prepend">
+          <span class="add-on" style="width:140px">Solar year offset</span>
+          <input type="text" v-model.number="input.solar_year_offset" style="width:120px" />
+        </div><br>
+
         <div class="input-prepend">
           <span class="add-on" style="width:140px">Solar capacity</span>
           <input type="text" v-model.number="input.solar_capacity" style="width:120px" />
@@ -297,6 +302,7 @@ var input = {
     solar_existing: false,
     solar_capacity: 3000,

+    solar_year_offset: 0,
     battery_capacity: 8.5,
     battery_max_charge_rate: 3000,
     battery_max_discharge_rate: 3000,
@@ -384,6 +390,10 @@ function process_month(d) {

    var month = ["January","February","March","April","May","June","July","August","September","October","November","December"];

+   sd = new Date(d);
+   sd.setFullYear(sd.getFullYear()-input.solar_year_offset);
+   s_month_name = month[sd.getMonth()]+" "+sd.getFullYear();
+
     var month = {
         name: month[d.getMonth()]+" "+d.getFullYear(),
         total_consumption: 0,
@@ -416,23 +426,27 @@ function process_month(d) {
     d.setMonth(d.getMonth()+1);

     end = d.getTime();
+    end = start + Math.round( (end-start) / 3600000.0 / 24.0 ) * 24 * 3600000; // Round to an integral number of days

     if (cache_use[month.name]==undefined) {
-        use_data_month = feed.getdata(config.app.use.value,start,end-(interval*1000),interval,0,1,0,0);
+        use_data_month = feed.getdata(config.app.use.value,start,end-(interval*1000)-1,interval,0,1,0,0);
         cache_use[month.name] = use_data_month;
     } else {
         use_data_month = cache_use[month.name];
     }
-
-    if (cache_solar[month.name]==undefined) {
+
+    s_start = start - input.solar_year_offset * 31536000000;
+    s_end   = end   - input.solar_year_offset * 31536000000;
+
+    if (cache_solar[s_month_name]==undefined) {
         if (config.app.solar.value!='disable' && config.app.solar.value>0) {
-            solar_data_month = feed.getdata(config.app.solar.value,start,end-(interval*1000),interval,0,1,0,0);
+            solar_data_month = feed.getdata(config.app.solar.value,s_start,s_end-(interval*1000)-1,interval,0,1,0,0);
         } else {
-            solar_data_month = getdataremote(config.app.public_solar_feed.value,start,end-(interval*1000),interval,0,1,0,0)
+            solar_data_month = getdataremote(config.app.public_solar_feed.value,s_start,s_end-(interval*1000)-1,interval,0,1,0,0)
         }
-        cache_solar[month.name] = solar_data_month;
+        cache_solar[s_month_name] = solar_data_month;
     } else {
-        solar_data_month = cache_solar[month.name];
+        solar_data_month = cache_solar[s_month_name];
     }

     var use = 0;

[pcjc2]

I'm doing this so I can compare against historical data imported from a EU GIS database, but who's conveniently available data ends in 2020. E.g. I need to go back a few years (at least) to compare against this data.

Initially I just offset the timestamps of my imported data, but that feels wrong (and I can adjust the app to explicitly perform this function).

However - due to dates, leap-years etc... it seems most convenient to subtract an integer number of days worth of seconds from the data time-stamps per year (even if that is not super accurate over multiple years).

When I ran my initial attempts, I would often find the code crashing out due to the solar array data being one element shorter than the use data for a given month (probably the other way around was possible too, but the code would not error on this).

I found the start and end time-stamps being used in the app don't always land an integer number of hours apart.

Additionally, having "fixed" this with some rounding, I would still get the errors. My solar feed start timestamp isn't perfectly aligned with my usage data, so I thought that would potentially be contributing.

In the end, it was the "-1" on the end-(interval*1000) -1 that appears to fix this bug.

The feed engine PHP seems to use a "<= $end " criterion for its data processing loop, which seems to be where I was getting exactly one more (or one fewer?) data-points, depending on the exact alignment of everything.

Not sure if any of this is a bug, but it feels wrong not to get a very predictable length of data when querying two different feeds for the same delta of start-end, and interval.

Thoughts?