diff --git a/ROADMAP.md b/ROADMAP.md index 070552d07e..3bfb76cb33 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -2,7 +2,7 @@ **Stage: alpha** · next: beta, ~25 points of "ready for real work" and ~160–240 h away -> **Last reviewed:** 2026-10-11 · **Last updated:** 2026-10-12 · **Change:** minor (rows auto-fit oversized fonts on open and whole-row/column formatting) · **Target:** Microsoft Excel (Microsoft 365) +> **Last reviewed:** 2026-10-11 · **Last updated:** 2026-10-14 · **Change:** minor (What-If data tables: Data Table dialog, faster recalculation, checked in Excel) · **Target:** Microsoft Excel (Microsoft 365) GridCraft aims at full Microsoft Excel parity, and at being better in places: faster, open (XLSX is the native format), fully scriptable by agents (MCP, control channel, CLI), and on the web. @@ -132,6 +132,8 @@ Ranked; estimates in Opus 5.5 agent-hours. Detail and the full beta-gate list in | Date | What landed | |---|---| +| 2026-10-14 | Data › What-If Analysis › Data Table… opens Excel's dialog (Row input cell, Column input cell; German labels) and makes a live `{=TABLE()}` over the selection; the What-If menu follows Excel's order (Scenario Manager, Goal Seek, Data Table). Tables recalculate on one working copy each (over 10× faster after an edit on large tables), an edited input value recalculates only its own row or column, and a work budget keeps a hostile file from hanging the app. XLSX data tables written by GridCraft open live in Excel 365, and Excel's own form is read (builds on #548 by prxnvt) | +| 2026-10-12 | What-If data tables are live: `{=TABLE()}` results recalculate when their inputs or formulas change (on F9 in Automatic Except for Data Tables mode), edits of part of a table are refused, and tables round-trip through XLSX | | 2026-10-12 | Rows auto-fit oversized fonts: opening a file and whole-row/column formatting grow non-custom rows to their tallest font (#833) | | 2026-10-12 | Number formats: parsing a format code is linear in its length (long runs of `[`, `,` or `/` no longer take quadratic time), and an unclosed `[` is a literal when splitting sections as well as when tokenizing (#810) | | 2026-10-12 | Number formats: a fraction denominator with 65,536 zeros no longer crashes; format codes are capped at 255 characters like Excel (longer codes display as General, `TEXT` gives `#VALUE!`) (#807) | @@ -193,6 +195,8 @@ Ranked; estimates in Opus 5.5 agent-hours. Detail and the full beta-gate list in | Date | Change | Summary | |---|---|---| +| 2026-10-14 | minor | Progress log: Data Table dialog, data table recalculation speed, Excel round trip | +| 2026-10-12 | minor | Live What-If data tables; progress log row | | 2026-10-12 | minor | Rows auto-fit oversized fonts on open and whole-row/column formatting (#833); progress log row | | 2026-10-12 | minor | Progress log: multi-area row/column delete (#758) | | 2026-10-12 | trivial | Progress log: XLSX chart sheets kept on save (#727) | diff --git a/crates/calc/src/lib.rs b/crates/calc/src/lib.rs index 27c9e19610..1e9d094bb3 100644 --- a/crates/calc/src/lib.rs +++ b/crates/calc/src/lib.rs @@ -8,7 +8,7 @@ pub mod recalc; mod trimrange; pub use eval::{Area, SPECIAL_FUNCTIONS, is_known_function}; -pub use recalc::{Calc, Key, evaluate, now_serial}; +pub use recalc::{Calc, Key, Trial, evaluate, now_serial}; #[cfg(test)] mod tests; diff --git a/crates/calc/src/recalc.rs b/crates/calc/src/recalc.rs index b391821d72..09b0385a24 100644 --- a/crates/calc/src/recalc.rs +++ b/crates/calc/src/recalc.rs @@ -9,6 +9,10 @@ //! calculation is on: then a formula read while it is being evaluated gives its previous value, //! and the formulas that read one are evaluated again until the loop settles //! (`PassHost::iterate_loops`). +//! +//! What-If data tables (`{=TABLE()}`) are calculated after the formulas: each result is a +//! [`Trial`] that substitutes an edge value into the input cell and recalculates the formula on +//! a copy of the workbook ([`Calc::refresh_tables`]). use std::collections::{BTreeSet, VecDeque}; use std::hash::{BuildHasherDefault, Hasher}; @@ -42,7 +46,7 @@ type HashSet = std::collections::HashSet>; use gridcraft_core::{Array, CellError, CellRef, RangeRef, Value}; use gridcraft_formula::Expr; -use gridcraft_model::{CalcMode, Workbook}; +use gridcraft_model::{CalcMode, DataTable, TableTrial, Workbook}; use crate::eval::{Area, Evaluator, Host, precedents}; @@ -371,11 +375,14 @@ pub struct Calc { /// Cells whose formulas have errors from cycles. pub circular: Vec, pub last_recalc_cells: usize, + /// Set inside a what-if trial, where data tables are not calculated (a table whose formulas + /// depend on a data table would otherwise recurse). + in_trial: bool, } impl Default for Calc { fn default() -> Self { - Calc { graph: Graph::default(), rng: 0x9E37_79B9_7F4A_7C15, fixed_now: None, circular: vec![], last_recalc_cells: 0 } + Calc { graph: Graph::default(), rng: 0x9E37_79B9_7F4A_7C15, fixed_now: None, circular: vec![], last_recalc_cells: 0, in_trial: false } } } @@ -436,8 +443,9 @@ impl<'a> PassHost<'a> { } /// Whether an array result at `k` can't spill: it is in a table, or its area runs off the sheet, -/// overlaps merged cells, holds other cells, or another formula's spill (unless that formula is -/// recalculated in this pass, and so may spill elsewhere now). +/// overlaps merged cells or a data table's results (formula cells in Excel, even while blank), +/// holds other cells, or another formula's spill (unless that formula is recalculated in this +/// pass, and so may spill elsewhere now). fn spill_blocked(host: &PassHost<'_>, (sheet, anchor): Key, a: &Array) -> bool { let Some(sh) = host.wb.sheet(sheet) else { return false }; let end = @@ -452,6 +460,7 @@ fn spill_blocked(host: &PassHost<'_>, (sheet, anchor): Key, a: &Array) -> bool { || end.col >= gridcraft_core::MAX_COLS || sh.merges.iter().any(|m| m.intersects(&range)) || sh.tables.iter().any(|t| t.range.contains(anchor)) + || sh.data_tables.iter().any(|t| t.body().is_some_and(|b| b.intersects(&range))) || sh.cells.iter_range(range).any(|(c, cell)| c != anchor && (!cell.value.is_empty() || cell.formula.is_some())) || sh.spill.range(range.start..=range.end).any(|(c, _)| range.contains(*c) && spilled_over(*c)) } @@ -809,7 +818,7 @@ impl Calc { self.graph.settle(); } - /// Recalculates every formula (F9 / Ctrl+Alt+F9, and after loading). + /// Recalculates every formula (F9 / Ctrl+Alt+F9, and after loading), data tables included. pub fn recalc_all(&mut self, wb: &mut Workbook) { self.rebuild(wb); // Spills are laid out again from the formulas where they are now (a structural edit moves @@ -824,6 +833,10 @@ impl Calc { } let all: Vec = self.graph.nodes.keys().copied().collect(); self.run(wb, all, true); + // `last_recalc_cells` keeps counting the workbook's formulas, not the last table pass. + let cells = self.last_recalc_cells; + self.refresh_tables(wb, None); + self.last_recalc_cells = cells; } /// Updates the graph for cells whose content changed and recalculates what depends on them. @@ -908,13 +921,31 @@ impl Calc { let dirty = self.dirty_closure(wb, &seeds); let t1 = prof_now(); let n = dirty.len(); + // Data tables whose edges changed are calculated after the formulas (not in Automatic + // Except for Data Tables mode, where they wait for F9). + let tables = !self.in_trial && wb.calc.mode == CalcMode::Automatic && wb.sheets.iter().any(|s| !s.data_tables.is_empty()); + let mut trigger: Option = tables.then(|| Changes { + cells: changed.iter().chain(dirty.iter()).copied().collect(), + spills: cleared.iter().copied().chain(spill_areas(wb, &dirty)).collect(), + }); + let anchors: Vec = if tables { dirty.clone() } else { Vec::new() }; self.run(wb, dirty, false); - if std::env::var_os("GRIDCRAFT_PROFILE").is_some() { + if let Some(mut t) = trigger.take() { + t.spills.extend(spill_areas(wb, &anchors)); + self.refresh_tables(wb, Some(t)); + } + if profiling() && !self.in_trial { eprintln!("recalc: {} seeds, closure {} cells {:.1} ms, run {:.1} ms", seeds.len(), n, t1 - t0, prof_now() - t1); } } fn dirty_closure(&self, _wb: &Workbook, seeds: &[Key]) -> Vec { + self.closure(seeds, true) + } + + /// The formulas that depend on `seeds` (and the seeds that are formulas), transitively; with + /// `dynamic`, also every formula whose references are only known while evaluating. + fn closure(&self, seeds: &[Key], dynamic: bool) -> Vec { let mut seen: HashSet = HashSet::default(); let mut queue: VecDeque = VecDeque::new(); for k in seeds { @@ -929,7 +960,7 @@ impl Calc { } } } - for k in &self.graph.dynamic { + for k in self.graph.dynamic.iter().filter(|_| dynamic) { if seen.insert(*k) { queue.push_back(*k); } @@ -949,6 +980,8 @@ impl Calc { /// Evaluates the dirty set and writes results (and spills) back. fn run(&mut self, wb: &mut Workbook, mut dirty: Vec, full: bool) { + // What-if substitutions run this once each: they stay quiet. + let profile = profiling() && !self.in_trial; let mut dynamic_again = !self.graph.dynamic.is_empty(); for _round in 0..8 { if dirty.is_empty() { @@ -973,14 +1006,14 @@ impl Calc { } if host.iterate && !host.members.is_empty() { let passes = host.iterate_loops(&dirty, wb.calc.max_iterations, wb.calc.max_change); - if std::env::var_os("GRIDCRAFT_PROFILE").is_some() { + if profile { eprintln!(" iterative calculation: {} passes over {} formulas", passes, host.tainted.len()); } } (host.results, host.spills, host.cycles) }; self.rng = rng; - if std::env::var_os("GRIDCRAFT_PROFILE").is_some() { + if profile { eprintln!(" eval {} formulas {:.1} ms", results.len(), prof_now() - tp); } let tw = prof_now(); @@ -1018,7 +1051,8 @@ impl Calc { || end.col >= gridcraft_core::MAX_COLS || range.iter().any(|c| c != k.1 && (sheet.cells.has(c) || sheet.spill.contains_key(&c))) || sheet.merges.iter().any(|m| m.intersects(&range)) - || sheet.tables.iter().any(|t| t.range.contains(k.1)); + || sheet.tables.iter().any(|t| t.range.contains(k.1)) + || sheet.data_tables.iter().any(|t| t.body().is_some_and(|b| b.intersects(&range))); if blocked { if let Some(cell) = sheet.cells.get_mut(k.1) { cell.value = Value::Error(CellError::Spill); @@ -1065,7 +1099,7 @@ impl Calc { if full { // Everything was already evaluated once; spill dependents need another pass. } - if std::env::var_os("GRIDCRAFT_PROFILE").is_some() { + if profile { eprintln!(" write-back {:.1} ms", prof_now() - tw); } spill_changes.sort_by_key(|(s, c)| (*s, c.row, c.col)); @@ -1085,6 +1119,255 @@ pub fn now_serial() -> f64 { gridcraft_core::date::unix_now_ms().map_or(46302.5, gridcraft_core::date::serial_from_unix_ms) } +// ---------------------------------------------------------------- what-if trials and data tables + +/// Evaluates a workbook with some cells replaced by constants, without changing it: the workbook +/// is cloned (cheap, as sheets and cell bands are copy-on-write) and only the replaced cells' +/// dependents are recalculated with the dependency graph. Goal Seek, Scenario Manager and Solver +/// use it; data tables run the same substitution on one working copy per table +/// ([`Calc::compute_table`]). +pub struct Trial { + base: Workbook, + calc: Calc, +} + +impl Trial { + pub fn new(wb: &Workbook, calc: &Calc) -> Trial { + let mut base = wb.clone(); + // What-if analysis always recalculates, even in manual mode. + base.calc.mode = CalcMode::Automatic; + let mut calc = calc.clone(); + calc.in_trial = true; + Trial { base, calc } + } + + /// Sets `sets` to constants, recalculates their dependents and reads `reads`. + pub fn run(&mut self, sets: &[(usize, CellRef, Value)], reads: &[(usize, CellRef)]) -> Vec { + let mut wb = self.base.clone(); + substitute(&mut self.calc, &mut wb, sets, reads) + } +} + +/// One what-if substitution on `wb`: `sets` become constants, their dependents are recalculated +/// and `reads` are read. +fn substitute(calc: &mut Calc, wb: &mut Workbook, sets: &[(usize, CellRef, Value)], reads: &[(usize, CellRef)]) -> Vec { + let mut keys = Vec::with_capacity(sets.len()); + for (si, c, v) in sets { + if let Some(sh) = wb.sheet_mut(*si) { + let style = sh.style_id(*c); + sh.set_cell(*c, gridcraft_model::Cell { value: v.clone(), formula: None, style }); + keys.push((*si, *c)); + } + } + calc.cells_changed(wb, &keys); + reads.iter().map(|(si, c)| wb.sheet(*si).map(|s| s.value(*c)).unwrap_or_default()).collect() +} + +/// What a recalculation changed, for [`Calc::refresh_tables`]: cells whose content or value +/// changed, and the areas dynamic arrays spilled over before and after it (their values changed +/// too, without being formulas). +struct Changes { + cells: HashSet, + spills: Vec<(usize, RangeRef)>, +} + +impl Changes { + fn contains(&self, si: usize, c: CellRef) -> bool { + self.cells.contains(&(si, c)) || self.spills.iter().any(|(s, r)| *s == si && r.contains(c)) + } + + /// Whether a change is on `t`'s top row or left column. + fn touches(&self, si: usize, t: &DataTable) -> bool { + let r = t.range; + let top = RangeRef::new(r.start, CellRef::new(r.start.row, r.end.col)); + let left = RangeRef::new(r.start, CellRef::new(r.end.row, r.start.col)); + self.cells.iter().any(|(s, c)| *s == si && t.is_edge(*c)) + || self.spills.iter().any(|(s, a)| *s == si && (a.intersects(&top) || a.intersects(&left))) + } +} + +/// The areas the formulas in `keys` spill over now. +fn spill_areas(wb: &Workbook, keys: &[Key]) -> Vec<(usize, RangeRef)> { + keys.iter().filter_map(|(si, c)| Some((*si, *wb.sheet(*si)?.spill_ranges.get(c)?))).collect() +} + +/// Result cells one refresh of the data tables may calculate, over all tables and passes. Tables +/// past the budget keep their values until the next refresh (a hostile file can't hang a recalc). +const MAX_REFRESH_CELLS: u64 = 1_000_000; + +/// Formula evaluations one refresh may spend on data tables, over all tables and passes: each +/// substitution recalculates the formulas that depend on the input cells. Like the cell budget, a +/// table past it keeps its values until the next refresh. A refresh that spends all of it takes +/// some seconds; a hostile file asking for more can't hang the application. +const MAX_REFRESH_WORK: u64 = 25_000_000; + +/// Most passes of [`Calc::refresh_tables`]: a table whose formulas read another table's results +/// is calculated after it, in the next pass when it comes first. +const MAX_TABLE_PASSES: usize = 64; + +impl Calc { + /// Calculates some of one data table's substitutions: `(cell, value)` for each of their result + /// cells. They all set the same input cells, so they run one after another on a single working + /// copy of the workbook: each recalculates everything that depends on the input cells and so + /// reads what a fresh copy would. (A fresh copy per substitution would copy the 64-row cell + /// bands around the input cells and formulas once per result row, column or cell.) Input + /// cells that hold constants, the usual case, leave the dependency graph as it is, so this + /// calculation's own graph serves the substitutions; an input cell with a formula takes it + /// out of the graph, and then a copy of the graph is used. + fn compute_table(&mut self, wb: &Workbook, si: usize, trials: &[TableTrial]) -> Vec<(CellRef, Value)> { + let Some(sh) = wb.sheet(si) else { return vec![] }; + let Some(first) = trials.first() else { return vec![] }; + let mut work = wb.clone(); + // What-if analysis always recalculates, even in manual mode. + work.calc.mode = CalcMode::Automatic; + let formula_inputs = first.inputs.iter().any(|(c, _)| sh.cell(*c).is_some_and(|x| x.formula.is_some())); + let mut copy; + let calc: &mut Calc = if formula_inputs { + copy = self.clone(); + &mut copy + } else { + self + }; + // The substitutions leave no trace in this calculation's state. + let saved = (calc.rng, std::mem::take(&mut calc.circular), calc.last_recalc_cells, calc.in_trial); + calc.in_trial = true; + let mut out = Vec::with_capacity(trials.iter().map(|tt| tt.outputs.len()).sum()); + for tt in trials { + let sets: Vec<(usize, CellRef, Value)> = tt.inputs.iter().map(|(input, from)| (si, *input, sh.value(*from))).collect(); + let reads: Vec<(usize, CellRef)> = tt.outputs.iter().map(|(f, _)| (si, *f)).collect(); + let values = substitute(calc, &mut work, &sets, &reads); + out.extend(tt.outputs.iter().map(|(_, o)| *o).zip(values)); + } + (calc.rng, calc.circular, calc.last_recalc_cells, calc.in_trial) = saved; + out + } + + /// The data tables a refresh calculates, as `(sheet, table, circular)`: valid ones, within the + /// cell budget, flagged when the table's edge cells depend on its own results (directly or + /// through other tables). Like any circular reference, those give `#CIRC!`. Dependencies + /// through references only known while evaluating (INDIRECT, OFFSET) aren't followed here; + /// [`MAX_TABLE_PASSES`] bounds those loops. + fn table_plan(&self, wb: &Workbook) -> Vec<(usize, DataTable, bool)> { + let mut tables: Vec<(usize, DataTable)> = Vec::new(); + let mut budget = MAX_REFRESH_CELLS; + for (si, sh) in wb.sheets.iter().enumerate() { + for t in sh.data_tables.iter().filter(|t| t.is_valid()) { + let Some(rest) = t.body().and_then(|b| budget.checked_sub(b.count())) else { continue }; + budget = rest; + tables.push((si, t.clone())); + } + } + // feeds[i]: the tables whose edge formulas read table i's results. + let feeds: Vec> = tables + .iter() + .map(|(si, t)| { + let seeds: Vec = t.body().map(|b| b.iter().map(|c| (*si, c)).collect()).unwrap_or_default(); + let reach = self.closure(&seeds, false); + (0..tables.len()).filter(|&j| tables.get(j).is_some_and(|(sj, u)| reach.iter().any(|(s, c)| s == sj && u.is_edge(*c)))).collect() + }) + .collect(); + let circular = |i: usize| -> bool { + let mut seen = vec![false; tables.len()]; + let mut stack: Vec = feeds.get(i).cloned().unwrap_or_default(); + while let Some(j) = stack.pop() { + if j == i { + return true; + } + if let Some(s) = seen.get_mut(j) + && !*s + { + *s = true; + stack.extend(feeds.get(j).into_iter().flatten().copied()); + } + } + false + }; + let flags: Vec = (0..tables.len()).map(circular).collect(); + tables.into_iter().zip(flags).map(|((si, t), c)| (si, t, c)).collect() + } + + /// Calculates data tables: all of them (`trigger = None`), or those with an edge cell (input + /// value or formula) in `trigger`. A changed formula, or a changed precedent of one, calculates + /// the whole table again; changed input values only their own rows, columns or cells. Results + /// are written as values and their dependents recalculated; tables that read other tables' + /// results are calculated again in later passes. + fn refresh_tables(&mut self, wb: &mut Workbook, trigger: Option) { + if self.in_trial { + return; + } + if let Some(tr) = &trigger + && !wb.sheets.iter().enumerate().any(|(si, sh)| sh.data_tables.iter().any(|t| tr.touches(si, t))) + { + return; + } + let t0 = prof_now(); + let plan = self.table_plan(wb); + let mut trigger = trigger; + let mut work = MAX_REFRESH_WORK; + let mut substitutions = 0usize; + let circ = Value::Error(CellError::Circ); + for pass in 0..plan.len().clamp(1, MAX_TABLE_PASSES) { + let mut written: Vec = Vec::new(); + for (si, t, circular) in &plan { + let si = *si; + let Some(body) = t.body() else { continue }; + // Whether a table is in a loop can change with an edit of another table's + // formulas: circular tables, and tables that were circular, are checked whenever + // a refresh runs. + let was_circular = pass == 0 && wb.sheet(si).is_some_and(|sh| sh.cells.iter_range(body).any(|(_, x)| x.value == circ)); + let results: Vec<(CellRef, Value)> = if *circular { + body.iter().map(|c| (c, circ.clone())).collect() + } else { + let trials: Vec = match &trigger { + Some(tr) if !was_circular => { + if !tr.touches(si, t) { + continue; + } + let all = t.trials(); + if t.formulas().is_some_and(|f| f.iter().any(|c| tr.contains(si, c))) { + all + } else { + all.into_iter().filter(|tt| tt.inputs.iter().any(|(_, from)| tr.contains(si, *from))).collect() + } + } + _ => t.trials(), + }; + if trials.is_empty() { + continue; + } + // Each substitution recalculates what depends on the input cells. + let inputs: Vec = [t.row_input, t.col_input].into_iter().flatten().map(|c| (si, c)).collect(); + let per = self.closure(&inputs, true).len() as u64 + 1; + let Some(rest) = work.checked_sub((trials.len() as u64).saturating_mul(per)) else { continue }; + work = rest; + substitutions += trials.len(); + self.compute_table(wb, si, &trials) + }; + let Some(sh) = wb.sheet_mut(si) else { continue }; + for (c, v) in results { + if sh.value(c) != v || sh.cell(c).is_some_and(|x| x.formula.is_some()) { + let style = sh.style_id(c); + sh.set_cell(c, gridcraft_model::Cell { value: v, formula: None, style }); + written.push((si, c)); + } + } + } + if written.is_empty() { + break; + } + let dirty = self.dirty_closure(wb, &written); + let mut next = Changes { cells: written.iter().chain(dirty.iter()).copied().collect(), spills: spill_areas(wb, &dirty) }; + let anchors = dirty.clone(); + self.run(wb, dirty, false); + next.spills.extend(spill_areas(wb, &anchors)); + trigger = Some(next); + } + if profiling() && substitutions > 0 { + eprintln!("data tables: {} substitutions over {} tables {:.1} ms", substitutions, plan.len(), prof_now() - t0); + } + } +} + /// Evaluates a formula in the context of a cell without storing it (conditional formats, data /// validation, Evaluate Formula, Name Manager previews, the CLI `eval`). pub fn evaluate(wb: &Workbook, sheet: usize, at: CellRef, formula: &str) -> Value { @@ -1101,6 +1384,13 @@ pub fn evaluate_expr(wb: &Workbook, sheet: usize, at: CellRef, expr: &Expr) -> V ev.value(expr) } +/// Whether `GRIDCRAFT_PROFILE` is set. Read once: recalculation checks it several times per +/// pass, and a data table runs one pass per result row, column or cell. +fn profiling() -> bool { + static ON: std::sync::OnceLock = std::sync::OnceLock::new(); + *ON.get_or_init(|| std::env::var_os("GRIDCRAFT_PROFILE").is_some()) +} + #[allow(clippy::disallowed_methods)] // the clock is read only off wasm fn prof_now() -> f64 { #[cfg(not(target_arch = "wasm32"))] diff --git a/crates/calc/src/tests.rs b/crates/calc/src/tests.rs index 196c57de78..8df6aa67f5 100644 --- a/crates/calc/src/tests.rs +++ b/crates/calc/src/tests.rs @@ -1171,6 +1171,158 @@ fn trim_references() { assert_eq!(t.get("K20"), Value::Error(CellError::Calc)); } +fn table(range: &str, row_input: Option<&str>, col_input: Option<&str>) -> gridcraft_model::DataTable { + gridcraft_model::DataTable { range: gridcraft_core::RangeRef::parse(range).unwrap(), row_input: row_input.map(c), col_input: col_input.map(c) } +} + +#[test] +fn data_tables_recalculate_with_their_inputs_and_formulas() { + let mut t = T::new(); + // Rate B1, amount B2; a one-variable table with amounts down D3:D5 and two formulas. + t.set("B1", "0.1"); + t.set("B2", "1000"); + for (cell, v) in [("D3", "100"), ("D4", "200"), ("D5", "300")] { + t.set(cell, v); + } + t.set("E2", "=B2*(1+B1)"); + t.set("F2", "=B2*2"); + t.wb.sheet_mut(0).unwrap().data_tables.push(table("D2:F5", None, Some("B2"))); + t.set("H1", "=SUM(E3:E5)"); + t.calc.recalc_all(&mut t.wb); + assert!((t.num("E3") - 110.0).abs() < 1e-9); + assert_eq!(t.num("F5"), 600.0); + assert!((t.num("H1") - 660.0).abs() < 1e-9, "formulas over the results recalculate"); + assert_eq!(t.num("B2"), 1000.0, "the input cell keeps its value"); + // A precedent of the formulas changes: the results and their dependents follow. + t.set("B1", "0.5"); + assert_eq!(t.num("E4"), 300.0); + assert_eq!(t.num("H1"), 900.0); + // So does an input value on the edge. + t.set("D3", "1000"); + assert_eq!(t.num("E3"), 1500.0); + // And a formula on the edge. + t.set("F2", "=B2*3"); + assert_eq!(t.num("F4"), 600.0); + // Two-variable: rates across the top (row input B1), amounts down the side (column input B2). + t.set("J1", "=B2*B1"); + for (cell, v) in [("K1", "1"), ("L1", "2"), ("J2", "10"), ("J3", "20")] { + t.set(cell, v); + } + t.wb.sheet_mut(0).unwrap().data_tables.push(table("J1:L3", Some("B1"), Some("B2"))); + t.calc.recalc_all(&mut t.wb); + assert_eq!((t.num("K2"), t.num("L2"), t.num("K3"), t.num("L3")), (10.0, 20.0, 20.0, 40.0)); + t.set("J1", "=B2*B1+1"); + assert_eq!(t.num("L3"), 41.0); + t.set("L1", "3"); + assert_eq!(t.num("L3"), 61.0); + // Row-oriented one-variable table: rates across the top, a formula down the left column. + t.set("N2", "=B2*B1"); + for (cell, v) in [("O1", "1"), ("P1", "3")] { + t.set(cell, v); + } + t.wb.sheet_mut(0).unwrap().data_tables.push(table("N1:P2", Some("B1"), None)); + t.calc.recalc_all(&mut t.wb); + assert_eq!((t.num("O2"), t.num("P2")), (1000.0, 3000.0)); +} + +#[test] +fn data_tables_wait_for_f9_in_except_tables_and_manual_modes() { + for mode in [gridcraft_model::CalcMode::AutomaticExceptTables, gridcraft_model::CalcMode::Manual] { + let mut t = T::new(); + t.set("B1", "2"); + t.set("E1", "=B1*10"); + t.set("D2", "1"); + t.set("D3", "2"); + t.wb.sheet_mut(0).unwrap().data_tables.push(table("D1:E3", None, Some("B1"))); + t.calc.recalc_all(&mut t.wb); + assert_eq!(t.num("E3"), 20.0); + t.wb.calc.mode = mode; + t.set("E1", "=B1*100"); + assert_eq!(t.num("E3"), 20.0, "{mode:?}"); + t.calc.recalc_all(&mut t.wb); + assert_eq!(t.num("E3"), 200.0, "{mode:?}"); + } +} + +#[test] +fn data_tables_that_read_their_own_results_are_circular() { + let mut t = T::new(); + // A formula on the edge reads the table's own results: a circular reference. + t.set("B1", "1"); + t.set("E1", "=B1+SUM(E2:E3)"); + t.set("D2", "1"); + t.set("D3", "2"); + t.wb.sheet_mut(0).unwrap().data_tables.push(table("D1:E3", None, Some("B1"))); + t.calc.recalc_all(&mut t.wb); + assert_eq!(t.get("E2"), Value::Error(CellError::Circ)); + t.set("B1", "5"); + assert_eq!(t.get("E3"), Value::Error(CellError::Circ)); + // So do two tables that read each other's results. + t.set("E1", "=B1+SUM(H2:H3)"); + t.set("H1", "=B2+SUM(E2:E3)"); + t.set("G2", "1"); + t.set("G3", "2"); + t.wb.sheet_mut(0).unwrap().data_tables.push(table("G1:H3", None, Some("B2"))); + t.calc.recalc_all(&mut t.wb); + assert_eq!((t.get("E2"), t.get("H3")), (Value::Error(CellError::Circ), Value::Error(CellError::Circ))); + // Breaking the loop calculates both again. + t.set("H1", "=B2*2"); + assert_eq!((t.num("H3"), t.num("E2")), (4.0, 7.0)); +} + +#[test] +fn data_tables_that_read_tables_follow_them() { + let mut t = T::new(); + t.set("B1", "5"); + // A second table whose formula reads the first table's results follows it. + t.set("D2", "1"); + t.set("D3", "2"); + t.set("G1", "=E2*B1"); + t.set("H1", "=SUM(E2:E3)"); + t.set("F2", "1"); + t.set("F3", "2"); + t.set("E1", "=B1*10"); + // Listed first, so it waits for the second pass. + t.wb.sheet_mut(0).unwrap().data_tables.push(table("F1:H3", None, Some("B2"))); + t.wb.sheet_mut(0).unwrap().data_tables.push(table("D1:E3", None, Some("B1"))); + t.calc.recalc_all(&mut t.wb); + assert_eq!(t.num("H2"), 30.0); + t.set("E1", "=B1*100"); + assert_eq!((t.num("E3"), t.num("H3")), (200.0, 300.0)); +} + +#[test] +fn hostile_data_tables_are_not_calculated() { + let mut t = T::new(); + t.set("A1", "1"); + t.set("B1", "=A1"); + let sh = t.wb.sheet_mut(0).unwrap(); + // Too many result cells, no input cell, an input cell inside the table, a one-cell range. + sh.data_tables.push(table("A10:XFD1048576", None, Some("A1"))); + sh.data_tables.push(table("A2:B3", None, None)); + sh.data_tables.push(table("C2:D3", None, Some("D3"))); + sh.data_tables.push(table("F2:F2", None, Some("A1"))); + t.calc.recalc_all(&mut t.wb); + assert_eq!(t.get("B20"), Value::Empty); + assert_eq!(t.get("D3"), Value::Empty); + t.set("A1", "2"); + assert_eq!(t.num("B1"), 2.0); +} + +#[test] +fn data_tables_over_indirect_references_are_calculated() { + let mut t = T::new(); + t.set("B1", "2"); + t.set("E1", "=INDIRECT(\"B1\")*10"); + t.set("D2", "1"); + t.set("D3", "3"); + t.wb.sheet_mut(0).unwrap().data_tables.push(table("D1:E3", None, Some("B1"))); + t.calc.recalc_all(&mut t.wb); + assert_eq!((t.num("E2"), t.num("E3")), (10.0, 30.0)); + t.set("D3", "4"); + assert_eq!(t.num("E3"), 40.0); +} + #[test] fn cell_width_is_the_column_width_dialog_value_rounded() { // #735: Excel 365 values. Widths in px as `home.columnWidth {"chars"}` sets them @@ -1266,3 +1418,147 @@ fn offset_extreme_arguments_return_ref() { assert_eq!(crate::evaluate(&wb, 0, c("D1"), formula), Value::Number(expected), "{formula}"); } } + +/// Random edits around data tables (input values along the edges, the formulas, their precedents +/// and the input cells): after each, the incrementally recalculated sheet, where an edited input +/// value recalculates only its own row, column or cells, equals a full recalculation. +#[test] +fn data_tables_recalculated_in_part_agree_with_a_full_recalculation() { + let mut t = T::new(); + t.set("A1", "3"); + t.set("B1", "0.1"); + t.set("B2", "100"); + t.set("A2", "=B1*A1"); + // One-variable down a column (two formulas), across a row, and two-variable. + t.set("E2", "=B2*(1+B1)+A2"); + t.set("F2", "=B2-A1"); + t.set("H2", "=B1*B2"); + t.set("K1", "=B1*B2+A1"); + for (i, row) in (3..=6).enumerate() { + t.set(&format!("D{row}"), &(10 * (i + 1)).to_string()); + t.set(&format!("K{}", row - 1), &(i + 1).to_string()); + } + for (i, col) in ["I", "J", "K"].iter().enumerate() { + t.set(&format!("{col}1"), &format!("0.{}", i + 1)); + } + for (i, col) in ["L", "M"].iter().enumerate() { + t.set(&format!("{col}1"), &(i + 2).to_string()); + } + let sh = t.wb.sheet_mut(0).unwrap(); + sh.data_tables.push(table("D2:F6", None, Some("B2"))); + sh.data_tables.push(table("H1:J2", Some("B1"), None)); + sh.data_tables.push(table("K1:M5", Some("B1"), Some("B2"))); + t.set("O1", "=SUM(E3:F6)+SUM(L2:M5)+I2"); + t.calc.recalc_all(&mut t.wb); + let cells = ["D3", "D4", "D5", "D6", "I1", "J1", "K2", "K3", "L1", "M1", "A1", "B1", "B2", "E2", "F2", "H2", "K1"]; + let inputs = ["7", "0.25", "-3", "=A1*2", "=B2/10", "", "1e3", "=E3"]; + let formulas = ["=B2*(1+B1)", "=B1+B2+A1", "=A2*B2", "=B1"]; + let mut seed: u64 = 0x5EED; + let mut next = |n: usize| { + seed = seed.wrapping_mul(6364136223846793005).wrapping_add(1442695040888963407); + ((seed >> 33) as usize) % n + }; + for step in 0..80 { + let at = cells[next(cells.len())]; + let edge_formula = matches!(at, "E2" | "F2" | "H2" | "K1"); + let input = if edge_formula { formulas[next(formulas.len())] } else { inputs[next(inputs.len())] }; + t.set(at, input); + let mut full = t.wb.clone(); + Calc::new().recalc_all(&mut full); + for (c, cell) in full.sheet(0).unwrap().cells.iter() { + let got = t.wb.sheet(0).unwrap().value(c); + let same = match (&got, &cell.value) { + (Value::Number(a), Value::Number(b)) => (a - b).abs() <= 1e-9 * a.abs().max(1.0), + (a, b) => a == b, + }; + assert!(same, "step {step}: {at} := {input:?}; {} is {got:?}, a full recalculation gives {:?}", c.a1(), cell.value); + } + } +} + +/// A table whose calculation would take more formula evaluations than a refresh may spend keeps +/// its values (a hostile file can't hang the application); a table within it is calculated. +#[test] +fn data_tables_past_the_work_budget_keep_their_values() { + let mut t = T::new(); + // 400 formulas depend on B1: a 300 × 300 table would evaluate them 90,000 times. + let sh = t.wb.sheet_mut(0).unwrap(); + sh.cells.set(c("B1"), Cell::value(Value::Number(1.0))); + sh.cells.set(c("ZZ1"), Cell::formula(Formula::new("=B1+1"))); + for row in 2..=400 { + sh.cells.set(CellRef::new(row - 1, 701), Cell::formula(Formula::new(&format!("=ZZ{}+B1", row - 1)))); + } + sh.cells.set(c("A1"), Cell::formula(Formula::new("=ZZ400"))); + sh.data_tables.push(table("A1:KO301", Some("B1"), Some("B2"))); + // A small one-variable table on the same sheet. + sh.cells.set(c("QA1"), Cell::formula(Formula::new("=B3*2"))); + sh.cells.set(c("PZ2"), Cell::value(Value::Number(4.0))); + sh.cells.set(c("QA2"), Cell::value(Value::text("stale"))); + sh.data_tables.push(table("PZ1:QA2", None, Some("B3"))); + t.calc.recalc_all(&mut t.wb); + assert_eq!(t.get("B2"), Value::Empty); + assert_eq!(t.get("KO301"), Value::Empty, "too much work: not calculated"); + assert_eq!(t.num("QA2"), 8.0, "a table within the budget is calculated"); +} + +/// The substitutions of a table leave the calculation as they found it: circular references +/// found before are still listed, an input cell holding a formula keeps it (and its place in the +/// dependency graph), and formulas over it recalculate as before. +#[test] +fn data_tables_leave_the_calculation_as_they_found_it() { + let mut t = T::new(); + t.set("Z1", "=Z1+1"); + assert_eq!(t.calc.circular, vec![(0, c("Z1"))]); + t.set("A1", "5"); + t.set("B2", "=A1*2"); + t.set("E2", "=B2+1"); + t.set("D3", "100"); + t.set("D4", "200"); + t.wb.sheet_mut(0).unwrap().data_tables.push(table("D2:E4", None, Some("B2"))); + t.calc.recalc_all(&mut t.wb); + assert_eq!(t.calc.circular, vec![(0, c("Z1"))]); + assert_eq!((t.num("E3"), t.num("E4"), t.num("B2"), t.num("E2")), (101.0, 201.0, 10.0, 11.0)); + assert!(t.wb.sheet(0).unwrap().cell(c("B2")).unwrap().formula.is_some(), "the input cell keeps its formula"); + t.set("A1", "6"); + assert_eq!((t.num("B2"), t.num("E2"), t.num("E4")), (12.0, 13.0, 201.0)); + // With constant input cells, the table runs on this calculation's own graph. + t.set("B3", "1"); + t.set("H2", "=B3*10"); + t.set("G3", "2"); + t.wb.sheet_mut(0).unwrap().data_tables.push(table("G2:H3", None, Some("B3"))); + t.calc.recalc_all(&mut t.wb); + assert_eq!(t.calc.circular, vec![(0, c("Z1"))]); + assert_eq!(t.num("H3"), 20.0); + t.set("B3", "4"); + assert_eq!((t.num("H2"), t.num("H3")), (40.0, 20.0)); +} + +/// A data table's results are formula cells in Excel, so a dynamic array can't spill over them, +/// even while they're blank; it can spill along the table's edges, and the table follows the +/// spilled input values (both found by the command-sequence fuzz). +#[test] +fn dynamic_arrays_and_data_tables() { + let full = |t: &T| { + let mut wb = t.wb.clone(); + Calc::new().recalc_all(&mut wb); + wb + }; + let mut t = T::new(); + t.set("B1", "2"); + t.wb.sheet_mut(0).unwrap().data_tables.push(table("K20:M22", None, Some("B1"))); + t.calc.recalc_all(&mut t.wb); + t.set("K19", "=SEQUENCE(5,2)"); + assert_eq!(t.get("K19"), Value::Error(CellError::Spill)); + assert_eq!(full(&t).sheet(0).unwrap().value(c("K19")), Value::Error(CellError::Spill)); + // Down the left edge: the spilled values are the table's input values. + t.set("L20", "=B1*10"); + t.set("K19", "=SEQUENCE(4)"); + assert_eq!((t.num("K22"), t.num("L21"), t.num("L22")), (4.0, 30.0, 40.0)); + t.set("K19", "=SEQUENCE(4,1,5)"); + assert_eq!((t.num("L21"), t.num("L22")), (70.0, 80.0)); + let fresh = full(&t); + assert_eq!((fresh.sheet(0).unwrap().value(c("L21")), fresh.sheet(0).unwrap().value(c("L22"))), (Value::Number(70.0), Value::Number(80.0))); + // The spill shrinks off the edge: the input values are blank again. + t.set("K19", "=SEQUENCE(2)"); + assert_eq!((t.num("L21"), t.num("L22")), (0.0, 0.0)); +} diff --git a/crates/engine/src/cmd/edit.rs b/crates/engine/src/cmd/edit.rs index 82dbe0be72..77242ff6ba 100644 --- a/crates/engine/src/cmd/edit.rs +++ b/crates/engine/src/cmd/edit.rs @@ -151,6 +151,7 @@ pub(crate) fn input_to_cell(input: &str, sheet: usize, at: CellRef, wb: &mut gri let expr = parsed.map_err(|e| EngineError::Other(format!("There's a problem with this formula: {e}")))?; let expr = wb.name_call_case(expr); check_arg_counts(&expr)?; + check_no_table(&expr)?; let mut cell = Cell::formula(Formula::from_expr(expr)); cell.style = style; // Formulas whose result is a date/time get a format from functions like TODAY(). @@ -213,6 +214,19 @@ fn check_arg_counts(expr: &gridcraft_formula::Expr) -> Result<()> { } } +/// TABLE() is only made by What-If Analysis › Data Table; Excel refuses it typed into a cell. +fn check_no_table(expr: &gridcraft_formula::Expr) -> Result<()> { + let mut table = false; + expr.walk(&mut |e| { + if let gridcraft_formula::Expr::Call(name, _) = e + && name.eq_ignore_ascii_case("TABLE") + { + table = true; + } + }); + if table { Err(EngineError::Other("That function isn't valid.".into())) } else { Ok(()) } +} + fn cell_set(s: &mut Session, p: &Json) -> Result { let sheet = target_sheet(s, p)?; let at = match cell_param_on(s, p, "cell", sheet)? { @@ -1216,6 +1230,12 @@ pub(crate) fn rewrite_all_formulas(wb: &mut gridcraft_model::Workbook, target: & n.formula = gridcraft_formula::print(&new); } } + // Data tables ({=TABLE()}) are formulas of the target sheet too. + if let Some(i) = wb.sheet_index(target).filter(|&i| wb.sheet(i).is_some_and(|s| !s.data_tables.is_empty())) + && let Some(sh) = wb.sheet_mut(i) + { + super::whatif::adjust_data_tables(sh, e); + } } // ---------------------------------------------------------------- clear diff --git a/crates/engine/src/cmd/formulas.rs b/crates/engine/src/cmd/formulas.rs index 40df72df0c..3baa69d763 100644 --- a/crates/engine/src/cmd/formulas.rs +++ b/crates/engine/src/cmd/formulas.rs @@ -578,10 +578,12 @@ fn calc_options(s: &mut Session, p: &Json) -> Result { cx.wb.calc.set_max_change(v); } let c = &cx.wb.calc; - // Switching to automatic, or changing how circular references are calculated outside - // manual mode, recalculates the workbook. + // Switching to automatic (or from manual to automatic except for data tables), or + // changing how circular references are calculated outside manual mode, recalculates the + // workbook. let iteration_changed = (c.iterative, c.max_iterations, c.max_change) != (before.iterative, before.max_iterations, before.max_change); - if mode == Some(CalcMode::Automatic) || (iteration_changed && c.mode != CalcMode::Manual) { + let leaves_manual = before.mode == CalcMode::Manual && c.mode == CalcMode::AutomaticExceptTables; + if mode == Some(CalcMode::Automatic) || leaves_manual || (iteration_changed && c.mode != CalcMode::Manual) { cx.structural = true; } Ok(json!({"mode": format!("{:?}", c.mode), "iterative": c.iterative, "maxIterations": c.max_iterations, "maxChange": c.max_change})) diff --git a/crates/engine/src/cmd/inspect.rs b/crates/engine/src/cmd/inspect.rs index f92b48bd9d..980471b44c 100644 --- a/crates/engine/src/cmd/inspect.rs +++ b/crates/engine/src/cmd/inspect.rs @@ -9,7 +9,7 @@ pub fn specs() -> Vec { vec![ cmd!(query "document.inspect", "Inspect Workbook", [], None, "{} → sheets, selection, names, tables, charts, dirty, history", has_doc, inspect), cmd!(query "sheet.read", "Read Range", [], None, "{range?: \"A1:D10\" (default: used range), sheet?, formulas?: bool, formatted?: bool} → rows of values", has_doc, read), - cmd!(query "cell.get", "Get Cell", [], None, "{cell?, sheet?} → value, formula, display text, style, comment, link", has_doc, cell_get), + cmd!(query "cell.get", "Get Cell", [], None, "{cell?, sheet?} → value, formula, display text, style, comment, link, dataTable (the What-If data table whose result the cell is)", has_doc, cell_get), cmd!(query "selection.stats", "Selection Statistics", [], None, "{} → count, sum, average, min, max", has_doc, sel_stats), cmd!(query "history.list", "Undo History", [], None, "{} → undo/redo labels", has_doc, history), cmd!(query "app.commands", "List Commands", [], None, "{search?} → ids, labels, params", always, list_commands), @@ -126,7 +126,7 @@ fn cell_get(s: &mut Session, p: &Json) -> Result { "cell": c.a1(), "value": value_json(&sh.value(c)), "type": if sh.cell_pictures.contains_key(&c) { "picture" } else { match sh.value(c) { Value::Empty => "empty", Value::Number(_) => "number", Value::Text(_) => "text", Value::Bool(_) => "boolean", Value::Error(_) => "error", Value::Array(_) => "array" } }, - "formula": cell.and_then(|x| x.formula.as_ref()).map(|f| format!("={}", f.text)), + "formula": cell.and_then(|x| x.formula.as_ref()).map(|f| format!("={}", f.text)).or_else(|| sh.data_table_at(c).map(|t| format!("{{={}}}", t.formula_text()))), "text": crate::display::cell_text(&d.wb, sh, c), "input": sh.input_text(c), "picture": sh.cell_pictures.get(&c).map(|picture| json!({"mime": picture.mime, "alt": picture.alt, "bytes": picture.data.len()})), @@ -136,6 +136,11 @@ fn cell_get(s: &mut Session, p: &Json) -> Result { "merge": sh.merge_at(c).map(|m| m.a1()), "comment": sh.comments.get(&c), "hyperlink": sh.hyperlinks.get(&c), + "dataTable": sh.data_table_at(c).map(|t| json!({ + "range": t.range.a1(), + "rowInput": t.row_input.map(|c| c.a1()), + "columnInput": t.col_input.map(|c| c.a1()), + })), })) } diff --git a/crates/engine/src/cmd/mod.rs b/crates/engine/src/cmd/mod.rs index 50865a2b7d..b931b5eeb9 100644 --- a/crates/engine/src/cmd/mod.rs +++ b/crates/engine/src/cmd/mod.rs @@ -340,6 +340,7 @@ pub(crate) fn commit(d: &mut DocState, f: impl FnOnce(&mut Ctx) -> Result) if !structural { check_arrays(&d.wb, &wb)?; } + whatif::check_data_tables(&d.wb, &mut wb)?; if changed.len() <= 200_000 { fit_rows.extend(changed.iter().map(|(s, c)| (*s, c.row))); } diff --git a/crates/engine/src/cmd/sheet.rs b/crates/engine/src/cmd/sheet.rs index 7870dee4e9..efd7c15a63 100644 --- a/crates/engine/src/cmd/sheet.rs +++ b/crates/engine/src/cmd/sheet.rs @@ -256,6 +256,7 @@ fn insert_lines(s: &mut Session, p: &Json, axis: Axis) -> Result { } check_lines_protection(s, p, axis, at, count, true)?; let sheet = target_sheet(s, p)?; + check_data_table_lines(s, sheet, &Edit::Insert { axis, at, count })?; // Refuse to push data off the sheet. if let Some(u) = s.doc()?.wb.sheet(sheet).and_then(|sh| sh.used_range()) { let last = if axis == Axis::Rows { u.end.row } else { u.end.col }; @@ -331,6 +332,10 @@ fn delete_lines(s: &mut Session, p: &Json, axis: Axis) -> Result { check_lines_protection(s, p, axis, at, count, false)?; } let sheet = target_sheet(s, p)?; + // Areas are deleted last first, so each one is checked where it is now. + for &(at, count) in &areas { + check_data_table_lines(s, sheet, &Edit::Delete { axis, at, count })?; + } let out = edit(s, |cx| { let name = cx.wb.sheet(sheet).map(|s| s.name.clone()).unwrap_or_default(); cx.protection_checked = true; @@ -380,6 +385,8 @@ fn move_columns(s: &mut Session, p: &Json) -> Result { } check_lines_protection(s, &json!({"cols": RangeRef::cols(before, before).a1()}), Axis::Cols, before, count, true)?; check_lines_protection(s, &json!({"cols": RangeRef::cols(from, end - 1).a1()}), Axis::Cols, from, count, false)?; + check_data_table_lines(s, sheet, &Edit::Insert { axis: Axis::Cols, at: before, count })?; + check_data_table_lines(s, sheet, &Edit::Delete { axis: Axis::Cols, at: from, count })?; let moved_from = if before <= from { from + count } else { from }; if moved_from.checked_add(count).is_none_or(|end| end > MAX_COLS) { @@ -469,6 +476,7 @@ fn insert_cells(s: &mut Session, p: &Json) -> Result { let sheet = target_sheet(s, p)?; if let Some(sh) = s.doc()?.wb.sheet(sheet) { check_table_shift(sh, &e)?; + check_data_table_shift(sh, &e)?; } let right = shift == "right"; let out = edit(s, |cx| { @@ -500,6 +508,7 @@ fn delete_cells(s: &mut Session, p: &Json) -> Result { let sheet = target_sheet(s, p)?; if let Some(sh) = s.doc()?.wb.sheet(sheet) { check_table_shift(sh, &e)?; + check_data_table_shift(sh, &e)?; } let left = shift == "left"; let out = edit(s, |cx| { @@ -1048,6 +1057,42 @@ fn shift_table_columns(t: &mut gridcraft_model::Table, at: u32, count: u32, inse } } +fn check_data_table_lines(s: &Session, sheet: usize, e: &Edit) -> Result<()> { + match s.doc()?.wb.sheet(sheet) { + Some(sh) => check_data_table_shift(sh, e), + None => Ok(()), + } +} + +/// Like Excel, a structural edit never cuts through a data table: the table moves whole, or is +/// deleted whole. Inserting or deleting lines through it, and Insert/Delete Cells that shift part +/// of it, are refused. +pub(crate) fn check_data_table_shift(sh: &Sheet, e: &Edit) -> Result<()> { + for t in &sh.data_tables { + let tr = t.range; + let reshaped = map_range(tr, e).is_some_and(|n| n.width() != tr.width() || n.height() != tr.height()); + let partly_shifted = match *e { + Edit::InsertCells { axis: Axis::Rows, range: r } | Edit::DeleteCells { axis: Axis::Rows, range: r } => { + tr.end.row >= r.start.row + && tr.start.col <= r.end.col + && tr.end.col >= r.start.col + && (r.start.col > tr.start.col || r.end.col < tr.end.col) + } + Edit::InsertCells { axis: Axis::Cols, range: r } | Edit::DeleteCells { axis: Axis::Cols, range: r } => { + tr.end.col >= r.start.col + && tr.start.row <= r.end.row + && tr.end.row >= r.start.row + && (r.start.row > tr.start.row || r.end.row < tr.end.row) + } + _ => false, + }; + if reshaped || partly_shifted { + return Err(EngineError::Other(super::whatif::DATA_TABLE_PART.into())); + } + } + Ok(()) +} + /// Like Excel, Insert/Delete Cells never shifts part of a table: cells shifted up or down must /// hold whole table columns, cells shifted left or right whole table rows, and deleting cells /// never takes a table's header row without the whole table. diff --git a/crates/engine/src/cmd/solver.rs b/crates/engine/src/cmd/solver.rs index 4465874bc3..daddae8e4e 100644 --- a/crates/engine/src/cmd/solver.rs +++ b/crates/engine/src/cmd/solver.rs @@ -28,7 +28,7 @@ use gridcraft_core::{CellRef, RangeRef, Value}; use gridcraft_model::{Cell, DefinedName, Sheet, Workbook}; use serde_json::{Value as Json, json}; -use super::whatif::Trial; +use super::whatif::{Trial, trial}; use super::*; const CMD: &str = "data.solver"; @@ -1707,7 +1707,7 @@ fn title(out: &mut Lines, kind: ReportKind, sheet: &str) -> u32 { fn answer_report(d: &DocState, sh: &Sheet, inp: &ReportInput) -> Lines { let m = inp.model; - let mut trial = Trial::new(d); + let mut trial = trial(d); let read = |trial: &mut Trial, x: &[f64], cells: &[CellRef]| -> Vec { let sets: Vec<(usize, CellRef, Value)> = m.vars.iter().zip(x).map(|(c, v)| (m.sheet, *c, Value::number(*v))).collect(); let reads: Vec<(usize, CellRef)> = cells.iter().map(|c| (m.sheet, *c)).collect(); @@ -1810,7 +1810,7 @@ fn limits_report(d: &DocState, sh: &Sheet, inp: &ReportInput) -> Result { // Each variable is minimised and maximised with the others held at the solution. let per = (inp.budget / (2 * n as u64).max(1)).max(n as u64 + 3); let mut ev = Evaluator { - trial: Trial::new(d), + trial: trial(d), sheet: m.sheet, vars: m.vars.clone(), objective: m.objective, @@ -2014,7 +2014,7 @@ fn run(s: &mut Session, p: &Json) -> Result { let sh = d.wb.sheet(model.sheet).ok_or(EngineError::NoDocument)?; let x0: Vec = model.vars.iter().map(|c| as_num(&sh.value(*c))).map(|v| if v.is_finite() { v } else { 0.0 }).collect(); let mut ev = Evaluator { - trial: Trial::new(d), + trial: trial(d), sheet: model.sheet, vars: model.vars.clone(), objective: model.objective, diff --git a/crates/engine/src/cmd/whatif.rs b/crates/engine/src/cmd/whatif.rs index 09c716caba..966d72e178 100644 --- a/crates/engine/src/cmd/whatif.rs +++ b/crates/engine/src/cmd/whatif.rs @@ -8,9 +8,8 @@ use std::collections::{HashMap, HashSet}; -use gridcraft_calc::Calc; use gridcraft_core::{CellRef, RangeRef, Value}; -use gridcraft_model::{CalcMode, Cell, DefinedName, Sheet, StyleId, Workbook}; +use gridcraft_model::{CalcMode, Cell, DefinedName, MAX_DATA_TABLE_CELLS, Sheet, StyleId, Workbook}; use serde::{Deserialize, Serialize}; use serde_json::{Value as Json, json}; @@ -61,7 +60,7 @@ pub fn specs() -> Vec { "Data Table…", ["Data", "Forecast", "What-If Analysis"], None, - "{range: \"D2:E10\", rowInput?: \"B1\", columnInput?: \"B2\"} one-variable (values down the first column with columnInput, or across the top row with rowInput; formulas in the other edge) or two-variable (formula in the top-left corner). Computes every result by substitution and writes static values (a snapshot, not a live {=TABLE()}); ≤100,000 cells.", + "{range: \"D2:E10\", rowInput?: \"B1\", columnInput?: \"B2\", static?: bool} → {range, cells, mode, formula, static}. Creates a live data table {=TABLE(rowInput,columnInput)} over the results: one-variable (values down the first column with columnInput, or across the top row with rowInput; formulas along the other edge) or two-variable (formula in the top-left corner). Input cells are on the table's sheet, outside it. The results recalculate with the workbook (in Automatic Except for Data Tables mode, on F9); editing part of them is refused, clearing all of them removes the table. `static: true` instead writes the results once as values (input cells on any sheet). ≤100,000 result cells. From the ribbon, Data Table… opens Excel's dialog (Row input cell, Column input cell) over the selection.", has_doc, data_table ), @@ -180,34 +179,12 @@ fn value_json(v: &Value) -> Json { // ---------------------------------------------------------------- trials -/// Evaluates a workbook with some input cells replaced, without touching the document. -pub(crate) struct Trial { - base: Workbook, - calc: Calc, -} +/// What-if trials (shared with data tables, in calc, and Solver). +pub(crate) use gridcraft_calc::Trial; -impl Trial { - pub(crate) fn new(d: &DocState) -> Trial { - let mut base = (*d.wb).clone(); - // What-if analysis always recalculates, even in manual mode. - base.calc.mode = CalcMode::Automatic; - Trial { base, calc: d.calc.clone() } - } - - /// Sets `sets` to constants, recalculates their dependents and reads `reads`. - pub(crate) fn run(&mut self, sets: &[(usize, CellRef, Value)], reads: &[(usize, CellRef)]) -> Vec { - let mut wb = self.base.clone(); - let mut keys = Vec::with_capacity(sets.len()); - for (si, c, v) in sets { - if let Some(sh) = wb.sheet_mut(*si) { - let style = sh.style_id(*c); - sh.set_cell(*c, Cell { value: v.clone(), formula: None, style }); - keys.push((*si, *c)); - } - } - self.calc.cells_changed(&mut wb, &keys); - reads.iter().map(|(si, c)| wb.sheet(*si).map(|s| s.value(*c)).unwrap_or_default()).collect() - } +/// A trial on the document's workbook. +pub(crate) fn trial(d: &DocState) -> Trial { + Trial::new(&d.wb, &d.calc) } // ---------------------------------------------------------------- goal seek @@ -322,7 +299,7 @@ fn goal_seek(s: &mut Session, p: &Json) -> Result { } let x0 = d.wb.sheet(csi).map(|sh| sh.value(chg)).and_then(|v| v.as_f64()).unwrap_or(0.0); let (max_iter, tol) = (d.wb.calc.max_iterations, d.wb.calc.max_change); - let mut trial = Trial::new(d); + let mut trial = trial(d); let mut f = |x: f64| -> f64 { trial.run(&[(csi, chg, Value::number(x))], &[(ssi, set)]).first().and_then(Value::as_f64).map(|v| v - target).unwrap_or(f64::NAN) }; @@ -550,7 +527,7 @@ fn scenario_summary(s: &mut Session, p: &Json) -> Result { } let current: Vec = changing.iter().map(|c| sh.value(*c)).collect(); let current_results: Vec = results.iter().map(|c| sh.value(*c)).collect(); - let mut trial = Trial::new(d); + let mut trial = trial(d); let reads: Vec<(usize, CellRef)> = results.iter().map(|c| (si, *c)).collect(); let scenario_results: Vec> = per .iter() @@ -624,9 +601,153 @@ fn scenario_summary(s: &mut Session, p: &Json) -> Result { // ---------------------------------------------------------------- data tables -const MAX_TABLE_CELLS: u64 = 100_000; +/// Excel's message for an edit of some, not all, of a data table's results. +pub(crate) const DATA_TABLE_PART: &str = "Cannot change part of a data table."; + +/// Excel's message for an input cell that can't be used. +const BAD_INPUT: &str = "Input cell reference is not valid."; fn data_table(s: &mut Session, p: &Json) -> Result { + const CMD: &str = "data.dataTable"; + if bool_param(p, "static").unwrap_or(false) { + return data_table_static(s, p); + } + let (si, r) = range_arg(s, p, "range", CMD)?; + let input = |key: &str| -> Result> { + if !p.get(key).and_then(Json::as_str).is_some_and(|t| !t.trim().is_empty()) { + return Ok(None); + } + let (isi, c) = cell_arg(s, p, key, CMD)?; + // A live table substitutes into a cell of its own sheet, outside the table. + if isi != si || r.contains(c) { + return Err(EngineError::Other(BAD_INPUT.into())); + } + Ok(Some(c)) + }; + let (row_input, col_input) = (input("rowInput")?, input("columnInput")?); + if row_input.is_none() && col_input.is_none() { + return Err(bad(CMD, "give `rowInput`, `columnInput` or both")); + } + if r.height() < 2 || r.width() < 2 { + return Err(bad(CMD, "the table needs at least two rows and two columns")); + } + let table = gridcraft_model::DataTable { range: r, row_input, col_input }; + let Some(body) = table.body().filter(|b| b.count() <= MAX_DATA_TABLE_CELLS) else { + return Err(bad(CMD, format!("the table body is too large ({MAX_DATA_TABLE_CELLS} cells max)"))); + }; + let d = s.doc()?; + let sh = d.wb.sheet(si).ok_or(EngineError::NoDocument)?; + if sh.data_tables.iter().any(|t| t.range != r && t.range.intersects(&r)) { + return Err(EngineError::Other(DATA_TABLE_PART.into())); + } + let same = sh.data_tables.contains(&table); + let mode = match (row_input, col_input) { + (Some(_), Some(_)) => "twoVariable", + (Some(_), None) => "row", + _ => "column", + }; + let formula = format!("{{={}}}", table.formula_text()); + // Outside automatic calculation the edit alone wouldn't calculate the new table. + let full = d.wb.calc.mode != CalcMode::Automatic; + edit(s, |cx| { + let sh = cx.sheet_mut(si)?; + // Data Table over a table's own range replaces that table. + sh.data_tables.retain(|t| t.range != r); + if !same { + // The results replace what the body held. + for c in body.iter() { + let style = sh.style_id(c); + sh.set_cell(c, Cell { value: Value::Empty, formula: None, style }); + } + } + sh.data_tables.push(table); + for c in r.iter() { + cx.touch(si, c); + } + cx.structural = full; + Ok(()) + })?; + Ok(json!({"range": body.a1(), "cells": body.count(), "mode": mode, "formula": formula, "static": false})) +} + +/// Refuses an edit that changed what some, not all, of a data table's result cells hold, or that +/// left two tables overlapping. An edit that empties all of a table's results (Clear Contents over +/// them) removes the table. Tables the edit added, moved or replaced are the command's own and +/// their results aren't checked. +pub(crate) fn check_data_tables(old: &Workbook, new: &mut Workbook) -> Result<()> { + for si in 0..new.sheets.len() { + let Some(after) = new.sheets.get(si) else { continue }; + if after.data_tables.is_empty() { + continue; + } + let Some(before) = old.sheet_index(&after.name).and_then(|i| old.sheets.get(i)) else { continue }; + if std::sync::Arc::ptr_eq(before, after) { + continue; + } + // A cut and paste can bring one table onto another. + if after.data_tables != before.data_tables + && after.data_tables.iter().enumerate().any(|(i, t)| after.data_tables.iter().skip(i + 1).any(|u| u.range.intersects(&t.range))) + { + return Err(EngineError::Other(DATA_TABLE_PART.into())); + } + let changed: Vec = before.cells.diff(&after.cells).into_iter().filter(|c| !same_content(before.cell(*c), after.cell(*c))).collect(); + if changed.is_empty() { + continue; + } + let mut cleared: Vec = Vec::new(); + for t in after.data_tables.iter().filter(|t| before.data_tables.contains(t)) { + let Some(body) = t.body() else { continue }; + if !changed.iter().any(|c| body.contains(*c)) { + continue; + } + if after.cells.iter_range(body).any(|(_, x)| x.formula.is_some() || !x.value.is_empty()) { + return Err(EngineError::Other(DATA_TABLE_PART.into())); + } + cleared.push(t.range); + } + if !cleared.is_empty() + && let Some(sh) = new.sheet_mut(si) + { + sh.data_tables.retain(|t| !cleared.contains(&t.range)); + } + } + Ok(()) +} + +/// Moves a sheet's data tables with a structural edit or a cut on it: the range and the input +/// cells follow the cells. A table whose input cell is deleted, moves to another sheet or into the +/// table keeps its results as values; so does one the edit reshapes (the commands refuse those +/// edits first). +pub(crate) fn adjust_data_tables(sh: &mut Sheet, e: &gridcraft_formula::adjust::Edit) { + if sh.data_tables.is_empty() { + return; + } + let map = |r: RangeRef| -> Option { + match e { + gridcraft_formula::adjust::Edit::MoveToSheet { from, .. } => (!from.intersects(&r)).then_some(r), + _ => super::sheet::map_range(r, e), + } + }; + let cell = |c: Option| -> Option> { + match c { + Some(c) => map(RangeRef::cell(c)).map(|r| Some(r.start)), + None => Some(None), + } + }; + sh.data_tables = std::mem::take(&mut sh.data_tables) + .into_iter() + .filter_map(|t| { + let range = map(t.range).filter(|n| n.width() == t.range.width() && n.height() == t.range.height())?; + Some(gridcraft_model::DataTable { range, row_input: cell(t.row_input)?, col_input: cell(t.col_input)? }) + }) + // An input cell moved into the table, or the table moved over it. + .filter(gridcraft_model::DataTable::is_valid) + .collect(); +} + +/// The static form: every result is computed once by substitution and written as a value (a +/// snapshot, not a live table). Input cells may be on any sheet. +fn data_table_static(s: &mut Session, p: &Json) -> Result { const CMD: &str = "data.dataTable"; let (si, r) = range_arg(s, p, "range", CMD)?; let row_in = if p.get("rowInput").and_then(Json::as_str).is_some_and(|t| !t.is_empty()) { Some(cell_arg(s, p, "rowInput", CMD)?) } else { None }; @@ -639,8 +760,8 @@ fn data_table(s: &mut Session, p: &Json) -> Result { return Err(bad(CMD, "the table needs at least two rows and two columns")); } let body = RangeRef::new(CellRef::new(r.start.row + 1, r.start.col + 1), r.end); - if body.count() > MAX_TABLE_CELLS { - return Err(bad(CMD, format!("the table body is too large ({MAX_TABLE_CELLS} cells max)"))); + if body.count() > MAX_DATA_TABLE_CELLS { + return Err(bad(CMD, format!("the table body is too large ({MAX_DATA_TABLE_CELLS} cells max)"))); } for input in [row_in, col_in].into_iter().flatten() { if input.0 == si && r.contains(input.1) { @@ -650,7 +771,7 @@ fn data_table(s: &mut Session, p: &Json) -> Result { let d = s.doc()?; let sh = d.wb.sheet(si).ok_or(EngineError::NoDocument)?; let (r0, c0) = (r.start.row, r.start.col); - let mut trial = Trial::new(d); + let mut trial = trial(d); let mut out: Vec<(CellRef, Value)> = Vec::with_capacity(body.count() as usize); let mode = match (row_in, col_in) { (None, Some((isi, ic))) => { @@ -691,6 +812,8 @@ fn data_table(s: &mut Session, p: &Json) -> Result { }; let n = out.len(); edit(s, |cx| { + // A snapshot over a live table's range replaces the table. + cx.sheet_mut(si)?.data_tables.retain(|t| t.range != r); for (c, v) in out { let sh = cx.sheet_mut(si)?; let style = sh.style_id(c); @@ -1279,6 +1402,285 @@ mod tests { assert!(s.execute("data.dataTable", json!({"range": "L1:N3", "columnInput": "M2"})).is_err()); } + fn tables(s: &Session) -> Vec { + s.doc().unwrap().wb.active().unwrap().data_tables.clone() + } + + /// Rate B1, amount B2; amounts down D3:D5 and a formula in E2 (column input B2). + fn loan(s: &mut Session) { + s.execute("range.setValues", json!({"range": "B1", "values": [[0.05], [1000]]})).unwrap(); + s.execute("range.setValues", json!({"range": "D3", "values": [[100], [200], [300]]})).unwrap(); + s.execute("cell.set", json!({"cell": "E2", "input": "=B2*(1+B1)"})).unwrap(); + s.execute("cell.set", json!({"cell": "H1", "input": "=SUM(E3:E5)"})).unwrap(); + } + + #[test] + fn data_tables_are_live() { + let mut s = s(); + loan(&mut s); + let r = s.execute("data.dataTable", json!({"range": "$D$2:$E$5", "columnInput": "$B$2"})).unwrap(); + assert_eq!((r["formula"].as_str(), r["static"].as_bool(), r["cells"].as_u64()), (Some("{=TABLE(,B2)}"), Some(false), Some(3))); + assert!((n(&s, "H1") - 630.0).abs() < 1e-9); + // A precedent of the formula changes: the results and the formulas over them follow. + s.execute("cell.set", json!({"cell": "B1", "input": "0.1"})).unwrap(); + assert!((n(&s, "E3") - 110.0).abs() < 1e-9); + assert!((n(&s, "H1") - 660.0).abs() < 1e-9); + // So do an input value and the formula itself. + s.execute("cell.set", json!({"cell": "D5", "input": "1000"})).unwrap(); + assert!((n(&s, "E5") - 1100.0).abs() < 1e-9); + s.execute("cell.set", json!({"cell": "E2", "input": "=B2*2"})).unwrap(); + assert_eq!(n(&s, "E4"), 400.0); + assert_eq!(n(&s, "B2"), 1000.0, "the input cell keeps its value"); + let got = s.execute("cell.get", json!({"cell": "E4"})).unwrap(); + assert_eq!(got["formula"], "{=TABLE(,B2)}"); + // Undo goes back through the edits, then removes the table. + s.execute("edit.undo", json!({})).unwrap(); + assert!((n(&s, "E4") - 220.0).abs() < 1e-9); + s.execute("edit.undo", json!({"steps": 3})).unwrap(); + assert!(tables(&s).is_empty()); + assert_eq!(v(&s, "E4"), Value::Empty); + s.execute("edit.redo", json!({})).unwrap(); + assert_eq!(tables(&s).len(), 1); + assert!((n(&s, "E4") - 210.0).abs() < 1e-9); + + // Two-variable: rates across the top (row input B1), amounts down the side (column input B2). + s.execute("cell.set", json!({"cell": "L1", "input": "=B2*(1+B1)"})).unwrap(); + s.execute("range.setValues", json!({"range": "M1", "values": [[0.5, 1]]})).unwrap(); + s.execute("range.setValues", json!({"range": "L2", "values": [[10], [20]]})).unwrap(); + let r = s.execute("data.dataTable", json!({"range": "L1:N3", "rowInput": "B1", "columnInput": "B2"})).unwrap(); + assert_eq!((r["mode"].as_str(), r["formula"].as_str()), (Some("twoVariable"), Some("{=TABLE(B1,B2)}"))); + assert_eq!((n(&s, "M2"), n(&s, "N3")), (15.0, 40.0)); + s.execute("cell.set", json!({"cell": "L1", "input": "=B2*B1"})).unwrap(); + assert_eq!((n(&s, "M2"), n(&s, "N3")), (5.0, 20.0)); + s.execute("cell.set", json!({"cell": "N1", "input": "3"})).unwrap(); + assert_eq!(n(&s, "N3"), 60.0); + // Running Data Table again over the same range keeps one table. + s.execute("data.dataTable", json!({"range": "L1:N3", "rowInput": "B1", "columnInput": "B2"})).unwrap(); + assert_eq!(tables(&s).len(), 2); + assert_eq!(n(&s, "N3"), 60.0); + // Show Formulas shows the formula without the braces of the formula bar, as Excel does; + // `cell.get` also says which table the cell belongs to. + s.execute("formulas.showFormulas", json!({"on": true})).unwrap(); + let got = s.execute("cell.get", json!({"cell": "M2"})).unwrap(); + assert_eq!((got["text"].as_str(), got["formula"].as_str()), (Some("=TABLE(B1,B2)"), Some("{=TABLE(B1,B2)}"))); + assert_eq!(got["dataTable"], json!({"range": "L1:N3", "rowInput": "B1", "columnInput": "B2"})); + assert_eq!(s.execute("cell.get", json!({"cell": "L1"})).unwrap()["dataTable"], serde_json::Value::Null, "edges aren't results"); + } + + /// Editing one input value along an edge recalculates its own results; the others keep + /// theirs, and agree with a full recalculation. + #[test] + fn edge_values_recalculate_their_own_results() { + let mut s = s(); + loan(&mut s); + s.execute("data.dataTable", json!({"range": "D2:E5", "columnInput": "B2"})).unwrap(); + s.execute("cell.set", json!({"cell": "D4", "input": "1000"})).unwrap(); + assert!((n(&s, "E4") - 1050.0).abs() < 1e-9); + assert!((n(&s, "E3") - 105.0).abs() < 1e-9); + // An edge value that is a formula over another one follows it. + s.execute("cell.set", json!({"cell": "D5", "input": "=D4*2"})).unwrap(); + assert!((n(&s, "E5") - 2100.0).abs() < 1e-9); + s.execute("cell.set", json!({"cell": "D4", "input": "10"})).unwrap(); + assert!((n(&s, "E5") - 21.0).abs() < 1e-9); + let before: Vec = ["E3", "E4", "E5", "H1"].iter().map(|a| n(&s, a)).collect(); + s.execute("formulas.calculateNow", json!({})).unwrap(); + assert_eq!(["E3", "E4", "E5", "H1"].iter().map(|a| n(&s, a)).collect::>(), before); + } + + /// Manual mode: nothing waits for F9 but the tables; switching to Automatic Except for Data + /// Tables calculates the formulas edited meanwhile, and F9 the tables. + #[test] + fn data_tables_in_manual_and_except_tables_modes() { + let mut s = s(); + loan(&mut s); + s.execute("data.dataTable", json!({"range": "D2:E5", "columnInput": "B2"})).unwrap(); + s.execute("formulas.calculationOptions", json!({"mode": "manual"})).unwrap(); + s.execute("cell.set", json!({"cell": "B1", "input": "0.5"})).unwrap(); + s.execute("cell.set", json!({"cell": "G1", "input": "=B1*2"})).unwrap(); + assert!((n(&s, "E4") - 210.0).abs() < 1e-9, "manual mode waits for F9"); + let r = s.execute("formulas.calculationOptions", json!({"mode": "automaticExceptTables"})).unwrap(); + assert_eq!(r["mode"], "AutomaticExceptTables"); + assert_eq!(n(&s, "G1"), 1.0); + s.execute("cell.set", json!({"cell": "B1", "input": "1"})).unwrap(); + assert_eq!(n(&s, "G1"), 2.0, "formulas calculate"); + let stale = n(&s, "E4"); + s.execute("cell.set", json!({"cell": "D4", "input": "400"})).unwrap(); + assert_eq!(n(&s, "E4"), stale, "tables wait for F9"); + s.execute("formulas.calculateNow", json!({})).unwrap(); + assert_eq!(n(&s, "E4"), 800.0); + // Back to Automatic: tables follow every edit again. + s.execute("formulas.calculationOptions", json!({"mode": "automatic"})).unwrap(); + s.execute("cell.set", json!({"cell": "B1", "input": "0"})).unwrap(); + assert_eq!(n(&s, "E4"), 400.0); + } + + /// Deleting the whole results area (Delete over all of it) removes the table, with undo; + /// deleting part of it is refused. + #[test] + fn deleting_all_results_removes_the_table() { + let mut s = s(); + loan(&mut s); + s.execute("data.dataTable", json!({"range": "D2:E5", "columnInput": "B2"})).unwrap(); + let part = s.execute("edit.clearContents", json!({"range": "E4:E5"})); + assert!(matches!(part, Err(crate::EngineError::Other(m)) if m == super::DATA_TABLE_PART)); + // The whole table with its edges, as Excel allows. + s.execute("edit.clearContents", json!({"range": "D2:E5"})).unwrap(); + assert!(tables(&s).is_empty()); + assert_eq!(v(&s, "E4"), Value::Empty); + s.execute("edit.undo", json!({})).unwrap(); + assert_eq!(tables(&s).len(), 1); + assert!((n(&s, "E4") - 210.0).abs() < 1e-9); + s.execute("cell.set", json!({"cell": "B1", "input": "0.5"})).unwrap(); + assert!((n(&s, "E4") - 300.0).abs() < 1e-9, "live again after undo"); + } + + #[test] + fn data_table_results_cannot_be_edited_in_part() { + let mut s = s(); + loan(&mut s); + s.execute("data.dataTable", json!({"range": "D2:E5", "columnInput": "B2"})).unwrap(); + let part = |r: crate::Result| matches!(r, Err(crate::EngineError::Other(m)) if m == super::DATA_TABLE_PART); + assert!(part(s.execute("cell.set", json!({"cell": "E4", "input": "5"})))); + assert!(part(s.execute("range.setValues", json!({"range": "E4", "values": [[1], [2]]})))); + assert!(part(s.execute("edit.clearContents", json!({"range": "E3:E4"})))); + assert!(part(s.execute("data.sortDescending", json!({"range": "D3:E5"})))); + assert!(part(s.execute("data.dataTable", json!({"range": "E3:F6", "columnInput": "B1"}))), "overlaps a table"); + assert!(s.execute("cell.set", json!({"cell": "K1", "input": "=TABLE(B1,B2)"})).is_err()); + assert!((n(&s, "E4") - 210.0).abs() < 1e-9); + // Formatting results, and editing the edges, are fine. + s.execute("home.bold", json!({"range": "E3:E5"})).unwrap(); + s.execute("cell.set", json!({"cell": "D4", "input": "400"})).unwrap(); + assert!((n(&s, "E4") - 420.0).abs() < 1e-9); + // Rows and cells: cutting through the table is refused; above it, table and input move. + assert!(part(s.execute("home.insertRows", json!({"rows": "4:4"})))); + assert!(part(s.execute("home.deleteRows", json!({"rows": "4:5"})))); + assert!(part(s.execute("home.insertCells", json!({"range": "E3", "shift": "down"})))); + assert!(part(s.execute("home.deleteCells", json!({"range": "D1:D2", "shift": "up"})))); + s.execute("home.insertRows", json!({"rows": "1:1"})).unwrap(); + let t = tables(&s); + assert_eq!((t[0].range.a1(), t[0].col_input.map(|c| c.a1())), ("D3:E6".to_string(), Some("B3".to_string()))); + s.execute("cell.set", json!({"cell": "B2", "input": "0.2"})).unwrap(); + assert!((n(&s, "E4") - 120.0).abs() < 1e-9); + // Cutting the whole table moves it. + s.execute("edit.cut", json!({"range": "D3:E6"})).unwrap(); + s.execute("edit.paste", json!({"at": "G3"})).unwrap(); + assert_eq!(tables(&s)[0].range.a1(), "G3:H6"); + s.execute("cell.set", json!({"cell": "B2", "input": "0.5"})).unwrap(); + assert!((n(&s, "H4") - 150.0).abs() < 1e-9); + // Clearing every result removes the table; the cells then take values again. + s.execute("edit.clearContents", json!({"range": "H4:H6"})).unwrap(); + assert!(tables(&s).is_empty()); + s.execute("cell.set", json!({"cell": "H5", "input": "7"})).unwrap(); + s.execute("edit.undo", json!({"steps": 2})).unwrap(); + assert_eq!(tables(&s).len(), 1); + // Deleting the input cell leaves the results as values. + s.execute("home.deleteCells", json!({"range": "B3", "shift": "up"})).unwrap(); + assert!(tables(&s).is_empty()); + assert!((n(&s, "H4") - 150.0).abs() < 1e-9); + } + + #[test] + fn data_tables_that_read_their_results_are_circular() { + let mut s = s(); + s.execute("range.setValues", json!({"range": "B1", "values": [[1]]})).unwrap(); + s.execute("range.setValues", json!({"range": "D2", "values": [[1], [2]]})).unwrap(); + // The formula reads the table's own results. + s.execute("cell.set", json!({"cell": "E1", "input": "=B1+SUM(E2:E3)"})).unwrap(); + s.execute("data.dataTable", json!({"range": "D1:E3", "columnInput": "B1"})).unwrap(); + assert_eq!(v(&s, "E2"), Value::Error(gridcraft_core::CellError::Circ)); + s.execute("cell.set", json!({"cell": "B1", "input": "2"})).unwrap(); + assert_eq!(v(&s, "E3"), Value::Error(gridcraft_core::CellError::Circ)); + // Breaking the loop calculates the table again. + s.execute("cell.set", json!({"cell": "E1", "input": "=B1*3"})).unwrap(); + assert_eq!(n(&s, "E3"), 6.0); + // An input cell that is also one of the table's results is refused. + assert!(s.execute("data.dataTable", json!({"range": "G1:H3", "columnInput": "H2"})).is_err()); + } + + #[test] + fn data_tables_wait_for_f9_in_automatic_except_tables_mode() { + let mut s = s(); + loan(&mut s); + s.execute("formulas.calculationOptions", json!({"mode": "automaticExceptTables"})).unwrap(); + // A new table is calculated when it is made. + s.execute("data.dataTable", json!({"range": "D2:E5", "columnInput": "B2"})).unwrap(); + assert!((n(&s, "E4") - 210.0).abs() < 1e-9); + s.execute("cell.set", json!({"cell": "B1", "input": "0.5"})).unwrap(); + assert!((n(&s, "E4") - 210.0).abs() < 1e-9); + s.execute("formulas.calculateNow", json!({})).unwrap(); + assert!((n(&s, "E4") - 300.0).abs() < 1e-9); + } + + #[test] + fn hostile_data_tables_are_refused() { + let mut s = s(); + loan(&mut s); + s.execute("home.insertSheet", json!({"name": "Sheet2"})).unwrap(); + s.execute("sheet.activate", json!({"sheet": 0})).unwrap(); + for p in [ + json!({}), + json!({"range": "D2:E5"}), + json!({"range": "D2:E5", "columnInput": "E3"}), + json!({"range": "D2:E5", "columnInput": "Sheet2!B2"}), + json!({"range": "D2:E5", "columnInput": "B2:B3"}), + json!({"range": "D2:E5", "columnInput": "nonsense"}), + json!({"range": "D2:D5", "columnInput": "B2"}), + json!({"range": "A1:XFD1048576", "columnInput": "B2"}), + json!({"range": "D2:E100003", "columnInput": "B2"}), + ] { + assert!(s.execute("data.dataTable", p.clone()).is_err(), "{p}"); + } + assert!(tables(&s).is_empty()); + // The largest table allowed is calculated. + s.execute("data.dataTable", json!({"range": "D2:E100002", "columnInput": "B2"})).unwrap(); + assert!((n(&s, "E4") - 210.0).abs() < 1e-9); + } + + #[test] + fn data_tables_stay_live_through_xlsx() { + let mut s = s(); + loan(&mut s); + s.execute("data.dataTable", json!({"range": "D2:E5", "columnInput": "B2"})).unwrap(); + s.execute("cell.set", json!({"cell": "G1", "input": "=B2*B1"})).unwrap(); + s.execute("range.setValues", json!({"range": "H1", "values": [[1, 2]]})).unwrap(); + s.execute("range.setValues", json!({"range": "G2", "values": [[10], [20]]})).unwrap(); + s.execute("data.dataTable", json!({"range": "G1:I3", "rowInput": "B1", "columnInput": "B2"})).unwrap(); + let before = tables(&s); + let b64 = s.execute("file.saveBytes", json!({"format": "xlsx"})).unwrap()["base64"].as_str().unwrap().to_string(); + s.execute("file.open", json!({"name": "dt.xlsx", "base64": b64})).unwrap(); + let mut after = tables(&s); + after.sort_by_key(|t| t.range.start.col); + assert_eq!(after, before); + assert!((n(&s, "E4") - 210.0).abs() < 1e-9); + assert_eq!(n(&s, "I3"), 40.0); + assert_eq!(s.execute("cell.get", json!({"cell": "H2"})).unwrap()["formula"], "{=TABLE(B1,B2)}"); + // Still live after reopening. + s.execute("cell.set", json!({"cell": "B1", "input": "1"})).unwrap(); + assert_eq!(n(&s, "E4"), 400.0); + assert_eq!(n(&s, "I3"), 40.0); + s.execute("cell.set", json!({"cell": "G1", "input": "=B2*B1*2"})).unwrap(); + assert_eq!(n(&s, "I3"), 80.0); + } + + #[test] + fn static_data_tables_write_values() { + let mut s = s(); + loan(&mut s); + let r = s.execute("data.dataTable", json!({"range": "D2:E5", "columnInput": "B2", "static": true})).unwrap(); + assert_eq!(r["static"], true); + assert!(tables(&s).is_empty()); + assert!((n(&s, "E4") - 210.0).abs() < 1e-9); + s.execute("cell.set", json!({"cell": "B1", "input": "0.5"})).unwrap(); + assert!((n(&s, "E4") - 210.0).abs() < 1e-9, "a snapshot doesn't recalculate"); + s.execute("cell.set", json!({"cell": "E4", "input": "1"})).unwrap(); + // A snapshot over a live table's range replaces the table. + s.execute("data.dataTable", json!({"range": "D2:E5", "columnInput": "B2"})).unwrap(); + assert_eq!(tables(&s).len(), 1); + s.execute("data.dataTable", json!({"range": "D2:E5", "columnInput": "B2", "static": true})).unwrap(); + assert!(tables(&s).is_empty()); + assert_eq!(n(&s, "E4"), 300.0); + } + #[test] fn advanced_filter_in_place_and_copy() { let mut s = s(); diff --git a/crates/engine/src/display.rs b/crates/engine/src/display.rs index b90e42727b..a8323938ab 100644 --- a/crates/engine/src/display.rs +++ b/crates/engine/src/display.rs @@ -65,13 +65,14 @@ pub fn edit_text(wb: &Workbook, sheet: &Sheet, c: CellRef) -> String { } /// Whether `c` has a formula that must not be shown: its cell is formatted Hidden on a -/// protected sheet. A spilled cell follows its anchor. +/// protected sheet. A spilled cell follows its anchor; a data table's result cell has the table's formula. pub fn formula_hidden(wb: &Workbook, sheet: &Sheet, c: CellRef) -> bool { if !sheet.is_protected() { return false; } let at = match sheet.cell(c) { Some(x) if x.formula.is_some() => c, + _ if sheet.data_table_at(c).is_some() => c, _ => match sheet.spill_ranges.iter().find(|(_, r)| r.contains(c)) { Some((anchor, _)) => *anchor, None => return false, @@ -81,12 +82,16 @@ pub fn formula_hidden(wb: &Workbook, sheet: &Sheet, c: CellRef) -> bool { } /// The text a cell shows (full precision General; the grid narrows General to fit the column). +/// With Show Formulas on, a data table's result shows `=TABLE(,B2)`, without the braces the +/// formula bar adds (as Excel shows it). pub fn cell_text(wb: &Workbook, sheet: &Sheet, c: CellRef) -> String { - if sheet.show_formulas - && let Some(f) = sheet.cell(c).and_then(|x| x.formula.as_ref()) - && !formula_hidden(wb, sheet, c) - { - return format!("={}", f.text); + if sheet.show_formulas && !formula_hidden(wb, sheet, c) { + if let Some(f) = sheet.cell(c).and_then(|x| x.formula.as_ref()) { + return format!("={}", f.text); + } + if let Some(t) = sheet.data_table_at(c) { + return format!("={}", t.formula_text()); + } } value_text(wb, sheet, c) } diff --git a/crates/engine/tests/command_sequence_fuzz.rs b/crates/engine/tests/command_sequence_fuzz.rs index f8cb603cba..2714808424 100644 --- a/crates/engine/tests/command_sequence_fuzz.rs +++ b/crates/engine/tests/command_sequence_fuzz.rs @@ -781,6 +781,7 @@ const CURATED: &[(&str, usize)] = &[ ("data.filterBy", 2), ("data.clearFilter", 1), ("data.removeDuplicates", 1), + ("data.dataTable", 2), ("formulas.defineName", 2), ("formulas.deleteName", 1), ("edit.replace", 1), @@ -917,6 +918,24 @@ fn curated_params(rng: &mut Rng, wb: &Workbook, id: &str) -> Json { } } "data.removeDuplicates" => json!({"range": near_range(rng).a1(), "header": rng.chance(50)}), + "data.dataTable" => { + // Live (mostly) or static What-If data tables near the data: one- or two-variable, + // input cells near the data too (sometimes inside the table, which is refused). + let a = near_cell(rng); + let r = RangeRef::new(a, CellRef::new(a.row + rng.int(1, 6) as u32, a.col + rng.int(1, 3) as u32)); + let mut p = json!({"range": r.a1()}); + let which = rng.below(3); + if which != 1 { + p["columnInput"] = json!(near_cell(rng).a1()); + } + if which != 0 { + p["rowInput"] = json!(near_cell(rng).a1()); + } + if rng.chance(15) { + p["static"] = json!(true); + } + p + } "formulas.defineName" => { let sheet = other_sheet(rng, wb); let r = near_range(rng); diff --git a/crates/model/src/features.rs b/crates/model/src/features.rs index b51f7bc98a..497cfdbbc1 100644 --- a/crates/model/src/features.rs +++ b/crates/model/src/features.rs @@ -876,6 +876,107 @@ pub struct LineInfo { pub custom: bool, } +// ---------------------------------------------------------------- data tables + +/// Most result cells one What-If data table may have. +pub const MAX_DATA_TABLE_CELLS: u64 = 100_000; + +/// A What-If data table (`{=TABLE(row_input, column_input)}`), a multi-cell formula over its +/// results. The range's top row and left column are its edges: input values along one or both of +/// them, and the formulas along the other (for a two-variable table, one formula in the top-left +/// corner). The rest of the range is the body: each body cell holds a formula's result with the +/// edge value substituted into the input cell. Input cells are on the table's sheet, outside the +/// range. +#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)] +pub struct DataTable { + pub range: RangeRef, + /// Receives the values across the top row. + pub row_input: Option, + /// Receives the values down the left column. + pub col_input: Option, +} + +/// One substitution of a data table: each `(input, value_cell)` input takes the value of +/// `value_cell`, then each `(formula, result)` formula's value is copied into `result`. +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct TableTrial { + pub inputs: Vec<(CellRef, CellRef)>, + pub outputs: Vec<(CellRef, CellRef)>, +} + +impl DataTable { + /// The results area: the range without its top row and left column. + pub fn body(&self) -> Option { + let r = self.range; + (r.height() >= 2 && r.width() >= 2).then(|| RangeRef::new(CellRef::new(r.start.row + 1, r.start.col + 1), r.end)) + } + + /// Whether the table can be calculated: it has a body of at most [`MAX_DATA_TABLE_CELLS`] + /// cells and at least one input cell, outside the range. + pub fn is_valid(&self) -> bool { + self.body().is_some_and(|b| b.count() <= MAX_DATA_TABLE_CELLS) + && (self.row_input.is_some() || self.col_input.is_some()) + && [self.row_input, self.col_input].into_iter().flatten().all(|c| !self.range.contains(c)) + } + + /// Whether `c` is on the top row or the left column (the cells the results depend on). + pub fn is_edge(&self, c: CellRef) -> bool { + self.range.contains(c) && (c.row == self.range.start.row || c.col == self.range.start.col) + } + + /// The text body cells show: `TABLE(B1,B2)`, with an absent input left blank (`TABLE(,B2)`). + pub fn formula_text(&self) -> String { + let a = |c: Option| c.map(|c| c.a1()).unwrap_or_default(); + format!("TABLE({},{})", a(self.row_input), a(self.col_input)) + } + + /// The cells that hold the table's formulas: the top row of a table with values down its left + /// column, the left column of one with values across its top row, the top-left corner of a + /// two-variable table. Every result depends on all of them. + pub fn formulas(&self) -> Option { + let body = self.body()?; + let (r0, c0) = (self.range.start.row, self.range.start.col); + match (self.row_input, self.col_input) { + (None, Some(_)) => Some(RangeRef::new(CellRef::new(r0, body.start.col), CellRef::new(r0, body.end.col))), + (Some(_), None) => Some(RangeRef::new(CellRef::new(body.start.row, c0), CellRef::new(body.end.row, c0))), + (Some(_), Some(_)) => Some(RangeRef::cell(self.range.start)), + (None, None) => None, + } + } + + /// The substitutions that fill the body, grouped so that each is computed once. Empty for a + /// table that isn't [valid](Self::is_valid). + pub fn trials(&self) -> Vec { + let Some(body) = self.body().filter(|_| self.is_valid()) else { return vec![] }; + let (r0, c0) = (self.range.start.row, self.range.start.col); + match (self.row_input, self.col_input) { + // Values down the left column, formulas across the top row. + (None, Some(ci)) => (body.start.row..=body.end.row) + .map(|row| TableTrial { + inputs: vec![(ci, CellRef::new(row, c0))], + outputs: (body.start.col..=body.end.col).map(|col| (CellRef::new(r0, col), CellRef::new(row, col))).collect(), + }) + .collect(), + // Values across the top row, formulas down the left column. + (Some(ri), None) => (body.start.col..=body.end.col) + .map(|col| TableTrial { + inputs: vec![(ri, CellRef::new(r0, col))], + outputs: (body.start.row..=body.end.row).map(|row| (CellRef::new(row, c0), CellRef::new(row, col))).collect(), + }) + .collect(), + // One formula in the corner; every body cell is its own substitution. + (Some(ri), Some(ci)) => body + .iter() + .map(|c| TableTrial { + inputs: vec![(ri, CellRef::new(r0, c.col)), (ci, CellRef::new(c.row, c0))], + outputs: vec![(self.range.start, c)], + }) + .collect(), + (None, None) => vec![], + } + } +} + // ---------------------------------------------------------------- pivot tables /// How a PivotTable lays out its row fields. diff --git a/crates/model/src/sheet.rs b/crates/model/src/sheet.rs index b404b891cc..f1eaab7745 100644 --- a/crates/model/src/sheet.rs +++ b/crates/model/src/sheet.rs @@ -96,6 +96,9 @@ pub struct Sheet { /// PivotTables whose output lives on this sheet. #[serde(default)] pub pivots: Vec, + /// What-If data tables (`{=TABLE()}`) whose results are on this sheet. + #[serde(default, skip_serializing_if = "Vec::is_empty")] + pub data_tables: Vec, pub protection: Option, /// Allow Edit Ranges (meaningful while the sheet is protected). #[serde(default, skip_serializing_if = "Vec::is_empty")] @@ -157,6 +160,7 @@ impl Sheet { sparklines: vec![], slicers: vec![], pivots: vec![], + data_tables: vec![], protection: None, edit_ranges: vec![], print: PrintSettings::default(), @@ -167,6 +171,11 @@ impl Sheet { } } + /// The data table whose results area contains `c`. + pub fn data_table_at(&self, c: CellRef) -> Option<&DataTable> { + self.data_tables.iter().find(|t| t.body().is_some_and(|b| b.contains(c))) + } + pub fn cell(&self, c: CellRef) -> Option<&Cell> { self.cells.get(c) } diff --git a/crates/ui-egui/src/dialogs.rs b/crates/ui-egui/src/dialogs.rs index fb899c7026..de37e10d41 100644 --- a/crates/ui-egui/src/dialogs.rs +++ b/crates/ui-egui/src/dialogs.rs @@ -438,6 +438,11 @@ impl Dialog { json!({"header": true}), ), "goalSeek" => Dialog::custom("goalSeek", "Goal Seek", json!({"set": active, "to": "0", "changing": ""})), + // What-If Analysis › Data Table over the selection; OK runs `data.dataTable`, and an + // input cell Excel would refuse keeps the dialog open behind the message. + "dataTable" => { + Dialog::form("dataTable", "Data Table", "data.dataTable", vec![], json!({"range": sel, "rowInput": "", "columnInput": ""})) + } "forecastSheet" => Dialog::form( "forecastSheet", "Create Forecast Worksheet", @@ -1058,6 +1063,7 @@ pub fn show(app: &mut SheetApp, ctx: &egui::Context) { "selectData" => select_data(app, ui, &mut d, &mut confirm, &mut open), "commandSearch" => command_search(app, ui, &mut d, &mut confirm), "goalSeek" => goal_seek(app, ui, &mut d, &mut confirm), + "dataTable" => data_table(app, ui, &mut d, &mut confirm, &mut open), "analysisToolPak" => analysis_toolpak(app, ui, &mut d, &mut confirm, &mut open), "solver" => solver(ui, &mut d, &mut confirm, &mut open), "solverResults" => solver_results(app, ui, &mut d, &mut open), @@ -2307,6 +2313,25 @@ fn goal_seek(app: &mut SheetApp, ui: &mut egui::Ui, d: &mut Dialog, confirm: &mu } } +/// Data Table: Excel's two fields, the row input cell (receives the values across the top row) +/// and the column input cell (receives the values down the left column). Either may stay blank. +fn data_table(app: &SheetApp, ui: &mut egui::Ui, d: &mut Dialog, confirm: &mut bool, open: &mut bool) { + let lang = app.ui.language; + egui::Grid::new("dataTable").num_columns(2).spacing(vec2(8.0, 6.0)).show(ui, |ui| { + for (key, label) in [("rowInput", "Row input cell:"), ("columnInput", "Column input cell:")] { + ui.label(lang.tr(label)); + let mut text = d.values.get(key).and_then(Json::as_str).unwrap_or("").to_string(); + let response = ui.add(egui::TextEdit::singleline(&mut text).desired_width(150.0)); + response.widget_info(|| egui::WidgetInfo::labeled(egui::WidgetType::TextEdit, true, label)); + if response.changed() { + d.values.insert(key.into(), json!(text)); + } + ui.end_row(); + } + }); + ok_cancel(ui, confirm, open); +} + /// The fields of one Analysis ToolPak tool's form (input and options; output is drawn apart). fn toolpak_fields(tool: &str) -> Vec { use Field::*; diff --git a/crates/ui-egui/src/formula_bar.rs b/crates/ui-egui/src/formula_bar.rs index 7f1ad87fc1..703e5e470a 100644 --- a/crates/ui-egui/src/formula_bar.rs +++ b/crates/ui-egui/src/formula_bar.rs @@ -69,6 +69,15 @@ fn active_input(app: &SheetApp) -> String { if let Some(picture) = sh.cell_pictures.get(&a) { return if picture.alt.is_empty() { "Picture in cell".into() } else { format!("Picture: {}", picture.alt) }; } + // A data table's results show its formula in braces, `{=TABLE(B1,B2)}`, with the formula + // language's separators (`{=TABLE(B1;B2)}`) and function name. + if let Some(t) = sh.data_table_at(a) { + if gridcraft_engine::display::formula_hidden(&d.wb, sh, a) { + return String::new(); + } + let shown = crate::formula_locale::display(&format!("={}", t.formula_text()), app.ui.language.formula_locale(), &d.wb, d.wb.active_sheet); + return format!("{{{shown}}}"); + } if let Some(c) = sh.cell(a) { if c.formula.is_some() && d.wb.styles.get(c.style).protection.hidden && sh.is_protected() { return String::new(); diff --git a/crates/ui-egui/src/grid.rs b/crates/ui-egui/src/grid.rs index 97b787ef33..682211508f 100644 --- a/crates/ui-egui/src/grid.rs +++ b/crates/ui-egui/src/grid.rs @@ -531,6 +531,13 @@ fn display_text( { return (format!("={}", f.text), None, false, None); } + // A data table's results show its formula, `=TABLE(B1,B2)`. + if sh.show_formulas + && let Some(t) = sh.data_table_at(c) + && !gridcraft_engine::display::formula_hidden(wb, sh, c) + { + return (format!("={}", t.formula_text()), None, false, None); + } if !sh.show_zeros && v.as_f64() == Some(0.0) { return (String::new(), None, false, None); } @@ -1160,7 +1167,7 @@ pub(crate) fn paint_quadrant( let char_w = p.layout_no_wrap("0".to_string(), font.clone(), Color32::PLACEHOLDER).size().x.max(1.0); let avail_w = rect.right() - text_left; let (mut text, ncolor, numeric, fill_char) = display_text(wb, sh, c, &v, st, avail_w / z.max(0.1) * z, char_w); - if sh.show_formulas && sh.cell(c).is_some_and(|cell| cell.formula.is_some()) { + if sh.show_formulas && (sh.cell(c).is_some_and(|cell| cell.formula.is_some()) || sh.data_table_at(c).is_some()) { text = crate::formula_locale::display(&text, crate::i18n::current(p.ctx()).formula_locale(), wb, si); } if text.is_empty() { diff --git a/crates/ui-egui/src/i18n/de.rs b/crates/ui-egui/src/i18n/de.rs index c20c7d1b36..c0631d7563 100644 --- a/crates/ui-egui/src/i18n/de.rs +++ b/crates/ui-egui/src/i18n/de.rs @@ -152,7 +152,7 @@ pub(super) fn translate(text: &str) -> Option<&'static str> { "Calculate Now" => "Jetzt berechnen", "Calculate Sheet" => "Blatt berechnen", "Automatic" => "Automatisch", - "Automatic Except for Data Tables" => "Automatisch, außer für Datentabellen", + "Automatic Except for Data Tables" => "Automatisch außer bei Datentabellen", "Manual" => "Manuell", "Sort &\nFilter" => "Sortieren und\nFiltern", "Sort" => "Sortieren", @@ -175,6 +175,11 @@ pub(super) fn translate(text: &str) -> Option<&'static str> { "Subtotal" => "Teilergebnis", "What-If\nAnalysis" => "Was-wäre-wenn-\nAnalyse", "Data Table…" => "Datentabelle…", + "What-If Analysis" => "Was-wäre-wenn-Analyse", + "Data Table" => "Datentabelle", + "Row input cell:" => "Zeileneingabewert:", + "Column input cell:" => "Spalteneingabewert:", + "Cancel" => "Abbrechen", "Scenario Manager…" => "Szenario-Manager…", "Goal Seek…" => "Zielwertsuche…", "Spelling" => "Rechtschreibung", diff --git a/crates/ui-egui/src/keytips/keytip_table/generated.rs b/crates/ui-egui/src/keytips/keytip_table/generated.rs index 79920b5b8c..c9b1675053 100644 --- a/crates/ui-egui/src/keytips/keytip_table/generated.rs +++ b/crates/ui-egui/src/keytips/keytip_table/generated.rs @@ -312,7 +312,7 @@ pub const SEQUENCES: &[(&str, Entry)] = &[ ("AW", Entry::Continue), // group: group: What-If (goalSeek/scenarioManager/dataTable) ("AWG", Entry::Finish(Action::Dialog("goalSeek"))), // Excel: Goal Seek ("AWS", Entry::Finish(Action::Command("data.scenarioManager"))), // Excel: Scenario Manager - ("AWT", Entry::Finish(Action::Command("data.dataTable"))), // Excel: Data Table + ("AWT", Entry::Finish(Action::Dialog("dataTable"))), // Excel: Data Table ("AX", Entry::Finish(Action::Pending("data.existingConnections"))), // pending; Excel: Existing Connections; gap #12; reserved until the command lands ("AY", Entry::Finish(Action::Command("data.reapply"))), // Excel: Reapply Filter; containment(Reapply) ("RA1", Entry::Finish(Action::Dialog("accessibility"))), // Excel: Check Accessibility diff --git a/crates/ui-egui/src/keytips/keytips.json b/crates/ui-egui/src/keytips/keytips.json index 59aeadcfb8..0eff8b2508 100644 --- a/crates/ui-egui/src/keytips/keytips.json +++ b/crates/ui-egui/src/keytips/keytips.json @@ -2111,8 +2111,8 @@ }, { "seq": "AWT", - "kind": "command", - "target": "data.dataTable", + "kind": "dialog", + "target": "dataTable", "excel": "Data Table", "note": "" }, diff --git a/crates/ui-egui/src/lib.rs b/crates/ui-egui/src/lib.rs index f96cdac3c0..e788881580 100644 --- a/crates/ui-egui/src/lib.rs +++ b/crates/ui-egui/src/lib.rs @@ -880,8 +880,10 @@ impl SheetApp { let current = if sh.cell_pictures.contains_key(&at) { String::new() } else { - match sh.cell(at) { - Some(c) if c.formula.is_some() => formula_locale::display(&c.input_text(), locale, &d.wb, d.wb.active_sheet), + match (sh.cell(at), sh.data_table_at(at)) { + (Some(c), _) if c.formula.is_some() => formula_locale::display(&c.input_text(), locale, &d.wb, d.wb.active_sheet), + // A data table's result edits as its formula, `=TABLE(B1,B2)` (Enter keeps it). + (_, Some(t)) => formula_locale::display(&format!("={}", t.formula_text()), locale, &d.wb, d.wb.active_sheet), _ => gridcraft_engine::display::edit_text(&d.wb, sh, at), } }; @@ -916,6 +918,17 @@ impl SheetApp { } None => text, }; + // Enter on a data table's result with its `=TABLE()` formula unchanged changes nothing; + // any other entry there is refused by the engine ("Cannot change part of a data table."). + if let Some(d) = self.session.active() + && let Some(t) = d.wb.sheet(ed.sheet).and_then(|sh| sh.data_table_at(ed.cell)) + && text == format!("={}", t.formula_text()) + { + if dr != 0 || dc != 0 { + self.move_after_enter(dr, dc); + } + return true; + } // Data validation. if let Some(d) = self.session.active() && let Some((dv, msg)) = gridcraft_engine::cmd::data::check_validation(&d.wb, ed.sheet, ed.cell, &text) diff --git a/crates/ui-egui/src/ribbon.rs b/crates/ui-egui/src/ribbon.rs index 9c6d89eeb2..2e0334e0b3 100644 --- a/crates/ui-egui/src/ribbon.rs +++ b/crates/ui-egui/src/ribbon.rs @@ -1958,9 +1958,11 @@ fn data(app: &mut SheetApp, ui: &mut Ui) { ); }); sep(ui); + // Excel's Forecast group: What-If Analysis ▾ (Scenario Manager…, Goal Seek…, Data Table…), + // then Forecast Sheet. let wi = big_button(ui, Icon::Calc, "What-If\nAnalysis", "What-If Analysis", true); for sequence in - keytip_table::group_extension(app.keytips.prefix().unwrap_or_default(), &["dialog:goalSeek", "data.scenarioManager", "data.dataTable"]) + keytip_table::group_extension(app.keytips.prefix().unwrap_or_default(), &["data.scenarioManager", "dialog:goalSeek", "dialog:dataTable"]) { keytip(app, ui, sequence, wi.rect.center_bottom()); } @@ -1969,9 +1971,9 @@ fn data(app: &mut SheetApp, ui: &mut Ui) { app, ui, &[ - ("Goal Seek…", "dialog:goalSeek", json!({})), ("Scenario Manager…", "data.scenarioManager", json!({})), - ("Data Table…", "data.dataTable", json!({})), + ("Goal Seek…", "dialog:goalSeek", json!({})), + ("Data Table…", "dialog:dataTable", json!({})), ], ); }); diff --git a/crates/ui-egui/tests/data_table_dialog.rs b/crates/ui-egui/tests/data_table_dialog.rs new file mode 100644 index 0000000000..30f1f2c02f --- /dev/null +++ b/crates/ui-egui/tests/data_table_dialog.rs @@ -0,0 +1,138 @@ +//! Data › What-If Analysis › Data Table…: the dialog makes a live `{=TABLE()}` over the selection, +//! the formula bar and the cell editor show the table's formula, and results can't be edited one +//! by one. + +use egui::{accesskit::Role, vec2}; +use egui_kittest::kittest::Queryable; +use gridcraft_engine::Session; +use gridcraft_ui_egui::SheetApp; +use gridcraft_ui_egui::control::{Outcome, handle}; +use serde_json::{Value as Json, json}; + +type Harness = egui_kittest::Harness<'static, SheetApp>; + +/// Rate B1, amount B2; amounts down D3:D5 and the formula `=B2*(1+B1)` in E2; D2:E5 selected. +fn harness() -> Harness { + let mut session = Session::new(); + session.new_workbook(); + session.execute("range.setValues", json!({"range": "B1", "values": [[0.05], [1000]]})).unwrap(); + session.execute("range.setValues", json!({"range": "D3", "values": [[100], [200], [300]]})).unwrap(); + session.execute("cell.set", json!({"cell": "E2", "input": "=B2*(1+B1)"})).unwrap(); + session.execute("selection.set", json!({"range": "D2:E5"})).unwrap(); + let mut h = egui_kittest::Harness::builder().with_size(vec2(1200.0, 800.0)).build_ui_state( + |ui, app: &mut SheetApp| { + app.logic(&ui.ctx().clone()); + app.ui(ui); + }, + SheetApp::new(session, Default::default()), + ); + h.run_steps(6); + h +} + +fn control(h: &mut Harness, method: &str, params: Json) -> Json { + let ctx = h.ctx.clone(); + match handle(h.state_mut(), &ctx, method, ¶ms) { + Outcome::Done(v) => v, + Outcome::Screenshot { .. } => json!(null), + } +} + +fn value(h: &mut Harness, cell: &str) -> Json { + h.state_mut().run("cell.get", json!({"cell": cell})).unwrap()["value"].clone() +} + +fn tables(h: &Harness) -> usize { + h.state().session.active().unwrap().wb.active().unwrap().data_tables.len() +} + +#[test] +fn the_dialog_makes_a_live_table_over_the_selection() { + let mut h = harness(); + h.state_mut().open_dialog("dataTable", json!({})); + h.run_steps(6); + // Excel's two fields; the column input cell receives the values down the left column. + assert!(h.query_by_role_and_label(Role::TextInput, "Row input cell:").is_some()); + h.get_by_role_and_label(Role::TextInput, "Column input cell:").click(); + h.run_steps(2); + h.get_by_role_and_label(Role::TextInput, "Column input cell:").type_text("$B$2"); + h.run_steps(4); + h.get_by_label(" OK ").click(); + h.run_steps(6); + assert!(h.state().dialog.is_none(), "OK closes the dialog"); + assert!(h.state().message.is_none(), "{:?}", h.state().message); + assert_eq!(tables(&h), 1); + assert_eq!(value(&mut h, "E4"), json!(210.0)); + // The results follow their formula. + h.state_mut().run("cell.set", json!({"cell": "B1", "input": "0.5"})).unwrap(); + assert_eq!(value(&mut h, "E4"), json!(300.0)); +} + +#[test] +fn an_input_cell_excel_refuses_keeps_the_dialog_open_behind_the_message() { + let mut h = harness(); + control(&mut h, "ui.dialog", json!({"name": "dataTable"})); + h.run_steps(6); + // An input cell inside the table. + control(&mut h, "ui.dialog.set", json!({"field": "columnInput", "value": "E3"})); + h.run_steps(4); + h.get_by_label(" OK ").click(); + h.run_steps(6); + assert_eq!(h.state().message.as_ref().map(|(_, m)| m.as_str()), Some("Input cell reference is not valid.")); + assert!(h.state().dialog.is_some(), "the dialog stays open for another try"); + assert_eq!(tables(&h), 0); + // Over the control channel, the same command runs with the dialog's values. + control(&mut h, "ui.message.dismiss", json!({})); + control(&mut h, "ui.dialog.set", json!({"field": "columnInput", "value": "B2"})); + let r = control(&mut h, "ui.dialog.confirm", json!({})); + assert_eq!(r["ok"], json!(true), "{r}"); + assert_eq!(tables(&h), 1); +} + +#[test] +fn the_dialog_speaks_german() { + let mut h = harness(); + h.state_mut().run("app.language.set", json!({"language": "de"})).unwrap(); + h.state_mut().open_dialog("dataTable", json!({})); + h.run_steps(6); + assert!(h.query_by_label("Datentabelle").is_some(), "dialog title"); + assert!(h.query_by_role_and_label(Role::TextInput, "Row input cell:").is_some()); + assert!(!h.query_all_by_label("Zeileneingabewert:").collect::>().is_empty()); + assert!(!h.query_all_by_label("Spalteneingabewert:").collect::>().is_empty()); +} + +#[test] +fn the_formula_bar_shows_the_table_formula_in_the_formula_language() { + let mut h = harness(); + h.state_mut().run("data.dataTable", json!({"range": "D2:E5", "columnInput": "B2"})).unwrap(); + h.state_mut().run("selection.set", json!({"range": "E4"})).unwrap(); + h.run_steps(4); + assert!(h.query_by_label("{=TABLE(,B2)}").is_some(), "English formula bar"); + // A semicolon-separated formula language. + h.state_mut().run("app.language.set", json!({"language": "es"})).unwrap(); + h.run_steps(4); + assert!(h.query_by_label("{=TABLE(;B2)}").is_some(), "Spanish formula bar"); +} + +#[test] +fn editing_a_result_keeps_the_table_unless_it_changes() { + let mut h = harness(); + h.state_mut().run("data.dataTable", json!({"range": "D2:E5", "columnInput": "B2"})).unwrap(); + h.state_mut().run("selection.set", json!({"range": "E4"})).unwrap(); + h.run_steps(2); + // F2 shows the table's formula; Enter with it unchanged changes nothing. + control(&mut h, "ui.edit.begin", json!({})); + assert_eq!(h.state().editor.as_ref().map(|e| e.text.clone()), Some("=TABLE(,B2)".to_string())); + control(&mut h, "ui.edit.commit", json!({})); + h.run_steps(2); + assert!(h.state().message.is_none(), "{:?}", h.state().message); + assert!(h.state().editor.is_none()); + assert_eq!(tables(&h), 1); + // Typing over one result is refused with Excel's message. + h.state_mut().run("selection.set", json!({"range": "E4"})).unwrap(); + control(&mut h, "ui.edit.begin", json!({"text": "5"})); + control(&mut h, "ui.edit.commit", json!({})); + h.run_steps(2); + assert_eq!(h.state().message.as_ref().map(|(_, m)| m.as_str()), Some("Cannot change part of a data table.")); + assert_eq!(value(&mut h, "E4"), json!(210.0)); +} diff --git a/crates/ui-egui/tests/keytips_generated.rs b/crates/ui-egui/tests/keytips_generated.rs index 653554857e..0a31b586d0 100644 --- a/crates/ui-egui/tests/keytips_generated.rs +++ b/crates/ui-egui/tests/keytips_generated.rs @@ -2311,7 +2311,7 @@ fn keytip_aws_() { fn keytip_awt_() { let mut h = harness(blank()); sequence(&mut h, "AWT", false); - assert!(h.state().message.is_some() || h.state().toast.is_some(), "AWT must invoke data.dataTable and surface its missing-params error"); + assert!(h.state().dialog.is_some(), "AWT must open the dataTable dialog"); } #[test] diff --git a/crates/xlsx/src/sheet_read.rs b/crates/xlsx/src/sheet_read.rs index 0194d22688..31852f2c06 100644 --- a/crates/xlsx/src/sheet_read.rs +++ b/crates/xlsx/src/sheet_read.rs @@ -33,11 +33,15 @@ struct RowState { /// Dynamic-array spill ranges and legacy array ranges: cached values inside them (other than /// the anchor) are not stored, so the formula can fill them again. dynamic: Vec, + /// What-If data tables, from the `` on their first result cell. + data_tables: Vec, dropped: u64, rows: u64, } const MAX_HYPERLINK_CELLS: u64 = 10_000; +/// Most What-If data tables read per sheet. +const MAX_DATA_TABLES: usize = 1_000; pub fn read_sheet(cx: &mut Ctx<'_>, part: &str, name: &str) -> Result { let bytes = cx.pkg.read(part)?.ok_or_else(|| IoError::Format(format!("missing part {part}")))?; @@ -58,6 +62,7 @@ pub fn read_sheet(cx: &mut Ctx<'_>, part: &str, name: &str) -> Result 0 { cx.warn(format!("sheet \"{name}\": {} cells or rows outside the 1,048,576 × 16,384 grid were dropped", st.dropped)); } + sheet.data_tables = std::mem::take(&mut st.data_tables); drop(bytes); let mut fit_to_page = false; @@ -495,7 +500,7 @@ fn read_formula(cx: &mut Ctx<'_>, st: &mut RowState, f: &El, pos: CellRef, dynam Some(fm) } "dataTable" => { - cx.warn("what-if data tables are not supported; kept their values"); + read_data_table(cx, st, f, pos); None } _ => { @@ -508,6 +513,42 @@ fn read_formula(cx: &mut Ctx<'_>, st: &mut RowState, f: &El, pos: CellRef, dynam } } +/// A What-If data table, `` on its first +/// result cell. `ref` is the results area; the edges are the row above it and the column to its +/// left. Two-variable (`dt2D`): `r1` is the row input and `r2` the column input, whatever `dtr` +/// says (Excel sets it on two-variable tables). One-variable: `r1` is the input, a row input when +/// `dtr` is set. `del1`/`del2` mark a deleted input cell; `ca` (calculate always) needs nothing +/// here, as tables are calculated with their inputs anyway. The result cells keep their cached +/// values, and recalculation fills them again. +fn read_data_table(cx: &mut Ctx<'_>, st: &mut RowState, f: &El, pos: CellRef) { + let Some(body) = f.attr("ref").and_then(RangeRef::parse) else { + cx.warn("a what-if data table has no results area; kept its values"); + return; + }; + let input = |k: &str, deleted: &str| if f.flag(deleted, false) { None } else { f.attr(k).and_then(CellRef::parse) }; + let (row_input, col_input) = if f.flag("dt2D", false) { + (input("r1", "del1"), input("r2", "del2")) + } else if f.flag("dtr", false) { + (input("r1", "del1"), None) + } else { + (None, input("r1", "del1")) + }; + let (Some(r0), Some(c0)) = (body.start.row.checked_sub(1), body.start.col.checked_sub(1)) else { + cx.warn("a what-if data table has no room for its input values and formulas; kept its values"); + return; + }; + let table = gridcraft_model::DataTable { range: RangeRef::new(CellRef::new(r0, c0), body.end), row_input, col_input }; + if body.start != pos || !table.is_valid() || (f.flag("dt2D", false) && (row_input.is_none() || col_input.is_none())) { + cx.warn("a what-if data table is damaged, too large or has a deleted input cell; kept its values"); + } else if st.data_tables.len() >= MAX_DATA_TABLES { + cx.warn(format!("a sheet has more than {MAX_DATA_TABLES} what-if data tables; kept the values of the rest")); + } else if st.data_tables.iter().any(|t| t.range.intersects(&table.range)) { + cx.warn("what-if data tables overlap; kept the values of the later ones"); + } else { + st.data_tables.push(table); + } +} + /// `2024-03-15`, `2024-03-15T10:30:00Z`, `10:30:00` → serial. pub fn parse_iso_datetime(s: &str, sys: DateSystem) -> Option { let s = s.trim_end_matches('Z'); diff --git a/crates/xlsx/src/sheet_write.rs b/crates/xlsx/src/sheet_write.rs index 78e1afaaef..ca82a5b30a 100644 --- a/crates/xlsx/src/sheet_write.rs +++ b/crates/xlsx/src/sheet_write.rs @@ -261,6 +261,10 @@ pub fn write_sheet(wb: &Workbook, si: usize, selected: bool, out: &mut Out) -> ( keys.extend(sheet.cell_pictures.keys().copied()); keys.extend(extra.keys().copied()); keys.extend(overrides.keys().copied()); + // Each data table's `` is on its first result cell. + let table_at: BTreeMap = + sheet.data_tables.iter().filter(|t| t.is_valid()).filter_map(|t| t.body().map(|b| (b.start, t))).collect(); + keys.extend(table_at.keys().copied()); let mut rows: BTreeMap> = BTreeMap::new(); for k in keys { rows.entry(k.row).or_default().push(k); @@ -299,6 +303,10 @@ pub fn write_sheet(wb: &Workbook, si: usize, selected: bool, out: &mut Out) -> ( let cell = sheet.cells.get(*c); let ov = overrides.get(c); let ex = extra.get(c); + if let Some(t) = table_at.get(c) { + write_table_cell(&mut s, *c, cell, t, &sid, out); + continue; + } let array = dynamic.get(c).copied().or_else(|| { let f = cell?.formula.as_ref()?; (f.array.is_none() && crate::array_formula::needs_array(wb, si, *c, f)).then_some(RangeRef::cell(*c)) @@ -729,6 +737,64 @@ fn write_cell( } } +/// A data table's first result cell: the `dataTable` formula that describes the whole table, and +/// the cell's cached result. The other result cells are written as values. +fn write_table_cell( + s: &mut String, + c: CellRef, + cell: Option<&Cell>, + t: &gridcraft_model::DataTable, + sid: &dyn Fn(gridcraft_model::StyleId) -> u32, + out: &mut Out, +) { + let style = cell.map(|x| sid(x.style)).unwrap_or(0); + let _ = write!(s, " Value::Number(0.0), + Value::Error(CellError::Spill | CellError::Calc) => Value::Error(CellError::Value), + v => v, + }; + s.push_str(match &value { + Value::Text(_) => " t=\"str\"", + Value::Bool(_) => " t=\"b\"", + Value::Error(_) => " t=\"e\"", + _ => "", + }); + let body = t.body().unwrap_or(RangeRef::cell(c)); + // Excel writes `dtr="1"` on two-variable tables too, `r1` being the row input cell. + let (two, row, r1, r2) = match (t.row_input, t.col_input) { + (Some(r), Some(col)) => (true, true, Some(r), Some(col)), + (Some(r), None) => (false, true, Some(r), None), + (None, col) => (false, false, col, None), + }; + let _ = write!(s, ">"); + match &value { + Value::Number(n) => { + let _ = write!(s, "{}", num(*n)); + } + Value::Text(x) => { + let _ = write!(s, "{}", esc(&encode_xstring(x))); + } + Value::Bool(b) => { + let _ = write!(s, "{}", *b as u8); + } + Value::Error(e) => { + let _ = write!(s, "{}", e.as_str()); + } + _ => out.needs_calc = true, + } + s.push_str(""); +} + fn cfvo_xml(v: &CfValueKind) -> String { match v { CfValueKind::Min => "".into(), diff --git a/crates/xlsx/src/tests/data_table_tests.rs b/crates/xlsx/src/tests/data_table_tests.rs new file mode 100644 index 0000000000..8d90df8b0e --- /dev/null +++ b/crates/xlsx/src/tests/data_table_tests.rs @@ -0,0 +1,124 @@ +//! What-If data tables: `` on the first result cell, cached results in all of them. + +use gridcraft_core::{CellRef, RangeRef, Value}; +use gridcraft_model::{Cell, DataTable, Formula, Workbook}; + +use super::minimal; +use crate::package::Package; +use crate::xml::El; +use crate::{read_xlsx, write_xlsx}; + +fn at(s: &str) -> CellRef { + CellRef::parse(s).unwrap() +} + +fn table(range: &str, row_input: Option<&str>, col_input: Option<&str>) -> DataTable { + DataTable { range: RangeRef::parse(range).unwrap(), row_input: row_input.map(at), col_input: col_input.map(at) } +} + +fn cell<'a>(sheet: &'a El, address: &str) -> &'a El { + sheet.child("sheetData").unwrap().kids("row").flat_map(|r| r.kids("c")).find(|c| c.attr("r") == Some(address)).unwrap() +} + +fn attrs(f: &El) -> Vec<(String, String)> { + ["t", "ref", "dt2D", "dtr", "r1", "r2"].iter().filter_map(|k| f.attr(k).map(|v| (k.to_string(), v.to_string()))).collect() +} + +#[test] +fn data_tables_round_trip() { + let mut wb = Workbook::new(); + let sh = wb.sheet_mut(0).unwrap(); + for (a, v) in [("B1", 0.05), ("B2", 1000.0), ("D3", 100.0), ("D4", 200.0), ("E3", 105.0), ("E4", 210.0)] { + sh.cells.set(at(a), Cell::value(Value::Number(v))); + } + sh.cells.set(at("E2"), Cell::formula(Formula::new("B2*(1+B1)"))); + // One-variable down a column, two-variable, and one-variable across a row (text result). + sh.data_tables.push(table("D2:E4", None, Some("B2"))); + sh.data_tables.push(table("G1:I3", Some("B1"), Some("B2"))); + sh.cells.set(at("L2"), Cell::value(Value::text("low"))); + sh.data_tables.push(table("K1:M2", Some("B1"), None)); + let bytes = write_xlsx(&wb).unwrap(); + + let mut pkg = Package::open(&bytes).unwrap(); + let xml = pkg.read_xml("xl/worksheets/sheet1.xml").unwrap().unwrap(); + let f = |a: &str| attrs(cell(&xml, a).child("f").unwrap()); + let pairs = |v: &[(&str, &str)]| v.iter().map(|(k, v)| (k.to_string(), v.to_string())).collect::>(); + assert_eq!(f("E3"), pairs(&[("t", "dataTable"), ("ref", "E3:E4"), ("dt2D", "0"), ("dtr", "0"), ("r1", "B2")])); + // Excel sets `dtr` on two-variable tables as well. + assert_eq!(f("H2"), pairs(&[("t", "dataTable"), ("ref", "H2:I3"), ("dt2D", "1"), ("dtr", "1"), ("r1", "B1"), ("r2", "B2")])); + assert_eq!(f("L2"), pairs(&[("t", "dataTable"), ("ref", "L2:M2"), ("dt2D", "0"), ("dtr", "1"), ("r1", "B1")])); + // Cached results: on the formula cell and as values in the others. + assert_eq!(cell(&xml, "E3").child("v").unwrap().text, "105"); + assert_eq!(cell(&xml, "E4").child("v").unwrap().text, "210"); + assert!(cell(&xml, "E4").child("f").is_none()); + assert_eq!(cell(&xml, "L2").attr("t"), Some("str")); + + let (back, report) = read_xlsx(&bytes).unwrap(); + assert!(report.warnings.is_empty(), "{:?}", report.warnings); + let sh = back.sheet(0).unwrap(); + // Read in cell order. + let sorted = |mut v: Vec| { + v.sort_by_key(|t| (t.range.start.row, t.range.start.col)); + v + }; + assert_eq!(sorted(sh.data_tables.clone()), sorted(wb.sheet(0).unwrap().data_tables.clone())); + assert_eq!(sh.value(at("E3")), Value::Number(105.0)); + assert_eq!(sh.value(at("E4")), Value::Number(210.0)); + assert_eq!(sh.value(at("L2")), Value::text("low")); + assert!(sh.cell(at("E3")).unwrap().formula.is_none()); +} + +#[test] +fn damaged_data_tables_keep_their_values() { + let rows = [ + // A valid table. + r#"1"#, + r#"2"#, + // Overlaps the first one. + r#"3"#, + // Deleted input, input inside the table, a two-variable table with one input, no room for edges. + r#"456"#, + r#"7"#, + // Too many result cells, a `ref` that doesn't start at the cell, no `ref` at all. + r#"8910"#, + ]; + let bytes = minimal(&format!("{}", rows.join("")), &[], "", ""); + let (wb, report) = read_xlsx(&bytes).unwrap(); + let sh = wb.sheet(0).unwrap(); + assert_eq!(sh.data_tables, vec![table("D2:E4", None, Some("B2"))]); + assert!(!report.warnings.is_empty()); + for (a, v) in [("E5", 3.0), ("H10", 4.0), ("K10", 5.0), ("N10", 6.0), ("A1", 7.0), ("B20", 8.0), ("Q20", 9.0), ("T20", 10.0)] { + assert_eq!(sh.value(at(a)), Value::Number(v), "{a}"); + } +} + +/// The form Excel saves data tables in (observed black-box on Excel 365): the `dataTable` formula +/// on the first result cell with its attributes in Excel's order, `dtr="1"` on a two-variable +/// table, `ca="1"` on some, and the other results as plain cached values (text as a shared string). +/// All three kinds read back as live tables over the same ranges. +#[test] +fn data_tables_in_excels_form_are_read() { + let rows = [ + r#"0.050.10.2B1*B2500.10.2"#, + r#"1000B2*(1+B1)1050B2*B150100012"#, + r#"1001051024"#, + r#"20021020"#, + ]; + let sst = r#"two hundred"#; + let bytes = minimal( + &format!("{}", rows.join("")), + &[("xl/sharedStrings.xml", sst)], + r#""#, + "", + ); + let (wb, report) = read_xlsx(&bytes).unwrap(); + // (The minimal package has no styles part, which is all it warns about.) + assert!(!report.warnings.iter().any(|w| w.contains("data table")), "{:?}", report.warnings); + let sh = wb.sheet(0).unwrap(); + let mut got = sh.data_tables.clone(); + got.sort_by_key(|t| (t.range.start.row, t.range.start.col)); + assert_eq!(got, vec![table("G1:I2", Some("B1"), None), table("L1:N3", Some("B1"), Some("B2")), table("D2:E4", None, Some("B2"))]); + // The cached results are kept until the tables are calculated. + assert_eq!((sh.value(at("E4")), sh.value(at("I2")), sh.value(at("N3"))), (Value::Number(210.0), Value::text("two hundred"), Value::Number(4.0))); + assert!(sh.cell(at("M2")).unwrap().formula.is_none()); +} diff --git a/crates/xlsx/src/tests/mod.rs b/crates/xlsx/src/tests/mod.rs index 517113d0f6..f9e1f55204 100644 --- a/crates/xlsx/src/tests/mod.rs +++ b/crates/xlsx/src/tests/mod.rs @@ -3,6 +3,7 @@ mod array_export; mod chart_bool_default; mod csv_tests; +mod data_table_tests; mod encrypted_tests; mod fixtures; mod ink_roundtrip; diff --git a/docs/file-format-parity.md b/docs/file-format-parity.md index 9884724146..282f96ad29 100644 --- a/docs/file-format-parity.md +++ b/docs/file-format-parity.md @@ -1,6 +1,6 @@ # File-format parity -> **Last reviewed:** 2026-10-11 · **Last updated:** 2026-10-12 · **Change:** trivial (XLSX chart sheets are kept on save (#727); previously minor: CSV save keeps the separator it was opened with) · **Target:** Microsoft Excel (Microsoft 365), Excel for Mac 16.113.4 +> **Last reviewed:** 2026-10-11 · **Last updated:** 2026-10-14 · **Change:** minor (XLSX data tables checked against Excel 365 both ways) · **Target:** Microsoft Excel (Microsoft 365), Excel for Mac 16.113.4 Every format Excel opens or saves, with GridCraft's read and write support, fidelity and tests. Part of [target-app-parity.md](target-app-parity.md). **File formats: ~50% ready, 80–120 h** @@ -22,7 +22,7 @@ and repair-free opening remain unverified. | Format | Excel | GridCraft read | GridCraft write | Fidelity / notes | Tests | |---|---|---|---|---|---| -| `.xlsx` Excel Workbook (transitional) | R/W, native | yes | yes | Cells, shared/array/dynamic formulas, styles, themes, merges, CF, validation, tables, comments and notes, hyperlinks, names, freeze panes, outline, print settings, pictures, shapes, charts (simplified, see [chart-parity.md](chart-parity.md)), sparklines, PivotTables (caches and tables), protection, calc settings. **Kept unchanged on save (pass-through, `passthrough.rs`):** custom document properties, ribbon customisation and add-in task panes, custom XML, links to external workbooks (never followed or refreshed), data connections, sheet query tables, printer settings (#324), form controls, ActiveX controls, OLE objects and sheet backgrounds (with their VML shapes), and chart sheets (#727: the sheet, drawing, chart, style and colour parts are written back unchanged in their place, and left out when a sheet they plot was deleted or renamed or a worksheet took their name; they can't be edited or created); dropped when their sheet is deleted or their parts point at content GridCraft rewrites. **Not preserved:** slicers and slicer caches, timelines, data model, query tables attached to tables, sheet and workbook `extLst` entries we don't model, chart details; Strict Open XML files keep nothing. Never checked in Excel across a corpus: repair prompts unknown. 100 MB files fail (#175) | `roundtrip.rs`, `fixtures.rs`, `malformed.rs`, `pivot_tests.rs` | +| `.xlsx` Excel Workbook (transitional) | R/W, native | yes | yes | Cells, shared/array/dynamic formulas, styles, themes, merges, CF, validation, tables, comments and notes, hyperlinks, names, freeze panes, outline, print settings, pictures, shapes, charts (simplified, see [chart-parity.md](chart-parity.md)), sparklines, PivotTables (caches and tables), What-If data tables (`` with cached results, live again after opening; GridCraft's files open as live tables in Excel 365 and Excel's own form, with `dtr="1"` on two-variable tables and `ca`, is read), protection, calc settings. **Kept unchanged on save (pass-through, `passthrough.rs`):** custom document properties, ribbon customisation and add-in task panes, custom XML, links to external workbooks (never followed or refreshed), data connections, sheet query tables, printer settings (#324), form controls, ActiveX controls, OLE objects and sheet backgrounds (with their VML shapes), and chart sheets (#727: the sheet, drawing, chart, style and colour parts are written back unchanged in their place, and left out when a sheet they plot was deleted or renamed or a worksheet took their name; they can't be edited or created); dropped when their sheet is deleted or their parts point at content GridCraft rewrites. **Not preserved:** slicers and slicer caches, timelines, data model, query tables attached to tables, sheet and workbook `extLst` entries we don't model, chart details; Strict Open XML files keep nothing. Never checked in Excel across a corpus: repair prompts unknown. 100 MB files fail (#175) | `roundtrip.rs`, `fixtures.rs`, `malformed.rs`, `pivot_tests.rs` | | `.xlsx` Excel Workbook (transitional) | R/W, native | yes | yes | Cells, shared/array/dynamic formulas, styles, themes, merges, CF, validation, tables, notes (with their shown/hidden state) and threaded comments with replies, resolved state and timestamps (`threadedComments` and `persons` parts, #300), hyperlinks, names, freeze panes, outline, print settings, pictures, shapes, charts (simplified, see [chart-parity.md](chart-parity.md)), sparklines, PivotTables (caches and tables), protection, calc settings. **Not preserved:** unmodelled parts and extensions (form controls, slicer caches, timelines, external links, OLE objects, custom XML, data model, query tables, chart details). Never checked in Excel across a corpus: repair prompts unknown. 100 MB files fail (#175) | `roundtrip.rs`, `fixtures.rs`, `malformed.rs`, `pivot_tests.rs` | | `.xlsx` Excel Workbook (transitional) | R/W, native | yes | yes | Cells, shared/array/dynamic formulas, styles, themes, merges, CF, validation, tables, comments and notes, hyperlinks, names, freeze panes, outline, print settings, pictures, shapes, charts (simplified, see [chart-parity.md](chart-parity.md)), sparklines, PivotTables (caches and tables), protection, Allow Edit Ranges (`protectedRanges` with their password hashes, #307), calc settings. **Not preserved:** unmodelled parts and extensions (form controls, slicer caches, timelines, external links, OLE objects, custom XML, data model, query tables, chart details). Never checked in Excel across a corpus: repair prompts unknown. 100 MB files fail (#175) | `roundtrip.rs`, `fixtures.rs`, `malformed.rs`, `pivot_tests.rs` | | `.xlsx` Strict Open XML | R/W | yes | no (writes transitional) | Excel also saves transitional by default | fixtures | @@ -85,6 +85,8 @@ refresh. Counted under features (Get & Transform, ~10% ready), not here. | Date | Change | Summary | |---|---|---| +| 2026-10-14 | minor | XLSX data tables: two-variable tables are written with `dtr="1"` as Excel writes them; files written by GridCraft open in Excel 365 with live tables (one- and two-variable, text results), and Excel's form (`ca`, `dtr` on two-variable tables, shared-string results) is read; observed black-box | +| 2026-10-12 | minor | XLSX: What-If data tables are read and written (`` on the first result cell, cached results in every result cell); they were dropped to their values with a warning | | 2026-10-12 | trivial | XLSX chart sheets are kept on save (#727) | | 2026-10-12 | trivial | CSV and TSV export is capped at 512 MiB and refuses a larger used range with an error (#528) | | 2026-10-12 | minor | `.xls` import reads the calculation settings (CalcMode, CalcIter, CalcCount, CalcDelta, CalcPrecision), so circular models calculate instead of showing `#CIRC!` (#724) | diff --git a/docs/target-app-parity.md b/docs/target-app-parity.md index b623124d9f..6b483f0f0f 100644 --- a/docs/target-app-parity.md +++ b/docs/target-app-parity.md @@ -1,6 +1,6 @@ # Target-app parity: GridCraft vs Microsoft Excel -> **Last reviewed:** 2026-10-11 · **Last updated:** 2026-10-12 · **Change:** minor (rows auto-fit oversized fonts on open and row formatting) · **Target:** Microsoft Excel (Microsoft 365) +> **Last reviewed:** 2026-10-11 · **Last updated:** 2026-10-14 · **Change:** minor (What-If data tables: dialog, recalculation, Excel round trip) · **Target:** Microsoft Excel (Microsoft 365) This is the authoritative parity assessment. [`ROADMAP.md`](../ROADMAP.md) summarizes it, [`gaps.md`](gaps.md) lists every shortfall, and the area checklists hold the detail: @@ -87,7 +87,7 @@ performance 8, stability 8, localization 4, hardware 2, platforms 2, ecosystem 2 | PivotTables | 7 | ~65% | ~45% | 25–40 | Insert/Recommended, field list, three layouts, values on rows or columns, date grouping, show-values-as, refresh, calculated fields, XLSX round-trip. Missing: slicers, timelines, calculated items, PivotCharts, value/label filters, number grouping, GETPIVOTDATA, data-model pivots. Round-trip never checked in Excel | | Charts and sparklines | 8 | ~50% | ~30% | 40–60 | 25 chart kinds, all 2-D. The chart model holds title, legend position, a data-labels flag, gridlines flag and axis titles; no axis scaling, trendlines, error bars, per-point formatting, chart sheets, 3-D, surface, map, Pareto. Several kinds don't survive XLSX round trip (open PRs #158, #161). [chart-parity.md](chart-parity.md) | | Page layout, print, PDF | 5 | ~80% | ~55% | 12–20 | Page setup, print titles, breaks, Page Break Preview, headers/footers, PDF export and printing. No true Page Layout view; the PDF writer uses only the 14 standard fonts without embedding (no CJK or other non-Latin text in PDF); web print only exports (#141, #143) | -| What-if and analysis | 4 | ~50% | ~40% | 22–35 | Goal Seek, Scenario Manager, data tables, Analysis ToolPak (13 of 19 tools, no charts), Solver (Simplex LP, a local nonlinear method and Evolutionary; Answer and Limits reports; no Sensitivity report or options dialog). Missing: Forecast Sheet, Analyze Data | +| What-if and analysis | 4 | ~50% | ~40% | 22–35 | Goal Seek, Scenario Manager, live data tables (Data Table dialog, `{=TABLE()}` recalculated with the workbook, XLSX round trip checked in Excel 365), Analysis ToolPak (13 of 19 tools, no charts), Solver (Simplex LP, a local nonlinear method and Evolutionary; Answer and Limits reports; no Sensitivity report or options dialog). Missing: Forecast Sheet, Analyze Data | | Get & Transform, external data, data model | 5 | ~15% | ~10% | 60–100 | Import of CSV/TSV, JSON, HTML. No Power Query editor, no From Web, database or folder sources, no connections or refresh, no relationships or data model, no external workbook links (dropped with a warning) | | Insert objects | 3 | ~55% | ~45% | 25–40 | Pictures, 8 shape kinds (Excel has ~170), text boxes, own icon set, ink with Ink to Shape, hyperlinks, checkboxes. Missing: SmartArt, WordArt, equations, signature line, objects, 3-D models, screenshot, pictures in cells (PR #105) | | Review, comments, protection, collaboration | 4 | ~55% | ~40% | 35–55 | Threaded comments, notes, spelling (system dictionary), thesaurus, sheet and workbook protection, Allow Edit Ranges with Excel-compatible password hashes (#307). Missing: comment navigation, Show Changes, notes-to-comments conversion, accessibility checker, co-authoring, opening password-encrypted files (refused with a message since #17) | @@ -303,7 +303,7 @@ The inventory carried over from the 2026-10-07 ROADMAP.md, updated for what land - PivotTables: Insert/Recommended PivotTables, field list pane, compact/outline/tabular layouts, values on rows or columns, date grouping, sort, filters, show-values-as, refresh, and XLSX round-trip. - Page layout & print: Page Break Preview, print settings, PDF export and printing. -- What-if: Goal Seek, scenarios, data tables. Spelling (system word list) and thesaurus. +- What-if: Goal Seek, scenarios, live data tables (`{=TABLE()}`). Spelling (system word list) and thesaurus. - Draw: pen, highlighter, eraser, ink strokes and Ink to Shape. Task panes: Comments (threaded replies, resolve), Watch Window, Selection, Format Chart. - Charts: column, bar, line, area, pie, doughnut, scatter, bubble, radar, histogram, waterfall, @@ -332,6 +332,8 @@ The inventory carried over from the 2026-10-07 ROADMAP.md, updated for what land | Date | Change | Summary | |---|---|---| +| 2026-10-14 | minor | What-if: Data Table dialog in Excel's What-If Analysis menu order, data tables recalculate faster (one working copy per table, edited input values only their own results, a work budget) and their XLSX form is checked against Excel 365 both ways (no score change) | +| 2026-10-12 | minor | What-if: data tables are live `{=TABLE()}` formulas that recalculate with their inputs and formulas and round-trip through XLSX (they were static snapshots; no score change) | | 2026-10-12 | minor | Essentials AutoFit evidence: rows auto-fit oversized fonts on open and row formatting (#833) | | 2026-10-12 | minor | Sheet tabs: reveal the selected tab on navigation and resize while preserving manual scrolling (#681); no score change | | 2026-10-12 | minor | PivotTables: values on rows (Σ Values in Rows at any position, all three layouts; `pivot.moveField` and the field pane move it; `dataOnRows` round-trips) (no score change) | diff --git a/docs/ui-parity.md b/docs/ui-parity.md index 22e221e698..e2a6a2a1f4 100644 --- a/docs/ui-parity.md +++ b/docs/ui-parity.md @@ -1,6 +1,6 @@ # UI parity -> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-13 · **Change:** minor (horizontal scrollbar in the sheet tab bar, #840) · **Target:** Microsoft Excel (Microsoft 365), Excel for Mac 16.113.4 +> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-14 · **Change:** minor (Data Table dialog) · **Target:** Microsoft Excel (Microsoft 365), Excel for Mac 16.113.4 Interaction, controls and feel, against Excel. Part of [target-app-parity.md](target-app-parity.md). **UI/UX fidelity: ~55% ready, 50–80 h** (estimated). Ribbon command coverage is measured @@ -135,7 +135,7 @@ text, so AltGr symbols (e.g. `~` on a German layout) never trigger a shortcut. Format Cells, Insert Function/Formula Builder, Find and Replace, Go To and Go To Special, Sort, Name Manager, Data Validation, Paste Special, Conditional Formatting manager, Page Setup, Goal -Seek, Scenario Manager, Data Analysis, Forecast Sheet and others exist. Missing or partial: Options, +Seek, Scenario Manager, Data Table, Data Analysis, Forecast Sheet and others exist. Missing or partial: Options, Solver, Power Query editor, Insert Object, Equation editor, Protect/Allow Edit Ranges, Workbook Links, Accessibility checker. @@ -161,6 +161,7 @@ scrollbar bugs 2–4 · pixel/feel pass against observed Excel 7–12. | Date | Change | Summary | |---|---|---| +| 2026-10-14 | minor | Data › What-If Analysis › Data Table… dialog (Row input cell, Column input cell, German labels); the What-If menu in Excel's order; the formula bar, cell editor and Show Formulas show a data table's `=TABLE()` formula in the formula language | | 2026-10-12 | minor | The File view's recent list tells when each workbook was last changed, read off the UI thread | | 2026-10-13 | minor | Horizontal scrollbar in the sheet tab bar: drag, track click pages, frozen columns excluded from the page (#840) | | 2026-10-12 | minor | Ctrl+wheel zoom steps along a ladder of percentages about 20% apart that never skips 100%, gliding to each step with animations on, so a notch out undoes a notch in (it drifted: 100 → 122 → 101); Excel steps 15 points, see decisions.md |