Skip to content

Return the database value from Avg on integer fields - #2314

Open
Shaisolaris wants to merge 3 commits into
tortoise:developfrom
Shaisolaris:fix-avg-integer-field
Open

Shaisolaris wants to merge 3 commits into
tortoise:developfrom
Shaisolaris:fix-avg-integer-field

Conversation

@Shaisolaris

@Shaisolaris Shaisolaris commented Oct 12, 2026 •

Copy link
Copy Markdown

Avg keeps the type of the field it averages, so the average of an IntField, BigIntField or SmallIntField column goes through IntField.to_python_value and loses its fraction:

for score in (1, 2, 2):
    await Rating.create(score=score)   # score = fields.IntField()

await Rating.annotate(avg=Avg("score")).values("avg")
# before: [{'avg': 1}]
# after:  [{'avg': 1.6666666666666667}]   (SQLite; a Decimal on PostgreSQL)

The same happened for values_list(), group_by(...).values() and Avg(F("score") + 1).

Change

Avg.resolve now keeps the field type only for DecimalField and FloatField. For any other field the database value is returned as is, so nothing is truncated and nothing is converted through a lossy cast (a BigIntField average on PostgreSQL stays an exact Decimal).

Tests

  • New test_avg_of_integer_field_is_not_truncated in tests/test_aggregation.py (values, values_list, group_by, and an F expression). It fails on develop (assert 1.0 == 1.667) and passes with the change.
  • Result types checked before and after on SQLite and on PostgreSQL 16 (asyncpg), for integer, bigint, smallint, float and decimal columns, with NULLs, an empty table, and F expressions. Only the integer-family averages change. Avg on FloatField and DecimalField, Sum, Max, Min and Count return identical values and types.
  • tests/test_aggregation.py, tests/test_group_by.py, tests/fields/test_decimal.py and tests/test_f.py: 67 passed, 9 skipped (SQLite).
  • Whole tests directory on SQLite: 1 failed, 1847 passed, 145 skipped, 4 collection errors. The same failure (test_psycopg_connection_params) and the same 4 collection errors occur on unmodified develop in the same environment (missing optional dependencies and a missing time zone database), so they are not caused by this change. Only SQLite was used for the test suite; MySQL, Oracle and MSSQL were not run.

Arithmetic with other fields

Avg.resolve clears field_object, but the ResolveResult it returns still carries the integer field as output_field, and CombinedExpression compares operand field types. I compared a grouped query on SQLite with and without this change, using a model with integer, float and decimal columns:

  • F("f") + Avg("i"), Avg("i") + F("f") and F("d") + Avg("i") raise FieldError: Cannot use arithmetic expression between different field type, identically on develop and with this change. It is the same error F("i") + F("f") gives, so it comes from the existing rule on mixed field types and not from this change.
  • F("i") + Avg("i"), Avg("i") + F("i"), F("f") + Avg("f") and F("d") + Avg("d") give identical results before and after.

So the stale output_field causes no new error and no result differs, and I left it unchanged. This was checked on SQLite only; arithmetic between an integer column and an integer average was not run on PostgreSQL.

Follow-up validation — October 16

Follow-up validation (October 16): rebased onto current upstream, narrowed the conversion exception to IntField and its BigInt/SmallInt/IntEnum subclasses, and parameterized the regressions as requested. The assertion allows MySQL AVG(integer) rounding to four decimal places while rejecting integer truncation. Aggregation suite on SQLite: 21 passed, 1 skipped. Hosted database jobs still require verification.

Comment thread tortoise/functions.py Resolved
Comment thread tests/test_aggregation.py Resolved
@codspeed

codspeed Bot commented Oct 13, 2026 •

Copy link
Copy Markdown

Merging this PR will not alter performance

⚠️ 21 benchmarks spent significant time in system calls

System calls cannot be consistently instrumented, so they are not included in the measure, which understates the real cost. Please switch to the Walltime instrument to accurately measure system calls.

Measurement and system calls

✅ 24 untouched benchmarks


Comparing Shaisolaris:fix-avg-integer-field (12f724b) with develop (ef637d4)

Open in CodSpeed

Shaisolaris and others added 3 commits October 16, 2026 06:02
Avg kept the type of the averaged field, so the average of an IntField
column was truncated by IntField.to_python_value. Only decimal and float
fields keep their own type now; for other fields the database value is
returned as is.
@Shaisolaris
Shaisolaris force-pushed the fix-avg-integer-field branch from 6ea577c to 12f724b Compare October 16, 2026 13:07
@Shaisolaris
Shaisolaris force-pushed the fix-avg-integer-field branch from 12f724b to b5e18c1 Compare October 31, 2026 12:41

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants