Optimize SQL generation

Author: Schultzer

CHANGELOG.md

@@ -15,6 +15,7 @@
  - MySQL adapter [#5](https://github.com/elixir-dbvisor/sql/pull/5).
  - PostgreSQL adapter [#5](https://github.com/elixir-dbvisor/sql/pull/5).
  - TDS adapter [#5](https://github.com/elixir-dbvisor/sql/pull/5).
+ - Improve SQL generation with 4-600x compared to Ecto [#7](https://github.com/elixir-dbvisor/sql/pull/7).
 
 ### Deprecation
  - token_to_sql/2 is deprecated in favor of SQL.Token behaviour token_to_string/2 [#5](https://github.com/elixir-dbvisor/sql/pull/5).

bench.exs

@@ -1,10 +1,18 @@
 import SQL
+import Ecto.Query
+defmodule SQL.Repo do
+  use Ecto.Repo, otp_app: :sql, adapter: Ecto.Adapters.Postgres
+end
+Application.put_env(:sql, :ecto_repos, [SQL.Repo])
+Application.put_env(:sql, SQL.Repo, username: "postgres", password: "postgres", hostname: "localhost", database: "sql_test#{System.get_env("MIX_TEST_PARTITION")}", pool: Ecto.Adapters.SQL.Sandbox, pool_size: 10)
+SQL.Repo.start_link()
 range = 1..10_000
 Benchee.run(
   %{
     "to_stirng" => fn -> for _ <- range, do: to_string(~SQL[with recursive temp (n, fact) as (select 0, 1 union all select n+1, (n+1)*fact from temp where n < 9)]) end,
     "to_sql" => fn -> for _ <- range, do: SQL.to_sql(~SQL[with recursive temp (n, fact) as (select 0, 1 union all select n+1, (n+1)*fact from temp where n < 9)]) end,
-    "inspect" => fn -> for _ <- range, do: inspect(~SQL[with recursive temp (n, fact) as (select 0, 1 union all select n+1, (n+1)*fact from temp where n < 9)]) end
+    "inspect" => fn -> for _ <- range, do: inspect(~SQL[with recursive temp (n, fact) as (select 0, 1 union all select n+1, (n+1)*fact from temp where n < 9)]) end,
+    "ecto" => fn -> for _ <- range, do: SQL.Repo.to_sql(:all, "temp" |> recursive_ctes(true) |> with_cte("temp", as: ^union_all(select("temp", [t], %{n: 0, fact: 1}), ^where(select("temp", [t], [t.n+1, t.n+1*t.fact]), [t], t.n < 9))) |> select([t], [t.n])) end
   },
   time: 10,
   memory_time: 2

benchmarks/v0.2.0.md

@@ -0,0 +1,97 @@
+➜  sql git:(optimize-sql-generation) ✗ mix sql.bench
+Compiling 1 file (.ex)
+Generated sql app
+Operating System: macOS
+CPU Information: Apple M1 Max
+Number of Available Cores: 10
+Available memory: 64 GB
+Elixir 1.18.0
+Erlang 27.2
+JIT enabled: true
+
+Benchmark suite executing with the following configuration:
+warmup: 2 s
+time: 10 s
+memory time: 2 s
+reduction time: 0 ns
+parallel: 1
+inputs: none specified
+Estimated total run time: 56 s
+
+Benchmarking Ecto.Repo.to_sql ...
+Benchmarking inspect ...
+Benchmarking to_sql ...
+Benchmarking to_stirng ...
+Calculating statistics...
+Formatting results...
+
+Name                       ips        average  deviation         median         99th %
+to_sql                  4.74 K        0.21 ms    ±20.20%        0.20 ms        0.32 ms
+to_stirng               4.51 K        0.22 ms     ±3.60%        0.22 ms        0.26 ms
+inspect                 0.24 K        4.09 ms     ±2.96%        4.09 ms        4.36 ms
+Ecto.Repo.to_sql     0.00757 K      132.03 ms     ±1.45%      131.80 ms      139.43 ms
+
+Comparison:
+to_sql                  4.74 K
+to_stirng               4.51 K - 1.05x slower +0.0108 ms
+inspect                 0.24 K - 19.39x slower +3.88 ms
+Ecto.Repo.to_sql     0.00757 K - 626.20x slower +131.82 ms
+
+Memory usage statistics:
+
+Name                Memory usage
+to_sql                   0.38 MB
+to_stirng               0.153 MB - 0.40x memory usage -0.22888 MB
+inspect                  4.88 MB - 12.80x memory usage +4.50 MB
+Ecto.Repo.to_sql       179.35 MB - 470.13x memory usage +178.97 MB
+
+**All measurements for memory usage were the same**
+
+➜  sql git:(optimize-sql-generation) ✗ mix sql.bench
+Compiling 1 file (.ex)
+Generated sql app
+Operating System: macOS
+CPU Information: Apple M1 Max
+Number of Available Cores: 10
+Available memory: 64 GB
+Elixir 1.18.0
+Erlang 27.2
+JIT enabled: true
+
+Benchmark suite executing with the following configuration:
+warmup: 2 s
+time: 10 s
+memory time: 2 s
+reduction time: 0 ns
+parallel: 1
+inputs: none specified
+Estimated total run time: 56 s
+
+Benchmarking ecto ...
+Benchmarking inspect ...
+Benchmarking to_sql ...
+Benchmarking to_stirng ...
+Calculating statistics...
+Formatting results...
+
+Name                ips        average  deviation         median         99th %
+to_stirng         27.59       36.24 ms     ±0.75%       36.22 ms       37.21 ms
+to_sql            27.45       36.42 ms     ±0.47%       36.39 ms       36.93 ms
+inspect           24.53       40.76 ms     ±1.07%       40.75 ms       41.60 ms
+ecto               6.96      143.65 ms     ±1.37%      143.36 ms      148.62 ms
+
+Comparison:
+to_stirng         27.59
+to_sql            27.45 - 1.00x slower +0.181 ms
+inspect           24.53 - 1.12x slower +4.52 ms
+ecto               6.96 - 3.96x slower +107.41 ms
+
+Memory usage statistics:
+
+Name         Memory usage
+to_stirng         4.50 MB
+to_sql            4.73 MB - 1.05x memory usage +0.23 MB
+inspect           9.23 MB - 2.05x memory usage +4.73 MB
+ecto            202.87 MB - 45.11x memory usage +198.37 MB
+
+**All measurements for memory usage were the same**