Skip to content

Improve Order Book Analytics page #208

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
Jul 8, 2025
Merged

Improve Order Book Analytics page #208

Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
7cc86b3
Improve Order Book Analytics page
mtopolnik Jul 7, 2025
1de0ba8
Remove duplicate query
mtopolnik Jul 7, 2025
dfe8585
Add missing @
mtopolnik Jul 7, 2025
805903d
Remove redundant alias
mtopolnik Jul 7, 2025
223262f
Remove extra spaces
mtopolnik Jul 7, 2025
cb914f0
Add alias
mtopolnik Jul 8, 2025
a0a043b
Extract parameter to DECLARE
mtopolnik Jul 8, 2025
71f47b5
Merge branch 'main' into mt_improve-order-book-guide
goodroot Jul 8, 2025
de467c4
Merge branch 'main' into mt_improve-order-book-guide
goodroot Jul 8, 2025
73e2e52
make query demoable
goodroot Jul 8, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
215 changes: 116 additions & 99 deletions documentation/guides/order-book.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,52 +1,42 @@
# Order book analytics using arrays

In the following examples, we'll use the table schema below. It is a bare-bones
simplification of a realistic table, where we omit the otherwise essential
columns such as the symbol of the financial instrument. The goal is to
demonstrate the essential aspects of the analytical queries.

The order book is stored in a 2D array with two rows: the top row are the
prices, and the bottom row are the volumes at each price point.
In the following examples, we'll use the table schema below. The order book is
stored in a 2D array with two rows: the top row are the prices, and the bottom
row are the volumes at each price point.

```questdb-sql
CREATE TABLE order_book (
ts TIMESTAMP,
asks DOUBLE[][],
bids DOUBLE[][]
) TIMESTAMP(ts) PARTITION BY HOUR;
CREATE TABLE market_data (
timestamp TIMESTAMP,
symbol SYMBOL,
bids DOUBLE[][],
asks DOUBLE[][]
) TIMESTAMP(timestamp) PARTITION BY HOUR;
```

## Basic order book analytics

### What is the bid-ask spread at any moment?

```questdb-sql
SELECT ts, spread(bids[1][1], asks[1][1]) FROM order_book;
SELECT timestamp, spread(bids[1][1], asks[1][1]) spread
FROM market_data WHERE symbol='EURUSD';
```

#### Sample data and result

```questdb-sql
INSERT INTO order_book VALUES
('2025-07-01T12:00:00Z', ARRAY[ [10.1, 10.2], [0, 0] ], ARRAY[ [9.3, 9.2], [0, 0] ]),
('2025-07-01T12:00:01Z', ARRAY[ [10.3, 10.5], [0, 0] ], ARRAY[ [9.7, 9.4], [0, 0] ]);
INSERT INTO market_data VALUES
('2025-07-01T12:00:00Z', 'EURUSD', ARRAY[ [9.3, 9.2], [0, 0] ], ARRAY[ [10.1, 10.2], [0, 0] ]),
('2025-07-01T12:00:01Z', 'EURUSD', ARRAY[ [9.7, 9.4], [0, 0] ], ARRAY[ [10.3, 10.5], [0, 0] ]);
```

| ts | spread |
| timestamp | spread |
| ------------------- | ------ |
| 2025-07-01T12:00:00 | 0.8 |
| 2025-07-01T12:00:01 | 0.6 |

### How much volume is available within 1% of the best price?

```questdb-sql
SELECT ts, array_sum(
asks[2, 1:insertion_point(asks[1], 1.01 * asks[1, 1])]
) volume FROM order_book;
```

In a dense query like this, you can use `DECLARE` for better legibility:

```questdb-sql
DECLARE
@prices := asks[1],
Expand All @@ -55,21 +45,19 @@ DECLARE
@multiplier := 1.01,
@target_price := @multiplier * @best_price,
@relevant_volume_levels := @volumes[1:insertion_point(@prices, @target_price)]
SELECT asks,
ts,
array_sum(@relevant_volume_levels) total_volume
FROM order_book;
SELECT timestamp, array_sum(@relevant_volume_levels) total_volume
FROM market_data WHERE symbol='EURUSD';
```

#### Sample data and result

```questdb-sql
INSERT INTO order_book VALUES
('2025-07-01T12:00:00Z', ARRAY[ [10.00, 10.02, 10.04, 10.10, 10.12, 10.14], [10.0, 15, 13, 12, 18, 20] ], NULL),
('2025-07-01T12:00:01Z', ARRAY[ [20.00, 20.02, 20.04, 20.10, 20.12, 20.14], [1.0, 5, 3, 2, 8, 10] ], NULL);
INSERT INTO market_data VALUES
('2025-07-01T12:00:00Z', 'EURUSD', NULL, ARRAY[ [10.00, 10.02, 10.04, 10.10, 10.12, 10.14], [10.0, 15, 13, 12, 18, 20] ]),
('2025-07-01T12:00:01Z', 'EURUSD', NULL, ARRAY[ [20.00, 20.02, 20.04, 20.10, 20.12, 20.14], [1.0, 5, 3, 2, 8, 10] ]);
```

| ts | volume |
| timestamp | volume |
| ------------------- | ------ |
| 2025-07-01T12:00:00 | 50.0 |
| 2025-07-01T12:00:01 | 29.0 |
Expand All @@ -82,44 +70,54 @@ Find the order book level at which the price passes a threshold, and then sum
the sizes up to that level.

```questdb-sql
SELECT ts, array_sum(
asks[2, 1:insertion_point(asks[1], asks[1,1] + 0.1)]) volume
FROM order_book;
DECLARE
@prices := asks[1],
@volumes := asks[2],
@best_price := @prices[1],
@price_delta := 0.1,
@target_price := @best_price + @price_delta,
@relevant_volumes := @volumes[1:insertion_point(@prices, @target_price)]
SELECT timestamp, array_sum(@relevant_volumes) volume
FROM market_data WHERE symbol='EURUSD';
```

#### Sample data and result

```questdb-sql
INSERT INTO order_book VALUES
('2025-07-01T12:00:00Z', ARRAY[ [10.0, 10.02, 10.04, 10.10, 10.12, 10.14], [10.0, 15, 13, 12, 18, 20] ], NULL),
('2025-07-01T12:00:01Z', ARRAY[ [10.0, 10.10, 10.12, 10.14, 10.16, 10.18], [1.0, 5, 3, 2, 8, 10] ], NULL);
INSERT INTO market_data VALUES
('2025-07-01T12:00:00Z', 'EURUSD', NULL, ARRAY[ [10.0, 10.02, 10.04, 10.10, 10.12, 10.14], [10.0, 15, 13, 12, 18, 20] ]),
('2025-07-01T12:00:01Z', 'EURUSD', NULL, ARRAY[ [10.0, 10.10, 10.12, 10.14, 10.16, 10.18], [1.0, 5, 3, 2, 8, 10] ]);
```

| ts | volume |
| timestamp | volume |
| ------------------- | ------ |
| 2025-07-01T12:00:00 | 50.0 |
| 2025-07-01T12:00:01 | 6.0 |

### What price level will a buy order for the given volume reach?

```questdb-sql
DECLARE
@prices := asks[1],
@volumes := asks[2],
@target_volume := 30.0
SELECT
ts,
array_cum_sum(asks[2]) cum_volumes,
insertion_point(cum_volumes, 30.0, true) target_level,
asks[1, target_level] price
FROM order_book;
timestamp,
array_cum_sum(@volumes) cum_volumes,
insertion_point(cum_volumes, @target_volume, true) target_level,
@prices[target_level] price
FROM market_data WHERE symbol='EURUSD';
```

#### Sample data and result

```questdb-sql
INSERT INTO order_book VALUES
('2025-07-01T12:00:00Z', ARRAY[ [10.0, 10.02, 10.04, 10.10, 10.12, 10.14], [10.0, 15, 13, 12, 18, 20] ], NULL),
('2025-07-01T12:00:01Z', ARRAY[ [10.0, 10.02, 10.04, 10.10, 10.12, 10.14], [10.0, 5, 3, 12, 18, 20] ], NULL);
INSERT INTO market_data VALUES
('2025-07-01T12:00:00Z', 'EURUSD', NULL, ARRAY[ [10.0, 10.02, 10.04, 10.10, 10.12, 10.14], [10.0, 15, 13, 12, 18, 20] ]),
('2025-07-01T12:00:01Z', 'EURUSD', NULL, ARRAY[ [10.0, 10.02, 10.04, 10.10, 10.12, 10.14], [10.0, 5, 3, 12, 18, 20] ]);
```

| ts | cum_volumes | target_level | price |
| timestamp | cum_volumes | target_level | price |
| ------------------- | ----------------------------- | ------------ | ----- |
| 2025-07-01T12:00:00 | [10.0, 25.0, 38.0, 50.0, ...] | 3 | 10.04 |
| 2025-07-01T12:00:01 | [10.0, 15.0, 18.0, 30.0, ...] | 4 | 10.10 |
Expand All @@ -136,70 +134,77 @@ sellers at the top of the book).

```questdb-sql
SELECT
ts, bids[2, 1] / asks[2, 1] imbalance
FROM order_book;
timestamp, bids[2, 1] / asks[2, 1] imbalance
FROM market_data WHERE symbol='EURUSD';
```

#### Sample data and result

```questdb-sql
INSERT INTO order_book VALUES
('2025-07-01T12:00:00Z', ARRAY[ [0.0,0], [10.0, 15] ], ARRAY[ [0.0,0], [20.0, 25] ]),
('2025-07-01T12:00:01Z', ARRAY[ [0.0,0], [15.0, 2] ], ARRAY[ [0.0,0], [14.0, 45] ]);
INSERT INTO market_data VALUES
('2025-07-01T12:00:00Z', 'EURUSD', ARRAY[ [0.0,0], [20.0, 25] ], ARRAY[ [0.0,0], [10.0, 15] ]),
('2025-07-01T12:00:01Z', 'EURUSD', ARRAY[ [0.0,0], [14.0, 45] ], ARRAY[ [0.0,0], [15.0, 2] ]);
```

| ts | imbalance |
| timestamp | imbalance |
| ------------------- | --------- |
| 2025-07-01T12:00:00 | 2.0 |
| 2025-07-01T12:00:01 | 0.93 |

### Cumulative imbalance (Top 3 Levels)

```questdb-sql
DECLARE
@bid_volumes := bids[2],
@ask_volumes := asks[2]
SELECT
array_sum(asks[2, 1:4]) ask_vol,
array_sum(bids[2, 1:4]) bid_vol,
timestamp,
array_sum(@bid_volumes[1:4]) bid_vol,
array_sum(@ask_volumes[1:4]) ask_vol,
bid_vol / ask_vol ratio
FROM order_book;
FROM market_data WHERE symbol='EURUSD';
```

#### Sample data and result

```questdb-sql
INSERT INTO order_book VALUES
('2025-07-01T12:00:00Z', ARRAY[ [0.0,0,0,0], [10.0, 15, 13, 12] ], ARRAY[ [0.0,0,0,0], [20.0, 25, 23, 22] ]),
('2025-07-01T12:00:01Z', ARRAY[ [0.0,0,0,0], [15.0, 2, 20, 23] ], ARRAY[ [0.0,0,0,0], [14.0, 45, 22, 5] ]);
INSERT INTO market_data VALUES
('2025-07-01T12:00:00Z', 'EURUSD', ARRAY[ [0.0,0,0,0], [20.0, 25, 23, 22] ], ARRAY[ [0.0,0,0,0], [10.0, 15, 13, 12] ]),
('2025-07-01T12:00:01Z', 'EURUSD', ARRAY[ [0.0,0,0,0], [14.0, 45, 22, 5] ], ARRAY[ [0.0,0,0,0], [15.0, 2, 20, 23] ]);
```

| ts | ask_vol | bid_vol | ratio |
| timestamp | bid_vol | ask_vol | ratio |
| ------------------- | ------- | ------- | ----- |
| 2025-07-01T12:00:00 | 38.0 | 68.0 | 1.79 |
| 2025-07-01T12:00:01 | 37.0 | 81.0 | 2.19 |
| 2025-07-01T12:00:00 | 68.0 | 38.0 | 1.79 |
| 2025-07-01T12:00:01 | 81.0 | 37.0 | 2.19 |

### Detect quote stuffing/fading (Volume dropoff)

Detect where the order book thins out rapidly after the first two levels. This
signals lack of depth (fading) or fake orders (stuffing).

```questdb-sql
DECLARE
@volumes := asks[2],
@dropoff_ratio := 3.0
SELECT * FROM (
SELECT
ts,
array_avg(asks[2, 1:3]) top,
array_avg(asks[2, 3:6]) deep
FROM order_book)
WHERE top > 3 * deep;
timestamp,
array_avg(@volumes[1:3]) top,
array_avg(@volumes[3:6]) deep
FROM market_data WHERE symbol='EURUSD')
WHERE top > @dropoff_ratio * deep;
```

#### Sample data and result

```questdb-sql
INSERT INTO order_book VALUES
('2025-07-01T12:00:00Z', ARRAY[ [0.0,0,0,0,0,0], [20.0, 15, 13, 12, 18, 20] ], NULL),
('2025-07-01T12:00:01Z', ARRAY[ [0.0,0,0,0,0,0], [20.0, 25, 3, 7, 5, 2] ], NULL);
INSERT INTO market_data VALUES
('2025-07-01T12:00:00Z', 'EURUSD', NULL, ARRAY[ [0.0,0,0,0,0,0], [20.0, 15, 13, 12, 18, 20] ]),
('2025-07-01T12:00:01Z', 'EURUSD', NULL, ARRAY[ [0.0,0,0,0,0,0], [20.0, 25, 3, 7, 5, 2] ]);
```

| ts | top | deep |
| timestamp | top | deep |
| ------------------- | ---- | ---- |
| 2025-07-01T12:00:01 | 22.5 | 5.0 |

Expand All @@ -209,55 +214,67 @@ Look for cases where the top bid/ask volume dropped compared to the prior
snapshot — potential order withdrawal ahead of adverse movement.

```questdb-sql
DECLARE
@top_bid_volume := bids[2, 1],
@top_ask_volume := asks[2, 1],
@drop_ratio := 1.5
SELECT * FROM (
SELECT
ts ts,
lag(asks[2, 1]) OVER () prev_ask_vol,
asks[2, 1] curr_ask_vol,
lag(bids[2, 1]) OVER () prev_bid_vol,
bids[2, 1] curr_bid_vol
FROM order_book)
WHERE prev_bid_vol > curr_bid_vol * 1.5 OR prev_ask_vol > curr_ask_vol * 1.5;
timestamp,
lag(@top_bid_volume) OVER () prev_bid_vol,
@top_bid_volume curr_bid_vol,
lag(@top_ask_volume) OVER () prev_ask_vol,
@top_ask_volume curr_ask_vol
FROM market_data WHERE symbol='EURUSD')
WHERE prev_bid_vol > curr_bid_vol * @drop_ratio OR prev_ask_vol > curr_ask_vol * @drop_ratio;
```

#### Sample data and result

```questdb-sql
INSERT INTO order_book VALUES
('2025-07-01T12:00:00Z', ARRAY[ [0.0], [10.0] ], ARRAY[ [0.0], [10.0] ]),
('2025-07-01T12:00:01Z', ARRAY[ [0.0], [ 9.0] ], ARRAY[ [0.0], [ 9.0] ]),
('2025-07-01T12:00:02Z', ARRAY[ [0.0], [ 4.0] ], ARRAY[ [0.0], [ 8.0] ]),
('2025-07-01T12:00:03Z', ARRAY[ [0.0], [ 4.0] ], ARRAY[ [0.0], [ 4.0] ]);
INSERT INTO market_data VALUES
('2025-07-01T12:00:00Z', 'EURUSD', ARRAY[ [0.0], [10.0] ], ARRAY[ [0.0], [10.0] ]),
('2025-07-01T12:00:01Z', 'EURUSD', ARRAY[ [0.0], [ 9.0] ], ARRAY[ [0.0], [ 9.0] ]),
('2025-07-01T12:00:02Z', 'EURUSD', ARRAY[ [0.0], [ 8.0] ], ARRAY[ [0.0], [ 4.0] ]),
('2025-07-01T12:00:03Z', 'EURUSD', ARRAY[ [0.0], [ 4.0] ], ARRAY[ [0.0], [ 4.0] ]);
```

| ts | prev_ask_vol | curr_ask_vol | prev_bid_vol | curr_bid_vol |
| timestamp | prev_bid_vol | curr_bid_vol | prev_ask_vol | curr_ask_vol |
| ------------------- | ------------ | ------------ | ------------ | ------------ |
| 2025-07-01T12:00:02 | 9.0 | 4.0 | 9.0 | 8.0 |
| 2025-07-01T12:00:03 | 4.0 | 4.0 | 8.0 | 4.0 |
| 2025-07-01T12:00:02 | 9.0 | 8.0 | 9.0 | 4.0 |
| 2025-07-01T12:00:03 | 8.0 | 4.0 | 4.0 | 4.0 |

### Price-weighted volume imbalance

For each level, calculate the deviation from the mid price (midpoint between
best bid and best ask), and weight it by the volume at that level. This shows us
whether there's stronger buying or selling interest.

```questdb-sql
```questdb-sql demo
DECLARE
@bid_prices := bids[1],
@bid_volumes := bids[2],
@ask_prices := asks[1],
@ask_volumes := asks[2],
@best_bid_price := bids[1, 1],
@best_ask_price := asks[1, 1]
SELECT
round((asks[1][1] + bids[1][1]) / 2, 2) mid_price,
(asks[1] - mid_price) * asks[2] weighted_ask_pressure,
(mid_price - bids[1]) * bids[2] weighted_bid_pressure
FROM order_book;
timestamp,
round((@best_bid_price + @best_ask_price) / 2, 2) mid_price,
(mid_price - @bid_prices) * @bid_volumes weighted_bid_pressure,
(@ask_prices - mid_price) * @ask_volumes weighted_ask_pressure
FROM market_data WHERE symbol='EURUSD';
```

#### Sample data and result

```questdb-sql
INSERT INTO order_book VALUES
('2025-07-01T12:00:00Z', ARRAY[ [6.0, 6.1], [15.0, 25] ], ARRAY[ [5.0, 5.1], [10.0, 20] ]),
('2025-07-01T12:00:01Z', ARRAY[ [6.2, 6.4], [20.0, 9] ], ARRAY[ [5.1, 5.2], [20.0, 25] ]);
INSERT INTO market_data VALUES
('2025-07-01T12:00:00Z', 'EURUSD', ARRAY[ [5.0, 5.1], [10.0, 20] ], ARRAY[ [6.0, 6.1], [15.0, 25] ]),
('2025-07-01T12:00:01Z', 'EURUSD', ARRAY[ [5.1, 5.2], [20.0, 25] ], ARRAY[ [6.2, 6.4], [20.0, 9] ]);
```

| ts | mid_price | weighted_ask_pressure | weighted_bid_pressure |
| timestamp | mid_price | weighted_bid_pressure | weighted_ask_pressure |
| ------------------- | --------- | --------------------- | --------------------- |
| 2025-07-01T12:00:00 | 5.5 | [ 7.5, 15.0] | [ 5.0, 8.0] |
| 2025-07-01T12:00:01 | 5.65 | [11.0, 6.75] | [11.0, 11.25] |
| 2025-07-01T12:00:00 | 5.5 | [5.0, 8.0] | [7.5, 15.0] |
| 2025-07-01T12:00:01 | 5.65 | [11.0, 11.25] | [11.0, 6.75] |