Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,13 @@ end).
- **Result sets are framed in bulk.** One read with the remainder carried
forward, never a header read then a payload read: 61 reads over 5000 rows
rather than 10,006. See [the benchmark](bench/README.md).
- **The socket NIFs are the default transport, not the inet driver.** A round
trip costs about 97 reductions rather than 323, and lending a connection out
of the pool costs a NIF call rather than two port reassignments. At sixty-four
connections that is about a fifth more queries a second for about a sixth less
of the machine; at sixteen it is nothing at all, because the pool is the
ceiling before the client is. `transport => inet` is one option away. See
[the benchmark](bench/README.md).

## Guides

Expand Down
6 changes: 5 additions & 1 deletion bench/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,11 @@ clients in the same run, and anything under about ten percent is noise.
## A run

2000 iterations, OTP 29, PostgreSQL 17 in Docker on the same machine, times in
microseconds per operation, mean.
microseconds per operation, mean. The minato column is the `inet` transport,
which is what the default was when the run was made; `minato_bench` takes the
default, so a run made now is the `socket` transport and the whole column moves
together. What that changes is the last two sections, not this one - a percent
on a query against a local server is not where a transport shows up.

| workload | minato | epgsql | pgo | node-postgres | postgres.js | asyncpg | psycopg3 |
| --- | --- | --- | --- | --- | --- | --- | --- |
Expand Down
6 changes: 2 additions & 4 deletions bench/minato_saturation.erl
Original file line number Diff line number Diff line change
Expand Up @@ -204,10 +204,8 @@ teardown({epgsql, _How}, Connections) ->

sql() -> ~"SELECT $1::int4".

connection(inet) ->
base();
connection(socket) ->
(base())#{transport => socket}.
connection(Transport) ->
(base())#{transport => Transport}.

base() ->
#{
Expand Down
15 changes: 15 additions & 0 deletions guides/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ directly.
| `cancel_timeout` | `5000` | how long a cancelled statement has to acknowledge |
| `prepared_statements` | `64` | how many statements a connection keeps parsed; `0` disables |
| `socket_options` | `binary`, `{active,false}`, `{packet,raw}`, `{nodelay,true}` | each replaces one default |
| `transport` | `socket` | `inet` goes back to the driver every BEAM program uses |
| `frame_opts` | `#{max_message_length => 67108864}` | raise it only for a single value near PostgreSQL's 1 GB limit |

Two startup parameters are sent unless `parameters` overrides them:
Expand All @@ -34,6 +35,20 @@ unambiguous under ISO. Set `application_name` here - it is what
`pg_stat_activity` shows, and it is the difference between finding the query
that is hurting and guessing.

`transport` is `socket`, which is `minato_socket` over the socket NIFs. It does
everything the driver does on every machine minato runs on, costs about a
quarter less per query and about half less per pooled one, and holds more
throughput at the same CPU once a pool is deep enough for per-operation cost to
matter at all - see the [bench](https://github.com/Taure/minato/tree/main/bench)
for where that is and is not true. `inet` is the driver, and is one option away
for a machine or a workload that disagrees.

The two transports take the same `socket_options`, but the socket transport has
no driver behind it to hold the ones it does not use: `nodelay`, `keepalive`,
`recbuf` and `sndbuf` are set on the socket, `binary`, `{packet, raw}`,
`{active, false}` and `buffer` are accepted and ignored, and anything else is
refused at connect rather than silently dropped.

## A pool

| option | default | |
Expand Down
36 changes: 30 additions & 6 deletions src/minato_conn.erl
Original file line number Diff line number Diff line change
Expand Up @@ -202,10 +202,12 @@ to `prefer` under `ssl => true` and `disable` without it, since there is nothing
to bind to on a plain socket. `require` refuses to connect at all unless the
server offers `SCRAM-SHA-256-PLUS`.

`transport` is `inet`, the driver every BEAM program uses, or `socket`, which is
`m:minato_socket` over the socket NIFs. `socket` is experimental and faster per
operation, by about a quarter on a query and about a half on a pooled one, and
does everything the driver does on every machine minato runs on. See
`transport` is `socket`, which is `m:minato_socket` over the socket NIFs, or
`inet`, the driver every BEAM program uses. `socket` is the default: it does
everything the driver does on every machine minato runs on, costs about a
quarter less per query and about half less per pooled one, and holds more
throughput at the same CPU once a pool is deep enough for per-operation cost to
show. `inet` is there for a machine or a workload that disagrees. See
`m:minato_socket`.

`prepared_statements` is how many statements this connection will keep parsed on
Expand Down Expand Up @@ -316,8 +318,8 @@ opening(Opts) ->
end.

-spec transport(opts()) -> plain().
transport(#{transport := socket}) -> minato_socket;
transport(_Opts) -> gen_tcp.
transport(#{transport := inet}) -> gen_tcp;
transport(_Opts) -> minato_socket.

-doc """
Close a connection.
Expand Down Expand Up @@ -534,6 +536,13 @@ The connection is unusable for `next/1` until the messages stop coming, and the
two must not be mixed on one connection.
""".
-spec activate(conn()) -> {ok, conn()} | {error, error()}.
%% Asking twice is asking for the same thing: one message. `{active, once}`
%% twice is harmless on the driver, but a second read on a socket handle is a
%% second operation on the same handle, which a completion system refuses
%% outright. A caller that re-activates on every subscriber - which is what a
%% listener does - would otherwise work on POSIX and not on Windows.
activate(#conn{transport = minato_socket, waiting = Waiting} = Conn) when Waiting =/= undefined ->
{ok, Conn};
activate(#conn{transport = minato_socket} = Conn) ->
case minato_socket:activate(handle(Conn)) of
{ok, Data} -> {ok, delivered(Conn, Data)};
Expand Down Expand Up @@ -651,6 +660,21 @@ handle_message(
{'$socket', Socket, completion, {Handle, {error, Reason}}}
) ->
{closed, Reason};
%% The wait can also end with the machine taking it away rather than answering
%% it: a peer that closes while a read is outstanding aborts it on Windows
%% instead of completing it with an error. Ignoring that leaves an owner waiting
%% on a socket that will never speak again, which is a listener that never
%% reconnects.
handle_message(
#conn{transport = minato_socket, socket = Socket, waiting = {completion_info, _Tag, Handle}},
{'$socket', Socket, abort, {Handle, Reason}}
) ->
{closed, Reason};
handle_message(
#conn{transport = minato_socket, socket = Socket, waiting = {select_info, _Tag, Handle}},
{'$socket', Socket, abort, {Handle, Reason}}
) ->
{closed, Reason};
handle_message(#conn{transport = gen_tcp, socket = Socket}, {tcp_closed, Socket}) ->
{closed, closed};
handle_message(#conn{transport = gen_tcp, socket = Socket}, {tcp_error, Socket, Reason}) ->
Expand Down
54 changes: 38 additions & 16 deletions src/minato_socket.erl
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,11 @@
-moduledoc """
The `m:socket` module behind the same calls `m:gen_tcp` answers.

Experimental, and not the default. `m:minato_conn` speaks to its transport
through `connect/4`, `send/2`, `recv/3`, `close/1`, `controlling_process/2` and
The default transport. `m:minato_conn` speaks to its transport through
`connect/4`, `send/2`, `recv/3`, `close/1`, `controlling_process/2` and
`setopts/2`, and this module answers all six on top of the socket NIFs so that
the transport can be chosen per connection with nothing else changing.
`transport => inet` goes back to `m:gen_tcp`.

## Why

Expand Down Expand Up @@ -78,10 +79,12 @@ where it hands over the bytes.
Open a connection.

Takes the option list `m:gen_tcp` would take, so a caller does not have to know
which transport it got. `binary`, `{active, false}` and `{packet, raw}` are what
this module always does and are accepted and ignored; `{nodelay, Boolean}` and
`{buffer, _}` are the two that mean anything here, and anything else is refused
rather than silently dropped.
which transport it got. `binary`, `{mode, binary}`, `{active, false}`,
`{packet, raw}` and `{buffer, _}` are what this module always does or has no
equivalent of, and are accepted and ignored. `{nodelay, _}`, `{keepalive, _}`,
`{recbuf, _}` and `{sndbuf, _}` are set on the socket. Anything else is refused
rather than silently dropped: this is the default transport, and an option that
did something through the driver must not quietly stop doing it here.
""".
-spec connect(
inet:socket_address() | inet:hostname(), inet:port_number(), [term()], timeout()
Expand Down Expand Up @@ -159,11 +162,16 @@ for the calling process, so whoever wants the message has to make the call.
""".
-spec activate(socket()) -> {ok, binary()} | {waiting, waiting()} | {error, term()}.
activate(Socket) ->
case socket:recv(Socket, 0, nowait) of
try socket:recv(Socket, 0, nowait) of
{ok, Data} -> {ok, Data};
{select, Select} -> {waiting, Select};
{completion, Completion} -> {waiting, Completion};
{error, Reason} -> {error, Reason}
catch
%% A socket whose read was aborted from under it has no state left to
%% read in, and Windows raises rather than answering. It is closed, and
%% the caller asked a question that has an answer.
error:{invalid, state} -> {error, closed}
end.

-doc """
Expand All @@ -182,22 +190,36 @@ cancel(Socket, Waiting) ->
{error, _Too_late} -> pending
end.

-doc "Set what can be set. `{active, _}` is refused; see the module note.".
-doc """
Set what can be set.

`{active, _}` is refused; see the module note. So is an option this module has
no answer for, because the alternative is a connection that was configured and
silently was not.
""".
-spec setopts(socket(), [term()]) -> ok | {error, term()}.
setopts(Socket, [{active, false} | Rest]) ->
setopts(Socket, Rest);
setopts(_Socket, [{active, _Mode} | _Rest]) ->
{error, active_mode_unsupported};
setopts(Socket, [{nodelay, Value} | Rest]) ->
case socket:setopt(Socket, {tcp, nodelay}, Value) of
setopts(Socket, [Option | Rest]) ->
case setopt(Socket, Option) of
ok -> setopts(Socket, Rest);
{error, Reason} -> {error, Reason}
end;
setopts(Socket, [_Ignored | Rest]) ->
setopts(Socket, Rest);
setopts(_Socket, []) ->
ok.

-spec setopt(socket(), term()) -> ok | {error, term()}.
setopt(_Socket, {active, false}) -> ok;
setopt(_Socket, {active, _Mode}) -> {error, active_mode_unsupported};
setopt(Socket, {nodelay, Value}) -> socket:setopt(Socket, {tcp, nodelay}, Value);
setopt(Socket, {keepalive, Value}) -> socket:setopt(Socket, {socket, keepalive}, Value);
setopt(Socket, {recbuf, Value}) -> socket:setopt(Socket, {socket, rcvbuf}, Value);
setopt(Socket, {sndbuf, Value}) -> socket:setopt(Socket, {socket, sndbuf}, Value);
setopt(_Socket, binary) -> ok;
setopt(_Socket, {mode, binary}) -> ok;
setopt(_Socket, {packet, raw}) -> ok;
setopt(_Socket, {packet, 0}) -> ok;
setopt(_Socket, {buffer, _Driver_side}) -> ok;
setopt(_Socket, Option) -> {error, {unsupported_socket_option, Option}}.

%%----------------------------------------------------------------------
%% Opening
%%----------------------------------------------------------------------
Expand Down
Loading
Loading