"incremental" field - is it useful beyond incremental delivery?

[benjie]

Hey folks; I was reflecting on the recent payload structural changes, specifically the incremental field allowing multiple payloads, and was wondering if this may also be useful for subscriptions. As we know, streaming events to the client one at a time and having them parse and process each one individually can lead to client thrashing; the same can happen over subscriptions. So I'm wondering if the server might take a number of the subscription payloads and pop them together into a batch.

Imagining subscribing to a stock market ticker (subscription { stockChange { stock value time } }) with a sudden flurry of activity:

{data: {stockChange: {stock: "ANTO", value: 1078, time: "2022-09-23T09:00:00.000Z"}}}


{data: {stockChange: {stock: "BME", value: 331, time: "2022-09-23T09:00:00.012Z"}}}


{data: {stockChange: {stock: "BNZL", value: 2692, time: "2022-09-23T09:00:00.022Z"}}}


{data: {stockChange: {stock: "BT.A", value: 134.65, time: "2022-09-23T09:00:00.057Z"}}}

This could, should the server choose, instead be delivered via batch at a rate of 10 times per second (or respecting backpressure, or whatever it chooses really):

{incremental: [
  {data: {stockChange: {stock: "ANTO", value: 1078, time: "2022-09-23T09:00:00.000Z"}}},
  {data: {stockChange: {stock: "BME", value: 331, time: "2022-09-23T09:00:00.012Z"}}},
  {data: {stockChange: {stock: "BNZL", value: 2692, time: "2022-09-23T09:00:00.022Z"}}},
  {data: {stockChange: {stock: "BT.A", value: 134.65, time: "2022-09-23T09:00:00.057Z"}}}
]}

Of course these aren't really "incremental" results. So really, my question is... Is there a better term that allows for more utilisation of this batching capability in future?

[yaacovCR]

Hi! I think it's a great idea to introduce batching into subscriptions!

The incremental wrapper within the payload helps with batching, but also serves the purpose of Separating the incremental results portion from the initial result.

In subscriptions, since we do not have that requirement of separation (yet) could we not simply send an array of results?

I think that will eventually help us if we want to introduce incremental results into subscriptions again.

[benjie]

So, for example if we had subscription { stockChange { stock value time ... @defer { company { name } } } } you're proposing that a batch of streamed data might look like this?

[
  {data: {stockChange: {stock: "ANTO", value: 1078, time: "2022-09-23T09:00:00.000Z"}}},
  {incremental: [{data: {company: {name: "Antofagasta Plc"}}, path: ["stockChange"]}]},
  {data: {stockChange: {stock: "BME", value: 331, time: "2022-09-23T09:00:00.012Z"}}},
  {incremental: [{data: {company: {name: "B&M European Value Retail SA"}}, path: ["stockChange"]}]},
  {data: {stockChange: {stock: "BNZL", value: 2692, time: "2022-09-23T09:00:00.022Z"}}},
  {incremental: [{data: {company: {name: "Bunzl plc"}}, path: ["stockChange"]}]},
  {data: {stockChange: {stock: "BT.A", value: 134.65, time: "2022-09-23T09:00:00.057Z"}}},
  {incremental: [{data: {company: {name: "BT Group - CLASS A Common Stock"}}, path: ["stockChange"]}]}
]

[yaacovCR]

Something like that, although we have to take into account that we may want to enable "multiplexing" the incremental results within the response stream so that later events do not have to await delivery of all payloads from previous events.

Overall, I think the exact payload format should be left up to the transport, both with respect to batching and multiplexing, and that the main spec can pretty much stay silent about it, or perhaps a normative note can be added that it may be useful to transmit the response stream in chunks.

[benjie]

The spec itself tends to be pretty explicit on the result format - sure it doesn't detail the encoding (JSON, protobuf, etc) but it does detail the shape (map with these keys, etc).

[yaacovCR]

I agree => I guess where we differ is just whether multiple results in this context really requires a change in shape or just potentially can be thought of as varying the encoding.

Practically speaking, I am thinking it may be best to leave up to the transport how to best multiplex, and so that any "shape" specified in the main spec might "get in the way".

Perhaps it might be useful to think of how the solution for both batching and multiplexing would be solved together? And then think about where it should be specified?