Data-sharing delivery is more efficient than Shared Memory Transport?

[wjhtinger]

Right?
So, within the same machine, I should always to use Data-sharing delivery？ But when I need to use Shared Memory Transport?

[JesusPoderoso]

Although Data-sharing delivery uses shared memory, it differs from Shared Memory Transport in that Shared Memory is a full-compliant transport. That means that with Shared Memory Transport the data being transmitted must be copied from the DataWriter history to the transport and from the transport to the DataReader. With Data-sharing these copies can be avoided.

Use of Data-sharing delivery does not prevent data copies between the application and the DataReader and DataWriter. These can be avoided in some cases using Zero-Copy communication.

Answering your question, Shared memory transport should be used when the application does not meet the Data-sharing constraints.

[wjhtinger]

Why there need copying data from the DataWriter history to the transport in Shared Memory Transport? This copy greatly devalue the SHM transport. Why not use pointer between DataWriter history and transport?

[JesusPoderoso]

You are describing the something different from Shared Memory (SHM).
Here it is necessary to differentiate between the different copies that occur in the different communication layers involved.

On one side, the application data is serialized (and deserialized) to the DataWriter history (and from DataReader history, respectively). That serialization involves a copy (as well as deserialization involves another copy). It can be avoided using the loan mechanism (example described in Zero-Copy communication section).

Network based transports data copies

Then, if a network transport is involved (UDP or TCP), the serialized data requires an encapsulation, adding a header and other valuable information to the serialized data. That encapsulation requires a copy to the encapsulation buffer (and other copy from the received buffer to obtain the serialized data in the reception side).
Another copy is performed from the encapsulation buffer to the socket buffer. If multiple DataReaders matched and it is not a multicast scenario, the amount of this copy would increase with the number of DataReaders.

sequenceDiagram

  title Fast DDS Network based transports data copies

  participant send_app as Send application
  participant dw as DataWriter
  participant encapbuff as Encapsulation Buffer
  participant send_sock as Send Socket
  participant recv_sock as Reception Socket
  participant recv_buff as Reception Buffer
  participant dr as DataReader
  participant recv_app as Reception application
  
  send_app->>dw : serialization (if not using loans)
  dw->>encapbuff : encapsulation
  encapbuff->>send_sock : send() (once per DR if no multicast)
  send_sock->>recv_sock : OS specific (not controlled by Fast DDS)
  recv_sock->>recv_buff : recv_from()
  recv_buff->>dr : de-encapsulation (once per DR)
  dr->>recv_app : deserialization (it not using loans)

Loading

Take into account that between sockets, the number of copies are no longer controlled by the middleware, the OS is in charge of that.

Shared Memory transport data copies

In the SHM case, the encapsulation buffer and DataReader will write and read (respectively) from the segment. Together with the fact that no matter how many DataReaders there are, the encapsulated data is only written once in the segment; the number of copies is reduced compared to the other network transports.

sequenceDiagram

  title Fast DDS Shared Memory transport data copies

  participant send_app as Send application
  participant dw as DataWriter
  participant encapbuff as Encapsulation Buffer
  participant segm as Segment
  participant dr as DataReader
  participant recv_app as Reception application

  send_app->>dw : serialization (if not using loans)
  dw->>encapbuff : encapsulation
  encapbuff->>segm : write()
  segm->>dr : de-encapsulation (once per DR)
  dr->>recv_app : deserialization (it not using loans)

Loading

To avoid the encapsulation steps, the communication can be performed using Data-Sharing or Intra-Process if some of their constraints are met (see below table).

Data-Sharing data copies

In the Data-sharing case, the described constraints avoid the encapsulation, so the serialized data is directly written in the segment.

sequenceDiagram

  title Fast DDS Data-Sharing data copies

  participant send_app as Send application
  participant dw as DataWriter
  participant segm as Segment
  participant dr as DataReader
  participant recv_app as Reception application
  send_app->>segm : serialization (if not using loans)
  segm->>recv_app : deserialization (it not using loans)

Loading

Intra-Process data copies

In the Intra-Process case, the DataWriter and DataReader are in the same process and they share process memory. Therefore, they share the payload pool. The application data is serialized into the payload, and the DataReader reads it directly from there.

sequenceDiagram

  title Fast DDS Intra-Process data copies

  participant send_app as Send application
  participant dw as DataWriter
  participant pool as Shared Payload Pool
  participant dr as DataReader
  participant recv_app as Reception application

  send_app->>pool : serialization (if not using loans)
  pool->>recv_app : deserialization (it not using loans)

Loading

If you want to avoid data copies based in your description, you should consider use the Zero-Copy communication.
Zero-Copy is based on using loans to avoid the serialization and deserialization steps, plus one of the previous delivery mechanisms (Data-Sharing or Intra-process).
Loans can also be used with any network transports to reduce the number of copies.

The following table represent the number of copies formed per scenario, taking into account that loans are only used in the Zero-Copy case, and there are only one DataWriter and one DataReader.

Transport Layer	Serializations-Deserializations	RTPS encapsulation-decapsulation	Encapsulation-decapsulation Buffer	Fast DDS total copies	Constraints
Network	2	2	2	6	-
Shared memory	2	2	1	5	Same machine
Intra-process	2	0	0	2	Same process
Data-sharing	2	0	0	2	Same machine, bounded types without keys
Zero-Copy	0	0	0	0	Same machine, plain types, loans

Hope this explanations solve your doubts!

[wjhtinger]

Thank you very much, you are great!

[andreclerigo]

@JesusPoderoso
Hello,
How can one verify if the transport layer is using Shared memory, Data-sharing or Zero-Copy?
Would there be any changes needed to the code to specify zero-copy instead of data-sharing (I have seen some code examples for C++ but couldn't find anything for Python)

[zoeeer]

@JesusPoderoso in the pictures it says "if not using loans" at several places. I wander if it is possible to use loans on Intra-Process or UDP/TCP communication to reduce 2 copies occured on serialization, on condition that the plain and bounded types constraints are met?

[JesusPoderoso]

Hi @andreclerigo, sorry for the late response.
There is no such mechanism to verify which transport layer is being used. Sorry for the inconvenience.
The only way to know would be:

graph TD
    A[Does it meet the data-sharing constraints?]
    A -- NO --> B[Using SharedMemory]
    A -- YES --> C[Does it meet zero-copy constraints, AND does it use loans API?]
    C -- NO --> D[DataSharing]
    C -- YES --> E[ZeroCopy]

Loading

These links refers to the data-sharing constraints, zero-copy constraints and loans API.
You need to analyze your code before run-time to know which delivery mechanism is being used.
If it helps, a new v3.0.0 release is coming really soon, and includes this delivery mechanisms example that meets both data-sharing and zero-copy requirements, and uses the loans API.
Hope it helps!

[zoeeer]

Hi @JesusPoderoso . If we use RTPS directly to send/receive data, should we drop the serialization/deserialization counts (since these are handled in the user application)?

[JesusPoderoso]

Hi @zoeeer.
Yes, you can avoid the serialization copies by writing directly in the payload and reading directly from the payload.

[zoeeer]

Thanks, @JesusPoderoso . According to the table Intra-Process only has serialization/deserialization copies. But I tried to use Intra-Process on RTPS layer, I don't think I've achieved zero-copy. Here is my approach:

1. set library_settings.intraprocess_delivery to INTRAPROCESS_FULL;
2. set transport to BuiltinTransports::UDPv4 (turn off SHM and use UDP only, so that I can capture non-intra-process traffic with wireshark)
3. print the memory address of the data sent and the data received:

   // In RTPSWriter:
   printf("Sending: %s at %p\n", (char*)ch->serializedPayload.data, ch->serializedPayload.data);
   
   // In RTPSReader:
   printf("Received: %s at %p\n", change->serializedPayload.data, change->serializedPayload.data);

4. fastdds version is 2.14.1

And I get:

1. wireshark only captured PDP traffics, which indicates Intra-Process is used
2. memory address of data sent and data received don't match.

In contrast, running the delivery_mechanisms example with intra-process, and printing out the memory addresses, they match.

I think the difference may come from that delivery_mechanisms example used loaning, right?

Is it possible to use loaning on RTPS layer?
Is it possible to achieve zero-copy on RTPS layer?

[JesusPoderoso]

Hi @zoeeer,

Is it possible to use loaning on RTPS layer?
Is it possible to achieve zero-copy on RTPS layer?

the loans API is part of the DDS layer only, but answering the second question, there is a methodology to achieve zero-copy in the RTPS layer, by sharing the payload pool between RTPS endpoints.

[zoeeer]

@JesusPoderoso Thanks again for your reply.

I played a bit around the examples and read a bit of the library code, and have the impression that by loaning a sample in DataWriter, we can actually gain access to the payload buffer itself. While with RTPS we automatically achieve this by just writing to or reading from CacheChange.serializedPayload.data.

You also mentioned sharing payload pool to achieve zero-copy. I did notice in this reply #3654 (reply in thread) you said with Intra-Process, DataReader and DataWriter share the payload pool.
With my test case where I got different memory address of the message CacheChange.serializedPayload.data in Reader and Writer with Intra-Process, I want to check whether they have the save pointer address to the payload pools on both ends. However I failed to find a way to get the payload pool in RTPSReader. It seems only RTPSWriter provides API to access the payload pool with WriterHistory.get_payload_pool()

---- edit ----
Here is the code to demonstrate the test case I've mentioned above #3654 (reply in thread)

[JesusPoderoso]

Hi @zoeeer, while building the endpoint you can provide as an argument the payload pool.
Try to instanciate a PayloadPool in your main.cpp code and use it in the construction of the WriterHistory and ReaderHistory. That would pass this condition and avoid data copies in the rtps layer.