REST gRPC updated

Author: behnamasadi

.gitmodules

@@ -7,3 +7,6 @@
 [submodule "src/microservices/vcpkg"]
 	path = src/microservices/vcpkg
 	url = https://github.com/microsoft/vcpkg
+[submodule "src/microservices/REST/vcpkg"]
+	path = src/microservices/REST/vcpkg
+	url = https://github.com/microsoft/vcpkg

README.md

@@ -317,12 +317,16 @@ This change ensures that VSCode uses the "Ninja Multi-Config" generator by defau
    * [XML](docs/tinyxml2.md)  
 
 ## [REST API and Microservices with C++](#)
-   * [Monolithic Architecture vs REST API and Microservices](docs/REST_API_microservices.md)   
-   * [mock APIs with Mockoon](docs/mockoon.md)
-   * [gRPC Microservice](docs/gRPC_microservice.md)   
-   * [REST API with Crow](docs/REST_API_with_crow.md)   
-   * [Test API with Postman](docs/test_API_with_postman.md)    
-
+   * [Monolithic Architecture vs REST API and Microservices](docs/microservices/REST_API_microservices.md)   
+   * [gRPC](docs/microservices/grpc.md)  
+   * [REST API with crow](docs/microservices/REST_API_with_crow.md)   
+   * [mocking APIs with Mockoon](docs/microservices/mockoon.md)
+   * [Test API with Postman](docs/microservices/test_API_with_postman.md)    
+
+## [Event Streaming and Message Queuing with C++](#)
+   * [Apache Kafka](docs/event_streaming_message/apache_kafka.md)    
+   * [RabbitMQ](docs/event_streaming_message/rabbitMQ.md)    
+   
 ## [Logging With C++](#)
    * [spdlog](docs/spdlog.md)  
 

docs/event_streaming_message/apache_kafka.md

@@ -0,0 +1,124 @@
+## RabbitMQ, and Kafka 
+
+
+RabbitMQ, and Kafka are  designed for messaging or communication between systems, but they serve different purposes and operate in different environments. Let's break down how MAVLink compares to RabbitMQ and Kafka.
+
+### **RabbitMQ**
+- **Purpose**: RabbitMQ is a **message broker** designed for **asynchronous communication** between distributed systems. It is widely used for managing the distribution of tasks or messages between producer and consumer applications.
+- **Architecture**: RabbitMQ follows a **producer-consumer** architecture with queues that can hold and forward messages. Producers send messages to the broker, which distributes them to consumers. It is based on the **AMQP protocol** (Advanced Message Queuing Protocol).
+- **Transport**: RabbitMQ handles communication over **TCP/IP** using AMQP as the underlying protocol. It offers features like **message durability**, **acknowledgements**, and **routing** mechanisms (like topic and direct exchanges).
+- **Message Structure**: Messages are more loosely structured and can contain any type of payload (JSON, XML, binary data). RabbitMQ is better suited for **reliable delivery** and **high-throughput** tasks, but with some latency compared to MAVLink's real-time focus.
+
+### **Apache Kafka**
+- **Purpose**: Kafka is designed for **distributed event streaming** and high-throughput message handling, especially for applications requiring event logging, monitoring, and real-time data analytics. It is used to handle large volumes of data in real time, like **log aggregation**, **event sourcing**, and **stream processing**.
+- **Architecture**: Kafka uses a **publish-subscribe** model where producers publish messages to topics, and consumers subscribe to those topics to receive messages. Kafka is highly distributed and fault-tolerant, designed for scalability.
+- **Transport**: Kafka uses its own custom protocol built on **TCP/IP**. It supports **distributed storage** of messages and has mechanisms for **replication** and **high availability**.
+- **Message Structure**: Kafka messages are organized in **topics** and **partitions**, allowing for massive parallelism. Kafka emphasizes **durability**, **event ordering**, and **real-time data streaming**.
+
+### **Key Comparisons**
+
+| Feature                 | **RabbitMQ**                                   | **Kafka**                                     |
+|-------------------------|------------------------------------------------|----------------------------------------------|
+| **Primary Use Case**    | Asynchronous messaging between systems         | Distributed event streaming and data logging |
+| **Message Model**       | Producer-consumer with broker-based queues     | Publish-subscribe, distributed architecture  |
+| **Transport**           | TCP/IP, AMQP                                   | TCP/IP, custom Kafka protocol                 |
+| **Message Durability**  | Yes, with message persistence                  | Yes, messages are persisted in partitions     |
+| **Real-Time Focus**     | Not real-time, but supports asynchronous tasks | Real-time stream processing                   |
+| **Scalability**         | Can scale horizontally, but not for huge volumes | Designed for high scalability and throughput |
+| **Latency**             | Higher latency, but ensures reliability        | Low-latency with high-throughput focus        |
+
+### **Summary**
+
+- **RabbitMQ** is ideal for **asynchronous task distribution** and managing message delivery between producer-consumer applications. It's reliable and offers strong **durability** features but isn’t designed for real-time telemetry.
+- **Kafka** is designed for handling **massive amounts of streaming data** with high throughput and **durability**, often in **big data** or **log aggregation** scenarios. It’s less suited for low-latency, real-time control systems like those served by MAVLink.
+
+
+
+## Real-world Scenarios of RabbitMQ and Kafka
+
+---
+
+### **Scenario 1: Asynchronous Task Processing (RabbitMQ)**
+#### Example: Processing Image Uploads in a Web Application
+- **Use Case**: A user uploads an image to a web application, and the image needs to be processed (resized, watermarked, etc.). The web server sends the image to a **task queue** for processing by worker services, which handle the image asynchronously and then store the processed version.
+- **RabbitMQ Fit**: RabbitMQ excels here because the tasks (image processing) are:
+  - **Discrete**: Each image is an independent task.
+  - **Asynchronous**: Processing can be done independently from the user interaction. The user doesn't need to wait for the result immediately.
+  - **Reliable**: If the system crashes or a worker fails, RabbitMQ can ensure that tasks are not lost. It supports **message durability**, **acknowledgements**, and **retry mechanisms**.
+  
+- **Why Kafka Could Introduce Complexity**: 
+  - Kafka is designed for **high-throughput event streams** and doesn’t handle discrete, one-off tasks as efficiently. If you use Kafka, you would need to implement custom logic to ensure messages (images) are processed exactly once and not left behind or duplicated. Kafka doesn’t provide strong **task routing** or **priority** management like RabbitMQ does.
+  - **Overhead**: Kafka requires more infrastructure setup, such as managing topics and partitions. For a simple task queue scenario like this, Kafka's added complexity is unnecessary.
+
+#### Why RabbitMQ is a Better Fit:
+- **Worker Queue Pattern**: RabbitMQ’s core strength is in distributing tasks across workers using queues. It is simple to use and scales well for small to medium-scale task-based processing systems.
+- **Routing and Prioritization**: RabbitMQ supports routing rules that Kafka doesn’t natively support, such as sending high-priority messages to faster queues.
+
+---
+
+### **Scenario 2: Event-Driven Microservices (RabbitMQ)**
+#### Example: Order Processing in an E-Commerce System
+- **Use Case**: When a customer places an order, several systems (e.g., inventory, billing, shipping) need to be notified and act on the event.
+- **RabbitMQ Fit**: RabbitMQ is often used in **event-driven architectures** to:
+  - Distribute events between various microservices.
+  - Allow fine-grained control over how messages are routed to each service (e.g., fanout or topic exchanges).
+  - Support scenarios where different services process events at different speeds, with built-in message persistence and retry mechanisms.
+
+- **Why Kafka Could Cause Problems**:
+  - Kafka is better suited for **log aggregation** or event streaming rather than microservice orchestration.
+  - Kafka doesn’t support **message priority**, so time-sensitive orders could get stuck behind less critical events if all services are subscribed to the same topic.
+  - Kafka doesn’t offer built-in mechanisms for complex **routing** of messages, requiring additional infrastructure and logic to replicate RabbitMQ’s flexible routing capabilities.
+
+#### Why RabbitMQ is a Better Fit:
+- **Microservice Communication**: RabbitMQ supports both **fanout** (broadcast to all services) and **topic exchanges** (route based on conditions), which is ideal for microservices that need to handle specific events based on their role.
+- **Message TTL and Dead Letter Exchanges**: RabbitMQ offers message time-to-live (TTL) and dead-letter exchanges to handle delayed or failed messages, which is critical for systems like e-commerce.
+
+---
+
+### **Scenario 3: Real-Time Data Streaming (Kafka)**
+#### Example: Log Aggregation and Real-Time Monitoring
+- **Use Case**: A system needs to collect logs from thousands of servers, process them in real time (e.g., alerting on error rates), and analyze them for patterns over time (e.g., anomaly detection).
+- **Kafka Fit**: Kafka is designed to handle **high-throughput** log aggregation and **streaming** events in real time. It excels in:
+  - **Durability**: Kafka stores all logs for a specified time, allowing you to replay logs or process them at a different time.
+  - **Horizontal Scalability**: Kafka can handle huge streams of data by distributing logs across partitions, making it perfect for systems that need to scale to millions of events per second.
+  - **Replayability**: Kafka’s ability to replay streams is valuable for debugging or replaying logs to test new processing pipelines.
+
+- **Why RabbitMQ Would Struggle**:
+  - **Throughput**: RabbitMQ would struggle to handle the same level of throughput efficiently. It was not designed for massive, persistent streams of data like Kafka is.
+  - **Data Retention**: RabbitMQ messages are typically discarded after they are consumed, whereas Kafka retains messages for a specified period, which is critical for scenarios like log analysis or event stream processing.
+  - **Scalability**: RabbitMQ’s architecture isn’t as horizontally scalable as Kafka’s, especially when dealing with large-scale, distributed event streams.
+
+#### Why Kafka is a Better Fit:
+- **Event Streaming**: Kafka is designed for **continuous, high-volume event streams**, making it ideal for real-time log aggregation.
+- **Durability and Reprocessing**: Kafka keeps the event log for a long time, so consumers can reprocess or analyze historical data, a capability that RabbitMQ doesn't offer.
+- **Scaling**: Kafka’s partitioned architecture allows it to scale horizontally with very high throughput, ideal for applications needing to process millions of events per second.
+
+---
+
+### **Scenario 4: Financial Transactions Streaming (Kafka)**
+#### Example: Tracking Financial Transactions for Real-Time Fraud Detection
+- **Use Case**: A bank wants to track all financial transactions in real time and analyze them for patterns of fraudulent activity. Thousands of events are generated per second across multiple systems.
+- **Kafka Fit**:
+  - **High Throughput**: Kafka can ingest millions of financial transactions in real time, allowing immediate analysis by fraud detection systems.
+  - **Stream Processing**: Kafka works well with stream processing frameworks (like Kafka Streams or Apache Flink) to apply business rules in real time and flag suspicious transactions.
+  - **Durability**: Kafka's ability to store streams of events means that any missed transactions can be replayed and re-analyzed.
+  
+- **Why RabbitMQ Would Be a Poor Fit**:
+  - RabbitMQ lacks the throughput and durability features needed for this kind of high-volume data ingestion and analysis.
+  - The lack of **message replay** in RabbitMQ would be problematic if historical data needed to be re-analyzed.
+  - RabbitMQ’s more complex routing and message delivery guarantees are unnecessary in this scenario, where Kafka’s simpler event log and partitioning are better suited.
+
+#### Why Kafka is a Better Fit:
+- **Massive Scale and Replay**: Kafka is better suited for high-velocity data and allows the reprocessing of events, making it perfect for financial fraud detection.
+- **Durable Event Streams**: Kafka retains the entire event log, allowing for both real-time processing and later analysis, whereas RabbitMQ would delete messages after consumption.
+
+---
+
+### **Conclusion: Where Each Solution Fits Best**
+
+| Solution    | **Best for**                                                             | **Problems if Misused**                                                                                  |
+|-------------|---------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------|
+| **RabbitMQ**| **Asynchronous task processing**, **microservices**, **event-driven apps**| **Limited scalability** for high-throughput or event streaming; doesn’t support replay or long-term storage|
+| **Kafka**   | **High-throughput event streaming**, **log aggregation**, **real-time analytics**| **Overly complex** for simple tasks, lacks **routing** and **priority features**, more setup and management |
+
+RabbitMQ is ideal for cases where **reliability, routing, and discrete task management** are important, while Kafka excels at **high-throughput, durable event streams** where **scalability** and **real-time analytics** are the focus. Each platform serves different types of workloads, and using the wrong one for a particular job can result in added complexity and inefficiencies.

docs/REST_API_with_crow.md renamed to docs/event_streaming_message/rabbitMQ.md

@@ -196,7 +196,7 @@ int main() {
 In the microservice architecture, services would typically communicate using HTTP or gRPC. For example, `OrderService` might call `UserService` to retrieve user info, or `PaymentService` might be called to process payments after an order is placed.
 
 The implementation of communication could be handled using REST API frameworks like:
-- **CppRestSDK** (for RESTful communication)
+- **Crow** (for RESTful communication)
 - **gRPC** (for high-performance communication)
 
 

docs/gRPC_microservice.md

@@ -0,0 +1,129 @@
+### Microservice Example with Crow
+
+First source:
+```
+./vcpkg/bootstrap-vcpkg.sh
+```
+and then build it
+
+```
+cmake -S . -B build  -G "Ninja Multi-Config" -DCMAKE_TOOLCHAIN_FILE=./vcpkg/scripts/buildsystems/vcpkg.cmake
+```
+
+#### 1. **User Service** (user_service.cpp)
+
+```cpp
+#include "crow.h"
+#include <string>
+std::string getUserInfo() {
+  return "{\"name\": \"John Doe\", \"email\": \"[email protected]\"}";
+}
+
+int main() {
+
+  crow::SimpleApp app;
+
+  CROW_ROUTE(app, "/user")
+  ([]() { return crow::response(getUserInfo()); });
+
+  app.port(18080).multithreaded().run();
+}
+```
+the run:
+
+```bash
+curl http://localhost:18080/user`
+```
+
+#### 2. **Product Service** (product_service.cpp)
+
+```cpp
+#include "crow.h"
+#include <string>
+
+std::string getProductCatalog() {
+    return "[{\"name\": \"Laptop\", \"price\": 1200.50}, {\"name\": \"Headphones\", \"price\": 200.99}]";
+}
+
+int main() {
+    crow::SimpleApp app;
+
+    CROW_ROUTE(app, "/products")
+    ([]() {
+        return crow::response(getProductCatalog());
+    });
+
+    app.port(18081).multithreaded().run();
+}
+```
+the run:
+
+```bash
+curl http://localhost:18081/products
+```
+
+#### 3. **Order Service** (order_service.cpp)
+
+```cpp
+#include "crow.h"
+#include <iostream>
+
+void createOrder() {
+    std::cout << "Order created for John Doe with Laptop and Headphones" << std::endl;
+}
+
+int main() {
+    crow::SimpleApp app;
+
+    CROW_ROUTE(app, "/order")
+    ([]() {
+        createOrder();
+        return crow::response(200);
+    });
+
+    app.port(18082).multithreaded().run();
+}
+```
+the run:
+
+```bash
+curl http://localhost:18082/order
+```
+
+
+#### 4. **Payment Service** (payment_service.cpp)
+
+```cpp
+#include "crow.h"
+
+void processPayment(double amount) {
+    std::cout << "Processing payment of $" << amount << std::endl;
+}
+
+int main() {
+    crow::SimpleApp app;
+
+    CROW_ROUTE(app, "/payment/<double>")
+    ([](double amount) {
+        processPayment(amount);
+        return crow::response(200);
+    });
+
+    app.port(18083).multithreaded().run();
+}
+```
+the run:
+
+```bash
+curl http://localhost:18083/payment/10
+```
+
+
+### Running Services
+
+- **User Service** runs on port 18080
+- **Product Service** runs on port 18081
+- **Order Service** runs on port 18082
+- **Payment Service** runs on port 18083
+
+Each service runs independently and listens for HTTP requests. They can be deployed separately, scaled, or updated independently without affecting other services.