Real-world benchmarks

[cristaloleg]

Hi, thank you for the repository & data, it's very useful and quite unique. But there is one thing that can make these benchmarks even better.

What about to make some useful work inside a server method? Like:

• make a 1k, 10k, 100k, 1M iterations of a loop (with a simple math by example)
• allocate an array of 1k, 10k, 100k, 1M elements (of simple type as int or a heavy struct)
• acquire a semaphore with 10, 100, 1000 tickets (similar to DB connection pool)
• do a sleep for 10ms, 100ms, 1sec.
• ...

That will take time to implement for all implementations but the result will be more than just a benchmark of gRPC implementations. WDYT?

Thanks.

[LesnyRumcajs]

Hey, thanks for the kind words. While I agree that results from such benchmarks would be interesting, I don't think implementing them here will fly:

• they would make benchmarks more complicated, less generic and harder to understand. Most likely, fewer people would be willing to contribute if each change required changes to each of those scenarios.
• They would test some internals of the language itself, not the underlying gRPC library. While applicable on their own, it might be simpler to test them outside of any gRPC framework.
• running all of such scenarios would require a lot of time. Even the current set of benchmarks, for up to 6 CPUs, IIRC takes > 12 hours to complete.
• I am not able to invest much time in such a massive endeavour

That said, it might make sense to fork the repository and implement the changes you suggested.

[cristaloleg]

Fair enough, thank you. Closing to not bother you.