CSV format for source?

[MikhailMS]

Hello there,

Just discovered this application and it is a great timing - I was looking exactly for this type of functionality

However, the issue I am facing atm is that there seems to be no support for CSV formatted data in any of the sources (I use Kafka)

Would appreciate if someone can point me in the right direction around the code to see if support for CSV format could be added (brief scan around didn't help) and unless there are some limitations in the core/engine of the project I would try to add this functionality (or if there are already attempts around this area would be great to test those)

Ta

[lmatz]

in the right direction around the code to see if support for CSV format could be added

Hi @MikhailMS, Risingwave supports CSV format:
https://github.com/risingwavelabs/risingwave/blob/main/src/connector/src/parser/csv_parser.rs

But right now CSV format is only supported in the S3 connector: https://www.risingwave.dev/docs/current/data-ingestion/

I guess there is no extra difficulty supporting in other types of connectors, just because we didn't encounter a Kafka use case that reads CSV data.

[neverchanje]

@tabVersion Maybe we can add the 'BYTES' format at first and allow users to use https://www.risingwave.dev/docs/current/sql-function-string/ split_part ( input_string , delimiter_string, int_n ) → varchar to get the csv? Anyways, this requirement sounds valid.
I believe lmatz has been tracking this issue here: #9858

[MikhailMS]

This look awesome - would be keeping my eyes on the release where this one is available :)

Appreciate your help @lmatz @neverchanje

[lmatz]

No problem, I am enabling it in #9875, soon to be merged