[FEA] write a grep utility with cuDF

[karthikeyann]

Is your feature request related to a problem? Please describe.
Regex features are available in cudf string APIs and nvtext APIs.
grep is widely used utility. It would cool to try creating cudfgrep with cudf/nvtext and document the throughput.

Describe the solution you'd like
Create cudfgrep utility in our python codebase or a C++ example with libcudf APIs.

Additional context
Benchmark it without GDS (most common usecase).
Benchmark on GPU in PCIE, and on Grace systems with fast C2C.

[vyasr]

Would this live in the cudf repo as an example? Not sure what else we would do.

[karthikeyann]

Yes. Similar to CLI utilities in some python packages, I would like this to be available as part of cudf python.
So, if someone wanted to try GPU grep, they could just install using conda, and try it.

[davidwendt]

Are we picturing grep CLI parameters in the form pattern filenames where the files are loaded through cuIO?
Would the files be any of text, csv, parquet, json, orc?
If multiple columns are in the file which column is targeted? A separate parameter or perhaps all columns?
Would this apply only to strings columns or would matching need to occur on int, float, timestamp, etc types as well?
What about nested columns?
Also, what should the output look like? That is, should it include row numbers or the full row data (or both perhaps).

For strings columns, the pattern could support regex similar to how grep does.
One thing that may be missing from the cudf API is a function that returns the (position, size) of the match so those characters could be highlighted perhaps in the output. There is a find_re API which gives the start position but the size would be nice to return as well.

Also, regular grep will report multiple matches on a single line. Would the cudf-grep returning multiple matches on a row? This would likely require returning a lists column. This may mean we create a new grep-like API which is not difficult but would have to account for all the questions above.

[karthikeyann]

Yes. It will be loaded through cuIO.
Initial idea is to grep only text files. 1 single string per line. String only. (no int, float, timestamp etc).
To Supporting additional formats, we could consider introducing column mode, where regex match is done for a specific column in the input table.
Picture using this utility to grep multi Gigabytes log file. That's the end user goal.

Multiple matches per single line! - Yes.

[JayYarlagadda]

Hey @karthikeyann @vyasr — mind if I take this one? I saw #21942 and #22206 are both closed and it's been quiet for a couple months, and vyasr mentioned it was fine for someone to pick it back up.

Planning to stick to the scope from the thread: plain text logs, one string per line, strings only, regex through cuDF, and multiple matches per line. I'll keep the CLI as close to grep as I can (-i, -n, -c, -v, -o, and --help working properly). I'll also wire up the GDS toggle both ways (env var + CLI flag) like was asked, and add regex tests that cover the common cases instead of just one or two.

The main goal is grepping a multi-GB log fast, so I'll benchmark without GDS and post the throughput numbers when I open the PR. I'll make sure pre-commit is clean too.

[karthikeyann]

Sounds good, please go ahead. Thanks for picking this up, @JayYarlagadda! The scope you outlined matches what I had in mind.
If this is your first PR, I recommend reading https://github.com/rapidsai/cudf/blob/main/CONTRIBUTING.md, and cudf documentation.

[JayYarlagadda]

Thanks @karthikeyann! Opened the PR — #23286. Stuck to the scope we discussed and read through CONTRIBUTING. Benchmark numbers (4 GB log on a T4) are in the description. Left it as a draft for now since it's my first PR and CI needs an ok-to-test.