Skip to content

Decompress the kernel module if it's compressed#202

Open
ldzhong wants to merge 2 commits intoSUSE:develfrom
ldzhong:decompress
Open

Decompress the kernel module if it's compressed#202
ldzhong wants to merge 2 commits intoSUSE:develfrom
ldzhong:decompress

Conversation

@ldzhong
Copy link
Copy Markdown

@ldzhong ldzhong commented Mar 16, 2026

If the kernel module is present in the compressed format, an error will bail out that said "Module {str(obj_path)} doesn't exists. Aborting" when running the klp-build setup command. In this patch we'll decompress it instead of aborting. Currently it supports .xz, .gz and .zst.

@vmezzela
Copy link
Copy Markdown
Collaborator

Hi Lidong, currently the modules are already extracted during setup. However, it can happen sometimes that if setup fails or is interrupted during the extraction of the modules, some of them are left unextracted. If you were having some issue with it, that's most likely the cause. Unfortunately the error handling is still missing. A quick workaround is to use klp-build data --download --force --filter <broken-cs> to force the download and extraction of the broken cs.

Regarding the PR, there are three approaches we can have:

  1. Extract everything during setup (the current approach)
  2. Extract objects on demand (the one you implemented)
  3. IIRC @marcosps, @fgyanz and I were leaning toward implementing an approach where modules are not extracted on the disk at all, but rather extracted in memory only when needed. This would allow to save even more space on disk.

If we were to go with option 2, I'd vote to not extract any module during setup, given that we would extract them on demand when needed. This approach should save more space on the disk with respect to option 1.

@marcosps, @fgyanz , what do you think?

@marcosps
Copy link
Copy Markdown
Collaborator

In the past, when we still supported clang-extract as well, we used to extract them modules in memory, take a look into 804b101. This would be the perfect world, with clang-extract also supporting it.

In this way, we could dream of using klp-build even on future upstream kernels, and enable people to extract code in their running systems. Given that we stopped supporting clang-extract, and that klp-ccp is basically in maintenance mode + some efforts, I dropped the code since it wasn't used anymore.

@ldzhong a good approach would be make klp-ccp to also extract the modules in memory, using the same approach than clang-extract, so we don't need to extract anymore on disk anymore.

Otherwise, everything is like @vmezzela said: we are doing this on setup phase, and whenever a module is not extracted was an interrupted process or a bug somewhere, but lately we only see issues regarding a previously interrupted setup.

@fgyanz
Copy link
Copy Markdown
Collaborator

fgyanz commented Mar 16, 2026

As I see it, the simplest solution would be to improve data --download to do an integrity check as the first step before downloading anything. Any uncompressed file should probably be deleted, because otherwise we cannot guarantee that the file is not corrupted (aka interrupted previous download).

The ideal solution should be the 2. or 3., as @vmezzela noted :)

@marcosps
Copy link
Copy Markdown
Collaborator

@ldzhong I would like to check if you would like to reimplement 804b101 on top of the current devel branch. I talked with @vmezzela and @fgyanz and we could also make klp-ccp to extract the modules in memory, instead of doing it manually when downloading new codestreams.

We could, for now, hide the new in memory extraction behind an env var, until we have it implemented on klp-ccp. What do you think?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants