[peft] Define PEFT base class and LoRA transform

[ananthsub]

Base class implementation leans heavily on implementations from PeFT in NeMo: https://github.com/NVIDIA/NeMo/blob/ec9c486557f1b5fb211a903e299d4cb5be1fd3b9/nemo/lightning/pytorch/callbacks/peft.py#L44

PEFT base class differences:

1. PEFT is an abstract dataclass so that implementations neatly plug into the ConfigContainer (params_to_save is now set in the post init, which subclasses need to explicitly initialize)
2. We explicitly pass a training flag to __call__ and freeze_model as we cannot rely on the presence of a lightning trainer object to indicate what stage is being requested
3. The base class does not yet concern itself with checkpointing outside of setting the params to save and the adapter key filter. This save/restore paths will be sent out in follow up PRs as part of integrating peft into the main training loop flow

LoRA is otherwise nearly identical to what's in NeMo minus the following:

• dropout_recompute is not enabled, since it adds an dependency on thunder ([peft] Port base adapter wrapper and lora utils #44 (comment))
• we don't use this attribute here (https://github.com/NVIDIA/NeMo/blob/ec9c486557f1b5fb211a903e299d4cb5be1fd3b9/nemo/collections/llm/peft/lora.py#L442) which is set in the base class here (https://github.com/NVIDIA/NeMo/blob/0af8b7df793ad7538f149ad9bcb8c2cae5134c1a/nemo/lightning/pytorch/callbacks/peft.py#L150)

LoRAMerge is identical to what's in NeMo

This PR includes fixes to walk_utils found from new unit tests

See #68 for the ModuleMatcher, which should get merged first as it's independently testable

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[ananthsub]

/ok to test 46f3623

[marcromeyn]

What's the timeline for adding PEFT to the hub? In the work I was previously doing I have re-implemented PEFT and was thinking to contribute that. But I first need to finish the bridge, so I can likely start with that in a few weeks. The new implementation would also leverage the bridge to support 2-way binding with HF.

The new implementation I worked on doesn't really change the external API, it's mostly internal. So could make sense to first merge a nemo-inspired PEFT and them iterate on it.

[ananthsub]

@marcromeyn I'd like to have some basic PEFT support in for 25.07, so as soon as possible. for now I am following the existing nemo pattern as closely as possible to simplify the migration

[ananthsub]

/ok to test 09c7d73

[ananthsub]

/ok to test 4256a0b

[ananthsub]

/ok to test c9c5b74