wehub-resource-sync
caf324b09d
Build documentation / build (push) Failing after 0s
Deploy "method_comparison" Gradio to Spaces / deploy (push) Has been cancelled
Deploy "PEFT shop" Gradio app to Spaces / deploy (push) Has been cancelled
tests on transformers main / tests (push) Has been cancelled
tests / check_code_quality (push) Has been cancelled
tests / tests (ubuntu-latest, 3.10) (push) Has been cancelled
tests / tests (ubuntu-latest, 3.11) (push) Has been cancelled
tests / tests (ubuntu-latest, 3.12) (push) Has been cancelled
tests / tests (ubuntu-latest, 3.13) (push) Has been cancelled
tests / tests (windows-latest, 3.10) (push) Has been cancelled
tests / tests (windows-latest, 3.11) (push) Has been cancelled
tests / tests (windows-latest, 3.12) (push) Has been cancelled
tests / tests (windows-latest, 3.13) (push) Has been cancelled
Secret Leaks / trufflehog (push) Has been cancelled
CI security linting / zizmor latest via Cargo (push) Has been cancelled
42 lines
1.4 KiB
Markdown
42 lines
1.4 KiB
Markdown
<!--⚠️ Note that this file is in Markdown but contain specific syntax for our doc-builder (similar to MDX) that may not be
|
|
rendered properly in your Markdown viewer.
|
|
-->
|
|
|
|
# Helper methods
|
|
|
|
A collection of helper functions for PEFT.
|
|
|
|
## Checking if a model is a PEFT model
|
|
|
|
[[autodoc]] helpers.check_if_peft_model
|
|
- all
|
|
|
|
## Temporarily Rescaling Adapter Scale in LoraLayer Modules
|
|
|
|
[[autodoc]] helpers.rescale_adapter_scale
|
|
- all
|
|
|
|
## Context manager to disable input dtype casting in the `forward` method of LoRA layers
|
|
|
|
[[autodoc]] helpers.disable_input_dtype_casting
|
|
- all
|
|
|
|
## Context manager to enable DoRA caching (faster at inference time but requires more memory)
|
|
|
|
[[autodoc]] helpers.DoraCaching
|
|
- all
|
|
|
|
## KappaTune target selection
|
|
|
|
`KappaTuneSelector` and `find_kappa_target_modules` implement a general target selection process from the [KappaTune paper](https://arxiv.org/abs/2506.16289).
|
|
|
|
The method identifies modules with higher flexibility (higher output differential entropy) and lower specialization (lower sensitivity to specific input directions).
|
|
|
|
These properties make the selected modules good candidates for mitigating catastrophic forgetting in any adaptation method that adds trainable parameters, including LoRA, DoRA, LoHa, AdaLoRA, and even direct fine-tuning of the original weights.
|
|
|
|
[[autodoc]] helpers.KappaTuneSelector
|
|
- all
|
|
|
|
[[autodoc]] helpers.find_kappa_target_modules
|
|
- all
|