huggingface--peft/docs/source/package_reference/deft.md

DEFT: Decompositional Efficient Fine-Tuning for Text-to-Image Models

DEFT (Decompositional Efficient Fine-Tuning) is a parameter-efficient fine-tuning method for text-to-image models. It decomposes the update of a frozen weight matrix W into two trainable components: a projection that removes a low-rank subspace from W, and a low-rank update that injects new content into that subspace. This formulation is designed to balance aligning with a target distribution, learning new concepts from a few images (personalization), and preserving the pretrained model's instruction-following ability and editability.

Concretely, DEFT combines two trainable low-rank components: (1) a projection onto the complement of a low-rank subspace spanned by a low-rank matrix, and (2) a low-rank update. The first low-rank matrix defines the subspace, while the second enables flexible parameter adaptation within that subspace.

When to use DEFT: it is best suited to adapting a model to new data or concepts while retaining and even improving the base model's instruction-following ability and keeping forgetting of its previous capabilities to a minimum.

Per target layer, DEFT learns a projection direction P (shape out_features x r) and an injection matrix R (shape r x in_features). The effective weight is the residual projection

W' = (I - P_proj) @ W + Q_P @ R

The projector P_proj is derived from P according to decomposition_method:

• "relu" (default): Q_P = P, P_proj = P @ relu(P).T — a non-orthogonal projection.
• "qr": Q_P = qr(P), P_proj = Q_P @ Q_P.T — an orthogonal projection.

The (I - P_proj) @ W term removes a sub-space of the pretrained weight while Q_P @ R injects new content into it. By default (init_weights=True) R is initialized so that the update is an exact identity at initialization (W' == W), so training starts from the pretrained weights and learns the injection. The update is equivalent to a low-rank additive delta Q_P @ (R - right.T @ W), which is computed without ever forming the out x out projection matrix and can be merged into the base weights for inference-free deployment.

Setting para=True selects the PaRa (Parameter Rank Reduction) variant: a removal-only update W' = (I - P_proj) @ W that keeps just the subspace-removal term and drops the injection. Only the projection P is trained (no injection matrix R), so the adapter is not an identity at initialization. PaRa was introduced for personalizing text-to-image diffusion models and is available here as a special case of DEFT.

DEFT is currently implemented for torch.nn.Linear and Conv1D (e.g. gpt-2, via fan_in_fan_out) layers. The original implementation and the experiments from the paper (Dreambooth, Dreambench Plus, InsDet, VisualCloze, on Stable Diffusion and a unified model) are available at github.com/MAXNORM8650/DEFT.

If you use DEFT in your work, please cite the paper:

@article{kumar2026deft,
  title={DEFT: Decompositional Efficient Fine-Tuning for Text-to-Image Models},
  author={Kumar, Komal and Anwer, Rao and Shahbaz Khan, Fahad and Khan, Salman and Laptev, Ivan and Cholakkal, Hisham},
  journal={Advances in Neural Information Processing Systems},
  volume={38},
  pages={102009--102035},
  year={2026}
}

If you use the PaRa variant (para=True), please also cite:

@inproceedings{chen2025personalizing,
  title={Para: Personalizing text-to-image diffusion via parameter rank reduction},
  author={Chen, Shangyu and Pan, Zizheng and Cai, Jianfei and Phung, Dinh},
  booktitle={International Conference on Learning Representations},
  year={2025}
}

DeftConfig

autodoc tuners.deft.config.DeftConfig

DeftModel

autodoc tuners.deft.model.DeftModel