en.wikipedia.org/wiki/Fine-tuning_(machine_learning)
1 correction found
Unlike parameter-efficient fine-tuning (PEFT) methods, which mainly focus on updating weights, ReFT targets representations.
This misclassifies ReFT. ReFT is itself a parameter-efficient fine-tuning (PEFT) method; it differs from other PEFT methods by editing representations rather than weights.
Full reasoning
The sentence incorrectly contrasts ReFT with PEFT as if ReFT were outside that category. But the Stanford ReFT project page explicitly says ReFT should be viewed as "another PEFT method," and a later ACL paper describes ReFT as "a recently proposed parameter-efficient fine-tuning (PeFT) method."
So the problem is not the second half of the sentence—ReFT does target representations—but the taxonomy in the first half. A correct formulation would be closer to: "Unlike many other PEFT methods, which mainly focus on updating weights, ReFT targets representations."
Why this matters: readers could come away thinking ReFT is an alternative to PEFT rather than a representation-based subtype within PEFT.
2 sources
- ReFT: Representation Finetuning for Language Models
We also hope ReFT can not only be viewed as another PEFT method to benchmark against for your future work...
- Punctuation-Steered Representation Fine-Tuning - ACL Anthology
Representation Fine-tuning (ReFT), a recently proposed parameter-efficient fine-tuning (PeFT) method, significantly improves parameter efficiency by modifying the representation space alone.