折腾LoRA之前，先把它到底省在哪搞明白

[nadleeh]

LoRA 省的就一件事：原权重不动，旁边挂组低秩小矩阵，只训这块。要更新的参数量压到万分之一量级，显存砍三分之二，老卡最明显；推理时合并回原权重，不多耗时。全参微调跑不动，就冲这。低秩分解的数学得啃，不然不知道为啥 work

[xiaomulao]

I also had to grind through the papers on low-rank stuff before, reading your take made it click a lot more

[seedhunter]

Bookmarked

[hana_2]

合并回权重之后我测过，前后速度基本没差

[tuqiangg]

Cutting VRAM by two-thirds is really sweet, my 12G card only runs because of this

[aoi_render]

What rank should I set, I’ve just been going by feel

[hutao_mj]

Bump