{"id":2170,"date":"2026-08-11T14:09:59","date_gmt":"2026-08-11T14:09:59","guid":{"rendered":"https:\/\/convly.ai\/?p=2170"},"modified":"2026-08-11T14:09:59","modified_gmt":"2026-08-11T14:09:59","slug":"lora-fine-tuning-explained","status":"publish","type":"post","link":"https:\/\/convly.ai\/it\/lora-fine-tuning-explained\/","title":{"rendered":"Fine-tuning con LoRA: guida pratica"},"content":{"rendered":"<div class=\"convly-tldr\">\n<ul>\n<li><strong>LoRA fine tuning<\/strong> trains a tiny set of adapter weights instead of the full model \u2014 typically 1\u20135% of total parameters \u2014 so you can fine-tune a 7B model on a single consumer GPU.<\/li>\n<li><strong>QLoRA<\/strong> adds 4-bit quantisation to the frozen base model, cutting VRAM further: a 7B model fits in ~6 GB, a 13B in ~10 GB.<\/li>\n<li><strong>Rank (r) and alpha<\/strong> are the two knobs that control how much the adapter can change the model&#8217;s behaviour. Start at r=16, alpha=32.<\/li>\n<li><strong>Fine-tuning is often the wrong tool.<\/strong> If your problem is missing knowledge, use RAG. If it&#8217;s formatting or tone, improve your system prompt first.<\/li>\n<\/ul>\n<\/div>\n<p>LoRA fine tuning (Low-Rank Adaptation) is a parameter-efficient method for adapting a pretrained language model to a specific task. Instead of updating every weight in the model, LoRA freezes the original weights and injects small trainable matrices into the attention layers. The result is an <em>adapter<\/em> \u2014 a file often under 100 MB \u2014 that snaps onto the base model at inference time. You get specialised behaviour without retraining billions of parameters.<\/p>\n<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_86 counter-flat ez-toc-counter ez-toc-container-direction\">\n<label for=\"ez-toc-cssicon-toggle-item-6a7c5f8e97f2d\" class=\"ez-toc-cssicon-toggle-label\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #000000;color:#000000\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewBox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #000000;color:#000000\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewBox=\"0 0 24 24\" version=\"1.2\" baseProfile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/label><input type=\"checkbox\"  id=\"ez-toc-cssicon-toggle-item-6a7c5f8e97f2d\"  aria-label=\"Toggle\" \/><nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/convly.ai\/it\/lora-fine-tuning-explained\/#How_LoRA_Works\" >How LoRA Works<\/a><\/li><li class='ez-toc-page-1'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/convly.ai\/it\/lora-fine-tuning-explained\/#LoRA_vs_QLoRA\" >LoRA vs QLoRA<\/a><\/li><li class='ez-toc-page-1'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/convly.ai\/it\/lora-fine-tuning-explained\/#Rank_and_Alpha_What_They_Actually_Do\" >Rank and Alpha: What They Actually Do<\/a><\/li><li class='ez-toc-page-1'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/convly.ai\/it\/lora-fine-tuning-explained\/#Realistic_VRAM_and_Time_Requirements\" >Realistic VRAM and Time Requirements<\/a><\/li><li class='ez-toc-page-1'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/convly.ai\/it\/lora-fine-tuning-explained\/#Tooling_How_to_Actually_Run_a_LoRA_Fine-Tune\" >Tooling: How to Actually Run a LoRA Fine-Tune<\/a><\/li><li class='ez-toc-page-1'><a class=\"ez-toc-link ez-toc-heading-6\" href=\"https:\/\/convly.ai\/it\/lora-fine-tuning-explained\/#When_Fine-Tuning_Is_the_Wrong_Tool\" >When Fine-Tuning Is the Wrong Tool<\/a><\/li><li class='ez-toc-page-1'><a class=\"ez-toc-link ez-toc-heading-7\" href=\"https:\/\/convly.ai\/it\/lora-fine-tuning-explained\/#Frequently_Asked_Questions\" >Frequently Asked Questions<\/a><\/li><\/ul><\/nav><\/div>\n<h2><span class=\"ez-toc-section\" id=\"How_LoRA_Works\"><\/span>How LoRA Works<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>A standard transformer weight matrix might be 4096 \u00d7 4096. LoRA decomposes the <em>update<\/em> to that matrix into two much smaller matrices: one of shape 4096 \u00d7 r and one of r \u00d7 4096, where <em>r<\/em> is the rank (commonly 4\u201364). During training, only these low-rank matrices are updated. At inference, the product of the two small matrices is added back to the frozen original \u2014 no extra latency in most implementations, because the adapter is merged before deployment.<\/p>\n<p>This matters for hardware: because the base model weights are frozen, they don&#8217;t need optimiser states or gradients. Only the adapter parameters do. That is why VRAM requirements drop so dramatically compared to full fine-tuning.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"LoRA_vs_QLoRA\"><\/span>LoRA vs QLoRA<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<table>\n<thead>\n<tr>\n<th>Method<\/th>\n<th>Base model precision<\/th>\n<th>Adapter precision<\/th>\n<th>7B VRAM (training)<\/th>\n<th>13B VRAM (training)<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>Full fine-tune<\/td>\n<td>bf16\/fp16<\/td>\n<td>\u2014<\/td>\n<td>~60 GB<\/td>\n<td>~110 GB<\/td>\n<\/tr>\n<tr>\n<td>LoRA<\/td>\n<td>bf16\/fp16<\/td>\n<td>bf16\/fp16<\/td>\n<td>~16 GB<\/td>\n<td>~28 GB<\/td>\n<\/tr>\n<tr>\n<td>QLoRA<\/td>\n<td>4-bit (NF4)<\/td>\n<td>bf16\/fp16<\/td>\n<td>~6 GB<\/td>\n<td>~10 GB<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>QLoRA, introduced by Dettmers et al. (2023), loads the base model in 4-bit NormalFloat (NF4) quantisation and keeps the adapter in full precision. Training is slower than standard LoRA because of dequantisation overhead on each forward pass, but the VRAM savings make 13B and 70B models trainable on hardware most people actually own. Quality versus full LoRA is usually negligible for task-specific fine-tunes; for complex reasoning tasks, some degradation is possible.<\/p>\n<p>Before committing to hardware, run your target model through the <a href=\"https:\/\/convly.ai\/llm-vram-calculator\/\">VRAM calculator<\/a> \u2014 it accounts for batch size and sequence length, both of which move the numbers significantly.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"Rank_and_Alpha_What_They_Actually_Do\"><\/span>Rank and Alpha: What They Actually Do<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p><strong>Rank (r)<\/strong> controls the expressiveness of the adapter. A rank of 4 adds very few parameters and produces subtle changes; rank 64 gives the adapter more capacity to reshape model behaviour but increases VRAM and overfitting risk.<\/p>\n<p><strong>Alpha (\u03b1)<\/strong> is a scaling factor applied to the LoRA output before it is added to the frozen weights. The effective learning rate of the adapter scales with \u03b1 \/ r. Keeping alpha at 2\u00d7 rank (e.g., r=16, alpha=32) is the most common starting point and works well in practice.<\/p>\n<table>\n<thead>\n<tr>\n<th>Use case<\/th>\n<th>Recommended r<\/th>\n<th>Recommended alpha<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>Style \/ tone shift<\/td>\n<td>4\u20138<\/td>\n<td>8\u201316<\/td>\n<\/tr>\n<tr>\n<td>Domain-specific Q&amp;A<\/td>\n<td>16<\/td>\n<td>32<\/td>\n<\/tr>\n<tr>\n<td>New task format (e.g. function calling)<\/td>\n<td>32\u201364<\/td>\n<td>64\u2013128<\/td>\n<\/tr>\n<tr>\n<td>Complex behaviour change<\/td>\n<td>64<\/td>\n<td>128<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>Higher rank does not always mean better results. For most instruction-following fine-tunes, r=16 is sufficient. If validation loss is not improving, increase rank or add more data before increasing epochs.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"Realistic_VRAM_and_Time_Requirements\"><\/span>Realistic VRAM and Time Requirements<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>The figures below assume QLoRA with a batch size of 1 and sequence length of 2048. Multi-GPU setups scale roughly linearly with VRAM but require FSDP or DeepSpeed configuration.<\/p>\n<table>\n<thead>\n<tr>\n<th>Model size<\/th>\n<th>Minimum GPU<\/th>\n<th>Comfortable GPU<\/th>\n<th>~1000 steps (A100)<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>3B<\/td>\n<td>RTX 3060 (12 GB)<\/td>\n<td>RTX 4070 (12 GB)<\/td>\n<td>~5 min<\/td>\n<\/tr>\n<tr>\n<td>7B<\/td>\n<td>RTX 3060 (12 GB)<\/td>\n<td>RTX 4080 (16 GB)<\/td>\n<td>~15 min<\/td>\n<\/tr>\n<tr>\n<td>13B<\/td>\n<td>RTX 3090 (24 GB)<\/td>\n<td>RTX 4090 (24 GB)<\/td>\n<td>~30 min<\/td>\n<\/tr>\n<tr>\n<td>34B<\/td>\n<td>2\u00d7 RTX 3090<\/td>\n<td>A100 40 GB<\/td>\n<td>~90 min<\/td>\n<\/tr>\n<tr>\n<td>70B<\/td>\n<td>2\u00d7 A100 40 GB<\/td>\n<td>4\u00d7 A100<\/td>\n<td>~4 hrs<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>Cloud costs vary. A single A100 80 GB on Lambda Labs runs roughly $1.50\u2013$2.00\/hr as of mid-2026. A 7B QLoRA fine-tune on 50k examples typically completes in under two hours \u2014 well under $5. For GPU purchase decisions, see <a href=\"https:\/\/convly.ai\/best-gpus-for-local-llms-2026\/\">the GPU guide for local LLMs<\/a>.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"Tooling_How_to_Actually_Run_a_LoRA_Fine-Tune\"><\/span>Tooling: How to Actually Run a LoRA Fine-Tune<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>The two dominant frameworks are <strong>Hugging Face TRL + PEFT<\/strong> and <strong>Axolotl<\/strong>. Both support LoRA and QLoRA. Unsloth is a popular third option that achieves 2\u00d7 faster training via custom CUDA kernels.<\/p>\n<h3>Minimal TRL + PEFT Example (Python)<\/h3>\n<pre><code>from transformers import AutoModelForCausalLM, BitsAndBytesConfig\nfrom peft import LoraConfig, get_peft_model\nfrom trl import SFTTrainer, SFTConfig\nimport torch\n\nbnb_config = BitsAndBytesConfig(\n    load_in_4bit=True,\n    bnb_4bit_quant_type=\"nf4\",\n    bnb_4bit_compute_dtype=torch.bfloat16,\n)\n\nmodel = AutoModelForCausalLM.from_pretrained(\n    \"meta-llama\/Meta-Llama-3-8B-Instruct\",\n    quantization_config=bnb_config,\n    device_map=\"auto\",\n)\n\nlora_config = LoraConfig(\n    r=16,\n    lora_alpha=32,\n    target_modules=[\"q_proj\", \"v_proj\"],\n    lora_dropout=0.05,\n    task_type=\"CAUSAL_LM\",\n)\n\nmodel = get_peft_model(model, lora_config)\n\ntrainer = SFTTrainer(\n    model=model,\n    train_dataset=your_dataset,  # expects \"text\" column\n    args=SFTConfig(output_dir=\".\/output\", num_train_epochs=3),\n)\ntrainer.train()\nmodel.save_pretrained(\".\/my-lora-adapter\")\n<\/code><\/pre>\n<p>The adapter saved to <code>.\/my-lora-adapter<\/code> is typically 50\u2013300 MB. Merge it into the base model for faster inference with <code>model.merge_and_unload()<\/code> before saving.<\/p>\n<h3>Axolotl (Config-Driven)<\/h3>\n<p>Axolotl drives the whole pipeline from a YAML file, which makes it easier to reproduce runs. Install with <code>pip install axolotl<\/code>, then:<\/p>\n<pre><code># config.yml\nbase_model: meta-llama\/Meta-Llama-3-8B-Instruct\nload_in_4bit: true\nadapter: lora\nlora_r: 16\nlora_alpha: 32\nlora_dropout: 0.05\ndatasets:\n  - path: .\/data\/train.jsonl\n    type: alpaca\noutput_dir: .\/output\nnum_epochs: 3\n<\/code><\/pre>\n<pre><code>accelerate launch -m axolotl.cli.train config.yml\n<\/code><\/pre>\n<h2><span class=\"ez-toc-section\" id=\"When_Fine-Tuning_Is_the_Wrong_Tool\"><\/span>When Fine-Tuning Is the Wrong Tool<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>LoRA fine tuning solves a specific problem: changing <em>how<\/em> a model behaves or responds. It does not reliably inject factual knowledge. If your use case falls into one of these categories, a different approach will produce better results for less effort:<\/p>\n<ul>\n<li><strong>The model lacks current or proprietary knowledge.<\/strong> Use retrieval-augmented generation (RAG). Fine-tuning on facts produces models that hallucinate confidently on anything outside the training slice.<\/li>\n<li><strong>You need the model to follow specific instructions.<\/strong> Try a detailed system prompt first. A well-engineered prompt on a capable base model often outperforms a fine-tuned smaller model on the same task.<\/li>\n<li><strong>You have fewer than ~500 high-quality examples.<\/strong> The signal-to-noise ratio is too low; the model will likely overfit. Curate more data or use few-shot prompting instead.<\/li>\n<li><strong>You are prototyping.<\/strong> Fine-tuning locks in a behaviour. Use the API and iterate on prompts until the behaviour is stable, then consider fine-tuning to reduce token costs at scale. The <a href=\"https:\/\/convly.ai\/ai-api-cost-calculator\/\">API cost calculator<\/a> helps quantify when fine-tuning a local model becomes cheaper than paying per token.<\/li>\n<\/ul>\n<p>If you are weighing a local fine-tuned model against a hosted API long-term, the <a href=\"https:\/\/convly.ai\/self-hosting-vs-api-calculator\/\">self-hosting vs API break-even calculator<\/a> gives you the crossover point based on your volume and GPU costs.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"Frequently_Asked_Questions\"><\/span>Frequently Asked Questions<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<h3>How much data do I need for LoRA fine tuning?<\/h3>\n<p>For instruction following or style changes, 500\u20132000 high-quality, diverse examples are often enough. For complex domain adaptation, 5000\u201320000 examples produce more robust results. Quality matters far more than quantity \u2014 200 carefully curated examples outperform 2000 noisy ones.<\/p>\n<h3>Can I run LoRA inference on consumer hardware?<\/h3>\n<p>Yes. A merged adapter adds no inference overhead over the base model. An unmerged adapter adds a small amount of computation per forward pass. Both llama.cpp and Ollama support loading GGUF-converted LoRA adapters directly. See the <a href=\"https:\/\/convly.ai\/vram-requirements-every-major-llm-2026\/\">VRAM requirements guide<\/a> for inference-only memory figures.<\/p>\n<h3>What is the difference between LoRA and full fine-tuning?<\/h3>\n<p>Full fine-tuning updates every weight in the model and requires storing optimiser states for all of them \u2014 roughly 16\u201320 bytes per parameter in mixed precision with Adam. LoRA updates only the low-rank adapter matrices, reducing trainable parameters by 10\u20131000\u00d7. The trade-off is capacity: full fine-tuning can reshape the model more completely, but for most practical tasks LoRA matches it.<\/p>\n<h3>Which layers should I target with LoRA?<\/h3>\n<p>The attention projection layers (<code>q_proj<\/code> and <code>v_proj<\/code>) are the most common targets and work well for most tasks. Adding <code>k_proj<\/code>, <code>o_proj<\/code>, and the MLP layers (<code>gate_proj<\/code>, <code>up_proj<\/code>, <code>down_proj<\/code>) increases capacity at the cost of more VRAM and slightly longer training. If VRAM is tight, start with just q and v projections.<\/p>\n<h3>Does QLoRA produce a worse model than full LoRA?<\/h3>\n<p>For most task-specific fine-tunes the difference is negligible. Published benchmarks show QLoRA within 1\u20132 percentage points of full LoRA on standard evals. The gap can widen on complex reasoning tasks with very small datasets, because quantisation noise compounds with limited signal. If accuracy is critical and you have the VRAM, use LoRA over a bf16 base model.<\/p>\n<h3>How do I evaluate whether my fine-tune actually helped?<\/h3>\n<p>Hold out 10\u201320% of your data as a validation set and track validation loss during training. Stop when validation loss stops improving (early stopping). Then run task-specific evals: for classification, measure accuracy on held-out examples; for generation, use human review or an LLM-as-judge setup on 50\u2013100 examples. A drop in validation loss that does not translate to better task performance is a sign of distribution mismatch between your training data and real inputs.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>LoRA fine tuning trains a tiny set of adapter weights instead of the full model \u2014 typically 1\u20135% of total [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":2171,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"site-sidebar-layout":"default","site-content-layout":"","ast-site-content-layout":"default","site-content-style":"default","site-sidebar-style":"default","ast-global-header-display":"","ast-banner-title-visibility":"","ast-main-header-display":"","ast-hfb-above-header-display":"","ast-hfb-below-header-display":"","ast-hfb-mobile-header-display":"","site-post-title":"","ast-breadcrumbs-content":"","ast-featured-img":"","footer-sml-layout":"","ast-disable-related-posts":"","theme-transparent-header-meta":"","adv-header-id-meta":"","stick-header-meta":"","header-above-stick-meta":"","header-main-stick-meta":"","header-below-stick-meta":"","astra-migrate-meta-layouts":"default","ast-page-background-enabled":"default","ast-page-background-meta":{"desktop":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"ast-content-background-meta":{"desktop":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"footnotes":""},"categories":[7],"tags":[],"class_list":["post-2170","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai-news"],"_links":{"self":[{"href":"https:\/\/convly.ai\/it\/wp-json\/wp\/v2\/posts\/2170","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/convly.ai\/it\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/convly.ai\/it\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/convly.ai\/it\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/convly.ai\/it\/wp-json\/wp\/v2\/comments?post=2170"}],"version-history":[{"count":1,"href":"https:\/\/convly.ai\/it\/wp-json\/wp\/v2\/posts\/2170\/revisions"}],"predecessor-version":[{"id":2172,"href":"https:\/\/convly.ai\/it\/wp-json\/wp\/v2\/posts\/2170\/revisions\/2172"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/convly.ai\/it\/wp-json\/wp\/v2\/media\/2171"}],"wp:attachment":[{"href":"https:\/\/convly.ai\/it\/wp-json\/wp\/v2\/media?parent=2170"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/convly.ai\/it\/wp-json\/wp\/v2\/categories?post=2170"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/convly.ai\/it\/wp-json\/wp\/v2\/tags?post=2170"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}