{"id":2608,"date":"2026-09-10T20:40:34","date_gmt":"2026-09-10T20:40:34","guid":{"rendered":"https:\/\/convly.ai\/?p=2608"},"modified":"2026-09-10T20:40:34","modified_gmt":"2026-09-10T20:40:34","slug":"gguf-model","status":"publish","type":"post","link":"https:\/\/convly.ai\/ar\/gguf-model\/","title":{"rendered":"\u0635\u064a\u063a\u0629 \u0646\u0645\u0648\u0630\u062c GGUF: \u0645\u0627 \u0647\u064a \u0648\u0643\u064a\u0641\u064a\u0629 \u062a\u0634\u063a\u064a\u0644 \u0646\u0645\u0648\u0630\u062c \u0645\u0646\u0647\u0627"},"content":{"rendered":"<div class=\"convly-tldr\">\n<ul>\n<li><strong>GGUF<\/strong> (GPT-Generated Unified Format) \u0647\u064a \u0635\u064a\u063a\u0629 \u062b\u0646\u0627\u0626\u064a\u0629 \u0645\u0644\u0641 \u0648\u0627\u062d\u062f \u0644\u0646\u0645\u0627\u0630\u062c \u0627\u0644\u0644\u063a\u0627\u062a \u0627\u0644\u0643\u0628\u064a\u0631\u0629 \u0627\u0644\u0645\u064f\u0643\u0645\u064e\u0651\u0646\u0629\u060c \u0648\u0642\u062f \u0642\u062f\u0651\u0645\u062a\u0647\u0627 \u0645\u0634\u0631\u0648\u0639 llama.cpp \u0641\u064a \u0623\u063a\u0633\u0637\u0633 2023 \u0628\u0627\u0639\u062a\u0628\u0627\u0631\u0647\u0627 \u0627\u0644\u062e\u0644\u0641 \u0644\u0635\u064a\u063a\u0629 GGML.<\/li>\n<li>\u0648\u0647\u064a \u062a\u0636\u0645 \u0627\u0644\u0623\u0648\u0632\u0627\u0646 \u0648\u0645\u064f\u0641\u0643\u0650\u0651\u0643 \u0627\u0644\u0631\u0645\u0648\u0632 (tokenizer) \u0648\u0646\u0645\u0648\u0630\u062c \u0627\u0644\u0645\u062d\u0627\u062f\u062b\u0629 (chat template) \u0648\u0627\u0644\u0628\u064a\u0627\u0646\u0627\u062a \u0627\u0644\u0648\u0635\u0641\u064a\u0629 (metadata) \u0641\u064a \u0645\u0644\u0641 \u0648\u0627\u062d\u062f <code>.gguf<\/code> \u064a\u064f\u0645\u0643\u0646 \u062a\u0634\u063a\u064a\u0644\u0647 \u0639\u0644\u0649 \u0648\u062d\u062f\u0629 \u0627\u0644\u0645\u0639\u0627\u0644\u062c\u0629 \u0627\u0644\u0645\u0631\u0643\u0632\u064a\u0629 (CPU) \u0623\u0648 \u0648\u062d\u062f\u0629 \u0645\u0639\u0627\u0644\u062c\u0629 \u0627\u0644\u0631\u0633\u0648\u0645\u064a\u0627\u062a (GPU) \u0623\u0648 \u0643\u0644\u064a\u0647\u0645\u0627 \u0645\u0639\u064b\u0627 \u0639\u0628\u0631 <strong>llama.cpp<\/strong>, <strong>Ollama<\/strong>, <strong>LM Studio<\/strong>, <strong>KoboldCpp<\/strong> \u0648 <strong>text-generation-webui<\/strong>.<\/li>\n<li>\u0645\u0633\u062a\u0648\u064a\u0627\u062a \u0627\u0644\u062a\u0643\u0645\u064a\u0646 \u0645\u062b\u0644 <code>Q4_K_M<\/code>, <code>Q5_K_M<\/code> \u0648 <code>Q8_0<\/code> \u0627\u0644\u062a\u064a \u062a\u0636\u062d\u064a \u0628\u0627\u0644\u062c\u0648\u062f\u0629 \u0645\u0646 \u0623\u062c\u0644 \u0627\u0644\u062d\u062c\u0645 \u2014 \u0625\u0630 \u064a\u062d\u062a\u0627\u062c \u062a\u0643\u0645\u064a\u0646 4 \u0628\u062a \u0644\u0646\u0645\u0648\u0630\u062c \u0628\u062d\u062c\u0645 8 \u0645\u0644\u064a\u0627\u0631 \u0645\u0639\u0644\u0645\u0629 (8B) \u0625\u0644\u0649 \u0646\u062d\u0648 5 \u063a\u064a\u063a\u0627\u0628\u0627\u064a\u062a \u0645\u0646 \u0627\u0644\u0630\u0627\u0643\u0631\u0629 \u0627\u0644\u0639\u0634\u0648\u0627\u0626\u064a\u0629 (RAM) \u0623\u0648 \u0630\u0627\u0643\u0631\u0629 \u0648\u062d\u062f\u0629 \u0645\u0639\u0627\u0644\u062c\u0629 \u0627\u0644\u0631\u0633\u0648\u0645\u064a\u0627\u062a (VRAM).<\/li>\n<li>\u0627\u062d\u0635\u0644 \u0639\u0644\u0649 \u0627\u0644\u0645\u0644\u0641\u0627\u062a \u0645\u0646 Hugging Face (\u0627\u0628\u062d\u062b \u0639\u0646 \u00abGGUF\u00bb)\u060c \u0648\u062d\u0645\u0651\u0644\u0647\u0627 \u0628\u0627\u0633\u062a\u062e\u062f\u0627\u0645 <code>llama-cli -m model.gguf<\/code> \u0623\u0648 <code>ollama run<\/code>\u060c \u0648\u0627\u062e\u062a\u0631 \u0645\u0633\u062a\u0648\u0649 \u0627\u0644\u062a\u0643\u0645\u064a\u0646 \u0627\u0644\u0630\u064a \u064a\u062a\u0648\u0627\u0641\u0642 \u0645\u0639 \u0623\u062c\u0647\u0632\u062a\u0643 \u0628\u0627\u0633\u062a\u062e\u062f\u0627\u0645 <a href=\"https:\/\/convly.ai\/ar\/llm-vram-calculator\/\">\u062d\u0627\u0633\u0628\u0629 \u0627\u0644\u0630\u0627\u0643\u0631\u0629 VRAM<\/a>.<\/li>\n<\/ul>\n<\/div>\n<p>\u0623 <strong>\u0646\u0645\u0648\u0630\u062c GGUF<\/strong> \u0647\u0648 \u0646\u0645\u0648\u0630\u062c \u0644\u063a\u0648\u064a \u0643\u0628\u064a\u0631 \u0645\u0639\u0628\u0651\u0623 \u0628\u0635\u064a\u063a\u0629 \u0645\u0644\u0641 GGUF \u2014 \u0623\u064a \u062d\u0627\u0648\u064a\u0629 \u0645\u0644\u0641 \u0648\u0627\u062d\u062f \u062a\u062d\u062a\u0648\u064a \u0627\u0644\u0623\u0648\u0632\u0627\u0646 \u0627\u0644\u0645\u0643\u0645\u064e\u0651\u0646\u0629 \u0648\u0645\u064f\u0641\u0643\u0650\u0651\u0643 \u0627\u0644\u0631\u0645\u0648\u0632 \u0648\u0627\u0644\u0645\u064f\u0639\u0637\u064a\u0627\u062a \u0627\u0644\u0641\u0627\u0626\u0642\u0629 (hyperparameters) \u0648\u0646\u0645\u0648\u0630\u062c \u0627\u0644\u0645\u0648\u062c\u0647 (prompt template) \u0627\u0644\u062e\u0627\u0635 \u0628\u0627\u0644\u0646\u0645\u0648\u0630\u062c. \u0648\u0642\u062f \u0623\u0646\u0634\u0623\u0647 \u062c\u0648\u0631\u062c\u064a \u062c\u064a\u0631\u063a\u0627\u0646\u0648\u0641 \u0648\u0645\u0634\u0631\u0648\u0639 <a href=\"https:\/\/github.com\/ggml-org\/llama.cpp\" rel=\"noopener\" target=\"_blank\">llama.cpp<\/a> project in August 2023 to replace the older GGML format. GGUF is what makes it possible to download one file, point a runtime at it, and get a working local LLM on a laptop or workstation.<\/p>\n<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_87_1 counter-flat ez-toc-counter ez-toc-container-direction\">\n<label for=\"ez-toc-cssicon-toggle-item-6aa323fd1fd06\" class=\"ez-toc-cssicon-toggle-label\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">\u062a\u0628\u062f\u064a\u0644<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #000000;color:#000000\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewbox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #000000;color:#000000\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewbox=\"0 0 24 24\" version=\"1.2\" baseprofile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/label><input type=\"checkbox\"  id=\"ez-toc-cssicon-toggle-item-6aa323fd1fd06\"  aria-label=\"\u062a\u0628\u062f\u064a\u0644\" \/><nav><ul class='ez-toc-list ez-toc-list-level-1' ><li class='ez-toc-page-1'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/convly.ai\/ar\/gguf-model\/#What_GGUF_Actually_Is\" >\u0645\u0627\u0647\u064a\u0629 \u0635\u064a\u063a\u0629 GGUF \u0641\u0639\u0644\u064a\u064b\u0651\u0627<\/a><\/li><li class='ez-toc-page-1'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/convly.ai\/ar\/gguf-model\/#Quantization_The_Naming_Scheme\" >\u0627\u0644\u062a\u0643\u0645\u064a\u0646: \u0646\u0638\u0627\u0645 \u0627\u0644\u062a\u0633\u0645\u064a\u0629<\/a><\/li><li class='ez-toc-page-1'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/convly.ai\/ar\/gguf-model\/#Where_to_Get_GGUF_Files\" >\u0623\u064a\u0646 \u064a\u0645\u0643\u0646 \u0627\u0644\u062d\u0635\u0648\u0644 \u0639\u0644\u0649 \u0645\u0644\u0641\u0627\u062a GGUF<\/a><\/li><li class='ez-toc-page-1'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/convly.ai\/ar\/gguf-model\/#Running_a_GGUF_Model\" >\u062a\u0634\u063a\u064a\u0644 \u0646\u0645\u0648\u0630\u062c GGUF<\/a><\/li><li class='ez-toc-page-1'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/convly.ai\/ar\/gguf-model\/#GGUF_vs_Safetensors_vs_AWQ_vs_GPTQ\" >GGUF \u0645\u0642\u0627\u0628\u0644 Safetensors \u0645\u0642\u0627\u0628\u0644 AWQ \u0645\u0642\u0627\u0628\u0644 GPTQ<\/a><\/li><li class='ez-toc-page-1'><a class=\"ez-toc-link ez-toc-heading-6\" href=\"https:\/\/convly.ai\/ar\/gguf-model\/#When_GGUF_Is_Not_the_Right_Answer\" >\u0645\u062a\u0649 \u0644\u0627 \u062a\u0643\u0648\u0646 \u0635\u064a\u063a\u0629 GGUF \u0647\u064a \u0627\u0644\u062d\u0644 \u0627\u0644\u0645\u0646\u0627\u0633\u0628<\/a><\/li><li class='ez-toc-page-1'><a class=\"ez-toc-link ez-toc-heading-7\" href=\"https:\/\/convly.ai\/ar\/gguf-model\/#Converting_a_Model_to_GGUF\" >\u062a\u062d\u0648\u064a\u0644 \u0646\u0645\u0648\u0630\u062c \u0625\u0644\u0649 \u0635\u064a\u063a\u0629 GGUF<\/a><\/li><li class='ez-toc-page-1'><a class=\"ez-toc-link ez-toc-heading-8\" href=\"https:\/\/convly.ai\/ar\/gguf-model\/#Frequently_Asked_Questions\" >\u0627\u0644\u0623\u0633\u0626\u0644\u0629 \u0627\u0644\u0634\u0627\u0626\u0639\u0629<\/a><\/li><\/ul><\/nav><\/div>\n<h2><span class=\"ez-toc-section\" id=\"What_GGUF_Actually_Is\"><\/span>\u0645\u0627\u0647\u064a\u0629 \u0635\u064a\u063a\u0629 GGUF \u0641\u0639\u0644\u064a\u064b\u0651\u0627<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>GGUF \u062a\u0639\u0646\u064a <em>GPT-Generated Unified Format<\/em>. \u0648\u0647\u064a \u0645\u0646 \u0627\u0644\u0646\u0627\u062d\u064a\u0629 \u0627\u0644\u0628\u0646\u0627\u0626\u064a\u0629 \u0645\u0644\u0641 \u062b\u0646\u0627\u0626\u064a \u064a\u062a\u0636\u0645\u0651\u0646 \u0631\u0623\u0633\u064b\u0627 (header) \u0648\u0643\u062a\u0644\u0629 \u0628\u064a\u0627\u0646\u0627\u062a \u0648\u0635\u0641\u064a\u0629 (metadata) \u0630\u0627\u062a \u0645\u0641\u062a\u0627\u062d-\u0642\u064a\u0645\u0629 (key-value)\u060c \u0648\u0641\u0647\u0631\u0633 \u0627\u0644\u062a\u0646\u0633\u0648\u0631\u0627\u062a (tensor index)\u060c \u0648\u0628\u064a\u0627\u0646\u0627\u062a \u0627\u0644\u062a\u0646\u0633\u0648\u0631\u0627\u062a \u0646\u0641\u0633\u0647\u0627. \u0648\u064a\u062d\u062f\u0651\u062f <a href=\"https:\/\/github.com\/ggml-org\/ggml\/blob\/master\/docs\/gguf.md\" rel=\"noopener\" target=\"_blank\">\u0627\u0644\u0648\u0635\u0641 \u0627\u0644\u0631\u0633\u0645\u064a \u0644\u0635\u064a\u063a\u0629 GGUF<\/a> \u0641\u064a \u0645\u0633\u062a\u0648\u062f\u0639 ggml \u0627\u0644\u062a\u0631\u062a\u064a\u0628 \u0627\u0644\u0628\u0646\u064a\u0648\u064a \u0644\u0647\u0627.<\/p>\n<p>\u064a\u0648\u062c\u062f \u062e\u0627\u0635\u064a\u062a\u0627\u0646 \u0623\u0633\u0627\u0633\u064a\u062a\u0627\u0646 \u062a\u0647\u0645\u0651 \u0627\u0644\u0645\u0633\u062a\u062e\u062f\u0645\u064a\u0646:<\/p>\n<ul>\n<li><strong>\u0645\u0633\u062a\u0642\u0644\u0629 \u0630\u0627\u062a\u064a\u064b\u0651\u0627 (Self-contained).<\/strong> \u064a\u062d\u0645\u0644 \u0627\u0644\u0645\u0644\u0641 \u0645\u064f\u0641\u0643\u0650\u0651\u0643 \u0627\u0644\u0631\u0645\u0648\u0632 \u0648\u0627\u0644\u0631\u0645\u0648\u0632 \u0627\u0644\u062e\u0627\u0635\u0629 \u0648\u0645\u0639\u0631\u0651\u0641\u0627\u062a \u0646\u0647\u0627\u064a\u0629 \u0627\u0644\u0633\u0644\u0633\u0644\u0629\/\u0628\u062f\u0627\u064a\u0629 \u0627\u0644\u0633\u0644\u0633\u0644\u0629 (EOS\/BOS IDs) \u0648\u0646\u0645\u0648\u0630\u062c \u0627\u0644\u0645\u062d\u0627\u062f\u062b\u0629 (chat template) \u0648\u0627\u0644\u0628\u064a\u0627\u0646\u0627\u062a \u0627\u0644\u0648\u0635\u0641\u064a\u0629 \u0627\u0644\u062e\u0627\u0635\u0629 \u0628\u0627\u0644\u0628\u0646\u064a\u0629 (architecture metadata). \u0648\u0628\u0627\u0644\u062a\u0627\u0644\u064a\u060c \u0644\u0627 \u062a\u062d\u062a\u0627\u062c \u0625\u0644\u0649 \u0648\u062c\u0648\u062f \u0645\u0644\u0641 \u0645\u0646\u0641\u0635\u0644 <code>tokenizer.json<\/code> \u0623\u0648 <code>config.json<\/code> \u0628\u062c\u0627\u0646\u0628\u0647.<\/li>\n<li><strong>\u0642\u0627\u0628\u0644\u0629 \u0644\u0645\u064f\u0637\u0627\u0628\u0642\u0629 \u0627\u0644\u0630\u0627\u0643\u0631\u0629 (Memory-mappable).<\/strong> \u062a\u0633\u062a\u062e\u062f\u0645 \u0628\u064a\u0626\u0627\u062a \u0627\u0644\u062a\u0634\u063a\u064a\u0644 \u062f\u0627\u0644\u0629 <code>mmap()<\/code> \u0644\u062a\u062d\u0645\u064a\u0644 \u0627\u0644\u0645\u0644\u0641\u060c \u0644\u0630\u0627 \u064a\u0643\u0648\u0646 \u0648\u0642\u062a \u0628\u062f\u0621 \u0627\u0644\u062a\u0634\u063a\u064a\u0644 \u0634\u0628\u0647 \u0641\u0648\u0631\u064a\u060c \u0648\u062a\u0642\u0648\u0645 \u0646\u0638\u0627\u0645 \u0627\u0644\u062a\u0634\u063a\u064a\u0644 \u0628\u062a\u062d\u0645\u064a\u0644 \u0623\u0648\u0632\u0627\u0646 \u0627\u0644\u0646\u0645\u0648\u0630\u062c \u0639\u0646\u062f \u0627\u0644\u062d\u0627\u062c\u0629 (on demand). \u0648\u0644\u0630\u0644\u0643 \u064a\u0641\u062a\u062d \u0645\u0644\u0641 GGUF \u0628\u062d\u062c\u0645 40 \u063a\u064a\u063a\u0627\u0628\u0627\u064a\u062a \u0641\u064a \u063a\u0636\u0648\u0646 \u062b\u0648\u0627\u0646\u064d \u062d\u062a\u0649 \u0639\u0644\u0649 \u0642\u0631\u0635 SSD \u0628\u0637\u064a\u0621.<\/li>\n<\/ul>\n<p>\u0627\u0633\u062a\u064f\u0628\u062f\u0644\u062a \u0635\u064a\u063a\u0629 GGUF \u0628\u0635\u064a\u063a\u0629 GGML \u0644\u0623\u0646 \u0627\u0644\u0623\u062e\u064a\u0631\u0629 \u0644\u0645 \u062a\u0643\u0646 \u062a\u062f\u0639\u0645 \u0627\u0644\u062a\u0631\u0642\u064a\u0645 \u0627\u0644\u0625\u0635\u062f\u0627\u0631\u064a (versioning)\u060c \u0648\u0643\u0627\u0646\u062a \u062a\u062e\u0644\u0637 \u0628\u064a\u0646 \u0627\u0644\u0628\u064a\u0627\u0646\u0627\u062a \u0627\u0644\u0648\u0635\u0641\u064a\u0629 \u0648\u0627\u0644\u0623\u0648\u0632\u0627\u0646\u060c \u0643\u0645\u0627 \u0643\u0627\u0646\u062a \u062a\u062a\u0648\u0642\u0641 \u0639\u0646 \u0627\u0644\u0639\u0645\u0644 \u0641\u064a \u0643\u0644 \u0645\u0631\u0629 \u064a\u062a\u0645 \u0641\u064a\u0647\u0627 \u0625\u062f\u062e\u0627\u0644 \u0628\u0646\u064a\u0629 \u062c\u062f\u064a\u062f\u0629. \u0623\u0645\u0627 \u0635\u064a\u063a\u0629 GGUF \u0641\u0647\u064a \u0645\u064f\u0631\u0642\u064e\u0651\u0645\u0629 \u0648\u0642\u0627\u0628\u0644\u0629 \u0644\u0644\u062a\u0648\u0633\u064a\u0639.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"Quantization_The_Naming_Scheme\"><\/span>\u0627\u0644\u062a\u0643\u0645\u064a\u0646: \u0646\u0638\u0627\u0645 \u0627\u0644\u062a\u0633\u0645\u064a\u0629<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>\u0645\u0639\u0638\u0645 \u0645\u0644\u0641\u0627\u062a GGUF \u0627\u0644\u062a\u064a \u062a\u0642\u0648\u0645 \u0628\u062a\u0646\u0632\u064a\u0644\u0647\u0627 \u062a\u0643\u0648\u0646 <em>\u0645\u0643\u0645\u064e\u0651\u0646\u0629<\/em> \u2014 \u0623\u064a \u0623\u0646 \u0627\u0644\u0623\u0648\u0632\u0627\u0646 \u0645\u062e\u0632\u064e\u0651\u0646\u0629 \u0628\u0639\u062f\u062f \u0628\u062a\u0627\u062a \u0623\u0642\u0644 \u0645\u0646 \u0627\u0644\u0623\u0635\u0644\u064a\u0629 (FP16). \u0648\u064a\u0634\u064a\u0631 \u0627\u0644\u0644\u0627\u062d\u0642\u0629 \u0641\u064a \u0627\u0633\u0645 \u0627\u0644\u0645\u0644\u0641 \u0625\u0644\u0649 \u0646\u0638\u0627\u0645 \u0627\u0644\u062a\u0643\u0645\u064a\u0646 \u0627\u0644\u0645\u0633\u062a\u062e\u062f\u0645. \u0648\u0623\u064f\u062f\u0631\u062c\u062a \u0639\u0627\u0626\u0644\u0627\u062a \u0627\u0644\u062a\u0643\u0645\u064a\u0646 \u0627\u0644\u062d\u0627\u0644\u064a\u0629 \u0641\u064a <a href=\"https:\/\/github.com\/ggml-org\/llama.cpp\/blob\/master\/tools\/quantize\/README.md\" rel=\"noopener\" target=\"_blank\">\u0645\u0644\u0641 README \u0627\u0644\u062e\u0627\u0635 \u0628\u0623\u062f\u0627\u0629 quantize \u0641\u064a \u0645\u0634\u0631\u0648\u0639 llama.cpp<\/a>.<\/p>\n<table>\n<thead>\n<tr>\n<th>\u0627\u0644\u0643\u0645\u0651<\/th>\n<th>\u0639\u062f\u062f \u0627\u0644\u0628\u062a\u0627\u062a \u0644\u0643\u0644 \u0648\u0632\u0646 (\u062a\u0642\u0631\u064a\u0628\u064a)<\/th>\n<th>\u0627\u0644\u0627\u0633\u062a\u062e\u062f\u0627\u0645 \u0627\u0644\u0646\u0645\u0648\u0630\u062c\u064a<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>Q2_K<\/td>\n<td>~2.6<\/td>\n<td>\u0623\u0635\u063a\u0631 \u062d\u062c\u0645 \u0645\u0645\u0643\u0646 \u0645\u0639 \u0641\u0642\u062f\u0627\u0646 \u0645\u0644\u062d\u0648\u0638 \u0641\u064a \u0627\u0644\u062c\u0648\u062f\u0629<\/td>\n<\/tr>\n<tr>\n<td>Q3_K_M<\/td>\n<td>~3.9<\/td>\n<td>\u0636\u063a\u0637 \u0639\u062f\u0648\u0627\u0646\u064a<\/td>\n<\/tr>\n<tr>\n<td>Q4_K_M<\/td>\n<td>~4.8<\/td>\n<td>\u0627\u0644\u0627\u0641\u062a\u0631\u0627\u0636\u064a \u0627\u0644\u0623\u0643\u062b\u0631 \u0634\u064a\u0648\u0639\u064b\u0627 \u2014 \u062a\u0648\u0627\u0632\u0646 \u062c\u064a\u062f \u0628\u064a\u0646 \u0627\u0644\u062d\u062c\u0645 \u0648\u0627\u0644\u062c\u0648\u062f\u0629<\/td>\n<\/tr>\n<tr>\n<td>Q5_K_M<\/td>\n<td>~5.7<\/td>\n<td>\u062c\u0648\u062f\u0629 \u0623\u0639\u0644\u0649 \u0645\u0639 \u062d\u062c\u0645 \u0645\u0644\u0641 \u0623\u0643\u0628\u0631<\/td>\n<\/tr>\n<tr>\n<td>Q6_K<\/td>\n<td>~6.6<\/td>\n<td>\u0634\u0628\u064a\u0647 \u0628\u0627\u0644\u0641\u0642\u062f\u0627\u0646 \u0627\u0644\u0635\u0641\u0631\u064a \u0645\u0642\u0627\u0631\u0646\u0629\u064b \u0628\u0640 FP16<\/td>\n<\/tr>\n<tr>\n<td>Q8_0<\/td>\n<td>~8.5<\/td>\n<td>\u0641\u0639\u0644\u064a\u064b\u0651\u0627 \u0628\u062f\u0648\u0646 \u0641\u0642\u062f\u0627\u0646\u060c \u0648\u0628\u062d\u062c\u0645 \u064a\u0642\u0627\u0631\u0628 \u0636\u0639\u0641 \u062d\u062c\u0645 Q4<\/td>\n<\/tr>\n<tr>\n<td>F16 \/ BF16<\/td>\n<td>16<\/td>\n<td>\u0645\u0631\u062c\u0639 \u063a\u064a\u0631 \u0645\u0643\u0645\u064e\u0651\u0646<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>\u0627\u0644\u0640 <code>_K<\/code> \u0627\u0644\u0623\u0646\u0648\u0627\u0639 \u0627\u0644\u0645\u062a\u063a\u064a\u0631\u0629 (k-quants) \u062a\u0633\u062a\u062e\u062f\u0645 \u062f\u0642\u0629 \u0645\u062a\u063a\u064a\u0631\u0629 \u0644\u0643\u0644 \u0643\u062a\u0644\u0629 \u062a\u0646\u0633\u0648\u0631. \u0648\u0631\u0645\u0632 <code>_M<\/code> \/ <code>_S<\/code> \/ <code>_L<\/code> \u062a\u0634\u064a\u0631 \u0627\u0644\u0644\u0627\u062d\u0642\u0629 \u0625\u0644\u0649 \u0625\u0639\u062f\u0627\u062f\u0627\u062a \u0645\u0633\u0628\u0642\u0629 \u0644\u0645\u0632\u062c \u0627\u0644\u0643\u062a\u0644 \u0645\u062a\u0648\u0633\u0637\u0629 \u0623\u0648 \u0635\u063a\u064a\u0631\u0629 \u0623\u0648 \u0643\u0628\u064a\u0631\u0629. \u0627\u0644\u0625\u0635\u062f\u0627\u0631\u0627\u062a \u0627\u0644\u0623\u062d\u062f\u062b <code>\u0628\u0627\u062f\u0626\u0627\u062a IQ<\/code> (\u0645\u062b\u0644\u060c) <code>IQ4_XS<\/code>\u062a\u0633\u062a\u062e\u062f\u0645 \u0645\u0639\u0627\u064a\u0631\u0629 \u0645\u0635\u0641\u0648\u0641\u0629 \u0627\u0644\u0623\u0647\u0645\u064a\u0629 \u0644\u062a\u062d\u0642\u064a\u0642 \u062c\u0648\u062f\u0629 \u0623\u0641\u0636\u0644 \u0639\u0646\u062f \u0646\u0641\u0633 \u0645\u064a\u0632\u0627\u0646\u064a\u0629 \u0627\u0644\u0628\u062a.<\/p>\n<p>\u0648\u0643\u0642\u0627\u0639\u062f\u0629 \u0639\u0627\u0645\u0629 \u0628\u0627\u0644\u0646\u0633\u0628\u0629 \u0644\u0630\u0627\u0643\u0631\u0629 VRAM\u060c \u062e\u0630 \u062d\u062c\u0645 \u0627\u0644\u0645\u0644\u0641 \u0639\u0644\u0649 \u0627\u0644\u0642\u0631\u0635 \u0648\u0623\u0636\u0641 \u062a\u0642\u0631\u064a\u0628\u064b\u0627 \u0661\u2013\u0663 \u062c\u064a\u062c\u0627\u0628\u0627\u064a\u062a \u0644\u0645\u062e\u0628\u0623 KV \u0641\u064a \u0627\u0644\u0633\u064a\u0627\u0642\u0627\u062a \u0627\u0644\u0642\u0635\u064a\u0631\u0629\u060c \u0648\u0627\u0644\u0645\u0632\u064a\u062f \u0641\u064a \u0627\u0644\u0633\u064a\u0627\u0642\u0627\u062a \u0627\u0644\u0637\u0648\u064a\u0644\u0629. \u0641\u0639\u0644\u0649 \u0633\u0628\u064a\u0644 \u0627\u0644\u0645\u062b\u0627\u0644\u060c \u064a\u062d\u062a\u0627\u062c <a href=\"https:\/\/convly.ai\/ar\/models\/\">Convly<\/a>: Llama 3.1 8B needs around 5&nbsp;GB at 4-bit, Llama 3.3 70B around 40&nbsp;GB, and Mistral 7B around 4.5&nbsp;GB. For anything larger, the <a href=\"https:\/\/convly.ai\/ar\/vram-requirements-every-major-llm-2026\/\">\u062c\u062f\u0648\u0644 \u0645\u062a\u0637\u0644\u0628\u0627\u062a \u0627\u0644\u0640 VRAM<\/a> \u0623\u0633\u0631\u0639 \u0637\u0631\u064a\u0642\u0629 \u0644\u0644\u0628\u062d\u062b.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"Where_to_Get_GGUF_Files\"><\/span>\u0623\u064a\u0646 \u064a\u0645\u0643\u0646 \u0627\u0644\u062d\u0635\u0648\u0644 \u0639\u0644\u0649 \u0645\u0644\u0641\u0627\u062a GGUF<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>\u064a\u062a\u0645 \u062a\u0648\u0632\u064a\u0639 \u0645\u0639\u0638\u0645 \u0645\u0644\u0641\u0627\u062a GGUF \u0639\u0644\u0649 \u0645\u0646\u0635\u0629 Hugging Face. \u0627\u0628\u062d\u062b \u0639\u0646 \u0627\u0633\u0645 \u0627\u0644\u0646\u0645\u0648\u0630\u062c \u0645\u0639 \u0625\u0636\u0627\u0641\u0629 \u0639\u0628\u0627\u0631\u0629 \u00abGGUF\u00bb. \u0648\u0645\u0646 \u0623\u062f\u0648\u0627\u062a \u0625\u0639\u0627\u062f\u0629 \u0627\u0644\u062a\u0643\u0645\u064a\u0646 \u0627\u0644\u0645\u0648\u062b\u0648\u0642\u0629: <strong>bartowski<\/strong>, <strong>\u0643\u0648\u064a\u0646<\/strong> (\u0627\u0644\u0631\u0633\u0645\u064a\u0629 \u0644\u0640 Qwen3)\u060c <strong>lmstudio-community<\/strong>, <strong>unsloth<\/strong> \u0648 <strong>MaziyarPanahi<\/strong>. Meta, Google, Mistral and Microsoft publish some official GGUF builds; for others the community quantizes from the released safetensors.<\/p>\n<p>\u0639\u0627\u062f\u0629\u064b \u0645\u0627 \u062a\u062d\u062a\u0648\u064a \u0627\u0644\u0645\u0633\u062a\u0648\u062f\u0639\u0627\u062a (repo) \u0639\u0644\u0649 \u0645\u0644\u0641 \u0648\u0627\u062d\u062f \u0644\u0643\u0644 \u0645\u0633\u062a\u0648\u0649 \u062a\u0643\u0645\u064a\u0646\u060c \u0645\u062b\u0644: <code>Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf<\/code>. \u0623\u0645\u0627 \u0628\u0627\u0644\u0646\u0633\u0628\u0629 \u0644\u0644\u0646\u0645\u0627\u0630\u062c \u0627\u0644\u062a\u064a \u064a\u062a\u062c\u0627\u0648\u0632 \u062d\u062c\u0645\u0647\u0627 ~\u0665\u0660 \u062c\u064a\u062c\u0627\u0628\u0627\u064a\u062a\u060c \u0641\u064a\u064f\u0642\u0633\u064e\u0651\u0645 \u0627\u0644\u0645\u0644\u0641 \u0625\u0644\u0649 \u0623\u062c\u0632\u0627\u0621 (shards) \u062a\u062d\u0645\u0644 \u0623\u0633\u0645\u0627\u0621 \u0645\u062b\u0644: <code>-00001-of-00003.gguf<\/code>\u061b \u0648\u062a\u0642\u0648\u0645 \u0628\u0631\u0627\u0645\u062c \u0627\u0644\u062a\u0634\u063a\u064a\u0644 (runtimes) \u062a\u0644\u0642\u0627\u0626\u064a\u064b\u0651\u0627 \u0628\u062a\u062d\u0645\u064a\u0644\u0647\u0627 \u0628\u062f\u0621\u064b\u0627 \u0645\u0646 \u0627\u0644\u062c\u0632\u0621 \u0627\u0644\u0623\u0648\u0644.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"Running_a_GGUF_Model\"><\/span>\u062a\u0634\u063a\u064a\u0644 \u0646\u0645\u0648\u0630\u062c GGUF<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<h3>\u0644\u064a\u0646\u0643\u0633<\/h3>\n<p>\u0627\u0628\u0646\u0650 llama.cpp \u0645\u0646 \u0627\u0644\u0645\u0635\u062f\u0631 \u0623\u0648 \u062b\u0628\u0650\u0651\u062a \u0627\u0644\u062d\u0632\u0645\u0629 \u0627\u0644\u062c\u0627\u0647\u0632\u0629 \u0627\u0644\u0645\u064f\u0633\u0628\u0642\u0629 \u0627\u0644\u062a\u062c\u0645\u064a\u0639. \u0645\u0639 \u062f\u0639\u0645 CUDA:<\/p>\n<pre><code>git clone https:\/\/github.com\/ggml-org\/llama.cpp\ncd llama.cpp\ncmake -B build -DGGML_CUDA=ON\ncmake --build build --config Release -j\n.\/build\/bin\/llama-cli -m ~\/models\/llama-3.1-8b-instruct-Q4_K_M.gguf \n    -p \"\u0627\u0634\u0631\u062d \u062a\u0646\u0633\u064a\u0642 GGUF \u0641\u064a \u0641\u0642\u0631\u0629 \u0648\u0627\u062d\u062f\u0629.\" -n 256 -ngl 99<\/code><\/pre>\n<p>\u0627\u0644\u0640 <code>(\u0639\u062f\u062f \u0637\u0628\u0642\u0627\u062a \u0648\u062d\u062f\u0629 \u0645\u0639\u0627\u0644\u062c\u0629 \u0627\u0644\u0631\u0633\u0648\u0645\u0627\u062a).<\/code> \u064a\u062d\u062f\u062f \u0627\u0644\u0639\u0644\u0645 (n-gpu-layers) \u0639\u062f\u062f \u0637\u0628\u0642\u0627\u062a \u0627\u0644\u0645\u062d\u0648\u0644 (transformer) \u0627\u0644\u062a\u064a \u062a\u064f\u062d\u0645\u064e\u0651\u0644 \u0639\u0644\u0649 \u0648\u062d\u062f\u0629 \u0645\u0639\u0627\u0644\u062c\u0629 \u0627\u0644\u0631\u0633\u0648\u0645\u0627\u062a (GPU). \u0627\u0636\u0628\u0637\u0647 \u0639\u0644\u0649 \u0642\u064a\u0645\u0629 \u0639\u0627\u0644\u064a\u0629 \u0628\u0645\u0627 \u064a\u0643\u0641\u064a \u0644\u062a\u0646\u0627\u0633\u0628 \u0627\u0644\u0646\u0645\u0648\u0630\u062c \u0628\u0623\u0643\u0645\u0644\u0647 \u0641\u064a \u0630\u0627\u0643\u0631\u0629 VRAM\u061b \u0648\u0625\u0630\u0627 \u0646\u0641\u062f \u0627\u0644\u0645\u0643\u0627\u0646\u060c \u0641\u0642\u0644\u0650\u0651\u0644 \u0647\u0630\u0647 \u0627\u0644\u0642\u064a\u0645\u0629 \u0644\u064a\u0638\u0644 \u0627\u0644\u062c\u0632\u0621 \u0627\u0644\u0645\u062a\u0628\u0642\u064a \u0639\u0644\u0649 \u0648\u062d\u062f\u0629 \u0627\u0644\u0645\u0639\u0627\u0644\u062c\u0629 \u0627\u0644\u0645\u0631\u0643\u0632\u064a\u0629 (CPU). \u0648\u0644\u062a\u0634\u063a\u064a\u0644 \u0648\u0627\u062c\u0647\u0629 \u0628\u0631\u0645\u062c\u0629 \u062a\u0637\u0628\u064a\u0642\u0627\u062a HTTP \u0645\u062a\u0648\u0627\u0641\u0642\u0629 \u0645\u0639 OpenAI\u060c \u0627\u0633\u062a\u062e\u062f\u0645 <code>llama-server<\/code> \u0648\u0627\u0644\u062a\u064a \u062a\u0639\u0645\u0644 \u0627\u0641\u062a\u0631\u0627\u0636\u064a\u064b\u0651\u0627 \u0639\u0644\u0649 \u0627\u0644\u0645\u0646\u0641\u0630 \u0668\u0660\u0668\u0660.<\/p>\n<h3>macOS<\/h3>\n<p>\u0639\u0644\u0649 \u0623\u062c\u0647\u0632\u0629 Apple Silicon\u060c \u064a\u0633\u062a\u062e\u062f\u0645 llama.cpp \u062a\u0644\u0642\u0627\u0626\u064a\u064b\u0651\u0627 \u0648\u0627\u062c\u0647\u0629 Metal. \u062b\u0628\u0650\u0651\u062a\u0647 \u0639\u0628\u0631 Homebrew:<\/p>\n<pre><code>brew install llama.cpp\nllama-cli -m ~\/models\/qwen3-8b-Q5_K_M.gguf -p \"\u0645\u0631\u062d\u0628\u064b\u0627\" -ngl 99<\/code><\/pre>\n<p>\u062a\u0639\u0646\u064a \u0627\u0644\u0630\u0627\u0643\u0631\u0629 \u0627\u0644\u0645\u0648\u062d\u064e\u0651\u062f\u0629 \u0623\u0646 <code>(\u0639\u062f\u062f \u0637\u0628\u0642\u0627\u062a \u0648\u062d\u062f\u0629 \u0645\u0639\u0627\u0644\u062c\u0629 \u0627\u0644\u0631\u0633\u0648\u0645\u0627\u062a).<\/code> ceiling is your total RAM minus the OS overhead. A 32&nbsp;GB M-series Mac comfortably runs Qwen3 14B or Gemma 3 12B at Q4_K_M &mdash; the database lists those at ~9&nbsp;GB and ~8&nbsp;GB of VRAM respectively.<\/p>\n<p>\u0623\u0645\u0627 \u0628\u0627\u0644\u0646\u0633\u0628\u0629 \u0644\u0648\u0627\u062c\u0647\u0629 \u0627\u0644\u0645\u0633\u062a\u062e\u062f\u0645 \u0627\u0644\u0631\u0633\u0648\u0645\u064a\u0629 (GUI)\u060c \u0641\u0625\u0646 <a href=\"https:\/\/convly.ai\/ar\/lm-studio-complete-guide-2026\/\">LM Studio<\/a> \u0647\u064a \u0627\u0644\u062e\u064a\u0627\u0631 \u0627\u0644\u0634\u0627\u0626\u0639 \u0639\u0644\u0649 \u0623\u062c\u0647\u0632\u0629 Mac. \u0641\u0647\u064a \u062a\u0642\u0648\u0645 \u062a\u0644\u0642\u0627\u0626\u064a\u064b\u0651\u0627 \u0628\u062a\u0646\u0632\u064a\u0644 \u0645\u0644\u0641\u0627\u062a GGUF \u0645\u0646 Hugging Face \u0648\u062a\u0642\u062f\u0650\u0651\u0645 \u062e\u0627\u062f\u0645\u064b\u0627 \u0645\u062d\u0644\u064a\u064b\u0651\u0627 \u0645\u062a\u0648\u0627\u0641\u0642\u064b\u0627 \u0645\u0639 \u0648\u0627\u062c\u0647\u0629 \u0628\u0631\u0645\u062c\u0629 \u062a\u0637\u0628\u064a\u0642\u0627\u062a OpenAI.<\/p>\n<h3>\u0648\u064a\u0646\u062f\u0648\u0632<\/h3>\n<p>\u0648\u0647\u0646\u0627\u0643 \u062b\u0644\u0627\u062b \u0637\u0631\u0642 \u0639\u0645\u0644\u064a\u0629:<\/p>\n<ul>\n<li><strong>Ollama.<\/strong> \u062b\u0628\u0650\u0651\u062a\u0647 \u0645\u0646 <a href=\"https:\/\/ollama.com\/download\" rel=\"noopener\" target=\"_blank\">ollama.com\/download<\/a>\u060c \u062b\u0645 <code>ollama run llama3.1:8b<\/code>. \u0648\u064a\u0642\u0648\u0645 Ollama \u062a\u0644\u0642\u0627\u0626\u064a\u064b\u0651\u0627 \u0628\u062a\u0646\u0632\u064a\u0644 \u0645\u0644\u0641\u0627\u062a GGUF \u0627\u0644\u062e\u0627\u0635\u0629 \u0628\u0647 \u0648\u0627\u0644\u0645\u064f\u062e\u062a\u0627\u0631\u0629 \u0628\u0639\u0646\u0627\u064a\u0629. \u0648\u0644\u062a\u062d\u0645\u064a\u0644 \u0645\u0644\u0641 \u0639\u0634\u0648\u0627\u0626\u064a\u060c \u0627\u0643\u062a\u0628 \u0645\u0644\u0641 \u0646\u0645\u0648\u0630\u062c (Modelfile) \u064a\u062d\u062a\u0648\u064a \u0639\u0644\u0649 \u0633\u0637\u0631 \u0645\u062b\u0644: <code>FROM .\/mymodel.gguf<\/code> \u062b\u0645 \u0634\u063a\u0650\u0651\u0644 \u0627\u0644\u0623\u0645\u0631: <code>ollama create mymodel -f Modelfile<\/code>. \u0631\u0627\u062c\u0639 \u062f\u0644\u064a\u0644\u0646\u0627 \u0627\u0644\u062e\u0627\u0635 \u0628\u0640 <a href=\"https:\/\/convly.ai\/ar\/how-to-install-ollama-2026\/\">\u062f\u0644\u064a\u0644 \u062a\u062b\u0628\u064a\u062a Ollama<\/a>.<\/li>\n<li><strong>LM Studio.<\/strong> \u062a\u062b\u0628\u064a\u062a \u0628\u0646\u0642\u0631\u0629 \u0648\u0627\u062d\u062f\u0629\u060c \u0648\u0627\u0633\u062a\u0639\u0631\u0627\u0636 \u0648\u062a\u0646\u0632\u064a\u0644 \u0645\u0644\u0641\u0627\u062a GGUF \u0639\u0628\u0631 \u0648\u0627\u062c\u0647\u0629 \u0645\u0633\u062a\u062e\u062f\u0645 \u0631\u0633\u0648\u0645\u064a\u0629\u060c \u0645\u0639 \u0634\u0631\u064a\u0637 \u062a\u0645\u0631\u064a\u0631 \u0644\u062a\u0641\u0639\u064a\u0644 \u0623\u0648 \u062a\u0639\u0637\u064a\u0644 \u062a\u0633\u0631\u064a\u0639 GPU.<\/li>\n<li><strong>\u0627\u0644\u062d\u0632\u0645 \u0627\u0644\u062c\u0627\u0647\u0632\u0629 \u0627\u0644\u0645\u064f\u0633\u0628\u0642\u0629 \u0627\u0644\u062a\u062c\u0645\u064a\u0639 \u0645\u0646 llama.cpp.<\/strong> \u064a\u0648\u0641\u0650\u0651\u0631 \u0627\u0644\u0645\u0634\u0631\u0648\u0639 \u0645\u0644\u0641\u0627\u062a \u0645\u0636\u063a\u0648\u0637\u0629 \u062c\u0627\u0647\u0632\u0629 \u0644\u0623\u0646\u0638\u0645\u0629 Windows \u0645\u062e\u0635\u0635\u0629 \u0644\u0648\u062d\u062f\u0627\u062a \u0627\u0644\u0645\u0639\u0627\u0644\u062c\u0629 \u0627\u0644\u0645\u0631\u0643\u0632\u064a\u0629 (CPU) \u0648CUDA \u0648Vulkan \u0639\u0644\u0649 \u0635\u0641\u062d\u0629 <a href=\"https:\/\/github.com\/ggml-org\/llama.cpp\/releases\" rel=\"noopener\" target=\"_blank\">\u0625\u0635\u062f\u0627\u0631\u0627\u062a GitHub<\/a>. \u0648\u0642\u0645 \u0628\u0627\u0633\u062a\u062e\u0631\u0627\u062c \u0627\u0644\u0645\u0644\u0641 \u0627\u0644\u0645\u0636\u063a\u0648\u0637 \u062b\u0645 \u0634\u063a\u0650\u0651\u0644 <code>llama-cli.exe<\/code> \u0628\u0627\u0644\u0637\u0631\u064a\u0642\u0629 \u0646\u0641\u0633\u0647\u0627 \u0627\u0644\u0645\u062a\u0628\u0639\u0629 \u0641\u064a \u0623\u0646\u0638\u0645\u0629 Linux.<\/li>\n<\/ul>\n<p>\u0648\u0628\u062e\u0635\u0648\u0635 \u0627\u062e\u062a\u064a\u0627\u0631 \u0648\u062d\u062f\u0629 \u0645\u0639\u0627\u0644\u062c\u0629 \u0627\u0644\u0631\u0633\u0648\u0645\u0627\u062a (GPU)\u060c \u0641\u0625\u0646 <a href=\"https:\/\/convly.ai\/ar\/best-gpus-for-local-llms-2026\/\">\u062f\u0644\u064a\u0644 GPU \u0627\u0644\u0645\u062d\u0644\u064a \u0644\u0644\u0646\u0645\u0627\u0630\u062c \u0627\u0644\u0644\u063a\u0648\u064a\u0629 \u0627\u0644\u0643\u0628\u064a\u0631\u0629 (LLM)<\/a> \u064a\u063a\u0637\u064a \u0627\u0644\u062e\u064a\u0627\u0631\u0627\u062a \u0627\u0644\u062d\u0627\u0644\u064a\u0629 \u0645\u0646 \u062d\u064a\u062b \u0627\u0644\u0633\u0639\u0631 \u0645\u0642\u0627\u0628\u0644 \u0633\u0639\u0629 VRAM.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"GGUF_vs_Safetensors_vs_AWQ_vs_GPTQ\"><\/span>GGUF \u0645\u0642\u0627\u0628\u0644 Safetensors \u0645\u0642\u0627\u0628\u0644 AWQ \u0645\u0642\u0627\u0628\u0644 GPTQ<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<table>\n<thead>\n<tr>\n<th>\u0627\u0644\u0635\u064a\u063a\u0629<\/th>\n<th>\u0628\u064a\u0626\u0629 \u0627\u0644\u062a\u0634\u063a\u064a\u0644 \u0627\u0644\u0623\u0633\u0627\u0633\u064a\u0629<\/th>\n<th>\u0627\u0644\u062f\u0642\u0629<\/th>\n<th>\u0627\u0644\u0623\u0641\u0636\u0644 \u0644\u0640<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>GGUF<\/td>\n<td>llama.cpp\u060c Ollama\u060c LM Studio<\/td>\n<td>\u0645\u0646 \u0662 \u0625\u0644\u0649 \u0668 \u0628\u062a + F16<\/td>\n<td>\u0648\u062d\u062f\u0629 \u0645\u0639\u0627\u0644\u062c\u0629 \u0645\u0631\u0643\u0632\u064a\u0629 (CPU)\u060c \u0623\u0648 \u0645\u0632\u064a\u062c \u0645\u0646 CPU\/GPU\u060c \u0623\u0648 Apple Silicon\u060c \u0623\u0648 \u0627\u0633\u062a\u062e\u062f\u0627\u0645 \u0641\u0631\u062f\u064a<\/td>\n<\/tr>\n<tr>\n<td>Safetensors (FP16\/BF16)<\/td>\n<td>Transformers\u060c vLLM<\/td>\n<td>\u0661\u0666 \u0628\u062a<\/td>\n<td>\u0627\u0644\u062a\u062f\u0631\u064a\u0628\u060c \u0648\u0627\u0644\u0628\u062d\u062b\u060c \u0648\u0627\u0644\u0627\u0633\u062a\u0646\u062a\u0627\u062c \u0628\u062f\u0642\u0629 \u0643\u0627\u0645\u0644\u0629<\/td>\n<\/tr>\n<tr>\n<td>AWQ<\/td>\n<td>vLLM\u060c AutoAWQ<\/td>\n<td>\u0664 \u0628\u062a<\/td>\n<td>\u0627\u0644\u0627\u0633\u062a\u062e\u062f\u0627\u0645 \u0639\u0627\u0644\u064a \u0627\u0644\u0625\u0646\u062a\u0627\u062c\u064a\u0629 \u0639\u0644\u0649 \u0648\u062d\u062f\u0627\u062a \u0645\u0639\u0627\u0644\u062c\u0629 \u0627\u0644\u0631\u0633\u0648\u0645\u0627\u062a (GPU)<\/td>\n<\/tr>\n<tr>\n<td>GPTQ<\/td>\n<td>vLLM\u060c ExLlamaV2<\/td>\n<td>\u0645\u0646 \u0663 \u0625\u0644\u0649 \u0668 \u0628\u062a<\/td>\n<td>\u0627\u0644\u0627\u0633\u062a\u0646\u062a\u0627\u062c \u0639\u0644\u0649 \u0648\u062d\u062f\u0629 \u0645\u0639\u0627\u0644\u062c\u0629 \u0627\u0644\u0631\u0633\u0648\u0645\u0627\u062a \u0641\u0642\u0637 \u0645\u0639 \u062f\u0639\u0645 \u0627\u0644\u062a\u062c\u0645\u064a\u0639 (batching)<\/td>\n<\/tr>\n<tr>\n<td>MLX<\/td>\n<td>MLX (Apple)<\/td>\n<td>4\u201316 \u0628\u062a<\/td>\n<td>\u0645\u062e\u0635\u0635 \u0644\u0631\u0642\u0627\u0642\u0627\u062a Apple Silicon \u0641\u0642\u0637<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>\u0627\u062e\u062a\u0631 \u062a\u0646\u0633\u064a\u0642 GGUF \u0639\u0646\u062f \u0627\u0633\u062a\u0647\u062f\u0627\u0641 \u0645\u0633\u062a\u062e\u062f\u0645 \u0648\u0627\u062d\u062f \u0639\u0644\u0649 \u0623\u062c\u0647\u0632\u0629 \u0645\u062a\u0646\u0648\u0639\u0629 \u0623\u0648 \u0639\u0646\u062f\u0645\u0627 \u062a\u0631\u063a\u0628 \u0641\u064a \u062a\u0634\u063a\u064a\u0644 \u0627\u0644\u0646\u0645\u0648\u0630\u062c \u0639\u0644\u0649 \u0648\u062d\u062f\u0629 \u0645\u0639\u0627\u0644\u062c\u0629 \u0645\u0631\u0643\u0632\u064a\u0629 (CPU) \u0628\u0623\u064a \u0634\u0643\u0644\u064d \u0645\u0646 \u0627\u0644\u0623\u0634\u0643\u0627\u0644. \u0623\u0645\u0627 \u0639\u0646\u062f \u062e\u062f\u0645\u0629 \u0639\u062f\u062f \u0643\u0628\u064a\u0631 \u0645\u0646 \u0627\u0644\u0645\u0633\u062a\u062e\u062f\u0645\u064a\u0646 \u0627\u0644\u0645\u062a\u0632\u0627\u0645\u0646\u064a\u0646 \u0639\u0644\u0649 \u0648\u062d\u062f\u0629 \u0645\u0639\u0627\u0644\u062c\u0629 \u0631\u0633\u0648\u0645\u064a\u0629 (GPU) \u0641\u064a \u0645\u0631\u0643\u0632 \u0628\u064a\u0627\u0646\u0627\u062a\u060c \u0641\u0627\u062e\u062a\u0631 vLLM \u0645\u0639 \u0645\u0644\u0641\u0627\u062a safetensors \u0627\u0644\u0645\u064f\u0643\u0645\u064e\u0651\u0646\u0629 \u0628\u0640 AWQ \u0623\u0648 GPTQ.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"When_GGUF_Is_Not_the_Right_Answer\"><\/span>\u0645\u062a\u0649 \u0644\u0627 \u062a\u0643\u0648\u0646 \u0635\u064a\u063a\u0629 GGUF \u0647\u064a \u0627\u0644\u062d\u0644 \u0627\u0644\u0645\u0646\u0627\u0633\u0628<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>\u064a\u0641\u062a\u0631\u0636 GGUF \u0627\u0633\u062a\u0646\u062a\u0627\u062c\u064b\u0627 \u0623\u062d\u0627\u062f\u064a \u0627\u0644\u062a\u062f\u0641\u0642 \u0648\u0628\u062d\u062c\u0645 \u062f\u0641\u0639\u0629 \u064a\u0633\u0627\u0648\u064a 1. \u0648\u0628\u0627\u0644\u062a\u0627\u0644\u064a \u0641\u0625\u0646 \u0645\u0639\u062f\u0644 \u0627\u0644\u0625\u0646\u062a\u0627\u062c\u064a\u0629 (throughput) \u064a\u0643\u0648\u0646 \u0623\u0642\u0644 \u0628\u0643\u062b\u064a\u0631 \u0645\u0642\u0627\u0631\u0646\u0629\u064b \u0628\u0640 vLLM \u0623\u0648 TensorRT-LLM \u062a\u062d\u062a \u062d\u0645\u0644 \u0645\u062a\u0632\u0627\u0645\u0646\u060c \u0643\u0645\u0627 \u0623\u0646\u0647 \u0644\u0627 \u064a\u062f\u0639\u0645 \u0622\u0644\u064a\u0629 \u0627\u0644\u0627\u0646\u062a\u0628\u0627\u0647 \u0627\u0644\u0645\u064f\u0641\u0635\u064e\u0651\u0644\u0629 (paged attention). \u0641\u0625\u0630\u0627 \u0643\u0646\u062a \u062a\u064f\u0634\u063a\u0650\u0651\u0644 \u0648\u0627\u062c\u0647\u0629 \u0628\u0631\u0645\u062c\u064a\u0629 (API) \u0644\u062a\u0644\u0628\u064a\u0629 \u0637\u0644\u0628\u0627\u062a \u062d\u0642\u064a\u0642\u064a\u0629\u060c \u0641\u0625\u0646 \u0646\u0634\u0631 \u0627\u0644\u0646\u0645\u0648\u0630\u062c \u0628\u062a\u0646\u0633\u064a\u0642 FP16 \u0623\u0648 AWQ \u0639\u0644\u0649 vLLM \u0633\u064a\u062a\u0641\u0648\u0642 \u0639\u0644\u0649 GGUF \u0645\u0646 \u062d\u064a\u062b \u0639\u062f\u062f \u0627\u0644\u0631\u0645\u0648\u0632 \u0627\u0644\u0645\u064f\u0648\u0644\u064e\u0651\u062f\u0629 \u0644\u0643\u0644 \u062b\u0627\u0646\u064a\u0629 \u0648\u0644\u0643\u0644 \u062f\u0648\u0644\u0627\u0631\u060c \u062d\u062a\u0649 \u0642\u0628\u0644 \u0623\u0646 \u062a\u064f\u062d\u0633\u0628 \u0648\u0642\u062a \u0627\u0644\u0645\u0637\u0648\u0631 \u0627\u0644\u0636\u0627\u0626\u0639.<\/p>\n<p>It also breaks down at the top end of model size. Kimi K3 needs ~1.4&nbsp;TB of VRAM at 4-bit and DeepSeek V4-Pro around 800&nbsp;GB &mdash; those are multi-node deployments, not desktop GGUF workloads. And for frontier hosted models like <a href=\"https:\/\/convly.ai\/ar\/models\/\">Claude Sonnet 4.6<\/a> \u0627\u0644\u062a\u064a \u062a\u0628\u0644\u063a \u062a\u0643\u0644\u0641\u0629 \u0627\u0644\u0645\u0639\u0627\u0644\u062c\u0629 \u0627\u0644\u062f\u0627\u062e\u0644\u0629 3 \u062f\u0648\u0644\u0627\u0631\u0627\u062a \u0648\u0627\u0644\u062e\u0627\u0631\u062c\u0629 15 \u062f\u0648\u0644\u0627\u0631\u064b\u0627 \u0644\u0643\u0644 \u0645\u0644\u064a\u0648\u0646 \u0631\u0645\u0632\u060c \u0623\u0648 Gemini 3.6 Flash \u0627\u0644\u062a\u064a \u062a\u0628\u0644\u063a \u062a\u0643\u0644\u0641\u062a\u0647\u0627 1.50 \u062f\u0648\u0644\u0627\u0631 \u062f\u0627\u062e\u0644\u0627\u064b \u06487.50 \u062f\u0648\u0644\u0627\u0631\u0627\u062a \u062e\u0627\u0631\u062c\u0627\u064b\u060c \u0641\u0644\u0627 \u062a\u0648\u062c\u062f \u0623\u0648\u0632\u0627\u0646 \u0645\u062d\u0644\u064a\u0629 (local weights) \u0642\u0627\u0628\u0644\u0629 \u0644\u0644\u062a\u062d\u0648\u064a\u0644 \u0623\u0635\u0644\u064b\u0627. \u0644\u0630\u0627 \u0627\u062d\u0633\u0628 \u0627\u0644\u0623\u0631\u0642\u0627\u0645 \u0628\u062f\u0642\u0629 \u0639\u0644\u0649 <a href=\"https:\/\/convly.ai\/ar\/self-hosting-vs-api-calculator\/\">\u062d\u0627\u0633\u0628\u0629 \u0627\u0644\u0645\u0642\u0627\u0631\u0646\u0629 \u0628\u064a\u0646 \u0627\u0644\u0627\u0633\u062a\u0636\u0627\u0641\u0629 \u0627\u0644\u0630\u0627\u062a\u064a\u0629 \u0648\u0648\u0627\u062c\u0647\u0629 \u0628\u0631\u0645\u062c\u0629 \u0627\u0644\u062a\u0637\u0628\u064a\u0642\u0627\u062a (API)<\/a> \u0642\u0628\u0644 \u0627\u0644\u0627\u0644\u062a\u0632\u0627\u0645 \u0628\u0634\u0631\u0627\u0621 \u0648\u062d\u062f\u0629 \u0645\u0639\u0627\u0644\u062c\u0629 \u0631\u0633\u0648\u0645\u064a\u0629.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"Converting_a_Model_to_GGUF\"><\/span>\u062a\u062d\u0648\u064a\u0644 \u0646\u0645\u0648\u0630\u062c \u0625\u0644\u0649 \u0635\u064a\u063a\u0629 GGUF<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>\u0625\u0630\u0627 \u0648\u064f\u062c\u062f \u0646\u0645\u0648\u0630\u062c \u0645\u0627 \u0639\u0644\u0649 Hugging Face \u0628\u0635\u064a\u063a\u0629 safetensors \u0644\u0643\u0646 \u0644\u0645 \u064a\u064f\u0643\u0645\u064e\u0651\u0646 \u0628\u0639\u062f\u064f \u0645\u0646 \u0642\u0650\u0628\u0644 \u0623\u062d\u062f\u060c \u0641\u0625\u0646 \u0633\u064a\u0631 \u0627\u0644\u0639\u0645\u0644 \u064a\u0643\u0648\u0646 \u0643\u0627\u0644\u062a\u0627\u0644\u064a:<\/p>\n<pre><code># \u0641\u064a \u0645\u0633\u062a\u0648\u062f\u0639 llama.cpp\npip install -r requirements.txt\npython convert_hf_to_gguf.py \/path\/to\/hf-model --outfile model-f16.gguf\n.\/build\/bin\/llama-quantize model-f16.gguf model-Q4_K_M.gguf Q4_K_M<\/code><\/pre>\n<p>\u064a\u062f\u0639\u0645 \u0646\u0635 \u0627\u0644\u062a\u062d\u0648\u064a\u0644 \u0641\u0642\u0637 \u0647\u064a\u0627\u0643\u0644 \u0627\u0644\u0646\u0645\u0627\u0630\u062c \u0627\u0644\u062a\u064a \u0639\u0644\u064e\u0651\u0645\u062a\u0647\u0627 llama.cpp \u2014 \u0645\u062b\u0644 llama \u0648mistral \u0648qwen2\/3 \u0648gemma \u0648phi\u060c \u0625\u0636\u0627\u0641\u0629\u064b \u0625\u0644\u0649 \u0642\u0627\u0626\u0645\u0629 \u0645\u062a\u0632\u0627\u064a\u062f\u0629 \u0645\u0646 \u0627\u0644\u0647\u064a\u0627\u0643\u0644 \u0627\u0644\u0623\u062e\u0631\u0649. \u0623\u0645\u0627 \u0627\u0644\u0647\u064a\u0627\u0643\u0644 \u0627\u0644\u062c\u062f\u064a\u062f\u0629 (novel architectures) \u0641\u062a\u062a\u0637\u0644\u0628 \u062a\u0639\u062f\u064a\u0644\u064b\u0627 \u0641\u064a \u0627\u0644\u0643\u0648\u062f \u0623\u0648\u0644\u064b\u0627. \u0648\u0642\u062f \u062a\u063a\u064a\u064e\u0651\u0631\u062a \u0623\u0633\u0645\u0627\u0621 \u0627\u0644\u0645\u0644\u0641\u0627\u062a \u0648\u0627\u0644\u0639\u0644\u0645\u0627\u062a (flags) \u0641\u064a \u0627\u0644\u0646\u0635 \u0628\u064a\u0646 \u0625\u0635\u062f\u0627\u0631\u0627\u062a llama.cpp\u060c \u0644\u0630\u0627 \u062a\u0623\u0643\u064e\u0651\u062f \u0645\u0646 \u0645\u0631\u0627\u062c\u0639\u0629 <code>python convert_hf_to_gguf.py --help<\/code> \u0628\u0627\u0644\u0646\u0633\u0628\u0629 \u0644\u0644\u0625\u0635\u062f\u0627\u0631 \u0627\u0644\u0630\u064a \u0642\u0645\u062a \u0628\u0628\u0646\u0627\u0626\u0647 \u0645\u0646 \u0627\u0644\u0645\u0635\u062f\u0631.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"Frequently_Asked_Questions\"><\/span>\u0627\u0644\u0623\u0633\u0626\u0644\u0629 \u0627\u0644\u0634\u0627\u0626\u0639\u0629<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<h3>\u0647\u0644 \u062a\u064f\u0639\u062f\u0651 \u0635\u064a\u063a\u0629 GGUF \u0645\u0645\u0627\u062b\u0644\u0629 \u0644\u0640 GGML\u061f<\/h3>\n<p>\u0643\u0644\u0627. \u0643\u0627\u0646 GGML \u0627\u0644\u062a\u0646\u0633\u064a\u0642 \u0627\u0644\u0623\u0642\u062f\u0645 \u0627\u0644\u0630\u064a \u0627\u0633\u062a\u062e\u062f\u0645\u062a\u0647 llama.cpp \u062d\u062a\u0649 \u0645\u0646\u062a\u0635\u0641 \u0639\u0627\u0645 2023. \u062b\u0645 \u062d\u0644\u064e\u0651 \u0645\u062d\u0644\u0647 GGUF \u0641\u064a \u0623\u063a\u0633\u0637\u0633 2023\u060c \u0644\u064a\u0636\u0645 \u0631\u0623\u0633\u064b\u0627 \u0645\u064f\u0631\u0642\u064e\u0651\u0645 \u0627\u0644\u0625\u0635\u062f\u0627\u0631 (versioned header)\u060c \u0648\u0628\u064a\u0627\u0646\u0627\u062a \u0648\u0635\u0641\u064a\u0629 \u0645\u0636\u0645\u0646\u0629 (embedded metadata)\u060c \u0648\u062a\u062e\u0637\u064a\u0637\u064b\u0627 \u062b\u0627\u0628\u062a\u064b\u0627 \u0644\u0644\u0645\u0648\u062a\u0631\u0627\u062a (stable tensor layout). \u0648\u0644\u0627 \u062a\u064f\u062d\u0645\u064e\u0651\u0644 \u0645\u0644\u0641\u0627\u062a GGML \u0627\u0644\u0642\u062f\u064a\u0645\u0629 \u0628\u0639\u062f \u0627\u0644\u0622\u0646 \u0641\u064a \u0623\u062d\u062f\u062b \u0625\u0635\u062f\u0627\u0631\u0627\u062a llama.cpp\u061b \u0628\u0644 \u064a\u062c\u0628 \u0639\u0644\u064a\u0643 \u0627\u0644\u0639\u062b\u0648\u0631 \u0639\u0644\u0649 \u0646\u0633\u062e\u0629 \u0645\u064f\u0643\u0645\u064e\u0651\u0646\u0629 \u0645\u062c\u062f\u062f\u064b\u0627 \u0628\u0635\u064a\u063a\u0629 GGUF. <code>\u0623\u0648 .bin<\/code> \u0648\u0644\u0627 \u062a\u064f\u062d\u0645\u064e\u0651\u0644 \u0645\u0644\u0641\u0627\u062a GGML \u0627\u0644\u0642\u062f\u064a\u0645\u0629 \u0628\u0639\u062f \u0627\u0644\u0622\u0646 \u0641\u064a \u0623\u062d\u062f\u062b \u0625\u0635\u062f\u0627\u0631\u0627\u062a llama.cpp\u061b \u0628\u0644 \u064a\u062c\u0628 \u0639\u0644\u064a\u0643 \u0627\u0644\u0639\u062b\u0648\u0631 \u0639\u0644\u0649 \u0646\u0633\u062e\u0629 \u0645\u064f\u0643\u0645\u064e\u0651\u0646\u0629 \u0645\u062c\u062f\u062f\u064b\u0627 \u0628\u0635\u064a\u063a\u0629 GGUF.<\/p>\n<h3>\u0623\u064a \u062a\u0643\u0645\u064a\u0646 \u064a\u062c\u0628 \u0623\u0646 \u0623\u062d\u0645\u0651\u0644\u0647\u061f<\/h3>\n<p>\u0627\u0628\u062f\u0623 \u0628\u0640 <code>Q4_K_M<\/code>\u0648\u0647\u0648 \u064a\u0646\u0627\u0633\u0628 \u0623\u0648\u0633\u0639 \u0646\u0637\u0627\u0642 \u0645\u0645\u0643\u0646 \u0645\u0646 \u0627\u0644\u0623\u062c\u0647\u0632\u0629\u060c \u0643\u0645\u0627 \u0623\u0646 \u0641\u0642\u062f\u0627\u0646 \u0627\u0644\u062c\u0648\u062f\u0629 \u0645\u0642\u0627\u0631\u0646\u0629\u064b \u0628\u0640 FP16 \u0636\u0626\u064a\u0644\u064c \u0628\u0627\u0644\u0646\u0633\u0628\u0629 \u0644\u0645\u0639\u0638\u0645 \u0627\u0644\u0646\u0645\u0627\u0630\u062c \u0627\u0644\u062a\u064a \u062a\u062a\u062c\u0627\u0648\u0632 7 \u0645\u0644\u064a\u0627\u0631 \u0645\u0639\u0644\u064e\u0651\u0645\u0629. \u0648\u0625\u0630\u0627 \u062a\u0648\u0641\u0631 \u0644\u062f\u064a\u0643 \u0645\u0633\u0627\u062d\u0629 \u0625\u0636\u0627\u0641\u064a\u0629 \u0641\u064a \u0630\u0627\u0643\u0631\u0629 VRAM \u0648\u0627\u0647\u062a\u0645\u0645\u062a \u0628\u062f\u0642\u0629 \u0627\u0644\u062a\u0631\u0645\u064a\u0632 \u0623\u0648 \u0627\u0644\u0627\u0633\u062a\u0646\u062a\u0627\u062c \u0627\u0644\u0645\u0646\u0637\u0642\u064a\u060c \u0641\u0627\u0646\u062a\u0642\u0644 \u0625\u0644\u0649 <code>Q5_K_M<\/code> \u0623\u0648 <code>Q6_K<\/code> \u0623\u0645\u0627 \u0625\u0630\u0627 \u0644\u0645 \u064a\u062a\u0646\u0627\u0633\u0628 \u0627\u0644\u0646\u0645\u0648\u0630\u062c \u0645\u0639 \u0627\u0644\u062c\u0647\u0627\u0632 \u0625\u0644\u0627 \u0639\u0646\u062f \u062e\u0641\u0636 \u0627\u0644\u062a\u0643\u0645\u064a\u0646 \u0623\u0643\u062b\u0631\u060c \u0641\u0627\u0646\u0632\u0644 \u0625\u0644\u0649 <code>Q3_K_M<\/code> \u0623\u0648 \u0625\u0644\u0649 \u0623\u062d\u062f \u062a\u0646\u0633\u064a\u0642\u0627\u062a <code>IQ3<\/code> \u0641\u0642\u0637 \u0639\u0646\u062f\u0645\u0627 \u0644\u0627 \u064a\u062a\u0646\u0627\u0633\u0628 \u0627\u0644\u0646\u0645\u0648\u0630\u062c \u0628\u0623\u064a \u0635\u064a\u063a\u0629 \u0623\u062e\u0631\u0649.<\/p>\n<h3>Can GGUF models use my GPU?<\/h3>\n<p>\u0646\u0639\u0645. \u064a\u062f\u0639\u0645 llama.cpp \u0648\u0627\u062c\u0647\u0629 CUDA (NVIDIA) \u0648ROCm (AMD) \u0648Metal (Apple) \u0648Vulkan (\u0645\u062a\u0639\u062f\u062f \u0627\u0644\u0645\u0648\u0631\u062f\u064a\u0646) \u0648SYCL (Intel). <code>(\u0639\u062f\u062f \u0637\u0628\u0642\u0627\u062a \u0648\u062d\u062f\u0629 \u0645\u0639\u0627\u0644\u062c\u0629 \u0627\u0644\u0631\u0633\u0648\u0645\u0627\u062a).<\/code> \u064a\u064f\u0641\u0639\u0650\u0651\u0644 \u0647\u0630\u0627 \u0627\u0644\u0639\u0644\u0645 \u0646\u0642\u0644 \u0627\u0644\u0637\u0628\u0642\u0627\u062a \u0625\u0644\u0649 \u0648\u062d\u062f\u0629 \u0645\u0639\u0627\u0644\u062c\u0629 \u0627\u0644\u0631\u0633\u0648\u0645\u0627\u062a (GPU). \u0648\u0625\u0630\u0627 \u0627\u0633\u062a\u0648\u0639\u0628\u062a \u0627\u0644\u0630\u0627\u0643\u0631\u0629 \u0627\u0644\u0639\u0634\u0648\u0627\u0626\u064a\u0629 \u0644\u0648\u062d\u062f\u0629 \u0645\u0639\u0627\u0644\u062c\u0629 \u0627\u0644\u0631\u0633\u0648\u0645\u0627\u062a (VRAM) \u0627\u0644\u0646\u0645\u0648\u0630\u062c \u0628\u0627\u0644\u0643\u0627\u0645\u0644\u060c \u0641\u0625\u0646 \u0645\u0639\u062f\u0644 \u0627\u0644\u0625\u0646\u062a\u0627\u062c\u064a\u0629 \u064a\u0642\u062a\u0631\u0628 \u0645\u0646 \u0645\u0639\u062f\u0644 \u0627\u0644\u0625\u0646\u062a\u0627\u062c\u064a\u0629 \u0627\u0644\u0645\u062d\u0642\u064e\u0651\u0642 \u0639\u0646\u062f \u0627\u0644\u062a\u0634\u063a\u064a\u0644 \u0639\u0644\u0649 \u0648\u062d\u062f\u0627\u062a \u0645\u0639\u0627\u0644\u062c\u0629 \u0631\u0633\u0648\u0645\u0627\u062a \u0645\u062e\u0635\u0635\u0629\u061b \u0623\u0645\u0627 \u0641\u064a \u062d\u0627\u0644\u0629 \u0627\u0644\u0646\u0642\u0644 \u0627\u0644\u062c\u0632\u0626\u064a \u0644\u0644\u0637\u0628\u0642\u0627\u062a\u060c \u0641\u064a\u062a\u062d\u062f\u062f \u0645\u0639\u062f\u0644 \u0627\u0644\u062a\u0634\u063a\u064a\u0644 \u0648\u0641\u0642\u064b\u0627 \u0644\u0623\u0628\u0637\u0623 \u0637\u0628\u0642\u0629\u064d \u0645\u0648\u062c\u0648\u062f\u0629\u064d \u0641\u064a \u0623\u064a \u0645\u0633\u062a\u0648\u0649 \u0645\u0646 \u0627\u0644\u0645\u0633\u062a\u0648\u064a\u0627\u062a.<\/p>\n<h3>\u0647\u0644 \u062a\u0639\u0645\u0644 \u0635\u064a\u063a\u0629 GGUF \u0645\u0639 \u0646\u0645\u0627\u0630\u062c \u0627\u0644\u0631\u0624\u064a\u0629 \u0623\u0648 \u0627\u0644\u0635\u0648\u062a\u061f<\/h3>\n<p>\u062c\u0632\u0626\u064a\u064b\u0627. \u062a\u062f\u0639\u0645 llama.cpp \u0646\u0645\u0627\u0630\u062c \u0627\u0644\u0631\u0624\u064a\u0629 \u0645\u062a\u0639\u062f\u062f\u0629 \u0627\u0644\u0648\u0633\u0627\u0626\u0637 \u0639\u0644\u0649 \u063a\u0631\u0627\u0631 LLaVA \u0639\u0628\u0631 \u0645\u0644\u0641 <code>mmproj<\/code> GGUF \u0645\u064f\u0631\u0641\u0642 \u064a\u062d\u062a\u0648\u064a \u0639\u0644\u0649 \u0645\u0634\u0631\u0648\u0639 \u0627\u0644\u0631\u0624\u064a\u0629 (vision projector). \u0648\u0642\u062f \u0623\u064f\u0636\u064a\u0641 \u0627\u0644\u062f\u0639\u0645 \u062a\u062f\u0631\u064a\u062c\u064a\u064b\u0651\u0627 \u0644\u0646\u0645\u0627\u0630\u062c \u0645\u062b\u0644 Qwen-VL \u0648Gemma 3 vision \u0648\u063a\u064a\u0631\u0647\u0627\u060c \u0644\u0643\u0646\u0647 \u064a\u062a\u0623\u062e\u0631 \u0639\u0646 \u0625\u0635\u062f\u0627\u0631\u0627\u062a \u0627\u0644\u0646\u0635\u0648\u0635 \u0641\u0642\u0637. \u0623\u0645\u0627 \u0646\u0645\u0627\u0630\u062c \u0627\u0644\u0635\u0648\u062a \u0645\u062b\u0644 Whisper \u0641\u0647\u064a \u062a\u0633\u062a\u062e\u062f\u0645 \u062a\u0646\u0633\u064a\u0642\u064b\u0627 \u0645\u0631\u062a\u0628\u0637\u064b\u0627 \u062e\u0627\u0635\u064b\u0627 \u0628\u0647\u0627 \u064a\u064f\u062f\u0627\u0631 \u0628\u0648\u0627\u0633\u0637\u0629 <code>whisper.cpp<\/code>\u060c \u0648\u0644\u064a\u0633 \u0628\u0648\u0627\u0633\u0637\u0629 \u0627\u0644\u0628\u0631\u0646\u0627\u0645\u062c \u0627\u0644\u062a\u0646\u0641\u064a\u0630\u064a \u0627\u0644\u0631\u0626\u064a\u0633\u064a \u0644\u0640 llama.cpp.<\/p>\n<h3>\u0643\u064a\u0641 \u0623\u062e\u062a\u0627\u0631 \u0646\u0645\u0648\u0630\u062c GGUF \u064a\u062a\u0648\u0627\u0641\u0642 \u0645\u0639 \u0623\u062c\u0647\u0632\u062a\u064a\u061f<\/h3>\n<p>\u0627\u0628\u062d\u062b \u0639\u0646 \u0627\u0644\u0646\u0645\u0648\u0630\u062c \u0641\u064a <a href=\"https:\/\/convly.ai\/ar\/models\/\">Convly<\/a> \u0644\u0644\u0645\u0639\u0631\u0641\u0629 \u0627\u0644\u0645\u0637\u0644\u0648\u0628\u0629 \u0645\u0646 \u0630\u0627\u0643\u0631\u0629 VRAM \u0639\u0646\u062f \u0627\u0644\u062a\u0643\u0645\u064a\u0646 \u0628\u0640 4 \u0628\u062a\u060c \u062b\u0645 \u0627\u0637\u0631\u062d 1\u20133 \u062c\u064a\u062c\u0627\u0628\u0627\u064a\u062a \u0643\u0645\u0633\u0627\u062d\u0629 \u0627\u062d\u062a\u064a\u0627\u0637\u064a\u0629 (headroom) \u0644\u0644\u0633\u064a\u0627\u0642 \u0648\u062a\u0643\u0627\u0644\u064a\u0641 \u0646\u0638\u0627\u0645 \u0627\u0644\u062a\u0634\u063a\u064a\u0644. \u0641\u0639\u0644\u0649 \u0633\u0628\u064a\u0644 \u0627\u0644\u0645\u062b\u0627\u0644\u060c \u0648\u0628\u0637\u0627\u0642\u0629 \u0633\u0639\u0629 24 \u062c\u064a\u062c\u0627\u0628\u0627\u064a\u062a\u060c \u0641\u0625\u0646 \u0646\u0645\u0648\u0630\u062c Qwen3 32B (\u062d\u0648\u0627\u0644\u064a 20 \u062c\u064a\u062c\u0627\u0628\u0627\u064a\u062a) \u0623\u0648 Gemma 3 27B (\u062d\u0648\u0627\u0644\u064a 16 \u062c\u064a\u062c\u0627\u0628\u0627\u064a\u062a) \u0639\u0646\u062f \u062a\u0643\u0645\u064a\u0646 Q4 \u0633\u064a\u0643\u0648\u0646\u0627\u0646 \u0645\u0646\u0627\u0633\u0628\u064a\u0646 \u062a\u0645\u0627\u0645\u064b\u0627. \u0623\u0645\u0627 <a href=\"https:\/\/convly.ai\/ar\/llm-vram-calculator\/\">\u062d\u0627\u0633\u0628\u0629 \u0627\u0644\u0630\u0627\u0643\u0631\u0629 VRAM<\/a> \u0641\u064a\u062a\u0639\u0627\u0645\u0644 \u0645\u0639 \u0627\u0644\u0633\u064a\u0627\u0642\u0627\u062a \u0627\u0644\u0623\u0637\u0648\u0644\u060c \u0648\u0627\u0644\u062a\u064a \u062a\u0648\u0633\u0651\u0639 \u0630\u0627\u0643\u0631\u0629 KV cache \u0628\u0634\u0643\u0644 \u0645\u0644\u062d\u0648\u0638.<\/p>\n<h3>\u0647\u0644 \u0645\u0644\u0641\u0627\u062a GGUF \u0622\u0645\u0646\u0629 \u0644\u0644\u062a\u0634\u063a\u064a\u0644\u061f<\/h3>\n<p>The weights themselves are just numbers &mdash; unlike Python pickle-based checkpoints, GGUF does not execute code on load. The risk is a maliciously crafted file triggering a parser bug in the runtime. Stick to known Hugging Face uploaders (bartowski, unsloth, lmstudio-community, official vendor accounts) and keep llama.cpp, Ollama or LM Studio updated.<\/p>","protected":false},"excerpt":{"rendered":"<p>GGUF (GPT-Generated Unified Format) is a single-file binary format for quantized large language models, introduced by the llama.cpp project in [\u2026]<\/p>\n","protected":false},"author":1,"featured_media":2609,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"site-sidebar-layout":"default","site-content-layout":"","ast-site-content-layout":"default","site-content-style":"default","site-sidebar-style":"default","ast-global-header-display":"","ast-banner-title-visibility":"","ast-main-header-display":"","ast-hfb-above-header-display":"","ast-hfb-below-header-display":"","ast-hfb-mobile-header-display":"","site-post-title":"","ast-breadcrumbs-content":"","ast-featured-img":"","footer-sml-layout":"","ast-disable-related-posts":"","theme-transparent-header-meta":"","adv-header-id-meta":"","stick-header-meta":"","header-above-stick-meta":"","header-main-stick-meta":"","header-below-stick-meta":"","astra-migrate-meta-layouts":"default","ast-page-background-enabled":"default","ast-page-background-meta":{"desktop":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"ast-content-background-meta":{"desktop":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"footnotes":""},"categories":[7],"tags":[],"class_list":["post-2608","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai-news"],"_links":{"self":[{"href":"https:\/\/convly.ai\/ar\/wp-json\/wp\/v2\/posts\/2608","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/convly.ai\/ar\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/convly.ai\/ar\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/convly.ai\/ar\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/convly.ai\/ar\/wp-json\/wp\/v2\/comments?post=2608"}],"version-history":[{"count":1,"href":"https:\/\/convly.ai\/ar\/wp-json\/wp\/v2\/posts\/2608\/revisions"}],"predecessor-version":[{"id":2610,"href":"https:\/\/convly.ai\/ar\/wp-json\/wp\/v2\/posts\/2608\/revisions\/2610"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/convly.ai\/ar\/wp-json\/wp\/v2\/media\/2609"}],"wp:attachment":[{"href":"https:\/\/convly.ai\/ar\/wp-json\/wp\/v2\/media?parent=2608"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/convly.ai\/ar\/wp-json\/wp\/v2\/categories?post=2608"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/convly.ai\/ar\/wp-json\/wp\/v2\/tags?post=2608"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}