{"id":2605,"date":"2026-09-09T20:14:45","date_gmt":"2026-09-09T20:14:45","guid":{"rendered":"https:\/\/convly.ai\/?p=2605"},"modified":"2026-09-12T17:11:25","modified_gmt":"2026-09-12T17:11:25","slug":"gguf-models","status":"publish","type":"post","link":"https:\/\/convly.ai\/ar\/gguf-models\/","title":{"rendered":"\u0646\u0645\u0627\u0630\u062c GGUF: \u0645\u0627 \u0647\u064a \u0648\u0643\u064a\u0641\u064a\u0629 \u062a\u0634\u063a\u064a\u0644\u0647\u0627"},"content":{"rendered":"<div class=\"convly-tldr\">\n<ul>\n<li><strong>GGUF \u0647\u0648 \u0635\u064a\u063a\u0629 \u062d\u0627\u0648\u064a\u0629 \u0645\u0644\u0641 \u0648\u0627\u062d\u062f \u0644\u0644\u0646\u0645\u0627\u0630\u062c \u0627\u0644\u0645\u0643\u0645\u0645\u0629<\/strong>\u060c \u062a\u0645 \u0625\u0646\u0634\u0627\u0624\u0647\u0627 \u0628\u0648\u0627\u0633\u0637\u0629 <a href=\"https:\/\/github.com\/ggml-org\/llama.cpp\" rel=\"noopener\" target=\"_blank\">llama.cpp<\/a>. \u0645\u0644\u0641 \u0648\u0627\u062d\u062f <code>.gguf<\/code> \u064a\u062d\u062a\u0648\u064a \u0639\u0644\u0649 \u0627\u0644\u0623\u0648\u0632\u0627\u0646 \u0648\u0627\u0644\u0645\u0631\u0645\u0632\u0627\u062a \u0648\u0646\u0645\u0648\u0630\u062c \u0627\u0644\u062f\u0631\u062f\u0634\u0629 \u0648\u0627\u0644\u0628\u064a\u0627\u0646\u0627\u062a \u0627\u0644\u0648\u0635\u0641\u064a\u0629 \u2014 \u0628\u062f\u0648\u0646 \u0645\u062c\u0644\u062f \u0625\u0639\u062f\u0627\u062f\u0627\u062a\u060c \u0628\u062f\u0648\u0646 \u0645\u0644\u0641\u0627\u062a \u0645\u0631\u0645\u0632\u0627\u062a \u0645\u0646\u0641\u0635\u0644\u0629.<\/li>\n<li><strong>GGUF models are what Ollama, LM Studio, KoboldCpp, Jan and llama.cpp actually load.<\/strong> \u0625\u0630\u0627 \u0642\u0645\u062a \u0628\u062a\u0634\u063a\u064a\u0644 \u0646\u0645\u0648\u0630\u062c \u0645\u062d\u0644\u064a\u064b\u0627 \u0639\u0644\u0649 \u0623\u062c\u0647\u0632\u0629 \u0627\u0644\u0645\u0633\u062a\u0647\u0644\u0643\u060c \u0641\u0623\u0646\u062a \u0628\u0643\u0644 \u062a\u0623\u0643\u064a\u062f \u062a\u0634\u063a\u0644 GGUF.<\/li>\n<li><strong>\u0627\u0644\u062e\u064a\u0627\u0631 \u0627\u0644\u0627\u0641\u062a\u0631\u0627\u0636\u064a: <code>Q4_K_M<\/code>.<\/strong> \u062a\u0642\u0631\u064a\u0628\u064b\u0627 0.6 GB \u0645\u0646 \u0627\u0644\u0645\u0644\u0641 \u0644\u0643\u0644 \u0645\u0644\u064a\u0627\u0631 \u0645\u0639\u0627\u0645\u0644 \u2014 \u064a\u0635\u0644 \u0646\u0645\u0648\u0630\u062c 8B \u0625\u0644\u0649 \u062d\u0648\u0627\u0644\u064a 5 GB\u060c \u0648\u0647\u0648 \u0645\u0627 \u064a\u0637\u0627\u0628\u0642 \u0631\u0642\u0645 4-bit \u0628\u062d\u0648\u0627\u0644\u064a 5 GB \u0627\u0644\u0630\u064a <a href=\"https:\/\/convly.ai\/ar\/models\/\">Convly<\/a> \u064a\u0633\u0631\u062f\u0647 Llama 3.1 8B.<\/li>\n<li><strong>GGUF \u0645\u062e\u0635\u0635 \u0644\u0644\u0627\u0633\u062a\u062f\u0644\u0627\u0644 \u0627\u0644\u0645\u062d\u0644\u064a \u0644\u0645\u0633\u062a\u062e\u062f\u0645 \u0648\u0627\u062d\u062f.<\/strong> For concurrent serving use safetensors with vLLM or an API instead.<\/li>\n<\/ul>\n<\/div>\n<p>GGUF (GPT-Generated Unified Format) \u0647\u064a \u0635\u064a\u063a\u0629 \u0627\u0644\u0645\u0644\u0641 \u0627\u0644\u062a\u064a \u064a\u0633\u062a\u062e\u062f\u0645\u0647\u0627 \u0646\u0638\u0627\u0645 llama.cpp\/ggml \u0644\u062a\u062e\u0632\u064a\u0646 \u0646\u0645\u0648\u0630\u062c \u062c\u0627\u0647\u0632 \u0644\u0644\u0627\u0633\u062a\u062f\u0644\u0627\u0644. \u0645\u0644\u0641 \u0648\u0627\u062d\u062f <code>.gguf<\/code> \u064a\u062d\u062a\u0648\u064a \u0639\u0644\u0649 \u0627\u0644\u0645\u0648\u062a\u0631\u0627\u062a \u0627\u0644\u0645\u0643\u0645\u0645\u0629 \u0628\u0627\u0644\u0625\u0636\u0627\u0641\u0629 \u0625\u0644\u0649 \u0643\u0644 \u0634\u064a\u0621 \u0645\u0637\u0644\u0648\u0628 \u0644\u0627\u0633\u062a\u062e\u062f\u0627\u0645\u0647\u0627: \u0627\u0644\u0645\u0641\u0631\u062f\u0627\u062a \u0648\u0625\u0639\u062f\u0627\u062f\u0627\u062a \u0627\u0644\u0645\u0631\u0645\u0632\u0627\u062a \u0648\u0646\u0645\u0648\u0630\u062c \u0627\u0644\u062f\u0631\u062f\u0634\u0629 \u0648\u0628\u064a\u0627\u0646\u0627\u062a \u0648\u0635\u0641 \u0627\u0644\u0628\u0646\u064a\u0629. \u062d\u0644 \u0645\u062d\u0644 \u0635\u064a\u063a\u0629 GGML \u0627\u0644\u0623\u0642\u062f\u0645 \u0641\u064a \u0623\u063a\u0633\u0637\u0633 2023 \u0648\u0647\u0648 \u0627\u0644\u0622\u0646 \u0645\u0639\u064a\u0627\u0631 \u0641\u0639\u0644\u064a \u0644\u062a\u0634\u063a\u064a\u0644 \u0627\u0644\u0646\u0645\u0627\u0630\u062c \u0639\u0644\u0649 \u0623\u062c\u0647\u0632\u0629 \u0627\u0644\u0643\u0645\u0628\u064a\u0648\u062a\u0631 \u0627\u0644\u0645\u062d\u0645\u0648\u0644\u0629 \u0648\u0623\u062c\u0647\u0632\u0629 \u0633\u0637\u062d \u0627\u0644\u0645\u0643\u062a\u0628 \u0648\u0627\u0644\u0645\u0639\u0627\u0644\u062c\u0627\u062a.<\/p>\n<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_88 counter-flat ez-toc-counter ez-toc-container-direction\">\n<label for=\"ez-toc-cssicon-toggle-item-6add687aa6908\" class=\"ez-toc-cssicon-toggle-label\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">\u062a\u0628\u062f\u064a\u0644<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #000000;color:#000000\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewbox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #000000;color:#000000\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewbox=\"0 0 24 24\" version=\"1.2\" baseprofile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/label><input type=\"checkbox\"  id=\"ez-toc-cssicon-toggle-item-6add687aa6908\"  aria-label=\"\u062a\u0628\u062f\u064a\u0644\" \/><nav><ul class='ez-toc-list ez-toc-list-level-1' ><li class='ez-toc-page-1'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/convly.ai\/ar\/gguf-models\/#What_is_actually_inside_a_gguf_file\" >\u0645\u0627 \u0647\u0648 \u0628\u0627\u0644\u0636\u0628\u0637 \u062f\u0627\u062e\u0644 \u0645\u0644\u0641 .gguf<\/a><\/li><li class='ez-toc-page-1'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/convly.ai\/ar\/gguf-models\/#How_to_read_a_GGUF_filename\" >\u0643\u064a\u0641\u064a\u0629 \u0642\u0631\u0627\u0621\u0629 \u0627\u0633\u0645 \u0645\u0644\u0641 GGUF<\/a><\/li><li class='ez-toc-page-1'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/convly.ai\/ar\/gguf-models\/#Sizing_which_GGUF_models_fit_your_GPU\" >\u0627\u0644\u062a\u062d\u062c\u064a\u0645: \u0623\u064a \u0646\u0645\u0627\u0630\u062c GGUF \u062a\u0646\u0627\u0633\u0628 GPU \u0627\u0644\u062e\u0627\u0635 \u0628\u0643<\/a><\/li><li class='ez-toc-page-1'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/convly.ai\/ar\/gguf-models\/#Where_to_download_GGUF_models\" >\u0623\u064a\u0646 \u062a\u062d\u0645\u0644 \u0646\u0645\u0627\u0630\u062c GGUF<\/a><\/li><li class='ez-toc-page-1'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/convly.ai\/ar\/gguf-models\/#Running_GGUF_models_on_Linux\" >\u062a\u0634\u063a\u064a\u0644 \u0646\u0645\u0627\u0630\u062c GGUF \u0639\u0644\u0649 Linux<\/a><\/li><li class='ez-toc-page-1'><a class=\"ez-toc-link ez-toc-heading-6\" href=\"https:\/\/convly.ai\/ar\/gguf-models\/#Running_GGUF_models_on_macOS\" >\u062a\u0634\u063a\u064a\u0644 \u0646\u0645\u0627\u0630\u062c GGUF \u0639\u0644\u0649 macOS<\/a><\/li><li class='ez-toc-page-1'><a class=\"ez-toc-link ez-toc-heading-7\" href=\"https:\/\/convly.ai\/ar\/gguf-models\/#Running_GGUF_models_on_Windows\" >\u062a\u0634\u063a\u064a\u0644 \u0646\u0645\u0627\u0630\u062c GGUF \u0639\u0644\u0649 Windows<\/a><\/li><li class='ez-toc-page-1'><a class=\"ez-toc-link ez-toc-heading-8\" href=\"https:\/\/convly.ai\/ar\/gguf-models\/#Loading_an_arbitrary_GGUF_into_Ollama\" >\u062a\u062d\u0645\u064a\u0644 GGUF \u0639\u0634\u0648\u0627\u0626\u064a \u0641\u064a Ollama<\/a><\/li><li class='ez-toc-page-1'><a class=\"ez-toc-link ez-toc-heading-9\" href=\"https:\/\/convly.ai\/ar\/gguf-models\/#When_GGUF_is_the_wrong_answer\" >\u0639\u0646\u062f\u0645\u0627 \u0644\u0627 \u064a\u0643\u0648\u0646 GGUF \u0647\u0648 \u0627\u0644\u0625\u062c\u0627\u0628\u0629 \u0627\u0644\u0635\u062d\u064a\u062d\u0629<\/a><\/li><li class='ez-toc-page-1'><a class=\"ez-toc-link ez-toc-heading-10\" href=\"https:\/\/convly.ai\/ar\/gguf-models\/#GGUF_vs_safetensors_AWQ_and_GPTQ\" >GGUF \u0645\u0642\u0627\u0628\u0644 safetensors \u0648 AWQ \u0648 GPTQ<\/a><\/li><li class='ez-toc-page-1'><a class=\"ez-toc-link ez-toc-heading-11\" href=\"https:\/\/convly.ai\/ar\/gguf-models\/#Converting_a_Model_to_GGUF\" >\u062a\u062d\u0648\u064a\u0644 \u0646\u0645\u0648\u0630\u062c \u0625\u0644\u0649 \u0635\u064a\u063a\u0629 GGUF<\/a><\/li><li class='ez-toc-page-1'><a class=\"ez-toc-link ez-toc-heading-12\" href=\"https:\/\/convly.ai\/ar\/gguf-models\/#Frequently_asked_questions\" >\u0627\u0644\u0623\u0633\u0626\u0644\u0629 \u0627\u0644\u0634\u0627\u0626\u0639\u0629<\/a><\/li><\/ul><\/nav><\/div>\n<h2><span class=\"ez-toc-section\" id=\"What_is_actually_inside_a_gguf_file\"><\/span>\u0645\u0627 \u0647\u0648 \u0628\u0627\u0644\u0636\u0628\u0637 \u062f\u0627\u062e\u0644 \u0645\u0644\u0641 .gguf<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>\u0645\u0644\u0641 GGUF \u064a\u062a\u0643\u0648\u0646 \u0645\u0646 \u0631\u0623\u0633 (header)\u060c \u0648\u0643\u062a\u0644\u0629 \u0628\u064a\u0627\u0646\u0627\u062a \u0648\u0635\u0641\u064a\u0629 (metadata) \u0645\u0641\u062a\u0627\u062d-\u0642\u064a\u0645\u0629\u060c \u062b\u0645 \u0628\u064a\u0627\u0646\u0627\u062a \u0627\u0644\u0645\u0648\u062a\u0651\u0631\u0627\u062a (tensor). \u0627\u0644\u0628\u064a\u0627\u0646\u0627\u062a \u0627\u0644\u0648\u0635\u0641\u064a\u0629 \u0647\u064a \u0627\u0644\u062c\u0632\u0621 \u0627\u0644\u0645\u0647\u0645 \u0639\u0645\u0644\u064a\u064b\u0627 \u2014 \u0648\u0647\u0630\u0627 \u0627\u0644\u0633\u0628\u0628 \u0641\u064a \u0623\u0646 \u0646\u0645\u0648\u0630\u062c GGUF \u0644\u0627 \u064a\u062d\u062a\u0627\u062c \u0625\u0644\u0649 \u0645\u0644\u0641\u0627\u062a \u0645\u0631\u0627\u0641\u0642\u0629. \u062a\u0648\u062b\u064a\u0642 Hugging Face <a href=\"https:\/\/huggingface.co\/docs\/hub\/en\/gguf\" rel=\"noopener\" target=\"_blank\">GGUF<\/a> \u064a\u0635\u0641 \u0627\u0644\u062a\u062e\u0637\u064a\u0637 \u0648\u0645\u0634\u0627\u0647\u062f \u0628\u064a\u0627\u0646\u0627\u062a GGUF \u0627\u0644\u0645\u062f\u0645\u062c\u0629 \u0641\u064a Hub\u060c \u0627\u0644\u0630\u064a \u064a\u062a\u064a\u062d \u0644\u0643 \u0641\u062d\u0635 \u0646\u0648\u0639 \u0627\u0644\u062a\u0643\u0645\u064a\u0645\u060c \u0637\u0648\u0644 \u0627\u0644\u0633\u064a\u0627\u0642\u060c \u0648\u0646\u0645\u0648\u0630\u062c \u0627\u0644\u062f\u0631\u062f\u0634\u0629 \u0644\u0644\u0645\u0644\u0641 \u0642\u0628\u0644 \u062a\u0646\u0632\u064a\u0644 \u0639\u062f\u0629 \u062c\u064a\u062c\u0627\u0628\u0627\u064a\u062a \u0645\u0646\u0647.<\/p>\n<p>\u0627\u0644\u0645\u0641\u0627\u062a\u064a\u062d \u0627\u0644\u0648\u0635\u0641\u064a\u0629 \u0627\u0644\u0646\u0645\u0637\u064a\u0629 \u062a\u0634\u0645\u0644 \u0627\u0644\u0639\u0645\u0627\u0631\u0629 (<code>Llama<\/code>, <code>qwen3<\/code>, <code>gemma3<\/code>, <code>phi3<\/code>)\u060c \u0637\u0648\u0644 \u0633\u064a\u0627\u0642 \u0627\u0644\u062a\u062f\u0631\u064a\u0628\u060c \u0625\u0639\u062f\u0627\u062f\u0627\u062a RoPE\u060c \u0627\u0644\u0645\u0641\u0631\u062f\u0627\u062a \u0627\u0644\u0643\u0627\u0645\u0644\u0629\u060c \u0648\u0646\u0645\u0648\u0630\u062c \u062f\u0631\u062f\u0634\u0629 Jinja. \u064a\u0642\u0631\u0623 \u0627\u0644\u0645\u064f\u062d\u0645\u0651\u0644 (loader) \u062a\u0644\u0643 \u0627\u0644\u0643\u062a\u0644\u0629 \u0648\u064a\u0636\u0628\u0637 \u0646\u0641\u0633\u0647\u061b \u0623\u0646\u062a \u0644\u0627 \u062a\u0645\u0631\u0631 tokenizer \u0623\u0648 \u0635\u064a\u063a\u0629 prompt \u064a\u062f\u0648\u064a\u064b\u0627.<\/p>\n<h3>\u0645\u0642\u0627\u0631\u0646\u0629 \u0628\u064a\u0646 GGUF \u0648safetensors<\/h3>\n<table>\n<thead>\n<tr>\n<th><\/th>\n<th>GGUF<\/th>\n<th>safetensors<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>\u0648\u0642\u062a \u0627\u0644\u062a\u0634\u063a\u064a\u0644 \u0627\u0644\u0623\u0633\u0627\u0633\u064a<\/td>\n<td>llama.cpp, Ollama, LM Studio<\/td>\n<td>PyTorch, vLLM, Transformers, TGI<\/td>\n<\/tr>\n<tr>\n<td>\u0645\u0644\u0641\u0627\u062a \u0644\u0643\u0644 \u0646\u0645\u0648\u0630\u062c<\/td>\n<td>\u0645\u0644\u0641 \u0648\u0627\u062d\u062f (\u0623\u0648 \u0639\u064f\u0646\u062f \u0645\u0631\u0642\u0645\u0629)<\/td>\n<td>\u0623\u0648\u0632\u0627\u0646 \u0628\u0627\u0644\u0625\u0636\u0627\u0641\u0629 \u0625\u0644\u0649 \u0645\u062c\u0644\u062f \u0627\u0644\u0625\u0639\u062f\u0627\u062f\u0627\u062a\/\u0627\u0644\u0645\u062d\u0644\u0644 \u0627\u0644\u0644\u063a\u0648\u064a<\/td>\n<\/tr>\n<tr>\n<td>Quantization<\/td>\n<td>\u0645\u062f\u0645\u062c (Q4_K_M, Q8_0, IQ\u2026)<\/td>\n<td>\u0639\u0627\u062f\u0629 FP16\/BF16\u060c \u0623\u0648 \u0645\u062a\u063a\u064a\u0631\u0627\u062a GPTQ\/AWQ<\/td>\n<\/tr>\n<tr>\n<td>CPU \u0628\u0627\u0644\u0625\u0636\u0627\u0641\u0629 \u0625\u0644\u0649 \u0625\u0632\u0627\u062d\u0629 GPU \u062c\u0632\u0626\u064a\u0629<\/td>\n<td>\u0646\u0639\u0645\u060c \u0647\u062f\u0641 \u0627\u0644\u062a\u0635\u0645\u064a\u0645 \u0627\u0644\u0623\u0633\u0627\u0633\u064a<\/td>\n<td>\u0645\u062d\u062f\u0648\u062f \u0648\u0628\u0637\u064a\u0621<\/td>\n<\/tr>\n<tr>\n<td>\u062e\u062f\u0645\u0629 \u062f\u0641\u0639\u064a\u0629 \u0645\u062a\u0632\u0627\u0645\u0646\u0629<\/td>\n<td>\u0636\u0639\u064a\u0641<\/td>\n<td>\u0642\u0648\u064a<\/td>\n<\/tr>\n<tr>\n<td>\u0627\u0644\u0636\u0628\u0637 \u0627\u0644\u062f\u0642\u064a\u0642<\/td>\n<td>\u0644\u0627 (\u062a\u062d\u0648\u064a\u0644 \u0645\u0631\u0629 \u0623\u062e\u0631\u0649 \u0623\u0648\u0644\u0627\u064b)<\/td>\n<td>\u0646\u0639\u0645<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<h2><span class=\"ez-toc-section\" id=\"How_to_read_a_GGUF_filename\"><\/span>\u0643\u064a\u0641\u064a\u0629 \u0642\u0631\u0627\u0621\u0629 \u0627\u0633\u0645 \u0645\u0644\u0641 GGUF<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>\u0639\u0627\u062f\u0629 \u0645\u0627 \u062a\u064f\u0633\u0645\u0649 \u0627\u0644\u0645\u0644\u0641\u0627\u062a <code>Model-Name-8B-Instruct-Q4_K_M.gguf<\/code>. \u0627\u0644\u0644\u0627\u062d\u0642\u0629 \u0647\u064a \u0645\u0632\u064a\u062c \u0627\u0644\u062a\u0643\u0645\u064a\u0645. \u0627\u0644\u0631\u0642\u0645 \u0647\u0648 \u0639\u0631\u0636 \u0627\u0644\u0628\u062a \u0627\u0644\u0627\u0633\u0645\u064a\u061b <code>_K<\/code> \u062a\u0639\u0646\u064a k-quants\u060c \u0627\u0644\u062a\u064a \u062a\u062d\u0627\u0641\u0638 \u0639\u0644\u0649 \u0645\u0648\u062a\u0651\u0631\u0627\u062a \u0627\u0644\u0627\u0647\u062a\u0645\u0627\u0645 \u0648\u0627\u0644\u062a\u0636\u0645\u064a\u0646 \u0628\u062f\u0642\u0629 \u0623\u0639\u0644\u0649 \u0645\u0646 \u0623\u0648\u0632\u0627\u0646 \u0627\u0644\u062a\u063a\u0630\u064a\u0629 \u0627\u0644\u0623\u0645\u0627\u0645\u064a\u0629 \u0627\u0644\u0631\u0626\u064a\u0633\u064a\u0629\u061b <code>S<\/code>\/<code>M<\/code>\/<code>L<\/code> \u0645\u062a\u063a\u064a\u0631\u0627\u062a \u0635\u063a\u064a\u0631\u0629\/\u0645\u062a\u0648\u0633\u0637\u0629\/\u0643\u0628\u064a\u0631\u0629 \u0645\u0646 \u0647\u0630\u0627 \u0627\u0644\u0645\u0632\u064a\u062c.<\/p>\n<table>\n<thead>\n<tr>\n<th>\u0627\u0644\u062a\u0643\u0645\u064a\u0645<\/th>\n<th>\u0628\u062a \u062a\u0642\u0631\u064a\u0628\u064a \u0644\u0643\u0644 \u0648\u0632\u0646<\/th>\n<th>\u0627\u0644\u062d\u062c\u0645 \u0627\u0644\u062a\u0642\u0631\u064a\u0628\u064a\u060c \u0646\u0645\u0648\u0630\u062c 8B<\/th>\n<th>\u0645\u062a\u0649 \u062a\u0633\u062a\u062e\u062f\u0645\u0647<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>Q8_0<\/td>\n<td>~8.5<\/td>\n<td>~8.5 GB<\/td>\n<td>\u0645\u0631\u062c\u0639 \u0642\u0631\u064a\u0628 \u0645\u0646 \u0639\u062f\u0645 \u0627\u0644\u0641\u0642\u062f\u0627\u0646\u061b \u0646\u0645\u0627\u0630\u062c \u0635\u063a\u064a\u0631\u0629 \u0641\u0642\u0637<\/td>\n<\/tr>\n<tr>\n<td>Q6_K<\/td>\n<td>~6.6<\/td>\n<td>~6.6 GB<\/td>\n<td>\u0644\u062f\u064a\u0643 VRAM \u0627\u062d\u062a\u064a\u0627\u0637\u064a<\/td>\n<\/tr>\n<tr>\n<td>Q5_K_M<\/td>\n<td>~5.7<\/td>\n<td>~5.7 GB<\/td>\n<td>\u0627\u0644\u0627\u0641\u062a\u0631\u0627\u0636\u064a \u0627\u0644\u0645\u064a\u0644 \u0646\u062d\u0648 \u0627\u0644\u062c\u0648\u062f\u0629<\/td>\n<\/tr>\n<tr>\n<td>Q4_K_M<\/td>\n<td>~4.9<\/td>\n<td>~4.9 GB<\/td>\n<td><strong>\u0627\u0644\u0627\u0641\u062a\u0631\u0627\u0636\u064a \u0627\u0644\u0645\u0639\u062a\u0627\u062f<\/strong><\/td>\n<\/tr>\n<tr>\n<td>Q4_0<\/td>\n<td>~4.5<\/td>\n<td>~4.5 \u063a\u064a\u063a\u0627\u0628\u0627\u064a\u062a<\/td>\n<td>\u0648\u0631\u0627\u062b\u064a\u061b \u0628\u0639\u0636 \u0627\u0644\u0645\u0633\u0631\u0639\u0627\u062a \u062a\u0641\u0636\u0644\u0647<\/td>\n<\/tr>\n<tr>\n<td>Q3_K_M<\/td>\n<td>~3.9<\/td>\n<td>~4.0 GB<\/td>\n<td>\u062f\u0633 \u0646\u0645\u0648\u0630\u062c \u0623\u0643\u0628\u0631 \u0641\u064a VRAM \u0635\u063a\u064a\u0631<\/td>\n<\/tr>\n<tr>\n<td>Q2_K \/ IQ2<\/td>\n<td>~2.5\u20133.4<\/td>\n<td>~2.5\u20133.4 GB<\/td>\n<td>\u0622\u062e\u0631 \u0645\u0644\u0627\u0630\u061b \u062a\u062f\u0647\u0648\u0631 \u0645\u0644\u062d\u0648\u0638<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>\u062a\u0639\u0627\u0645\u0644 \u0645\u0639 \u0639\u0645\u0648\u062f \u0627\u0644\u0628\u062a-\u0644\u0643\u0644-\u0627\u0644\u0648\u0632\u0646 \u0643\u062a\u0642\u0631\u064a\u0628. \u062a\u062a\u063a\u064a\u0631 \u0627\u0644\u0623\u062d\u062c\u0627\u0645 \u0627\u0644\u0641\u0639\u0644\u064a\u0629 \u0645\u0639 \u0639\u0645\u0627\u0631\u0629 \u0627\u0644\u0646\u0645\u0648\u0630\u062c (\u0642\u0627\u0645\u0648\u0633 \u0643\u0628\u064a\u0631 \u064a\u0646\u062a\u0641\u062e \u0645\u0648\u062a\u0651\u0631\u0627\u062a \u0627\u0644\u062a\u0636\u0645\u064a\u0646) \u0648\u0625\u0635\u062f\u0627\u0631 llama.cpp\u060c \u0644\u0623\u0646 \u0645\u0632\u064a\u062c \u0627\u0644\u062a\u0643\u0645\u064a\u0645 \u064a\u062a\u0645 \u0636\u0628\u0637\u0647 \u0628\u0645\u0631\u0648\u0631 \u0627\u0644\u0648\u0642\u062a. \u0639\u0627\u0626\u0644\u0629 <code>IQ<\/code> IQ (\u0645\u0646 IQ2_XXS \u0625\u0644\u0649 IQ4_NL) \u062a\u0633\u062a\u062e\u062f\u0645 \u0645\u0639\u0627\u064a\u0631\u0629 \u0645\u0635\u0641\u0648\u0641\u0629 \u0627\u0644\u0623\u0647\u0645\u064a\u0629 \u0644\u0644\u062d\u0641\u0627\u0638 \u0639\u0644\u0649 \u062c\u0648\u062f\u0629 \u0623\u0641\u0636\u0644 \u0639\u0646\u062f \u0639\u0631\u0648\u0636 \u0628\u062a \u0645\u0646\u062e\u0641\u0636\u0629 \u062c\u062f\u064b\u0627\u060c \u0648\u0627\u0644\u0645\u0643\u0645\u0645\u0627\u062a \u0645\u062b\u0644 Bartowski \u0648 Unsloth \u062a\u0646\u0634\u0631 \u0645\u062a\u063a\u064a\u0631\u0627\u062a IQ \u062c\u0646\u0628\u064b\u0627 \u0625\u0644\u0649 \u062c\u0646\u0628 \u0645\u0639 K-quants \u0627\u0644\u0642\u064a\u0627\u0633\u064a\u0629.<\/p>\n<p>\u0625\u062c\u0645\u0627\u0639 \u0627\u0644\u0645\u062c\u062a\u0645\u0639\u060c \u0644\u0627 \u0642\u064a\u0627\u0633 Convly\u060c \u0647\u0648 \u0623\u0646 Q4_K_M \u0647\u0648 \u0627\u0644\u0646\u0642\u0637\u0629 \u0627\u0644\u062d\u0644\u0648\u0629 \u0648\u0623\u0646 \u0627\u0644\u062c\u0648\u062f\u0629 \u062a\u0646\u062e\u0641\u0636 \u0628\u0634\u0643\u0644 \u062d\u0627\u062f \u0623\u0642\u0644 \u0645\u0646 \u062d\u0648\u0627\u0644\u064a 3 \u0628\u062a \u0644\u0643\u0644 \u0648\u0632\u0646 \u2014 \u0627\u0644\u0646\u0645\u0627\u0630\u062c \u0627\u0644\u0635\u063a\u064a\u0631\u0629 \u062a\u062a\u062f\u0647\u0648\u0631 \u0623\u0633\u0631\u0639 \u0645\u0646 \u0627\u0644\u0646\u0645\u0627\u0630\u062c \u0627\u0644\u0643\u0628\u064a\u0631\u0629 \u0628\u0646\u0641\u0633 \u0627\u0644\u062a\u0643\u0645\u064a\u0645.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"Sizing_which_GGUF_models_fit_your_GPU\"><\/span>\u0627\u0644\u062a\u062d\u062c\u064a\u0645: \u0623\u064a \u0646\u0645\u0627\u0630\u062c GGUF \u062a\u0646\u0627\u0633\u0628 GPU \u0627\u0644\u062e\u0627\u0635 \u0628\u0643<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>\u062d\u062c\u0645 \u0627\u0644\u0645\u0644\u0641 \u0647\u0648 \u0627\u0644\u062d\u062f \u0627\u0644\u0623\u062f\u0646\u0649\u060c \u0648\u0644\u064a\u0633 \u0627\u0644\u0625\u062c\u0645\u0627\u0644\u064a. \u0623\u0636\u0641 \u0630\u0627\u0643\u0631\u0629 \u0627\u0644\u062a\u062e\u0632\u064a\u0646 \u0627\u0644\u0645\u0624\u0642\u062a KV \u0644\u0637\u0648\u0644 \u0627\u0644\u0633\u064a\u0627\u0642 \u0627\u0644\u062e\u0627\u0635 \u0628\u0643 \u0628\u0627\u0644\u0625\u0636\u0627\u0641\u0629 \u0625\u0644\u0649 \u062d\u0648\u0627\u0644\u064a 500 MB\u20131 GB \u0645\u0646 \u0627\u0644\u062d\u0645\u0644 \u0627\u0644\u0625\u0636\u0627\u0641\u064a. \u062a\u0623\u062a\u064a \u0647\u0630\u0647 \u0627\u0644\u0623\u0631\u0642\u0627\u0645 4-bit \u0645\u0646 <a href=\"https:\/\/convly.ai\/ar\/models\/\">Convly<\/a>:<\/p>\n<table>\n<thead>\n<tr>\n<th>\u0627\u0644\u0646\u0645\u0648\u0630\u062c<\/th>\n<th>VRAM \u0639\u0646\u062f \u062a\u0643\u0645\u064a\u0645 4 \u0628\u062a<\/th>\n<th>\u0627\u0644\u0633\u064a\u0627\u0642<\/th>\n<th>GPU \u0648\u0627\u0642\u0639\u064a<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>Gemma 3 4B<\/td>\n<td>~3 \u062c\u064a\u062c\u0627\u0628\u0627\u064a\u062a<\/td>\n<td>128 \u0623\u0644\u0641 \u0631\u0645\u0632<\/td>\n<td>\u0623\u064a \u0628\u0637\u0627\u0642\u0629 6 GB\u060c GPU \u0645\u062a\u0643\u0627\u0645\u0644\u0629<\/td>\n<\/tr>\n<tr>\n<td>Mistral 7B<\/td>\n<td>~4.5 \u063a\u064a\u063a\u0627\u0628\u0627\u064a\u062a<\/td>\n<td>32K<\/td>\n<td>\u0628\u0637\u0627\u0642\u0629 8 GB<\/td>\n<\/tr>\n<tr>\n<td>Llama 3.1 8B<\/td>\n<td>~5 \u063a\u064a\u063a\u0627\u0628\u0627\u064a\u062a<\/td>\n<td>128 \u0623\u0644\u0641 \u0631\u0645\u0632<\/td>\n<td>\u0628\u0637\u0627\u0642\u0629 8 GB<\/td>\n<\/tr>\n<tr>\n<td>Qwen3 14B<\/td>\n<td>~9 \u062c\u064a\u062c\u0627\u0628\u0627\u064a\u062a<\/td>\n<td>128 \u0623\u0644\u0641 \u0631\u0645\u0632<\/td>\n<td>\u0628\u0637\u0627\u0642\u0629 12 GB<\/td>\n<\/tr>\n<tr>\n<td>Phi-4<\/td>\n<td>~9 \u062c\u064a\u062c\u0627\u0628\u0627\u064a\u062a<\/td>\n<td>16 \u0623\u0644\u0641 \u0631\u0645\u0632<\/td>\n<td>\u0628\u0637\u0627\u0642\u0629 12 GB<\/td>\n<\/tr>\n<tr>\n<td>Gemma 3 27B<\/td>\n<td>\u062d\u0648\u0627\u0644\u064a 16 \u062c\u064a\u062c\u0627\u0628\u0627\u064a\u062a<\/td>\n<td>128 \u0623\u0644\u0641 \u0631\u0645\u0632<\/td>\n<td>\u0628\u0637\u0627\u0642\u0629 24 GB<\/td>\n<\/tr>\n<tr>\n<td>Qwen3 32B<\/td>\n<td>~20 \u062c\u064a\u062c\u0627\u0628\u0627\u064a\u062a<\/td>\n<td>128 \u0623\u0644\u0641 \u0631\u0645\u0632<\/td>\n<td>\u0628\u0637\u0627\u0642\u0629 24 GB<\/td>\n<\/tr>\n<tr>\n<td>Llama 3.3 70B<\/td>\n<td>~40 \u062c\u064a\u062c\u0627\u0628\u0627\u064a\u062a<\/td>\n<td>128 \u0623\u0644\u0641 \u0631\u0645\u0632<\/td>\n<td>2\u00d724 GB\u060c \u0623\u0648 48 GB \u0630\u0627\u0643\u0631\u0629 \u0645\u0648\u062d\u062f\u0629<\/td>\n<\/tr>\n<tr>\n<td>DeepSeek R1 (\u0643\u0627\u0645\u0644\u0629)<\/td>\n<td>\u0646\u062d\u0648 400 \u062c\u064a\u062c\u0627\u0628\u0627\u064a\u062a<\/td>\n<td>128 \u0623\u0644\u0641 \u0631\u0645\u0632<\/td>\n<td>\u062e\u0627\u062f\u0645 \u0645\u062a\u0639\u062f\u062f GPU \u0641\u0642\u0637<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>Mixture-of-experts models are the trap here. Qwen3 30B-A3B activates only ~3B parameters per token but still needs all ~18 GB resident. At the extreme, the database puts Kimi K3 at ~1.4 TB at 4-bit \u2014 a GGUF exists in principle, but no single machine you own will hold it. For an exact figure at your context length, use the <a href=\"https:\/\/convly.ai\/ar\/llm-vram-calculator\/\">\u062d\u0627\u0633\u0628\u0629 VRAM<\/a> \u0623\u0648 \u0627\u0644\u062a\u0641\u0635\u064a\u0644 \u062d\u0633\u0628 \u0627\u0644\u0646\u0645\u0648\u0630\u062c \u0641\u064a <a href=\"https:\/\/convly.ai\/ar\/vram-requirements-every-major-llm-2026\/\">\u0645\u062a\u0637\u0644\u0628\u0627\u062a \u0630\u0627\u0643\u0631\u0629 VRAM \u0644\u0643\u0644 \u0646\u0645\u0648\u0630\u062c \u0644\u063a\u0648\u064a \u0631\u0626\u064a\u0633\u064a<\/a>. \u0625\u0630\u0627 \u0643\u0646\u062a \u0644\u0627 \u062a\u0632\u0627\u0644 \u062a\u062e\u062a\u0627\u0631 \u0627\u0644\u0623\u062c\u0647\u0632\u0629\u060c <a href=\"https:\/\/convly.ai\/ar\/best-gpus-for-local-llms-2026\/\">\u0623\u0641\u0636\u0644 GPUs \u0644\u0644\u0646\u0645\u0627\u0630\u062c \u0627\u0644\u0644\u063a\u0648\u064a\u0629 \u0627\u0644\u0645\u062d\u0644\u064a\u0629<\/a> \u064a\u063a\u0637\u064a \u0645\u0642\u0627\u0628\u0644\u0629 VRAM-per-dollar.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"Where_to_download_GGUF_models\"><\/span>\u0623\u064a\u0646 \u062a\u062d\u0645\u0644 \u0646\u0645\u0627\u0630\u062c GGUF<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<ul>\n<li><strong>Hugging Face<\/strong> \u2014 \u0635\u0641\u0651 \u0642\u0627\u0626\u0645\u0629 \u0627\u0644\u0646\u0645\u0648\u0630\u062c \u0645\u0639 <a href=\"https:\/\/huggingface.co\/models?library=gguf\" rel=\"noopener\" target=\"_blank\">library=gguf<\/a>. \u0645\u0639\u0638\u0645 \u0627\u0644\u0645\u0633\u062a\u0648\u062f\u0639\u0627\u062a \u062a\u0648\u0641\u0631 \u0643\u0644 \u0627\u0644\u062a\u0643\u0645\u064a\u0645\u0627\u062a \u0641\u064a \u0645\u0633\u062a\u0648\u062f\u0639 \u0648\u0627\u062d\u062f\u061b \u062d\u0645\u0651\u0644 \u0627\u0644\u0645\u0644\u0641 \u0627\u0644\u0630\u064a \u062a\u062d\u062a\u0627\u062c\u0647 \u0641\u0642\u0637\u060c \u0648\u0644\u064a\u0633 \u0627\u0644\u0645\u0633\u062a\u0648\u062f\u0639 \u0643\u0627\u0645\u0644\u0627\u064b.<\/li>\n<li><strong>\u0645\u0643\u062a\u0628\u0629 Ollama<\/strong> \u2014 <a href=\"https:\/\/ollama.com\/library\" rel=\"noopener\" target=\"_blank\">ollama.com\/library<\/a> \u064a\u0648\u0641\u0631 GGUF \u0645\u0639\u0628\u0623\u0629 \u0645\u0633\u0628\u0642\u0627\u064b \u0645\u0639 \u0642\u0627\u0644\u0628 \u0627\u0644\u062f\u0631\u062f\u0634\u0629 \u0627\u0644\u0645\u062f\u0645\u062c \u0628\u0627\u0644\u0641\u0639\u0644. \u0627\u0646\u0638\u0631 \u0625\u0644\u0649 <a href=\"https:\/\/convly.ai\/ar\/ollama-models-list-2026\/\">\u0642\u0627\u0626\u0645\u0629 \u0646\u0645\u0627\u0630\u062c Ollama<\/a>.<\/li>\n<li><strong>LM Studio<\/strong> \u2014 \u0639\u0644\u0627\u0645\u0629 \u0627\u0644\u062a\u0628\u0648\u064a\u0628 \"\u0627\u0643\u062a\u0634\u0641\" \u0641\u064a \u0627\u0644\u062a\u0637\u0628\u064a\u0642 \u062a\u0628\u062d\u062b \u0641\u064a \u0645\u0633\u062a\u0648\u062f\u0639\u0627\u062a Hugging Face GGUF \u0648\u062a\u062d\u062f\u062f \u0627\u0644\u062a\u0643\u0645\u064a\u0645\u0627\u062a \u0627\u0644\u0645\u0646\u0627\u0633\u0628\u0629 \u0644\u0630\u0627\u0643\u0631\u0629 \u0627\u0644\u0648\u0635\u0648\u0644 \u0627\u0644\u0639\u0634\u0648\u0627\u0626\u064a\/VRAM \u0627\u0644\u0645\u0643\u062a\u0634\u0641\u0629 \u0644\u062f\u064a\u0643. \u0634\u0631\u062d: <a href=\"https:\/\/convly.ai\/ar\/lm-studio-complete-guide-2026\/\">\u062f\u0644\u064a\u0644 LM Studio \u0627\u0644\u0643\u0627\u0645\u0644<\/a>.<\/li>\n<\/ul>\n<p>\u062a\u0635\u0644 \u0627\u0644\u0646\u0645\u0627\u0630\u062c \u0627\u0644\u0643\u0628\u064a\u0631\u0629 \u0645\u0642\u0633\u0645\u0629 \u0625\u0644\u0649 <code>model-00001-of-00003.gguf<\/code> \u0648\u0647\u0643\u0630\u0627. \u062d\u0645\u0651\u0644 \u0643\u0644 \u062c\u0632\u0621 \u0641\u064a \u0646\u0641\u0633 \u0627\u0644\u0645\u062c\u0644\u062f \u0648\u0623\u0634\u0631 \u0645\u062d\u0645\u0644 \u0627\u0644\u0628\u064a\u0627\u0646\u0627\u062a \u0625\u0644\u0649 \u0627\u0644\u062c\u0632\u0621 00001 \u2014 \u0633\u064a\u062c\u062f \u0627\u0644\u0628\u0642\u064a\u0629 \u0628\u0646\u0641\u0633\u0647.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"Running_GGUF_models_on_Linux\"><\/span>\u062a\u0634\u063a\u064a\u0644 \u0646\u0645\u0627\u0630\u062c GGUF \u0639\u0644\u0649 Linux<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>\u0628\u0646\u0627\u0621 llama.cpp \u0645\u0639 \u062f\u0639\u0645 CUDA:<\/p>\n<pre><code>git clone https:\/\/github.com\/ggml-org\/llama.cpp\ncd llama.cpp\ncmake -B build -DGGML_CUDA=ON\ncmake --build build --config Release -j<\/code><\/pre>\n<p>\u062a\u064f\u0648\u0636\u0639 \u0627\u0644\u0645\u0644\u0641\u0627\u062a \u0627\u0644\u062b\u0646\u0627\u0626\u064a\u0629 \u0641\u064a <code>build\/bin\/<\/code>. \u0627\u0628\u062f\u0623 \u062e\u0627\u062f\u0645 \u0645\u062a\u0648\u0627\u0641\u0642 \u0645\u0639 OpenAI:<\/p>\n<pre><code>.\/build\/bin\/llama-server -m ~\/models\/model-Q4_K_M.gguf -c 8192 -ngl 99 --port 8080<\/code><\/pre>\n<p><code>-ngl<\/code> (<code>--n-gpu-layers<\/code>) \u0647\u0648 \u0642\u0631\u0635 \u0627\u0644\u0625\u0641\u0631\u0627\u063a: <code>99<\/code> \u064a\u0639\u0646\u064a \u00ab\u062c\u0645\u064a\u0639 \u0627\u0644\u0637\u0628\u0642\u0627\u062a \u0639\u0644\u0649 GPU\u00bb\u060c <code>0<\/code> \u064a\u0639\u0646\u064a CPU \u0641\u0642\u0637\u060c \u0648\u0627\u0644\u0642\u064a\u0645 \u0627\u0644\u0648\u0633\u064a\u0637\u0629 \u062a\u0642\u0633\u0645 \u0627\u0644\u0646\u0645\u0648\u0630\u062c \u0639\u0646\u062f\u0645\u0627 \u0644\u0627 \u064a\u0646\u0627\u0633\u0628 \u0627\u0644\u0630\u0627\u0643\u0631\u0629. <code>-c<\/code> \u064a\u0639\u064a\u0651\u0646 \u0627\u0644\u0633\u064a\u0627\u0642\u061b \u062a\u0631\u0643\u0647 \u0639\u0646\u062f \u0627\u0644\u0633\u064a\u0627\u0642 \u0627\u0644\u0645\u062f\u0631\u0628 \u0627\u0644\u0643\u0627\u0645\u0644 \u0644\u0644\u0646\u0645\u0648\u0630\u062c \u0642\u062f \u064a\u0643\u0644\u0641 VRAM \u0623\u0643\u062b\u0631 \u0645\u0646 \u0627\u0644\u0623\u0648\u0632\u0627\u0646. \u0627\u0644\u0637\u0631\u0641 \u0627\u0644\u0646\u0647\u0627\u0626\u064a \u0647\u0648 \u0625\u0630\u0627\u064b <code>http:\/\/localhost:8080\/v1\/chat\/completions<\/code>.<\/p>\n<p>\u062a\u062d\u0630\u064a\u0631\u0627\u062a \u0627\u0644\u0625\u0635\u062f\u0627\u0631: \u0639\u0644\u0645 CMake \u0643\u0627\u0646 <code>LLAMA_CUBLAS<\/code> \u0641\u064a \u0627\u0644\u0625\u0635\u062f\u0627\u0631\u0627\u062a \u0627\u0644\u0623\u0642\u062f\u0645\u060c \u0648\u0623\u0639\u064a\u062f\u062a \u062a\u0633\u0645\u064a\u0629 \u0627\u0644\u0645\u0644\u0641\u0627\u062a \u0627\u0644\u062b\u0646\u0627\u0626\u064a\u0629 (\u00ab<code>main<\/code> \u2192 <code>llama-cli<\/code>, <code>server<\/code> \u2192 <code>llama-server<\/code>\u00bb) \u0641\u064a 2024. \u062a\u062d\u0642\u0642 \u0645\u0646 \u0645\u0644\u0641 README \u0641\u064a \u0627\u0644\u0645\u0633\u062a\u0648\u062f\u0639 \u0644\u0644\u0628\u0646\u0627\u0621 \u0627\u0644\u0630\u064a \u0627\u0633\u062a\u0646\u0633\u062e\u062a\u0647. \u0628\u0627\u0644\u0646\u0633\u0628\u0629 \u0644\u0645\u0639\u0627\u0644\u062c\u0627\u062a AMD \u0623\u0648 Intel\u060c \u0627\u0633\u062a\u0628\u062f\u0644 \u0639\u0644\u0645 \u0627\u0644\u0648\u0627\u062c\u0647\u0629 \u0627\u0644\u062e\u0644\u0641\u064a\u0629 ROCm\/Vulkan\/SYCL \u0627\u0644\u0645\u0648\u062b\u0642 \u0647\u0646\u0627\u0643 \u0628\u062f\u0644\u0627\u064b \u0645\u0646 \u0627\u0644\u062a\u062e\u0645\u064a\u0646.<\/p>\n<p>\u0625\u0630\u0627 \u0627\u0633\u062a\u062e\u062f\u0645\u062a Ollama \u0628\u062f\u0644\u0627\u064b \u0645\u0646 \u0630\u0644\u0643\u060c \u062a\u0639\u064a\u0634 \u0627\u0644\u0646\u0645\u0627\u0630\u062c \u0641\u064a <code>\/usr\/share\/ollama\/.ollama\/models<\/code> \u0639\u0646\u062f\u0645\u0627 \u064a\u0639\u0645\u0644 \u0643\u062e\u062f\u0645\u0629 \u0646\u0638\u0627\u0645\u060c \u0623\u0648 <code>~\/.ollama\/models<\/code> \u0644\u062a\u062b\u0628\u064a\u062a \u0627\u0644\u0645\u0633\u062a\u062e\u062f\u0645. \u0627\u0646\u0638\u0631 \u0625\u0644\u0649 <a href=\"https:\/\/convly.ai\/ar\/how-to-install-ollama-2026\/\">\u0643\u064a\u0641\u064a\u0629 \u062a\u062b\u0628\u064a\u062a \u0645\u0646\u0635\u0629 Ollama<\/a>.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"Running_GGUF_models_on_macOS\"><\/span>\u062a\u0634\u063a\u064a\u0644 \u0646\u0645\u0627\u0630\u062c GGUF \u0639\u0644\u0649 macOS<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Apple Silicon \u0627\u0633\u062a\u062b\u0646\u0627\u0626\u064a\u0629 \u062c\u062f\u0627\u064b \u0644\u0640 GGUF \u0644\u0623\u0646 \u0627\u0644\u0630\u0627\u0643\u0631\u0629 \u0627\u0644\u0645\u0634\u062a\u0631\u0643\u0629 \u062a\u0639\u0646\u064a \u0623\u0646 GPU \u064a\u0645\u0643\u0646\u0647\u0627 \u0627\u0644\u0648\u0635\u0648\u0644 \u0625\u0644\u0649 \u0645\u0639\u0638\u0645 \u0630\u0627\u0643\u0631\u0629 \u0627\u0644\u0646\u0638\u0627\u0645 \u2014 \u064a\u0645\u0643\u0646 \u0644\u0640 Mac \u0628\u0640 64 GB \u0623\u0646 \u064a\u0633\u062a\u0648\u0639\u0628 \u0646\u0645\u0648\u0630\u062c ~40 GB \u0628\u0640 4-bit 70B \u0644\u0646 \u064a\u062a\u0633\u0639 \u0641\u064a \u0628\u0637\u0627\u0642\u062a\u064a GPU \u0628\u0640 24 GB.<\/p>\n<pre><code>brew install llama.cpp\nllama-server -m ~\/models\/model-Q4_K_M.gguf -c 8192 --port 8080<\/code><\/pre>\n<p>\u064a\u062a\u0645 \u062a\u0641\u0639\u064a\u0644 \u0625\u0641\u0631\u0627\u063a Metal \u0641\u064a \u0628\u0646\u0627\u0621 Homebrew\u060c \u0644\u0630\u0627 \u0639\u0627\u062f\u0629 \u0644\u0627 \u062a\u062d\u062a\u0627\u062c \u0625\u0644\u0649 <code>-ngl<\/code>. \u064a\u062d\u062f macOS \u0645\u0646 \u0645\u0642\u062f\u0627\u0631 \u0627\u0644\u0630\u0627\u0643\u0631\u0629 \u0627\u0644\u0639\u0634\u0648\u0627\u0626\u064a\u0629 \u0627\u0644\u062a\u064a \u0642\u062f \u064a\u0637\u0644\u0628\u0647\u0627 GPU (\u0642\u0627\u0628\u0644 \u0644\u0644\u062a\u0639\u062f\u064a\u0644 \u0639\u0628\u0631 <code>iogpu<\/code> sysctl\u060c \u0648\u0627\u0644\u0630\u064a \u064a\u062e\u062a\u0644\u0641 \u062d\u0633\u0628 \u0625\u0635\u062f\u0627\u0631 macOS)\u060c \u0644\u0630\u0627 \u0627\u062a\u0631\u0643 \u0645\u062c\u0627\u0644\u0627\u064b \u0644\u0644\u0646\u0638\u0627\u0645. \u064a\u062e\u0632\u0646 Ollama \u0627\u0644\u0646\u0645\u0627\u0630\u062c \u0641\u064a <code>~\/.ollama\/models<\/code>\u061b \u064a\u0633\u062a\u062e\u062f\u0645 LM Studio <code>~\/.lmstudio\/models<\/code> \u0641\u064a \u0627\u0644\u0625\u0635\u062f\u0627\u0631\u0627\u062a \u0627\u0644\u062d\u0627\u0644\u064a\u0629 \u0648 <code>~\/.cache\/lm-studio\/models<\/code> \u0641\u064a \u0627\u0644\u0625\u0635\u062f\u0627\u0631\u0627\u062a \u0627\u0644\u0623\u0642\u062f\u0645 \u2014 \u0639\u0644\u0627\u0645\u0629 \u0627\u0644\u062a\u0628\u0648\u064a\u0628 My Models \u062a\u0639\u0631\u0636 \u0648\u062a\u063a\u064a\u0631 \u0627\u0644\u0645\u0633\u0627\u0631 \u0627\u0644\u0641\u0639\u0644\u064a.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"Running_GGUF_models_on_Windows\"><\/span>\u062a\u0634\u063a\u064a\u0644 \u0646\u0645\u0627\u0630\u062c GGUF \u0639\u0644\u0649 Windows<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>\u062b\u0644\u0627\u062b \u0637\u0631\u0642\u060c \u0627\u0644\u0623\u0633\u0647\u0644 \u0623\u0648\u0644\u0627\u064b:<\/p>\n<ol>\n<li><strong>LM Studio<\/strong> \u2014 \u0645\u062b\u0628\u062a \u0648\u0627\u062c\u0647\u0629 \u0631\u0633\u0648\u0645\u064a\u0629\u060c \u0628\u062d\u062b \u0627\u0644\u0646\u0645\u0648\u0630\u062c \u0641\u064a \u0627\u0644\u062a\u0637\u0628\u064a\u0642\u060c \u062a\u0628\u062f\u064a\u0644 \u062e\u0627\u062f\u0645 \u0645\u062d\u0644\u064a. \u0627\u0644\u062e\u064a\u0627\u0631 \u0627\u0644\u0627\u0641\u062a\u0631\u0627\u0636\u064a \u0627\u0644\u0623\u0641\u0636\u0644 \u0644\u0644\u0645\u0637\u0648\u0631\u064a\u0646 \u0628\u0644\u0627 \u062e\u0628\u0631\u0629.<\/li>\n<li><strong>Ollama<\/strong> \u2014 \u0645\u062b\u0628\u062a Windows \u0623\u0635\u0644\u064a\u061b \u062a\u0630\u0647\u0628 \u0627\u0644\u0646\u0645\u0627\u0630\u062c \u0625\u0644\u0649 <code>C:Users.ollamamodels<\/code>. \u0627\u0636\u0628\u0637 \u0645\u062a\u063a\u064a\u0631 \u0627\u0644\u0628\u064a\u0626\u0629 <code>OLLAMA_MODELS<\/code> \u0644\u0646\u0642\u0644\u0647\u0627 \u0628\u0639\u064a\u062f\u0627\u064b \u0639\u0646 \u0645\u062d\u0631\u0643 \u0627\u0644\u0646\u0638\u0627\u0645. \u0627\u0644\u062e\u0644\u0641\u064a\u0629: <a href=\"https:\/\/convly.ai\/ar\/what-is-ollama-complete-guide-2026\/\">\u0645\u0627 \u0647\u0648 Ollama<\/a>.<\/li>\n<li><strong>\u0645\u0644\u0641\u0627\u062a \u062b\u0646\u0627\u0626\u064a\u0629 \u0645\u064f\u062c\u0645\u0651\u0639\u0629 \u0645\u0633\u0628\u0642\u0627\u064b \u0644\u0640 llama.cpp<\/strong> \u2014 \u062a\u0646\u0634\u0631 \u0635\u0641\u062d\u0629 GitHub Releases \u0625\u0635\u062f\u0627\u0631\u0627\u062a Windows \u0645\u0636\u063a\u0648\u0637\u0629 \u0644\u0643\u0644 \u0648\u0627\u062c\u0647\u0629 \u062e\u0644\u0641\u064a\u0629 (CUDA \u0648 Vulkan \u0648 CPU). \u0641\u0643 \u0627\u0644\u0636\u063a\u0637 \u0648\u0627\u0644\u062a\u0634\u063a\u064a\u0644 <code>llama-server.exe -m model.gguf -ngl 99<\/code> \u0645\u0646 PowerShell. \u0627\u062e\u062a\u0631 \u0628\u0646\u0627\u0621 CUDA \u0644\u0640 NVIDIA\u061b Vulkan \u0647\u0648 \u0627\u0644\u0628\u062f\u064a\u0644 \u0627\u0644\u0622\u0645\u0646 \u0639\u0628\u0631 \u0627\u0644\u0628\u0627\u0626\u0639\u064a\u0646 \u0644\u0640 AMD \u0648 Intel Arc.<\/li>\n<\/ol>\n<p>\u0645\u0634\u0627\u0643\u0644 \u062e\u0627\u0635\u0629 \u0628\u0640 Windows: \u0627\u0644\u0645\u0633\u062d \u0627\u0644\u0641\u0648\u0631\u064a \u0641\u064a Defender \u064a\u0628\u0637\u0626 \u0627\u0644\u062a\u062d\u0645\u064a\u0644 \u0627\u0644\u0623\u0648\u0644 \u0644\u0645\u0644\u0641 \u0628\u0639\u062f\u0629 \u062c\u064a\u062c\u0627\u0628\u0627\u064a\u062a\u060c \u0648\u062a\u0634\u063a\u064a\u0644 llama.cpp \u062f\u0627\u062e\u0644 WSL2 \u064a\u0643\u0644\u0641\u0643 \u0634\u0631\u064a\u062d\u0629 \u0645\u0646 \u0627\u0644\u0630\u0627\u0643\u0631\u0629 \u0644\u0644\u062c\u0647\u0627\u0632 \u0627\u0644\u0627\u0641\u062a\u0631\u0627\u0636\u064a. \u0639\u0627\u062f\u0629\u064b \u0645\u0627 \u062a\u0643\u0648\u0646 \u0625\u0635\u062f\u0627\u0631\u0627\u062a Windows \u0627\u0644\u0623\u0635\u0644\u064a\u0629 \u0627\u0644\u0637\u0631\u064a\u0642 \u0627\u0644\u0623\u0628\u0633\u0637.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"Loading_an_arbitrary_GGUF_into_Ollama\"><\/span>\u062a\u062d\u0645\u064a\u0644 GGUF \u0639\u0634\u0648\u0627\u0626\u064a \u0641\u064a Ollama<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>\u064a\u0645\u0643\u0646 \u0644\u0625\u0635\u062f\u0627\u0631\u0627\u062a Ollama \u0627\u0644\u062d\u062f\u064a\u062b\u0629 \u0627\u0644\u0633\u062d\u0628 \u0645\u0628\u0627\u0634\u0631\u0629 \u0645\u0646 Hugging Face:<\/p>\n<pre><code>ollama run hf.co\/\/:Q4_K_M<\/code><\/pre>\n<p>\u0644\u0645\u0644\u0641 \u0645\u062d\u0644\u064a\u060c \u0627\u0643\u062a\u0628 Modelfile \u0641\u064a \u0646\u0641\u0633 \u0627\u0644\u0645\u062c\u0644\u062f:<\/p>\n<pre><code>FROM .\/model-Q4_K_M.gguf<\/code><\/pre>\n<p>\u062b\u0645 <code>ollama create my-model -f Modelfile<\/code> \u0648 <code>ollama run my-model<\/code>. \u0625\u0630\u0627 \u0643\u0627\u0646 GGUF \u064a\u0641\u062a\u0642\u062f \u0642\u0627\u0644\u0628 \u062f\u0631\u062f\u0634\u0629 \u0642\u0627\u0628\u0644\u0627\u064b \u0644\u0644\u0627\u0633\u062a\u062e\u062f\u0627\u0645\u060c \u0623\u0636\u0641 <code>TEMPLATE<\/code> \u0648 <code>PARAMETER stop<\/code> \u0633\u0637\u0631 \u2014 \u062a\u0648\u062c\u064a\u0647\u0627\u062a Modelfile \u0627\u0644\u062f\u0642\u064a\u0642\u0629 \u0632\u0627\u062f\u062a \u0639\u0628\u0631 \u0627\u0644\u0625\u0635\u062f\u0627\u0631\u0627\u062a\u060c \u0644\u0630\u0627 \u062a\u062d\u0642\u0642 \u0645\u0646 <code>ollama help create<\/code> \u0644\u0646\u0633\u062e\u062a\u0643. \u0627\u062e\u062a\u064a\u0627\u0631\u0627\u062a \u0645\u062e\u062a\u0627\u0631\u0629: <a href=\"https:\/\/convly.ai\/ar\/best-local-llms-to-run-on-ollama-2026\/\">\u0623\u0641\u0636\u0644 \u0646\u0645\u0627\u0630\u062c \u0627\u0644\u0644\u063a\u0629 \u0627\u0644\u0645\u062d\u0644\u064a\u0629 (LLMs) \u0644\u0640 Ollama<\/a>.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"When_GGUF_is_the_wrong_answer\"><\/span>\u0639\u0646\u062f\u0645\u0627 \u0644\u0627 \u064a\u0643\u0648\u0646 GGUF \u0647\u0648 \u0627\u0644\u0625\u062c\u0627\u0628\u0629 \u0627\u0644\u0635\u062d\u064a\u062d\u0629<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>GGUF optimizes one user at a time. Serving many concurrent requests, or needing tensor parallelism across GPUs, points to safetensors with vLLM or SGLang. And running locally is not automatically cheaper: hosted Llama 3.3 70B is $0.10 in \/ $0.32 out per 1M tokens, while frontier hosted models like Claude Sonnet 5 sit at $2.00 in \/ $10.00 out per 1M tokens for 1M context. Run your own volume through the <a href=\"https:\/\/convly.ai\/ar\/ai-api-cost-calculator\/\">\u062d\u0627\u0633\u0628\u0629 \u062a\u0643\u0644\u0641\u0629 \u0648\u0627\u062c\u0647\u0629 \u0628\u0631\u0645\u062c\u0629 \u0627\u0644\u062a\u0637\u0628\u064a\u0642\u0627\u062a (API)<\/a> \u0648 <a href=\"https:\/\/convly.ai\/ar\/self-hosting-vs-api-calculator\/\">self-hosting vs API break-even calculator<\/a> \u0642\u0628\u0644 \u0634\u0631\u0627\u0621 \u0648\u062d\u062f\u0629 GPU.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"GGUF_vs_safetensors_AWQ_and_GPTQ\"><\/span>GGUF \u0645\u0642\u0627\u0628\u0644 safetensors \u0648 AWQ \u0648 GPTQ<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<table class=\"convly-vs\">\n<thead>\n<tr>\n<th>\u0627\u0644\u0635\u064a\u063a\u0629<\/th>\n<th>\u0648\u0642\u062a \u0627\u0644\u062a\u0634\u063a\u064a\u0644 \u0627\u0644\u0623\u0633\u0627\u0633\u064a<\/th>\n<th>\u0627\u0644\u062f\u0642\u0629<\/th>\n<th>\u0627\u0644\u0623\u0641\u0636\u0644 \u0644\u0640<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>GGUF<\/td>\n<td>llama.cpp, Ollama, LM Studio<\/td>\n<td>\u0645\u0646 2 \u0625\u0644\u0649 8 \u0628\u062a + F16<\/td>\n<td>\u0648\u062d\u062f\u0629 \u0645\u0639\u0627\u0644\u062c\u0629 \u0645\u0631\u0643\u0632\u064a\u0629 (CPU)\u060c \u0623\u0648 \u0645\u0632\u064a\u062c \u0645\u0646 CPU\/GPU\u060c \u0623\u0648 Apple Silicon\u060c \u0623\u0648 \u0627\u0633\u062a\u062e\u062f\u0627\u0645 \u0641\u0631\u062f\u064a<\/td>\n<\/tr>\n<tr>\n<td>Safetensors (FP16\/BF16)<\/td>\n<td>Transformers\u060c vLLM<\/td>\n<td>16 \u0628\u062a<\/td>\n<td>\u0627\u0644\u062a\u062f\u0631\u064a\u0628\u060c \u0648\u0627\u0644\u0628\u062d\u062b\u060c \u0648\u0627\u0644\u0627\u0633\u062a\u0646\u062a\u0627\u062c \u0628\u062f\u0642\u0629 \u0643\u0627\u0645\u0644\u0629<\/td>\n<\/tr>\n<tr>\n<td>AWQ<\/td>\n<td>vLLM\u060c AutoAWQ<\/td>\n<td>4 \u0628\u062a<\/td>\n<td>\u0627\u0644\u0627\u0633\u062a\u062e\u062f\u0627\u0645 \u0639\u0627\u0644\u064a \u0627\u0644\u0625\u0646\u062a\u0627\u062c\u064a\u0629 \u0639\u0644\u0649 \u0648\u062d\u062f\u0627\u062a \u0645\u0639\u0627\u0644\u062c\u0629 \u0627\u0644\u0631\u0633\u0648\u0645\u0627\u062a (GPU)<\/td>\n<\/tr>\n<tr>\n<td>GPTQ<\/td>\n<td>vLLM\u060c ExLlamaV2<\/td>\n<td>\u0645\u0646 3 \u0625\u0644\u0649 8 \u0628\u062a<\/td>\n<td>\u0627\u0644\u0627\u0633\u062a\u0646\u062a\u0627\u062c \u0639\u0644\u0649 \u0648\u062d\u062f\u0629 \u0645\u0639\u0627\u0644\u062c\u0629 \u0627\u0644\u0631\u0633\u0648\u0645\u0627\u062a \u0641\u0642\u0637 \u0645\u0639 \u062f\u0639\u0645 \u0627\u0644\u062a\u062c\u0645\u064a\u0639 (batching)<\/td>\n<\/tr>\n<tr>\n<td>MLX<\/td>\n<td>MLX (Apple)<\/td>\n<td>4\u201316 \u0628\u062a<\/td>\n<td>\u0645\u062e\u0635\u0635 \u0644\u0631\u0642\u0627\u0642\u0627\u062a Apple Silicon \u0641\u0642\u0637<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>\u0627\u062e\u062a\u0631 \u062a\u0646\u0633\u064a\u0642 GGUF \u0639\u0646\u062f \u0627\u0633\u062a\u0647\u062f\u0627\u0641 \u0645\u0633\u062a\u062e\u062f\u0645 \u0648\u0627\u062d\u062f \u0639\u0644\u0649 \u0623\u062c\u0647\u0632\u0629 \u0645\u062a\u0646\u0648\u0639\u0629 \u0623\u0648 \u0639\u0646\u062f\u0645\u0627 \u062a\u0631\u063a\u0628 \u0641\u064a \u062a\u0634\u063a\u064a\u0644 \u0627\u0644\u0646\u0645\u0648\u0630\u062c \u0639\u0644\u0649 \u0648\u062d\u062f\u0629 \u0645\u0639\u0627\u0644\u062c\u0629 \u0645\u0631\u0643\u0632\u064a\u0629 (CPU) \u0628\u0623\u064a \u0634\u0643\u0644\u064d \u0645\u0646 \u0627\u0644\u0623\u0634\u0643\u0627\u0644. \u0623\u0645\u0627 \u0639\u0646\u062f \u062e\u062f\u0645\u0629 \u0639\u062f\u062f \u0643\u0628\u064a\u0631 \u0645\u0646 \u0627\u0644\u0645\u0633\u062a\u062e\u062f\u0645\u064a\u0646 \u0627\u0644\u0645\u062a\u0632\u0627\u0645\u0646\u064a\u0646 \u0639\u0644\u0649 \u0648\u062d\u062f\u0629 \u0645\u0639\u0627\u0644\u062c\u0629 \u0631\u0633\u0648\u0645\u064a\u0629 (GPU) \u0641\u064a \u0645\u0631\u0643\u0632 \u0628\u064a\u0627\u0646\u0627\u062a\u060c \u0641\u0627\u062e\u062a\u0631 vLLM \u0645\u0639 \u0645\u0644\u0641\u0627\u062a safetensors \u0627\u0644\u0645\u064f\u0643\u0645\u064e\u0651\u0646\u0629 \u0628\u0640 AWQ \u0623\u0648 GPTQ.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"Converting_a_Model_to_GGUF\"><\/span>\u062a\u062d\u0648\u064a\u0644 \u0646\u0645\u0648\u0630\u062c \u0625\u0644\u0649 \u0635\u064a\u063a\u0629 GGUF<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>\u0625\u0630\u0627 \u0648\u064f\u062c\u062f \u0646\u0645\u0648\u0630\u062c \u0645\u0627 \u0639\u0644\u0649 Hugging Face \u0628\u0635\u064a\u063a\u0629 safetensors \u0644\u0643\u0646 \u0644\u0645 \u064a\u064f\u0643\u0645\u064e\u0651\u0646 \u0628\u0639\u062f\u064f \u0645\u0646 \u0642\u0650\u0628\u0644 \u0623\u062d\u062f\u060c \u0641\u0625\u0646 \u0633\u064a\u0631 \u0627\u0644\u0639\u0645\u0644 \u064a\u0643\u0648\u0646 \u0643\u0627\u0644\u062a\u0627\u0644\u064a:<\/p>\n<pre><code># \u0641\u064a \u0645\u0633\u062a\u0648\u062f\u0639 llama.cpp\npip install -r requirements.txt\npython convert_hf_to_gguf.py \/path\/to\/hf-model --outfile model-f16.gguf\n.\/build\/bin\/llama-quantize model-f16.gguf model-Q4_K_M.gguf Q4_K_M<\/code><\/pre>\n<p>\u064a\u062f\u0639\u0645 \u0646\u0635 \u0627\u0644\u062a\u062d\u0648\u064a\u0644 \u0641\u0642\u0637 \u0647\u064a\u0627\u0643\u0644 \u0627\u0644\u0646\u0645\u0627\u0630\u062c \u0627\u0644\u062a\u064a \u0639\u0644\u064e\u0651\u0645\u062a\u0647\u0627 llama.cpp \u2014 \u0645\u062b\u0644 llama \u0648mistral \u0648qwen2\/3 \u0648gemma \u0648phi\u060c \u0625\u0636\u0627\u0641\u0629\u064b \u0625\u0644\u0649 \u0642\u0627\u0626\u0645\u0629 \u0645\u062a\u0632\u0627\u064a\u062f\u0629 \u0645\u0646 \u0627\u0644\u0647\u064a\u0627\u0643\u0644 \u0627\u0644\u0623\u062e\u0631\u0649. \u0623\u0645\u0627 \u0627\u0644\u0647\u064a\u0627\u0643\u0644 \u0627\u0644\u062c\u062f\u064a\u062f\u0629 (novel architectures) \u0641\u062a\u062a\u0637\u0644\u0628 \u062a\u0639\u062f\u064a\u0644\u064b\u0627 \u0641\u064a \u0627\u0644\u0643\u0648\u062f \u0623\u0648\u0644\u064b\u0627. \u0648\u0642\u062f \u062a\u063a\u064a\u064e\u0651\u0631\u062a \u0623\u0633\u0645\u0627\u0621 \u0627\u0644\u0645\u0644\u0641\u0627\u062a \u0648\u0627\u0644\u0639\u0644\u0645\u0627\u062a (flags) \u0641\u064a \u0627\u0644\u0646\u0635 \u0628\u064a\u0646 \u0625\u0635\u062f\u0627\u0631\u0627\u062a llama.cpp\u060c \u0644\u0630\u0627 \u062a\u0623\u0643\u064e\u0651\u062f \u0645\u0646 \u0645\u0631\u0627\u062c\u0639\u0629 <code>python convert_hf_to_gguf.py --help<\/code> \u0628\u0627\u0644\u0646\u0633\u0628\u0629 \u0644\u0644\u0625\u0635\u062f\u0627\u0631 \u0627\u0644\u0630\u064a \u0642\u0645\u062a \u0628\u0628\u0646\u0627\u0626\u0647 \u0645\u0646 \u0627\u0644\u0645\u0635\u062f\u0631.<\/p>\n<p>\u0644\u0645\u0639\u0631\u0641\u0629 \u0645\u0635\u062f\u0631 \u0627\u0644\u062a\u0646\u0633\u064a\u0642 \u0648\u0643\u064a\u0641 \u0627\u0633\u062a\u0628\u062f\u0644 GGUF GGML\u060c \u0627\u0646\u0638\u0631 <a href=\"\/ar\/what-is-gguf-format-explained\/\">\u0645\u0627 \u0647\u0648 GGUF?<\/a>.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"Frequently_asked_questions\"><\/span>\u0627\u0644\u0623\u0633\u0626\u0644\u0629 \u0627\u0644\u0634\u0627\u0626\u0639\u0629<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<h3>\u0645\u0627\u0630\u0627 \u064a\u0639\u0646\u064a GGUF\u060c \u0648\u0643\u064a\u0641 \u064a\u062e\u062a\u0644\u0641 \u0639\u0646 GGML\u061f<\/h3>\n<p>GGUF \u064a\u0631\u0645\u0632 \u0625\u0644\u0649 GPT-Generated Unified Format. GGML \u0643\u0627\u0646 \u0627\u0644\u062a\u0646\u0633\u064a\u0642 \u0627\u0644\u0623\u0642\u062f\u0645 \u0645\u0646 \u0646\u0641\u0633 \u0627\u0644\u0645\u0634\u0631\u0648\u0639\u061b \u0643\u0627\u0646 \u064a\u0643\u0633\u0631 \u0627\u0644\u062a\u0648\u0627\u0641\u0642\u064a\u0629 \u0643\u0644\u0645\u0627 \u0643\u0627\u0646\u062a \u0647\u0646\u0627\u0643 \u062d\u0627\u062c\u0629 \u0644\u0628\u064a\u0627\u0646\u0627\u062a \u0648\u0635\u0641\u064a\u0629 \u062c\u062f\u064a\u062f\u0629. \u0623\u0636\u0627\u0641 GGUF \u0643\u062a\u0644\u0629 \u0628\u064a\u0627\u0646\u0627\u062a \u0648\u0635\u0641\u064a\u0629 \u0642\u0627\u0628\u0644\u0629 \u0644\u0644\u062a\u0648\u0633\u0639 \u0628\u0645\u0641\u0627\u062a\u064a\u062d-\u0642\u064a\u0645 \u0628\u062d\u064a\u062b \u064a\u0645\u0643\u0646 \u0625\u0636\u0627\u0641\u0629 \u0645\u0639\u0645\u0627\u0631\u064a\u0627\u062a \u0648\u062e\u064a\u0627\u0631\u0627\u062a \u062c\u062f\u064a\u062f\u0629 \u062f\u0648\u0646 \u0625\u0628\u0637\u0627\u0644 \u0627\u0644\u0645\u0644\u0641\u0627\u062a \u0627\u0644\u0642\u062f\u064a\u0645\u0629. \u0645\u0644\u0641\u0627\u062a \u0645\u0627 \u0642\u0628\u0644 GGUF <code>\u0623\u0648 .bin<\/code> \u0645\u0644\u0641\u0627\u062a GGML \u0644\u0645 \u062a\u0639\u062f \u062a\u064f\u062d\u0645\u064e\u0651\u0644 \u0641\u064a llama.cpp \u0627\u0644\u062d\u0627\u0644\u064a\u0629.<\/p>\n<h3>\u0623\u064a \u062a\u0643\u0645\u064a\u0646 \u064a\u062c\u0628 \u0623\u0646 \u0623\u062d\u0645\u0651\u0644\u0647\u061f<\/h3>\n<p>\u0627\u0628\u062f\u0623 \u0628\u0640 Q4_K_M. \u0625\u0630\u0627 \u062a\u0631\u0643 \u0627\u0644\u0646\u0645\u0648\u0630\u062c \u0639\u062f\u0629 \u062c\u064a\u062c\u0627\u0628\u0627\u064a\u062a\u0627\u062a \u0645\u0646 VRAM \u062d\u0631\u0629\u060c \u0627\u0646\u062a\u0642\u0644 \u0625\u0644\u0649 Q5_K_M \u0623\u0648 Q6_K. \u0625\u0630\u0627 \u0644\u0645 \u064a\u0646\u0627\u0633\u0628\u060c \u0641\u0636\u0651\u0644 \u0646\u0645\u0648\u0630\u062c\u0627\u064b \u0623\u0635\u063a\u0631 \u0628\u0640 Q4_K_M \u0639\u0644\u0649 \u0646\u0641\u0633 \u0627\u0644\u0646\u0645\u0648\u0630\u062c \u0639\u0646\u062f Q2_K \u2014 \u0646\u0645\u0648\u0630\u062c 14B \u0628\u0640 4 \u0628\u062a \u064a\u062a\u0641\u0648\u0642 \u0639\u0645\u0648\u0645\u0627\u064b \u0639\u0644\u0649 32B \u0645\u0634\u0648\u0647 \u0644\u0640 2 \u0628\u062a. \u062a\u062d\u0642\u0642 \u0627\u0644\u0645\u0646\u0627\u0633\u0628\u0629 \u0645\u0639 <a href=\"https:\/\/convly.ai\/ar\/llm-vram-calculator\/\">\u062d\u0627\u0633\u0628\u0629 VRAM<\/a> \u0628\u0637\u0648\u0644 \u0627\u0644\u0633\u064a\u0627\u0642 \u0627\u0644\u0645\u0642\u0635\u0648\u062f.<\/p>\n<h3>\u0647\u0644 \u064a\u0645\u0643\u0646\u0646\u064a \u062a\u0634\u063a\u064a\u0644 \u0646\u0645\u0627\u0630\u062c GGUF \u0628\u062f\u0648\u0646 GPU\u061f<\/h3>\n<p>Yes \u2014 CPU-only inference is what GGUF was built for. Speed is bound by memory bandwidth, so expect single-digit tokens per second for a 7B\u20138B model at Q4_K_M on typical dual-channel desktop RAM, and slower for anything larger. Small models like Gemma 3 4B (~3 GB at 4-bit) are the practical CPU-only choice.<\/p>\n<h3>\u0647\u0644 \u064a\u0645\u0643\u0646\u0646\u064a \u062a\u062d\u0648\u064a\u0644 \u0646\u0645\u0648\u0630\u062c \u0645\u0646 Hugging Face \u0625\u0644\u0649 GGUF \u0628\u0646\u0641\u0633\u064a\u061f<\/h3>\n<p>\u0646\u0639\u0645. \u064a\u0623\u062a\u064a llama.cpp \u0645\u0639 \u0633\u0643\u0631\u064a\u0628\u062a \u062a\u062d\u0648\u064a\u0644 (\"<code>convert_hf_to_gguf.py<\/code> \u0641\u064a \u0627\u0644\u0625\u0635\u062f\u0627\u0631\u0627\u062a \u0627\u0644\u062d\u0627\u0644\u064a\u0629\u061b \u0627\u0633\u062a\u062e\u062f\u0645\u062a \u0627\u0633\u0645 \u0627\u0644\u0645\u0644\u0641 \u0634\u0631\u0637\u0627\u062a \u0641\u064a \u0627\u0644\u0625\u0635\u062f\u0627\u0631\u0627\u062a \u0627\u0644\u0623\u0642\u062f\u0645) \u064a\u0646\u062a\u062c GGUF F16\u060c \u062b\u0645 \u062b\u0646\u0627\u0626\u064a <code>llama-quantize<\/code> \u064a\u0636\u063a\u0637\u0647 \u0625\u0644\u0649 \u0643\u0645 \u0645\u0633\u062a\u0647\u062f\u0641. \u062a\u062d\u0642\u0642 \u0645\u0646 <code>--help<\/code> \u0641\u064a \u0627\u0644\u0646\u0633\u062e\u0629 \u0627\u0644\u062a\u064a \u0627\u0633\u062a\u0646\u0633\u062e\u062a\u0647\u0627\u060c \u0648\u0623\u0643\u062f \u0623\u0646 \u0645\u0639\u0645\u0627\u0631\u062a\u0643 \u0645\u062f\u0639\u0648\u0645\u0629 \u0642\u0628\u0644 \u0627\u0644\u0628\u062f\u0621 \u2014 \u0627\u0644\u0645\u0639\u0645\u0627\u0631\u064a\u0627\u062a \u063a\u064a\u0631 \u0627\u0644\u0645\u062f\u0639\u0648\u0645\u0629 \u062a\u0641\u0634\u0644 \u0641\u064a \u0627\u0644\u062a\u062d\u0648\u064a\u0644.<\/p>\n<h3>\u0647\u0644 \u062a\u062f\u0639\u0645 \u0646\u0645\u0627\u0630\u062c GGUF \u0627\u0644\u0631\u0624\u064a\u0629 \u0623\u0648 \u0627\u0633\u062a\u062f\u0639\u0627\u0621 \u0627\u0644\u0623\u062f\u0648\u0627\u062a\u061f<\/h3>\n<p>\u0627\u0633\u062a\u062f\u0639\u0627\u0621 \u0627\u0644\u0623\u062f\u0627\u0629 \u064a\u0639\u0645\u0644 \u062d\u064a\u062b \u064a\u0639\u0631\u0651\u0641 \u0642\u0627\u0644\u0628 \u0627\u0644\u062f\u0631\u062f\u0634\u0629 \u0627\u0644\u062e\u0627\u0635 \u0628\u0627\u0644\u0646\u0645\u0648\u0630\u062c \u0630\u0644\u0643 \u0648\u0645\u062d\u0645\u0651\u0644 \u0642\u064a\u0627\u0633\u0643 \u064a\u0634\u0631\u0651\u0641 \u0627\u0644\u0642\u0627\u0644\u0628. \u0627\u0644\u0646\u0645\u0627\u0630\u062c \u0627\u0644\u0645\u062a\u0639\u062f\u062f\u0629 \u0627\u0644\u0623\u0646\u0645\u0627\u0637 \u062a\u062d\u062a\u0627\u062c \u0645\u0644\u0641 \u062b\u0627\u0646\u064a \u2014 <code>mmproj<\/code> GGUF \u064a\u062d\u0645\u0644 \u0645\u062d\u0642\u0642 \u0627\u0644\u0631\u0624\u064a\u0629 \u2014 \u0645\u062d\u0645\u0651\u0644 \u062c\u0646\u0628\u0627\u064b \u0625\u0644\u0649 \u062c\u0646\u0628 \u0645\u0639 \u0623\u0648\u0632\u0627\u0646 \u0627\u0644\u0646\u0635. \u062a\u0645 \u0625\u0639\u0627\u062f\u0629 \u062a\u0633\u0645\u064a\u0629 \u0648\u0627\u0644\u062a\u0646\u0638\u064a\u0645 \u0623\u062f\u0648\u0627\u062a \u0627\u0644\u0648\u0633\u0627\u0626\u0637 \u0627\u0644\u0645\u062a\u0639\u062f\u062f\u0629 \u0641\u064a llama.cpp \u0623\u0643\u062b\u0631 \u0645\u0646 \u0645\u0631\u0629\u060c \u0644\u0630\u0627 \u0627\u062a\u0628\u0639 \u0645\u0633\u062a\u0646\u062f\u0627\u062a repo \u0627\u0644\u062d\u0627\u0644\u064a\u0629 \u0628\u062f\u0644\u0627\u064b \u0645\u0646 \u0628\u0631\u0646\u0627\u0645\u062c \u062a\u0639\u0644\u064a\u0645\u064a \u0642\u062f\u064a\u0645.<\/p>\n<h3>\u0647\u0644 \u064a\u063a\u064a\u0631 \u0627\u0644\u062a\u0643\u0645\u064a\u0645 \u0646\u0627\u0641\u0630\u0629 \u0627\u0644\u0633\u064a\u0627\u0642\u061f<\/h3>\n<p>\u0644\u0627. \u0627\u0644\u0633\u064a\u0627\u0642 \u0647\u0648 \u062e\u0627\u0635\u064a\u0629 \u0627\u0644\u0646\u0645\u0648\u0630\u062c\u060c \u0648\u0644\u064a\u0633 \u0627\u0644\u0643\u0645\u064a\u0629: Llama 3.3 70B \u0647\u0648 128K \u0648 Llama 4 Scout \u0647\u0648 10M \u0633\u0648\u0627\u0621 \u0642\u0645\u062a \u0628\u062a\u0634\u063a\u064a\u0644 F16 \u0623\u0648 Q4_K_M. \u0645\u0627 \u064a\u062a\u063a\u064a\u0631 \u0647\u0648 \u0645\u0627 \u0625\u0630\u0627 \u0643\u0627\u0646 \u0628\u0625\u0645\u0643\u0627\u0646\u0643 \u062a\u062d\u0645\u0644 \u0643\u0627\u0634 KV \u0644\u0647\u0630\u0627 \u0627\u0644\u0633\u064a\u0627\u0642 \u0641\u064a \u0627\u0644\u0630\u0627\u0643\u0631\u0629. \u0642\u0627\u0631\u0646 \u0627\u0644\u0633\u064a\u0627\u0642 \u0648\u0627\u0644\u062a\u0633\u0639\u064a\u0631 \u0639\u0628\u0631 \u0627\u0644\u0646\u0645\u0627\u0630\u062c \u0639\u0644\u0649 <a href=\"https:\/\/convly.ai\/ar\/llm-leaderboard\/\">\u0644\u0648\u062d\u0629 \u062a\u0635\u0646\u064a\u0641 \u0627\u0644\u0646\u0645\u0627\u0630\u062c \u0627\u0644\u0644\u063a\u0648\u064a\u0629 \u0627\u0644\u0643\u0628\u064a\u0631\u0629 (LLM)<\/a>.<\/p>","protected":false},"excerpt":{"rendered":"<p>GGUF is a single-file container format for quantized models, created for llama.cpp. One .gguf file holds the weights, the tokenizer, [\u2026]<\/p>\n","protected":false},"author":1,"featured_media":2606,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"site-sidebar-layout":"default","site-content-layout":"","ast-site-content-layout":"default","site-content-style":"default","site-sidebar-style":"default","ast-global-header-display":"","ast-banner-title-visibility":"","ast-main-header-display":"","ast-hfb-above-header-display":"","ast-hfb-below-header-display":"","ast-hfb-mobile-header-display":"","site-post-title":"","ast-breadcrumbs-content":"","ast-featured-img":"","footer-sml-layout":"","ast-disable-related-posts":"","theme-transparent-header-meta":"","adv-header-id-meta":"","stick-header-meta":"","header-above-stick-meta":"","header-main-stick-meta":"","header-below-stick-meta":"","astra-migrate-meta-layouts":"default","ast-page-background-enabled":"default","ast-page-background-meta":{"desktop":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"ast-content-background-meta":{"desktop":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"footnotes":""},"categories":[9],"tags":[],"class_list":["post-2605","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-tutorials"],"_links":{"self":[{"href":"https:\/\/convly.ai\/ar\/wp-json\/wp\/v2\/posts\/2605","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/convly.ai\/ar\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/convly.ai\/ar\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/convly.ai\/ar\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/convly.ai\/ar\/wp-json\/wp\/v2\/comments?post=2605"}],"version-history":[{"count":2,"href":"https:\/\/convly.ai\/ar\/wp-json\/wp\/v2\/posts\/2605\/revisions"}],"predecessor-version":[{"id":2685,"href":"https:\/\/convly.ai\/ar\/wp-json\/wp\/v2\/posts\/2605\/revisions\/2685"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/convly.ai\/ar\/wp-json\/wp\/v2\/media\/2606"}],"wp:attachment":[{"href":"https:\/\/convly.ai\/ar\/wp-json\/wp\/v2\/media?parent=2605"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/convly.ai\/ar\/wp-json\/wp\/v2\/categories?post=2605"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/convly.ai\/ar\/wp-json\/wp\/v2\/tags?post=2605"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}