Windows Desktop Application
GpuLLM is a free Windows application that lets you comprehensively evaluate local LLMs — measuring both inference speed (tokens/s, latency) and model capability (MMLU, C-Eval accuracy). Everything is 100% offline — chat, benchmark, GPU monitoring, model management — the only thing that touches the internet is downloading models. Find your ideal model-to-hardware match in minutes, not hours. No cloud dependency, no API keys, no data ever leaves your machine.
GpuLLM is the most complete free Windows suite for local LLM evaluation. Here's everything you get — no subscriptions, no paywalls.
Run GGUF-format large language models directly on your GPU using LLamaSharp with llama.cpp backend. Supports CUDA, Vulkan, and CPU fallback — automatically selects the best available backend.
Beyond the curated catalog, import any GGUF-format model from your local disk. Point GpuLLM to your own GGUF files and start chatting with custom or community models instantly — no cloud upload required.
Track GPU utilization, VRAM usage, temperature, and power draw in real time. Includes 60-second sparkline history charts and multi-GPU support with automatic device discovery.
Chat with any model and instantly measure your GPU's token generation speed (tokens/s), Time-to-First-Token, and total latency. The built-in cost estimator compares your local inference against cloud API pricing — showing exactly how much money you save by running models locally.
Browse 13 curated models including Qwen2.5-Coder, DeepSeek-R1, Llama 3.2, Gemma 3, and Mistral. Filter by category — Coding, Chat, or Reasoning — and check VRAM compatibility with one click.
Download models directly from HuggingFace Hub or ModelScope. Supports resumable downloads with SHA-256 verification, multi-source fallback, and license consent management.
Pioneer a new interaction pattern — assign two LLM roles in a single chat: a Responder that answers your questions, and a Reviewer that critically evaluates and improves the answer. This LLM-as-a-Judge / Agentic Debate pattern, widely discussed in AI research, lets you harness the strengths of two different models simultaneously for higher-quality, self-correcting conversations.
Prism Cat — a live animated mascot that responds to your GPU utilization levels. Watch the cat's emotional state and color shift in real time as inference workloads ramp up.
Benchmark your models against comprehensive question sets: 100 English questions (MMLU, sampled from 57 subjects) and 100 Chinese questions (C-Eval), covering 10+ dimensions including Mathematics, Physics, Chemistry, Biology, Computer Science, History, Literature, Economics, Philosophy, and Geography. Get per-category accuracy, difficulty-level breakdown, and exportable Markdown reports.
Each inference produces a detailed report with backend chain, GPU peak metrics, and token-level statistics. Export full chat conversations and reports as A4 PDF documents.
Most alternatives let you chat with models but don't measure performance, evaluate accuracy, or track GPU metrics. You're left guessing which model works best on your hardware — and whether you're actually saving money versus cloud APIs.
Every entry includes the correct HuggingFace repo, quantization, and file size — download with one click.
| Display Name | Creator | Category | Size | VRAM |
|---|---|---|---|---|
| Qwen2.5-Coder 3B | Alibaba | Coding | 1.9 GB | 8 GB |
| Qwen2.5-Coder 7B | Alibaba | Coding | 4.2 GB | 16 GB |
| DeepSeek-Coder V2 Lite | DeepSeek | Coding | 9.0 GB | 24 GB |
| Llama 3.2 1B Instruct | Meta | Chat | 0.7 GB | 4 GB |
| Llama 3.2 3B Instruct | Meta | Chat | 2.0 GB | 8 GB |
| Gemma 3 4B Instruct | Chat | 2.5 GB | 8 GB | |
| Mistral 7B Instruct | Mistral AI | Chat | 4.1 GB | 16 GB |
| Qwen2.5 7B Instruct | Alibaba | Chat | 4.7 GB | 8 GB |
| Qwen2.5 14B Instruct | Alibaba | Chat | 8.9 GB | 24 GB |
| DeepSeek-R1-Distill 1.5B | DeepSeek | Reasoning | 1.0 GB | 4 GB |
| DeepSeek-R1-Distill 7B | DeepSeek | Reasoning | 4.2 GB | 16 GB |
| DeepSeek-R1-Distill 14B | DeepSeek | Reasoning | 8.5 GB | 16 GB |
| DeepSeek-R1-Distill 32B | DeepSeek | Reasoning | 20 GB | 32 GB |
High-performance C++ inference engine with managed .NET bindings — the same backend that powers countless local AI applications worldwide.
Modern Windows desktop UI with Fluent Design System components. Dark/light theme support with glass effect materials and smooth animations.
CUDA 12 → Vulkan → CPU automatic fallback chain. The app detects available hardware and selects the fastest backend without manual configuration.
All model inference, data processing, and file operations happen exclusively on your local device. No telemetry, no cloud dependency, no network required once models are downloaded.
Tokens per second measures how fast a language model generates text. Higher tokens/s means more responsive conversations. GpuLLM measures both peak and sustained throughput to give you a complete picture of your GPU's LLM performance.
Time To First Token (TTFT) measures the latency between sending a prompt and receiving the first word of the response. Lower TTFT means snappier interactions. GpuLLM tracks TTFT to help you compare model responsiveness.
Massive Multitask Language Understanding (MMLU) is a standard benchmark testing knowledge across 57 subjects. GpuLLM includes 100 sampled questions to evaluate your model's breadth of knowledge and reasoning ability.
C-Eval is a Chinese evaluation suite covering 52 disciplines and four difficulty levels. GpuLLM includes 100 questions to test your model's Chinese language understanding and domain knowledge.
GpuLLM is a Windows desktop application. All inference, benchmark, and evaluation runs locally on your device. No model data or chat content is ever uploaded to any server. The download link opens Microsoft Store in your default browser.
All tools on fastool.io run entirely in your browser — zero data leaves your device. No personal data is collected, stored, or transmitted to any server. Solar calculations use SunCalc.js; lunar data uses JPL DE440 ephemeris; coordinate transforms use publicly documented EPSG/OGC standards. This site requires no signup, no account, and no cloud processing.