Will this local model fit on my machine?

I run local models on an M4 Pro Mac mini with 48 GB of memory. You might have a different Mac, a PC with a graphics card, or a laptop that already has most of its RAM in use. Copying my model choice won’t necessarily work for you.

Start here

1. Open your usual apps and check how much memory is left. On an Apple Silicon Mac, use available RAM. For a model running entirely on a PC’s separate graphics card, use available GPU memory. I’ll explain where to find these numbers below.
2. Pick the named model for your memory range in the table below. Download its 4-bit version in a format your model app supports.
3. Load one model and set its context to 4K if the app lets you choose. Context is how much text the model can take into account at once. Turn thinking off for this first check if the model supports that, and start with a short chat.
4. Try a task where you know what a useful answer looks like. Watch whether the rest of your computer stays responsive. If it struggles, reduce context or try the next smaller model.

If you need a model app, LM Studio lets you download and chat with models on Mac, Windows, and Linux. I use oMLX on Apple Silicon to make models available to my other apps. My setup post covers that connection.

Download size and memory use

You download a model to disk. To run it, the computer loads its learned values into RAM or GPU memory. These values are called weights, and “weight files” are the files that contain them. A model with 5 GB of weight files takes about 5 GB of disk space and needs roughly that much memory for the values when loaded. The exact amount depends on the model and app.

It needs extra memory to process your question and keep track of the conversation. A 5 GB model needs more than 5 GB of available memory to run comfortably. Your browser, operating system, and other apps also use memory. The computer’s total RAM isn’t all available for the model.

Find out how much memory you have left

On a Mac, open your usual apps, leave the model unloaded, and open Activity Monitor → Memory. Subtract Memory Used from Physical Memory for a rough starting budget. If you have 16 GB and are using 6 GB, plan around the remaining 10 GB. Check during normal work. A reading straight after restarting may look generous because fewer apps are running.

That budget will change as you work, and macOS can reclaim some memory used for cached data. Check the Memory Pressure graph too. If it’s yellow or red before you load a model, your existing apps are already putting the computer under pressure. Apple explains the readings.

On Apple Silicon Macs, the main processor and graphics processor share memory, so start with the RAM budget. Your model app can also impose a GPU memory limit, so the amount left in Activity Monitor isn’t a guarantee that a model will load.

On a PC with a separate graphics card, the card has its own memory, called VRAM. For a model running entirely on that card, use its available VRAM as your budget. In Windows Task Manager, open Performance → GPU, find Dedicated GPU memory, and subtract the amount in use from the total. The Shared GPU memory reading refers to system RAM, so don’t add it to the card’s VRAM budget. Microsoft explains these readings.

32 GB of RAM plus an 8 GB graphics card doesn’t give you one 40 GB pool. Some apps can split the model between the card and main memory, but that can slow it down. Ollama explains how to check where a model is loaded.

If you’re running the model on the CPU, use available system RAM instead. It may run more slowly.

What should I try with the memory I have left?

I’d start with a 4-bit model and a short conversation. A 4-bit version takes less space on disk and in memory than an 8-bit or 16-bit version of the same model. That leaves more memory for the conversation and your other apps.

The table uses versions labelled 4-bit as a common starting point. They don’t all compress the model in exactly the same way, and some values stay at higher precision. Quantization can reduce answer quality. Once the model runs comfortably, try an 8-bit version if you want to see whether the answers improve enough to justify the extra memory.

These are starting suggestions based on the weight-file sizes, with room left for short chats. I haven’t tested every row on different machines. They aren’t measured minimum requirements or a ranking of answer quality. In the names, B means billion parameters: the learned values that make up the model.

Pick the download format that matches your app:

• MLX works on Apple Silicon Macs, including my oMLX setup. LM Studio supports it on these Macs too.
• GGUF works in compatible apps such as LM Studio on Windows, Linux, or Mac. In the linked downloads, choose the file with Q4_K_M in its name. That’s the 4-bit version whose size I’ve listed.

You can paste a model page’s link into LM Studio’s model search and download it there. In oMLX, search for the linked MLX model in its model browser. Keep the app’s model runtime up to date and check that it supports the model; the right file format alone isn’t enough. Let the app handle the download, including any model split across several files.

Sizes are rounded decimal GB, checked on 15 October 2026. The MLX column counts the main model weight files; the GGUF column counts the named text-model file. Formats can differ in their compression, retained precision, and included components, so their sizes aren’t directly interchangeable. Image or audio support may need extra files, so start with text chat.

For Qwen3.6, the main MLX weights are 22.1 GB, as installed in my setup. Its repository also contains optional acceleration and vision weights. Downloading the entire repository now takes about 24.7 GB on disk. Check the total your app plans to download.

With 12 GB of memory left, I’d try Qwen3.5-9B in 4-bit. Its MLX files take about 6 GB, leaving about 6 GB for the conversation and calculations. An 11 GB download would leave too little spare memory for a first attempt.

If you’re near the bottom of a range or planning a long conversation, try the previous row’s model. With less than 4 GB left, close some apps first. Check the size shown in your model app before downloading, because the files can change.

Keep one model loaded while you test it. Loading several models or sending several requests at once needs more memory. Before moving to a larger model, check whether it gives better answers on your tasks.

What the model name tells you

Take Qwen3.6-35B-A3B-OptiQ-4bit:

The 35B is the language model’s total parameter count, according to Qwen’s model card. Extra components, such as its image encoder, add more parameters. More parameters give a model more capacity to learn, which often means better answers when comparing similar models. Training quality matters too, so a smaller model can outperform a larger one.

The A3B means it uses about 3 billion parameters for each piece of text it produces. It selects different groups of values as it works, an approach called mixture of experts, or MoE. That reduces the work for each piece of text. An ordinary setup still keeps the whole 35B model in memory.

Gemma’s E4B needs a closer look. The E means “effective”: Google counts about 4.5 billion parameters for the main processing, but about 8 billion including extra stored values used to represent text. Those extra values need memory too. Use the download size to plan your RAM budget. If you see it in a model name, it means the model is trained to follow instructions and chat.

How capable is my Qwen compared with familiar cloud models?

To get a sense of answer quality, I look at independent tests such as the Artificial Analysis Intelligence Index. It combines tests of reasoning, knowledge, coding, and completing tasks with tools. Higher scores mean better performance across those tests.

These familiar models score close to Qwen in the Artificial Analysis Intelligence Index. I checked the scores on 15 October 2026, using version 4.3.2:

The reasoning setting controls how much effort the model can spend working through an answer. Changing it can change the score.

That puts Qwen close to Haiku 4.5 and GPT-5 mini in these settings, with the same displayed score as GPT-6 Luna with reasoning disabled. The scores are rounded. If you’ve used those models, this gives you a rough comparison across the tested tasks. It doesn’t mean they’ll answer any particular question equally well.

These scores aren’t IQ scores or percentages. A score of 18 doesn’t mean “18% intelligent.” Similar overall scores can also hide different strengths on individual tasks.

Artificial Analysis tests hosted Qwen3.6 with reasoning enabled. My OptiQ 4-bit copy and its settings may give different results. The speed test below also had thinking switched off, so it doesn’t measure the reasoning setup behind this score. I still check its answers on the tasks I need it to do.

What 4-bit means

There isn’t one default bit count. Original model releases commonly use 16-bit values, while some use 32 bits or already come compressed. Bits measure how much data it takes to store a value. Reducing the precision to 4 bits is called quantization. It saves space on disk and in RAM or GPU memory, but can reduce answer quality.

Going from 16 bits to 4 makes those values roughly a quarter of the size. The complete download won’t shrink by exactly that amount because it includes extra information, and some parts may stay at higher precision to preserve quality.

Long conversations need more memory

The model needs memory for what you’re talking about as well as for the model itself. That includes earlier messages and any documents you paste in. A coding agent also sends instructions, code, and results from its tools.

The amount of text a model can take into account at once is its context window, measured in tokens. A token can be a whole word, part of a word, or a punctuation mark. The model reads and produces text in these pieces. 4,000 tokens doesn’t mean 4,000 words.

“4K context” means roughly 4,000 tokens in total, shared by your question, earlier messages, any other instructions or documents, and the model’s answer.

Start at 4K context for a short chat with thinking off if your app lets you choose. Try 8K once that runs comfortably. Thinking uses tokens too: Qwen’s authors recommend at least 128K context for its full thinking setup. The 4K starting point here is only a quick check that it runs. Documents and coding agents can need much more. You may have to choose a smaller model to leave enough memory for them.

A model advertising “128K context” can handle that much text in principle, provided the machine has enough memory. Its temporary conversation data, often called the KV cache, uses memory too. A quick question can work well, then pasting in a document can make the same model struggle.

Memory bandwidth affects speed

Memory capacity tells you how much data the computer can hold. Memory bandwidth tells you how quickly its processors can read and write that data. Capacity is measured in GB; bandwidth is measured in GB per second, written as GB/s.

While generating an answer, a model repeatedly reads its learned values and conversation data from memory. Those reads can limit how quickly the text arrives, especially in a single conversation. NVIDIA explains why generation often runs into this limit.

Apple publishes these figures in its Mac mini, M5, MacBook Pro, and Mac Studio specifications. They’re advertised bandwidth figures, not measurements of what a model app will sustain:

My M4 Pro can move more data per second than the base M4. That helps when memory reads are slowing generation down. It won’t make every model twice as fast. The model’s design, processing power, and app affect the result too.

More RAM lets you fit a larger model or a longer conversation. It doesn’t necessarily make memory faster. If you’re comparing hardware for speed, check the chip or graphics card’s bandwidth too.

4-bit gives the computer less model data to read, which can speed up generation if the app handles the format efficiently. Qwen’s expert selection reduces how much of the model it uses for each token too. It still needs memory for the whole model.

If memory pressure stays comfortable but output is slow, bandwidth may be part of the problem. Try a smaller model or a version that uses fewer bits and compare the speed. The wait before the answer starts matters too: reading a long input can depend more on computing power than bandwidth.

Test it on your own work

Use your model app for a few checks:

1. Load the model and ask a short question. Let this first request finish, because it may include time spent loading the model into memory. Later requests can be faster while the app keeps it loaded.
2. Start a fresh chat and ask it to summarize a document you’d actually use. Notice how long it takes to start answering, whether the answer arrives smoothly, and whether the rest of your computer stays responsive.
3. Repeat a few times with your usual apps open. Watch memory while it’s working.

If the app shows token counts, note those too. Try to keep the answer length similar between attempts. Repeating exactly the same input can be faster because the app may reuse earlier work.

Here’s why I watch the wait before the answer starts. On my Mac mini, the Qwen model above gave these results:

I measured these on 13 October 2026 with oMLX 0.6.4 and the model already loaded. Each number is the middle result from three attempts. I asked for memory-saving tips and added different amounts of sample text to make the inputs longer. Thinking was off, randomness was set to zero, and answers were limited to 128 tokens. The app reported no reuse of previously processed input. These settings describe this small timing test; they aren’t recommended settings for answer quality.

The output stayed fairly quick once it started. The bigger input made me wait longer because the model had more to read. That wait alone doesn’t prove it was running out of memory. Check the rest of the computer too.

When to try something smaller

On a Mac, watch Activity Monitor’s Memory Pressure graph. If it turns yellow or red while the model runs, especially alongside slow answers or a sluggish desktop, try a smaller workload.

Also watch Swap Used. Swap means the computer is using disk space to make room when RAM is tight. Some swap already being used is normal; if it keeps growing during your test and the computer slows down, that’s a more useful warning.

If short chats work but longer ones struggle, reduce context or start a fresh conversation. If even short chats make the computer sluggish, unload other models and try a smaller one. Change one thing at a time so you can tell what helped.

If memory looks comfortable but answers are consistently slow, the model may need more processing power or memory bandwidth than your machine provides. A model can fit and still be too slow for how you want to use it.

I’d rather keep a smaller model that answers well while my browser and editor stay usable. Move up when you need better answers and have enough memory to try it.