Industry News
Strata, Ollama, LM Studio: These systems utilize 8GB of video memory to enable 128K of context space. What the title doesn’t mention is that these devices also come equipped with 64GB of RAM.
“The claim that ‘8GB of video memory is sufficient to handle a context size of 128K’ in Strata 0.2.10 is not incorrect; official tests have confirmed that the system can indeed start up and maintain a context size of 131,072 tokens. However, the same update document also states that ‘32GB of memory is the minimum requirement, 48GB is recommended, and 64GB is ideal.’”
The 64GB that is not mentioned in the title is actually the real price associated with this specification.
Those who run models locally generally have three tools to choose from: Strata, Ollama, and LM Studio. These tools are often compared with each other in terms of performance, but the way they handle “context” varies significantly among them. In fact, this aspect is far more crucial than speed when determining whether your computer will experience lag or freeze while running these models.
Both items are labeled as “128K”; where are the corresponding bills recorded?
The main component of context storage is the KV cache, and its size increases linearly with the length of the data; it is not equivalent to the total number of parameters in the model. In the case of Ollama, some people have directly reduced the value of `num_ctx` for a 14B model to 128K, which caused the memory usage to surge to around 28GB. Just because a model “supports” such large amounts of context data does not mean that your hardware is capable of handling it.
There are also consequences for choosing a more compact version. When running the Qwen3 8B model on an RTX 3060 with 12GB of video memory, the amount of video memory used increases from approximately 6.0GB to 7.5GB as the context size is changed from 4K to 16K. Meanwhile, the processing speed drops from 55.2 token/s to 42.0 token/s, representing a reduction of around 24%. When using the 14B model in a 16K context, the speed further decreases to only 22.7 token/s. The initial price of this graphics card was 329 US dollars.
Strata's approach involves splitting the cache into different layers. When there is insufficient video memory, the KV window residing on the GPU is reduced in size, while the remaining data is stored in memory or, if necessary, on the hard drive. The expert weights are also distributed across video memory, memory, and a 28.8GB SSD, which is used to store lookup tables.
This is exactly why it can start operating with just 8GB of memory. The graphics video memory has been reallocated to the system RAM and the hard drive, so the total amount of available memory has not decreased; in fact, it has even increased.
| tool | How to set the context? | When you can't let go... | major weakness |
|---|---|---|---|
| Strata 0.2.10 | The default value is relatively long; it can be adjusted to 256K. | Cache is organized in layers, with data stored both in memory and on hard drives. | These processes consume memory and disk resources; by default, only one request is executed at a time. |
| Ollama | num_ctx: Determined based on the call, Modelfile, or environment variables. | Silence overflows into the memory without generating any errors. | The model was uninstalled after approximately 5 minutes, and the reloading process was very slow. |
| LM Studio | Slider, or the –context-length parameter in LMS loading. | Manually adjust the GPU allocation during loading. | Graphical interfaces consume more memory; Intel Macs can only use the CPU for such tasks. |
Who generated those 63 tokens?
There are two sets of official figures, and it’s very easy to simply remember the one that looks more favorable. When using the DFlash2 mode with an 8GB limit, the performance ranges from 63 to 73 tokens per second. However, when using the actual Intel Arc A750 graphics card with only 8GB of memory, the performance increases to between 12.7 and 14.8 tokens per second.
It is approximately five times lower.
The first set of numbers seems to indicate that a limit has been set on the amount of video memory available for high-end graphics cards, while the second set more closely reflects what ordinary users with 8GB of video memory would encounter. The statements from third-party sources also corroborate this: according to various tests, 8GB graphics cards have approximately 15 tok/s of performance, whereas the 12GB RTX 5070 achieves around 53 tok/s.
Moving on to the higher-end segment, a developer used an RTX 5090 with a starting price of $1,999 and 96GB of memory to run applications with a context size of 128K. In this setup, the benchmark score reached 159 tok/s. When the context size was increased to 256K, the average performance dropped from around 190 tok/s to approximately 175 tok/s, representing only a decrease of about 10%. This was because the expert cache hit rate remained stable at between 90% and 97%, and approximately 97% of the key-value read operations were still processed by the graphics card itself. However, conducting this evaluation was not easy: during the operation of the service, 88 to 92GB of memory were consumed, with about 48GB being allocated to the swap space. As a result, less than 800MB of video memory remained available, and even the application launcher could not be opened. To address this issue, the developer had to use the –vram-elastic option to allocate additional video memory to the desktop environment. In his conclusion, he stated that the available resources were “almost sufficient” but also complained that a context size of 128K was still quickly depleted during actual programming sessions, forcing agents to frequently compress data and discard unnecessary details.
128K is not sufficient for programming intelligent agents.
The most troublesome thing about Ollama is that it doesn’t make any sound at all.
When the model cannot fit within the available video memory, Ollama will quietly “overflow” into the system’s regular memory, causing its performance to drop significantly. However, no error messages are displayed on the interface. The question “Why is it so slow?” is perhaps the most frequently asked one in its community, and the answer can often only be obtained by running the “ollama ps” command—for example, when it shows that 60% of the CPU power and 40% of the GPU capacity are being used.
Another small trick is that the model is automatically uninstalled after approximately 5 minutes by default. If you ask again after some time, you will have to wait for it to be reloaded; in this case, you need to set the OLLAMAKEEP_ALIVE option.
Its benefits are indeed considerable. Once installed, it can be used as a background service, eliminating the need to handle APIs manually. In tests using the same model as the RTX 3060, it achieved the same results as llama.cpp, producing 64.5 tokens per second, and its encapsulation process barely had any negative impact on performance. In March of this year, the MLX engine for Apple Silicon was also added to it. It’s important to note the cloud-based functionality: when enabled, prompts will be processed through the Ollama server. The Pro version costs $20 per month, while the Max version costs $100 per month. Those who prefer to use local versions for privacy reasons should avoid enabling this feature.
LM Studio wins where you can see it.
Before loading the software, users can adjust the context by dragging the slider and view the GPU allocation; thus, issues related to data overflow are less likely to occur accidentally. Both the home and commercial versions are completely free. The 0.4.0 version released in January also included the interface-free service llmster, which addressed the previous limitation of requiring the software to be run within a window.
There’s really no need to argue about speed. Some comparative tests have shown a difference of 2% to 3% between the two; other blogs claim that LM Studio results in an additional 0.3% in overhead compared to the native llama.cpp implementation, while Ollama requires 10% more resources. Yet another evaluation found no significant difference at all. Given these varying findings, such minor differences are not enough to justify changing one tool for another.
The weaknesses lie in these areas: graphical interfaces consume more memory; on Intel Macs, only CPU-based rendering is available; on Windows systems running AMD hardware, Vulkan technology is used instead. The iPhone version of “Locally” doesn’t actually run on the phone itself but connects to the user’s home computer instead.
What makes Strata so special?
Unlike the other two, it is actually an engine designed specifically for large-scale MoE models. Qwen3.8-Flash-Next has approximately 125B parameters; IQ3_S, after quantization and the use of lookup tables, occupies around 84GB of memory. Additionally, the speculative decoding component takes up about 7GB more. When such a model is run on an RTX 3060 using Ollama, even models with sizes of 30B or 3AB would exceed the available 6.6GB of video memory. Its true value lies in enabling home computers to handle models of this scale, rather than in making smaller models run faster.
The list of costs is quite lengthy:
During the initial startup process, 35 to 55 GB of data must be loaded into memory, which may cause the computer to become temporarily unresponsive for 1 to 3 minutes. The disk space required initially is approximately 80 GB, but this amount can exceed 150 GB in some cases. The exact wording of the same prompt may vary when executed multiple times, as the specific requirements for dedicated video memory are subject to change. macOS, Intel Arc, and Strix Halo are still considered experimental versions and require building from source code.
Significant progress has been made in both multi-card performance and tool calling capabilities: In version 0.2.8, the decoding speed of Intel Arc Pro B65 cards improved by 17% to 23% in these two areas. Moreover, when tool calling is enabled by default, the success rate increased from 92.9% to 98.3%. However, these performance improvements are dependent on specific system configurations; single-card setups, A-series graphics cards, or systems with three or more graphics cards will not enjoy this same level of improvement.
Maturity is another matter altogether. The project was primarily developed by Niko Veit alone. In version 0.2.7, improvements were made to increase the performance of graphics cards with less than 12GB of video memory. However, these enhancements were later removed because repeated text appeared during long sessions, and users had to manually enable them again. The default DFlash2 configuration in this version reduced the amount of video memory allocated for rendering from approximately 5.2GB to around 625MiB – which is indeed a significant improvement. The developers also stated that this new approach is intended to be used primarily for single requests; for multiple concurrent tasks, the older MTP solution remains more efficient.
The newly added ISTA-DAS Coder is derived from the DeepSeek-V4.1-Flash framework. There have been very few third-party tests conducted on it, so for now, we can only rely on the official information provided by its developers.
It will depend on the type of memory module I have available.
If your system has less than 32GB of RAM, don’t even consider using Strata. For models with 8GB to 14GB of memory, you can use Ollama or LM Studio. Set the context size to around 16K, and the performance will still be acceptable.
If you want to use Ollama for both command-line operations and permanent services, remember to check the “ollama ps” command. If you prefer to examine the details before loading it, or if you want to try out various quantized versions available on Hugging Face, then choose LM Studio.
For systems with 64GB or more of RAM and at least 12GB of video memory, if you really want to use local models to train programming agents such as Strata, it’s worth giving it a try. Make sure to free up at least 100GB of disk space first. Also, don’t expect parallel sub-agents to work efficiently; by default, they will have to wait in line to be executed.
For those who only have 8GB of video memory but 64GB of system memory, Strata can still support 128K connections. A speed of around 15 tokens per second is sufficient for chatting, but if you want the agent to use this capacity to create a large database, you’ll need to be patient.
Relevant comparisons: “FLUX 3 Image Klein and Qwen-Image-2.1: The version claimed to be capable of running with 8GB of memory; the text encoder alone occupies 8.88GB of space,” “The weights used in Mistral Large 4 are consistent with those of GLM-5.3.”