Why Uluka
A model has to fit in a GPU's memory, and GPUs with more memory cost much more. We compress the model so it fits on a cheaper GPU.
GPU memory sets the price
A model runs inside a GPU's memory, so its size decides which GPU you rent. The price climbs quickly with memory: a 16 GB GPU costs $0.53 an hour and an 80 GB GPU costs $6.88.
With its image module, Google's 4-bit Gemma 4 26B is 15.6 GB. That leaves under 0.5 GB free on a 16 GB GPU, so it needs the 24 GB one. Uluka Dango, images included, is 8.7 GB and fits on the 16 GB GPU.
Price per hour, by GPU memory
16 GBT4. Fits Uluka Dango, 8.7 GB
24 GBL4. Fits Google's 4-bit, 15.6 GB
48 GBL40S. Room for larger models
80 GBH100. Fits the original, 51.6 GB
Large models need several GPUs
A model too big for one GPU has to be split across several, and the bill multiplies. Inkling-Small from Thinking Machines is a 266B model, about 10 times the size of Gemma 4 26B. Its official builds (NVFP4 and FP8) are 171 to about 270 GB and need two to four 96 GB GPUs.
Our custom compression architecture should get the model to about 80 GB, small enough for one GPU.
Inkling-Small in each format
OriginalBF16, Thinking Machines
FP8Red Hat
NVFP4Thinking Machines
Q4UD-Q4_K_XL, Unsloth
Q3UD-IQ3_XXS, Unsloth
Uluka 266Bour target
Where we are
What works today, and what we are building next.
- WorkingCompression and runtime work on two models: Uluka Dango, now live in Uluka chat, and the Uluka 7B proof of concept
- In progressServing more users at once, and using less memory in long conversations
- PlannedUluka 266B, compressed from Inkling-Small, and support for hardware beyond NVIDIA GPUs