# Quantization

> Storing a model’s weights at lower precision, such as 8-bit or 4-bit integers rather than 16-bit floats, so the model uses less memory and computes faster, at some cost in answer quality.

Performance

## In Quorum

Quantization research adds FP16, INT8 and INT4 to a GPU Node. INT8 serves 2× the tokens and INT4 3×, and each answer is worth 5% or 15% less. It pays when the GPU is the bottleneck and the answers can afford to be slightly worse.

## Design

[GPU Node and precision](https://wiki.playquorum.dev/design/05-components.md#gpu-node)

## Read next

[Model inference](https://wiki.playquorum.dev/glossary/model-inference.md), [Continuous batching](https://wiki.playquorum.dev/glossary/continuous-batching.md), [Utilization](https://wiki.playquorum.dev/glossary/utilization.md), [Cold start](https://wiki.playquorum.dev/glossary/cold-start.md)

## Learn more

- [Hugging Face docs: Quantization](https://huggingface.co/docs/transformers/main/en/quantization/overview)

## See also

- [Glossary](https://wiki.playquorum.dev/glossary.md)

Source: https://wiki.playquorum.dev/glossary/quantization