ReferenceResearch tree
Era 4 · Series BQuantization
A GPU Node runs INT8 or INT4 weights: faster and smaller, and its answers are worth 5% or 15% less.
Prerequisites#
Research these first: Model serving.
A GPU Node runs INT8 or INT4 weights: faster and smaller, and its answers are worth 5% or 15% less.
Research these first: Model serving.