Empfohlener Stack

Was bauen Sie? Customise a model

Eine Technologie pro Position, mit der Begründung und womit man sie ersetzen kann.

Base model

model

Family of decoder-only Transformer language models trained with a generative objective.

Why aus dem Graphen abgeleitet

The agents converse through a model backend, with the hosted models among the supported ones.

Alternativen

technique

Continue training a pre-trained model on task-specific data to adapt its behaviour.

Why aus dem Graphen abgeleitet

LoRA used for Fine-tuning

Alternativen

technique

Fine-tuning by training small low-rank adapter matrices instead of all weights.

Why aus dem Graphen abgeleitet

QLoRA adds 4-bit quantisation of the frozen base model on top of LoRA adapters.

Alternativen

technique

Cached attention keys and values that make generating one token at a time affordable.

Why aus dem Graphen abgeleitet

Reusing a shared prefix means reusing the cached keys and values for it.

Alternativen