Entdecken

13 technologies shown

• vLLM tool High-throughput inference server built around paged attention and continuous batching.
• Ollama tool Runs open-weight models locally behind a simple command and HTTP API.
• Tokenizer tool Component that converts text to tokens and back.
• LM Evaluation Harness tool Standard harness for running language model benchmarks.
• llama.cpp tool CPU-first inference for quantised models, in portable C++.
• Weights & Biases tool Experiment tracking and visualisation for training runs.
• MLflow tool Open platform for tracking experiments and managing model lifecycle.
• SGLang tool Serving engine built around reusing shared prompt prefixes.
• Triton Inference Server tool NVIDIA's server for deploying models from several frameworks behind one API.
• TensorRT-LLM tool NVIDIA's compiler and runtime for optimised LLM inference on its GPUs.
• Text Generation Inference tool Hugging Face's serving stack for transformer models.
• Ragas tool Evaluation library that scores retrieval and answer quality separately.
• LangSmith tool Tracing and evaluation for model applications, showing each step of a run.