AI developer tools

Open-source LLM tracing and evaluation, OpenTelemetry-native.

Open-source observability proxy for LLM apps.

Open-source LLM tracing, evals and prompt management.

Tracing, evals and monitoring from the LangChain team.

OpenLLMetry - open-source OpenTelemetry instrumentation for LLMs.

LLM app tracing and evals inside Weights & Biases.

Caching, rate-limiting and analytics in front of any LLM API.

AI plugins for the Kong API gateway - routing, prompt guarding.

Open-source proxy - one OpenAI-format API for 100+ providers.

AI gateway with routing, caching, guardrails and observability.

Evals, prompt playground and logging for LLM products.

Hosted platform for the open-source DeepEval framework.

Automated evaluation and guardrails for LLM systems.

Open-source CLI and library for LLM evals and red-teaming.

Open-source LLMOps - prompt playground, evals, observability.

Open-source prompt engineering platform with evals.

Prompt registry, versioning and analytics for teams.

Open-source framework for input/output validation of LLMs.

Prompt-injection and content-safety API for LLM apps.

ML/LLM security - model scanning, LLM firewall, red-teaming.

Fine-tune and deploy small models that beat GPT-4 on your task.

Fine-tune and serve open models (LoRA), built on Ludwig.

Fine-tune Llama, Qwen and more, then serve on Together.

Framework and cloud for orchestrating multi-agent crews.

Open-source platform for building and operating LLM apps/agents.

Deploy and manage long-running, stateful agents.

Managed parsing, indexing and retrieval from LlamaIndex.

Open-source data curation and feedback collection for LLMs.

Open-source data labeling for every data type, incl. LLM RLHF.