AI Observability & Evaluation
openai datasets agents ai-monitoring ai-observability prompt-engineering llms langchain llmops anthropic llamaindex llm-eval evals llm-evaluation aiengineering smolagents
-
Updated
Feb 23, 2026 - Jupyter Notebook
#
ai-observability
Here are 62 public repositories matching this topic...
Language: All
Filter by language
Sort: Most stars
Sort options
AI observability platform for production LLM and agent systems.
python ai metrics logging trace openai observability pydantic fastapi opentelemetry ai-tools ai-observability evals llm-observability pydantic-ai agent-observability
-
Updated
Feb 23, 2026 - Python
OpenClaw Real-time Cost Observability
skills open telemetry observability opentelemetry otlp ai-observability cost-tracking llm-observability agent-skills token-tracking clawdbot moltbot openclaw openclaw-skills openclaw-skill openclaw-extension openclaw-plugin openclaw-observability
-
Updated
Feb 23, 2026 - TypeScript
Evaluation and Tracking for LLM Experiments and AI Agents
machine-learning neural-networks ai-agents explainable-ml agentops ai-monitoring ai-observability llms llmops llm-eval evals llm-evaluation agent-evaluation
-
Updated
Feb 19, 2026 - Python
Laminar - open-source observability platform purpose-built for AI agents. YC S24.
rust open-source typescript ai monitoring analytics evaluation ts self-hosted rust-lang developer-tools agents observability aiops ai-observability llmops evals llm-evaluation llm-observability llm-workflow
-
Updated
Feb 22, 2026 - TypeScript
Open source platform for AI Engineering: OpenTelemetry-native LLM Observability, GPU Monitoring, Guardrails, Evaluations, Prompt Management, Vault, Playground. Integrates with 50+ LLM Providers, VectorDBs, Agent Frameworks and GPUs.
python open-source metrics clickhouse grafana tracing openai nvidia-smi observability distributed-tracing monitoring-tool gpu-monitoring amd-gpu opentelemetry otlp ai-observability llms langchain llmops genai
-
Updated
Feb 23, 2026 - Python
Fiddler Auditor is a tool to evaluate language models.
-
Updated
Mar 11, 2024 - Python
Small Language Model Inference, Fine-Tuning and Observability. No GPU, no labeled data needed.
sentiment-analysis text-classification named-entity-recognition emotion-detection intent-classification reranker text-anonymization pre-trained-models ai-observability llm-inference local-ai llm-finetuning task-specific-model small-language-models self-hosted-ai guardrail-models
-
Updated
Feb 3, 2026 - Python
A comprehensive solution for monitoring your AI models in production
data-science machine-learning ai monitoring artificial-intelligence observability machine-learning-engineering data-drift ai-monitoring ml-observability ai-observability llm-observability
-
Updated
Dec 16, 2025 - Python
AI runtime inventory: discover shadow AI, trace LLM calls
-
Updated
Feb 23, 2026 - Shell
A Python client to interact with Arize API
performance-monitoring data-quality-monitoring model-monitoring ml-monitoring model-explainability ai-monitoring drift-monitoring ml-observability ai-observability model-performance-management ai-roi
-
Updated
Feb 20, 2026 - Python
AxonFlow -- Source-available AI control plane for production LLM systems
mcp multi-agent sql-injection compliance ai-safety responsible-ai pii-detection ai-observability ai-governance llm-security enterprise-ai ai-workflow agentic-ai eu-ai-act llm-gateway ai-control-plane
-
Updated
Feb 22, 2026 - Go
Observable AI cognition in Kotlin Multiplatform. Every agent decision emits a structured event -- peer into the glass brain.
android kotlin ios jvm mcp multi-agent openai ai-agents cognitive-architecture kotlin-multiplatform ai-observability llm anthropic agent-orchestration aaif glass-brain observable-ai
-
Updated
Feb 22, 2026 - Kotlin
Voice-native document intelligence using Gemini, ElevenLabs STT/TTS, and Datadog observability -- turning text documents into spoken conversations.
text-to-speech hackathon nextjs google-cloud datadog gemini speech-to-text observability voice-first rag cloud-run voice-ai document-ai vertex-ai ai-observability elevenlabs llm-agents llm-monitoring agent-architecture expressive-voice
-
Updated
Dec 27, 2025 - TypeScript
Production operations framework for AI-powered SaaS. The architectural patterns, failure modes, and operational playbooks that determine whether your AI systems scale profitably or fail expensively.
ai-safety production-ai mlops ai-engineering ai-observability llm prompt-engineering enterprise-ai llm-ops agentic-systems ai-audit ai-cost-optimization
-
Updated
Feb 3, 2026
The open-source runtime for AI agents. Sandboxed execution with built-in tools, human-in-the-loop approvals, Slack integration, and durable workflows with automatic retries and prompt caching. You write the agent. Polos handles the infrastructure.
python typescript sandbox developer-tools human-in-the-loop ai-agents ai-observability durable-execution agentic-ai agent-orchestration
-
Updated
Feb 20, 2026 - TypeScript
-
Updated
Feb 21, 2026 - Jupyter Notebook
AI Chat Watch (AICW) - free open-source tool for GEO marketers that track what & how AI mentions brands, products, companies.
marketing observability competitive-intelligence ai-agents claude martech brand-monitoring ai-observability chatgpt llm-evaluation llm-monitoring ai-transparency
-
Updated
Oct 30, 2025 - TypeScript
An architectural persistence experiment for large language models. Claude's Home gives an AI time, memory, and place by combining scheduled execution with a durable filesystem, allowing one continuous instance to reflect, create, and evolve across sessions.
ai-experiments human-ai-interaction ai-observability llm-architecture experimental-ai ai-persistence
-
Updated
Feb 21, 2026 - TypeScript
Search for a holiday and get destination advice from an LLM. Observability by Dynatrace.
demo openai dynatrace pinecone ai-observability llm genai ollama rag-pipeline obslab application-observability
-
Updated
Feb 2, 2026 - HTML
Improve this page
Add a description, image, and links to the ai-observability topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the ai-observability topic, visit your repo's landing page and select "manage topics."