Architecture
Architecture Overview
KubeSense consists of sensor components that collect data, processing components that enrich and transform it, and storage components that persist it for querying and visualization.
Sensor Components
KubeSensor
An eBPF-based sensor that collects trace data directly from the kernel on every node. It captures network events, system calls, and process-level behavior with minimal overhead — no application code changes required.
- Kernel-level observability for system calls, network activity, and process execution
- End-to-end distributed trace data across services and components
LogSensor
A lightweight agent on each node that reads log data from containers, applications, and the OS. Logs are forwarded to the Log Aggregator for processing and storage.
- Auto-detects log files and sources via configuration or auto-discovery
- Supports multiple filesystems and custom log paths
- Fault-tolerant with retry mechanisms during network or aggregator outages
Metrics Scraper
Collects infrastructure and application metrics from Kubernetes APIs, Docker containers, cloud provider services (AWS CloudWatch, GCP Monitoring), and Prometheus-compatible endpoints. Scraped metrics are ingested into the Metrics Store.
Processing Components
Log Aggregator
Preprocesses log streams before storage — parsing, transforming, enriching with infrastructure metadata, redacting sensitive data, and filtering unwanted entries.
Kubecol
Processes and enriches trace data with infrastructure metadata and auto-tags. Integrates with Kubernetes, Docker, and cloud provider APIs for infrastructure monitoring. Batches trace data for efficient storage.
Alert Engine
Evaluates user-defined thresholds and conditions against metrics data. Forwards alerts to configured contact points — Email, Slack, PagerDuty, and webhooks. Maintains historical alert data for trend analysis.
Storage Components
Datastore
Purpose-built storage for logs and traces, optimized for:
- Full-text search and aggregation queries
- Compression with smart metadata management
- Tiered storage with automatic migration to cold/archival storage based on retention policies
- Replication for high availability and failover
Metrics Store
Time-series storage for infrastructure and application metrics:
- High write/read throughput for large-scale metrics ingestion
- Distributed querying layer that minimizes compute and memory usage
- Specialized indexing and compression for time-series data
Kubecol DB
Relational storage for configuration settings, infrastructure metadata, and supplementary data that supports efficient querying in the Datastore.
Application Layer
User Interface
The web-based dashboard for monitoring logs, traces, metrics, infrastructure, and alerts. Provides visualization, querying, and management capabilities.
API Service
Middleware layer between the UI and backend storage. Processes requests, queries the Datastore and Metrics Store, and returns structured responses.
KubeSense AI
AI-powered intelligence layer that provides:
- Root Cause Analysis — Correlates logs, traces, and metrics to automatically identify failure sources and reduce MTTR
- DevOps Bot — Natural language interface for querying telemetry data and simplifying DevOps workflows