EFK Stack for log
- E= Elasticsearch
- F= Fluentd
- K= Kibana
Fluentd (Demon sets): An open-source data collector and log forwarder. Fluentd collects logs from various sources and sends them to Elasticsearch for indexing.(Log collector)
Elastic search: A distributed search and analytics engine used to store and index logs. It allows you to search and analyze large volumes of log data quickly. (will store the logs from fluentd with a volume like EBS)
Kibana: A visualization tool that provides a user-friendly web interface for searching, analyzing, and visualizing logs stored in Elasticsearch. (will visualize the logs, UI)
Difference between Matrices and traces ?
Matrices
- Metrics are numerical data points that represent the state or performance of a system over time.
- Examples include CPU usage, memory consumption, request count, error rate, or response time we check it in Grafana.
Traces
- Traces represent a detailed path of a single request or transaction as it moves through various services or components in a distributed system.
- Traces we mostly used to monitor microservice application bcz traces can help to check flow of the request made between different services, so we can see when the issue is occurred and troubleshoot it
Key Differences:
| Aspect | Metrics | Traces |
|---|---|---|
| Focus | Aggregated system performance data. | Lifecycle of individual requests. |
| Granularity | High-level overview of system health. | Detailed and specific for a single request. |
| Use Case | Monitoring and alerting. | Debugging and root cause analysis. |
| Storage | Time-series databases. | Distributed tracing systems. |
| Visualization | Dashboards showing trends over time. | Trace maps showing request flows. |
| Example | “Avg response time: 200ms.” | “Request A spent 50ms in auth, 150ms in DB.” |