Kubernetes Monitoring: Distributed Clusters, Key Metrics, and Collection Strategies

](https://lh5.googleusercontent.com/tN8tzKcXWAFWF0TD9u9UkTFJakHsrdjtRx56WiF75UYwMKu8teFyr6LpLGjpuOWSr52M-l3do5r3a6VWi6VwhRWuaquCpGty8ksI585D9YuCL3t7DAcItJUwW6mlrM2jUw_jVq6A) In traditional, host-centric infrastructure, we were used to monitoring only two layers: applications and the hosts running them. Now with containers in the middle and Kubernetes itself needing to be monitored, there are four different components to monitor and collect metrics from. **Applications are constantly moving** Kubernetes schedules applications dynamically based on scheduling policy, so you don’t always know where applications are running. But they still need to be monitored. That’s why using a monitoring system or tool with service discovery is a must. It will automatically adapt metric collection to moving containers so applications can be continuously monitored without interruption. **Be prepared for distributed clusters** Kubernetes has the [ability](/docs/tasks/federation/federation-service-discovery/#hybrid-cloud-capabilities) to distribute containerized applications across multiple data centers and potentially different cloud providers. That means metrics must be collected and aggregated among all these different sources.    For more details about all these new monitoring challenges inherent to Kubernetes and how to overcome them, we recently published an [in-depth Kubernetes monitoring guide](https://www.datadoghq.com/blog/monitoring-kubernetes-era/). Part 1 of the series covers how to adapt your monitoring strategies to the Kubernetes era. **Metrics to monitor** Whether you use [Heapster](https://github.com/kubernetes/heapster) data or a monitoring tool integrating with Kubernetes and its different APIs, there are several key types of metrics that need to be closely tracked: - **Running pods** and their **deployments** - Usual **resource metrics** such as CPU, memory usage, and disk I/O - **Container-native [metrics](https://www.datadoghq.com/blog/monitoring-kubernetes-performance-metrics/)** - Application metrics for which a service discovery feature in your monitoring tool is essential  All these metrics should be aggregated using Kubernetes labels and correlated with events from Kubernetes and container technologies.   [Part 2](https://www.datadoghq.com/blog/monitoring-kubernetes-performance-metrics/) of our series on Kubernetes monitoring guides you through all the data that needs to be collected and tracked. **Collecting these metrics** Whether you want to track these key performance metrics by combining Heapster, a storage backend, and a graphing tool, or by integrating a monitoring tool with the different components of your infrastructure, [Part 3](https://www.datadoghq.com/blog/monitoring-kubernetes-with-datadog/), about Kubernetes metric collection, has you covered.   **Anchors aweigh!** Using Kubernetes drastically simplifies container management. But it requires us to rethink our monitoring strategies on several fronts, and to make sure all the key metrics from the different components are properly collected, aggregated, and tracked. We hope our monitoring guide will help you to effectively monitor your Kubernetes clusters. [Feedback and suggestions](https://github.com/DataDog/the-monitor) are more than welcome.   - Get involved with the Kubernetes project on [GitHub](https://github.com/kubernetes/kubernetes)  - Post questions (or answer questions) on [Stack Overflow](http://stackoverflow.com/questions/tagged/kubernetes)  - Connect with the community on [Slack](http://slack.k8s.io/) - Follow us on Twitter [@Kubernetesio](https://twitter.com/kubernetesio) for latest updates

This section elaborates on the challenges of Kubernetes monitoring, including distributed clusters across multiple data centers, and emphasizes the importance of collecting and aggregating metrics from all sources. It highlights key metrics to monitor, such as running pods, resource metrics, container-native metrics, and application metrics. It also points to a three-part Kubernetes monitoring guide that covers adapting monitoring strategies, identifying data to collect, and implementing metric collection using tools like Heapster or integrated monitoring tools. The section concludes by encouraging community involvement through GitHub, Stack Overflow, and Slack.