![](https://lh4.googleusercontent.com/B0M-LCr8iVTVRFC5u8Ni08-sXCu7BJAHXXFRTrT_ecPYi4ZHr7ylhkmbUlqwewNvRPmxH63DadNe72AA7jthoGm3cWChZclX7ARPdSFxNQbKqkwSgVLK2y6as02Y2hlQ2kSIcfpn) |
| Figure 3. Compare different test configurations. |



**Time Series and Tracing: Diving Into Performance Data**



Pod startup latency is an important metric for Kubelet, especially when creating a large number of pods per node. Using the dashboard you can see the change of latency, for example, when creating 105 pods, as shown in Figure 4. When you see the highly variable lines, you might expect that the variance is due to different builds. However, as these test here were run against the same Kubernetes code, we can conclude the variance is due to performance fluctuation. The variance is close to 40s when we compare the 99% latency of build #162 and #173, which is very large. To drill into the source of the fluctuation, let’s check out the "TIME SERIES" page.



| ![](https://lh5.googleusercontent.com/4WM9bX-Vzn-h2otSaVcES4FBPDeTFuIueo_uRXDctpKPO_lFAANjRj9QmezSn5x81QLcDAq8ui_Gvbik1edyjUwPKWQNKjbW7uSNwCFnGg7Bd1KqqU1U7B1gvwzK_X6Wo7DJjYH3)