Accelerators are a type of specialized hardware that enhances the performance of a virtual machine. These include Graphics Processing Units (GPUs), Field-Programmable Gate Arrays (FPGAs), and machine learning chips. Cloud Service Providers install accelerators in the hardware of high-intensity virtual machines. For AWS, this is the Accelerated Computing family, and for Azure, this is the N-series.
Neither provider delivers data on accelerator utilization to their customers as standard. In order to track accelerator utilization, customers of AWS and Azure must load additional software onto their instances. This would need to be automated by the organization’s DevOps team. When customers do have this software loaded, Tailpipe uses the utilization data directly from the Cloud Service Provider. At present, Tailpipe only calculates emissions for GPU accelerators. FPGAs and machine learning chips are not included in the Tailpipe methodology, due to the limited data on the power draw of these hardware components.
It matches this utilization data against the power draw of different accelerators at specific utilization loads. Tailpipe has lab-tested a range of accelerators using a load testing software that tracks utilization at 5% increments. This data reveals what proportion of an accelerator’s Thermal Design Power (TDP, measured in Watts) is drawn at each 5% utilization increment. Tailpipe uses this data to draw a generalized power consumption curve for accelerators depending on the utilization rate, which is matched to the client usage. An example of the power curve for NVIDIA GPUs is:
So, an NVIDIA GPU with a TDP of 300 W, being utilized at 50%, will draw 57% of the GPU’s maximum TDP:
0.57 * 300 = 171 W
The methodology used by Tailpipe for users who have enabled accelerator utilization tracking is therefore:
Accelerator Energy Consumption (Wh) = (TDP Power Factor * Accelerator TDP) * Hours of Utilization * Number of Accelerators
When customers do not have utilization tracking software enabled, Tailpipe assumes that accelerators are running at peak energy usage 50% of the time. Tailpipe uses an average accelerator power draw based on testing by Teads Engineering. Their analysis of 12 GPUs results in an average power draw of 133.54 W (see Appendix A). The methodology used by Tailpipe for users who do not track accelerator utilization is therefore:
Accelerator Energy Consumption (Wh) = 133.54 * (0.5 * Hours of Utilization) * Number of GPUs
