What are the parameters that needs to be considered for monitoring gpu

[u/technicalfault_]

I am seeking information regarding parameters for GPU monitoring for an internal tool. Specifically, I would like to identify the key metrics for monitoring GPUs and their associated agents, including relevant queries and performance indicators. Additionally, I am interested in exploring any open-source tools designed for this specific purpose.