The Transaction Processing Performance Council (TPC) has announced the formation of a new Working Group (TPC-AI), tasked with developing industry standard benchmarks for both hardware and software platforms associated with running Artificial Intelligence (AI) based workloads.
Current AI ecosystems contain a mix of technologies and products for everything from traditional servers and tools to specialized systems and libraries for building and deploying models into applications. As industry investment in AI continues to accelerate, and the number of AI platforms and tools continue to proliferate, vendors and customers increasingly require an impartial means of comparing performance and total cost of ownership across a widening array of hardware and software systems.
“Artificial Intelligence will become mainstream across industry,” said Raghunath Nambiar, chairman of the TPC-AI Working Group and chief technology officer at Cisco UCS. “These workloads are extremely data intensive adjacent to the core area of focus for the TPC. Our new AI Working Group’s priority is to develop a set of standardized metrics to enable fair price and performance comparisons across an array of different hardware and software stacks. And we’re actively encouraging industry experts and members of the research community to help us in this effort.”
“Interestingly, the TPC-AI Working Group has its origins in the TPC’s annual International Technology Conference Series on Performance Evaluation and Benchmarking (TPCTC),” continued Nambiar. “Every year, the TPC invites industry experts and researchers to identify pressing areas for benchmark development. Since 2009, the TPCTC has served as a forum for leading vendors and researchers to gather and identify innovative areas and methodologies for performance characterization and optimization beneficial to both industry and the research community. We have a solid history of selecting presentations for continued research. In fact, the TPC’s new benchmark standards on Big Data Analytics (BDA), hyperconverged infrastructure (HCI) and the Internet of Things (IoT) have early roots in TPCTC conferences.”
The next conference will be collocated with 44th International Conference on Very Large Data Bases (VLDB 2018) and will be held August 27 – 31 in Rio de Janeiro, Brazil. To view the call for papers visit http://www.tpc.org/tpctc/tpctc2018/default.asp.
Organizations that are interested in influencing the benchmarking development process, including benchmarks related to AI, are encouraged to become members of the TPC. Membership information is available online: http://www.tpc.org/information/about/join.asp.