Large Language Models have propelled the rapid development of artificial intelligence technology, greatly expanding the boundaries of machine intelligence. Global tech giants and innovative enterprises are intensively deploying strategies around LLMs. As products based on LLMs are continuously launched and iterated, the evaluation of their capabilities has gradually become the focus of the industry. However, how to conduct a systematic, comprehensive evaluation and quantification of LLMs through unified evaluation criteria still requires in-depth exploration. Some enterprises and institutions have established relevant evaluation benchmarks and rankings for general large models in fields such as education and finance but there is still a lack of guidance for LLMs in the electric power industry.
The electric power industry, a key sector related to the global economy and people’s livelihood, is rich in professional knowledge and business logic. The evaluation of LLMs in this sector should focus on issues such as what power-specific capabilities large language models should possess, the level these capabilities should reach, how to construct industry evaluation datasets, and how to conduct objective evaluations, so as to form evaluation guidelines.
The aim is to use these guidelines to provide comprehensive, professional, objective, and scientific evaluation results for LLMs in the electric power sector, understand the boundaries of their professional capabilities, and guide the research, development, optimization, and application of LLMs. However, the existing evaluation standards for general LLMs are difficult to assess the capabilities of LLMs in the electric power sector comprehensively and thoroughly. Therefore, it is necessary to construct an evaluation system for LLMs in the electric power sector , tailored to industry-specific characteristics, and to formulate corresponding evaluation guide.
