土木工程专业知识驱动大语言模型构建与评测体系

Construction and Evaluation Framework of Large Language Models Driven by Civil Engineering Domain Knowledge

摘要:

为解决通用大语言模型在土木工程领域应用中缺乏专业知识而导致的局限性问题，提出了一种专为土木工程领域设计的知识大模型―CivilGPT。通过多步骤的技术路线构建了CivilGPT模型，包括数据预处理、建立专业知识图谱、生成和优化自动化数据集、分阶段预训练和微调，以及与工程任务的对齐，确保模型能够在土木工程领域实现准确的知识表达和推理能力。此外，建立了基于土木工程资格考试的标准化评测框架Civil-Bench，涵盖13类注册工程师考试题目，包含14 823道客观题和269道主观题。通过测试15个国内外语言模型，结果显示CivilGPT在土木工程知识理解、推理能力和复杂问题解决方面具有显著优势。该研究为土木工程领域智能化发展奠定了技术基础，为其他专业领域模型开发提供了重要参考。

Abstract:

To address the limitations of general large language models （LLMs） in the field of civil engineering due to a lack of specialized knowledge， this study proposes a large knowledge model specifically designed for civil engineering， named CivilGPT. The development of CivilGPT follows a multi-step technical approach， including data preprocessing， the construction of a domain-specific knowledge graph， the generation and optimization of automated datasets， staged pre-training and fine-tuning， and alignment with engineering tasks to ensure that the model can accurately express and reason within the field of civil engineering. Additionally， this study introduces a standardized evaluation framework， Civil-Bench， based on civil engineering qualification exams. Civil-Bench encompasses 13 categories of professional engineering exam questions， including 14，823 objective questions and 269 subjective questions. Testing across 15 domestic and international language models demonstrates that CivilGPT exhibits significant advantages in civil engineering knowledge comprehension， reasoning ability， and solving complex problems. The outcomes of this research lay a technical foundation for the intelligent advancement of the civil engineering field and provide valuable insights for the development of models in other specialized domains.