Construction and Evaluation Framework of Large Language Models Driven by Civil Engineering Domain Knowledge

Abstract:

To address the limitations of general large language models （LLMs） in the field of civil engineering due to a lack of specialized knowledge， this study proposes a large knowledge model specifically designed for civil engineering， named CivilGPT. The development of CivilGPT follows a multi-step technical approach， including data preprocessing， the construction of a domain-specific knowledge graph， the generation and optimization of automated datasets， staged pre-training and fine-tuning， and alignment with engineering tasks to ensure that the model can accurately express and reason within the field of civil engineering. Additionally， this study introduces a standardized evaluation framework， Civil-Bench， based on civil engineering qualification exams. Civil-Bench encompasses 13 categories of professional engineering exam questions， including 14，823 objective questions and 269 subjective questions. Testing across 15 domestic and international language models demonstrates that CivilGPT exhibits significant advantages in civil engineering knowledge comprehension， reasoning ability， and solving complex problems. The outcomes of this research lay a technical foundation for the intelligent advancement of the civil engineering field and provide valuable insights for the development of models in other specialized domains.