Augmented reality-based assembly guidance systems superimpose digital information onto physical scenes to effectively guide complex assembly tasks. However, the gap between humans and the physical world in the assembly environment is huge, and the information to be fused into the physical world needs to be prepared in advance and triggered manually during the assembly process. The study of real-time and ubiquitous prompts is a hot research topic for complex assembly in augmented reality environment. In this paper, we propose an augmented reality assembly method based on the assistance of large language models (LLMs), the core of which is to use LLMs as another brain in the assembly process, providing ubiquitous assembly guidance and prompts support for technique information. Firstly, a system of LLMs-assisted augmented reality assembly method is established, and the elements and interrelationships of the system are analysed. Secondly, a matching process information model is constructed for LLMs environment. Then, the assisted guidance decision-making method and process based on LLMs are given. Finally, combined with a cable assembly expertise, a professional question answering system is developed to realize the smart assisted guidance of LLMs. Results show that the assembly pass rate increased by 15% and the effectiveness of the method is verified through several cases.
鲍劲松,李建军,袁轶,吕超凡,王森. 大语言模型辅助下的增强现实装配方法[J]. 航空制造技术, 2024, 67(16): 107-116.
BAO Jinsong, LI Jianjun, YUAN Yi, Lü Chaofan, WANG Sen. Augmented Reality Assembly Method Assisted by Large Language Models[J]. Aeronautical Manufacturing Technology, 2024, 67(16): 107-116.