This paper explores this critical question by subjecting Large Language Models (LLMs), specifically ChatGPT and Deepseek, to three key challenges drawn directly from the study of ancient Chinese literature. The investigation is centered around early narratives of the famous monkey figure, Sun Wukong, and his lesser-known antecedents found in pivotal texts like the Yuan dynasty play Xiyou ji zaju 西游记杂剧 and early zhiguai xiaoshuo 志怪小说 (tales of the strange). Through a series of focused "mini-experiments," we tested the AI's performance in textual tracing, translation from vernacular Chinese (baihua 白话), and the difficult interpretation of classical Chinese (wenyanwen 文言文). We discovered that while the AI models consistently demonstrated a high degree of elegance (ya雅)—producing fluent and natural-sounding English translations—they frequently stumbled when faced with tasks requiring deep cultural or historical knowledge. Specifically, the LLMs struggled to identify precise literary sources, misinterpreted religious or cultural terminology, and failed to navigate the complex, subject-omitted syntax common in wenyanwen. These errors reveal a persistent gap in faithfulness (xin 信) and expressiveness (da 达), the two cornerstones of rigorous translation according to Yan Fu’s established principles. Ultimately, the study concludes that AI is not yet ready to replace the human expert. Instead, it functions as a potent auxiliary tool, capable of generating preliminary drafts but critically dependent on the Sinologist. The human expert remains indispensable, serving as the necessary proofreader and cultural compass to guarantee the accuracy and intellectual rigor required for interpreting and translating ancient Chinese texts.

机器能成为汉学家吗?——基于中国古代文学的AI性能检验 / G. Ruscica. 新汉学视角下的人工智能发展——文化智慧与技术创新的交融 Beijing 2025.

机器能成为汉学家吗?——基于中国古代文学的AI性能检验

G. Ruscica
2025

Abstract

This paper explores this critical question by subjecting Large Language Models (LLMs), specifically ChatGPT and Deepseek, to three key challenges drawn directly from the study of ancient Chinese literature. The investigation is centered around early narratives of the famous monkey figure, Sun Wukong, and his lesser-known antecedents found in pivotal texts like the Yuan dynasty play Xiyou ji zaju 西游记杂剧 and early zhiguai xiaoshuo 志怪小说 (tales of the strange). Through a series of focused "mini-experiments," we tested the AI's performance in textual tracing, translation from vernacular Chinese (baihua 白话), and the difficult interpretation of classical Chinese (wenyanwen 文言文). We discovered that while the AI models consistently demonstrated a high degree of elegance (ya雅)—producing fluent and natural-sounding English translations—they frequently stumbled when faced with tasks requiring deep cultural or historical knowledge. Specifically, the LLMs struggled to identify precise literary sources, misinterpreted religious or cultural terminology, and failed to navigate the complex, subject-omitted syntax common in wenyanwen. These errors reveal a persistent gap in faithfulness (xin 信) and expressiveness (da 达), the two cornerstones of rigorous translation according to Yan Fu’s established principles. Ultimately, the study concludes that AI is not yet ready to replace the human expert. Instead, it functions as a potent auxiliary tool, capable of generating preliminary drafts but critically dependent on the Sinologist. The human expert remains indispensable, serving as the necessary proofreader and cultural compass to guarantee the accuracy and intellectual rigor required for interpreting and translating ancient Chinese texts.
dic-2025
Settore ASIA-01/F - Lingue e letterature della Cina e dell'Asia sud-orientale
机器能成为汉学家吗?——基于中国古代文学的AI性能检验 / G. Ruscica. 新汉学视角下的人工智能发展——文化智慧与技术创新的交融 Beijing 2025.
Conference Object
File in questo prodotto:
Non ci sono file associati a questo prodotto.
Pubblicazioni consigliate

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/2434/1208497
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus ND
  • ???jsp.display-item.citation.isi??? ND
  • OpenAlex ND
social impact