A Semantic Schema-Based Catalog for Identifying Joinable Columns via LLMs

Joinable table discovery consists of the identification of tabular datasets that can be joined with a given query dataset. The use of contextual information associated with the datasets and columns (tailored to the kinds of analyses the user intends to carry out) is seldom considered in the approaches proposed so far. In this paper, the generation of semantic task-oriented schema-based catalogs that facilitate the identification of joinable columns is proposed. By identifying a schema diagram that outlines the classes and relationship types for a certain kind of analysis, datasets are semantically annotated, and annotations are used to generate the catalog. The catalog, represented as a property graph, can then be leveraged for visual exploration, query formulation, and identification of joinable datasets useful for a specific analysis. The approach leverages the availability of metadata about datasets and their columns, combined with general-purpose large language models (LLMs). Initial experiments suggest that our approach is both practical and efficient, yielding promising results in terms of both accuracy and usability.