← All projects

EECS 584 Advanced Database Systems Fall 2023

LLM-Based Schema Matching

Team course project

Archived course report (PDF)

Schema matching identifies fields that represent the same concept across databases. This team project explored whether LLMs could match fields from schema metadata without task-specific training.

We compared GPT-3.5 and GPT-4 using direct prompts, in-context examples, step-by-step prompting, and self-refinement, alongside traditional schema matchers. The main evaluation matched Schema.org to 12 metadata schemas in the RDA Crosswalk benchmark.

GPT-4 prompting comparison
Prompting strategyMean F1
Direct prompting0.835
In-context examples0.868
Self-refinement0.782
Reported means across the 12 schemas in the 2023 course experiment (Table 3). Higher F1 is better.

Examples improved matching in this comparison, while self-refinement did not. More elaborate prompting was not consistently better; some model outputs also required manual parsing.

Project team: Houming Chen, Zhuocheng Sun, Yifan Zhu.