EECS 584 Advanced Database Systems Fall 2023
LLM-Based Schema Matching
Team course project
Archived course report (PDF)Schema matching identifies fields that represent the same concept across databases. This team project explored whether LLMs could match fields from schema metadata without task-specific training.
We compared GPT-3.5 and GPT-4 using direct prompts, in-context examples, step-by-step prompting, and self-refinement, alongside traditional schema matchers. The main evaluation matched Schema.org to 12 metadata schemas in the RDA Crosswalk benchmark.
| Prompting strategy | Mean F1 |
|---|---|
| Direct prompting | 0.835 |
| In-context examples | 0.868 |
| Self-refinement | 0.782 |
Examples improved matching in this comparison, while self-refinement did not. More elaborate prompting was not consistently better; some model outputs also required manual parsing.
Project team: Houming Chen, Zhuocheng Sun, Yifan Zhu.