Empirical Study of LLMs for Knowledge Graph to Text Generation
What it is
This paper benchmarks recent large language models against fine-tuned encoder-decoder models for the task of converting knowledge graph triples into natural language. It evaluates performance using both standard reference-based metrics and graph-grounded reference-free LLM-based evaluation, finding that LLMs perform better on graph-grounded metrics and produce fewer contradictions as graph size increases.
The problem it solves
Standard reference-based metrics fail to capture critical quality aspects like factual contradictions and triple omissions, and it is unclear how recent LLMs compare to established fine-tuned models on these deeper dimensions.
What you could use it for
- Selecting between LLMs and fine-tuned encoder-decoder models for KG-to-text generation tasks
- Evaluating the factual consistency and completeness of generated text against source knowledge graphs
- Benchmarking the robustness of text generation models against increasing graph complexity
Why it matters
Provides evidence that prompted LLMs may be more robust and factually consistent than fine-tuned models for knowledge graph verbalization, potentially shifting standard practices in the field.
Read the source ↗Back to the library →
Summary written by an AI model from the source; check the source before you rely on it.