EvALign-ICL
What it is
EvALign-ICL is an evaluation framework for Large Multimodal Models (LMMs) that assesses five axes: hallucinations, abstention, compositionality, explainability, and instruction following. It provides a benchmark for evaluating these flaws and introduces new In-Context Learning (ICL) variants like MultiTask-ICL and Self-Correcting-ICL to mitigate them.
The problem it solves
Current LMM evaluation benchmarks focus primarily on task performance (e.g., VQA accuracy) and fail to capture flaws like hallucinations and poor instruction following.
What you could use it for
- Evaluating object hallucination rates in LMMs
- Benchmarking instruction-following capabilities across different model scales
- Testing the impact of In-Context Learning (ICL) on model alignment
- Evaluating model explainability and answer abstention
Read the source ↗Back to the library →
Summary written by an AI model from the source; check the source before you rely on it.