MoI-Bench: A Benchmark for Image Generation and Editing on Mobile Devices
What it is
MoI-Bench is a benchmark designed to evaluate image generation and editing models specifically for mobile deployment. It uses a decoupled design of 16 operations and 8 visual targets to create 64 subtasks, allowing for fine-grained analysis of capability gaps between on-device and server-side models.
The problem it solves
Existing benchmarks provide limited guidance on identifying specific capability gaps between on-device and server-side models, often failing to account for mobile resource constraints and practical user needs.
What you could use it for
- Evaluating the specific editing capabilities of compact on-device image models
- Identifying priority capability gaps (e.g., text replacement vs. general editing) for mobile model development
- Assessing the trade-off between model quality and deployment efficiency using a two-dimensional capability map
Why it matters
Provides a structured framework for improving mobile image models by highlighting specific technical deficiencies (like identity preservation and text rendering) rather than relying on broad, single-score rankings.
Read the source ↗Back to the library →
Summary written by an AI model from the source; check the source before you rely on it.