FlowTT: Exploiting Computation Flow Reuse in Irregular Tensor-Train Embedding
What it is
FlowTT is a GPU execution framework designed to optimize Tensor-Train (TT) decomposition for embedding tables in recommendation models. It reformulates TT gather operations as prefix-shared irregular computation flows, utilizing fused execution paths, on-chip intermediate retention, and persistent-thread scheduling to minimize redundant operations and memory traffic.
The problem it solves
Existing TT-based embedding lookups are inefficient due to repeated materialization of intermediate results off-chip and failure to fully reuse partially shared computation flows across input indices, leading to high memory traffic and load imbalance.
What you could use it for
- Accelerating inference and training for large-scale recommendation systems using Tensor-Train compressed embeddings
- Reducing GPU memory footprint and latency in models with irregular, skewed embedding lookup workloads
Read the source ↗Back to the library →
Summary written by an AI model from the source; check the source before you rely on it.