LINGUAL: Language-INtegrated GUidance in Active Learning for Medical Image Segmentation WACV 2027
March 2025 to December 2025
Experts fix segmentation masks by typing what is wrong instead of redrawing pixels. An LLM turns each instruction into calls to EXPAND and SHRINK refinement tools.
- Cuts annotation interaction cost by up to about 25%.
- Matches or beats manual correction under domain shift and limited supervision.
How it works
- Problem
- Medical images have blurry, ambiguous boundaries. Drawing polygons around them is slow and tiring, and bigger regions cost more effort.
- Approach
- The expert types a short command such as “expand to bottom-right” or “remove small fragments”. GPT-3.5, using in-context examples, turns it into a program. The program calls refinement tools (EXPAND, SHRINK, REMOVE, FILL, SMOOTH) that edit the mask with no further human input.
- Why it scales
- The typed command stays about the same length whether the region is small or large, so expert effort stays nearly flat.
- Data
- Abdominal MRI (CHAOS, in-phase to out-of-phase: liver, kidneys, spleen) and breast ultrasound (BUSI to UDIAT).









