Md Shazid Islam

Md Shazid Islam

I build vision-language models that know where their answers come from, and learning systems that need far fewer labels.

Ph.D. candidate, Electrical and Computer Engineering, UC Riverside. Advised by Amit K. Roy-Chowdhury in the Vision and Learning Group.

  • LINGUAL accepted at WACV 2027.
  • Presented ODES at MICCAI 2025, supported by the MICCAI NIH registration grant.

About

I fine-tune VLMs and LLMs with preference optimization, build LLM pipelines that call executable tools, and design learning methods for segmentation, domain adaptation, and active learning. I take problems from formulation to experiments to open-source code.

Education

  • Ph.D., Electrical and Computer EngineeringUC Riverside, 2020 to present. Advisor: Amit K. Roy-Chowdhury
  • M.Sc., Electrical and Electronic EngineeringBUET, 2017 to 2020. Advisor: S.M. Mahbubur Rahman
  • B.Sc., Electrical and Electronic EngineeringBUET, 2013 to 2017. Advisor: Mohammad Ariful Haque

Research interests

  • Vision-language modelsModels that write grounded reports from medical images and know which words to doubt.
  • Annotation-efficient learningActive learning and language-guided labeling that cut how much experts must annotate.
  • Domain adaptationKeeping models accurate when the scanner, contrast, or hospital changes, even online and source-free.
  • Trustworthy medical AIUncertainty quantification and hallucination reduction, so clinicians can check model outputs.
  • 3D and biomedical image analysisSegmentation, registration, and cell tracking in volumetric and time-lapse data.
  • LLM agents and tool useLanguage models that turn instructions into executable programs and multi-step workflows.

Research

Medical images are expensive to label and risky to get wrong. My work cuts the labeling cost and makes model outputs easier to check.

LINGUAL refining a segmentation mask from a typed instruction
Demo coming soon

LINGUAL: Language-INtegrated GUidance in Active Learning for Medical Image Segmentation WACV 2027

March 2025 to December 2025

Experts fix segmentation masks by typing what is wrong instead of redrawing pixels. An LLM turns each instruction into calls to EXPAND and SHRINK refinement tools.

  • Cuts annotation interaction cost by up to about 25%.
  • Matches or beats manual correction under domain shift and limited supervision.
How it works
Problem
Medical images have blurry, ambiguous boundaries. Drawing polygons around them is slow and tiring, and bigger regions cost more effort.
Approach
The expert types a short command such as “expand to bottom-right” or “remove small fragments”. GPT-3.5, using in-context examples, turns it into a program. The program calls refinement tools (EXPAND, SHRINK, REMOVE, FILL, SMOOTH) that edit the mask with no further human input.
Why it scales
The typed command stays about the same length whether the region is small or large, so expert effort stays nearly flat.
Data
Abdominal MRI (CHAOS, in-phase to out-of-phase: liver, kidneys, spleen) and breast ultrasound (BUSI to UDIAT).
83.4%less annotation time, MRI
79.9%less annotation time, ultrasound
ODES adapting a segmentation model to a new scanner
Demo coming soon

ODES: Online Domain Adaptation with Expert Guidance for Medical Image Segmentation MICCAI 2025

July 2023 to June 2024

Source-free, single-pass online domain adaptation for medical image segmentation. It asks an expert only about the images that matter and stores nothing extra.

  • Image pruning plus a diversity-aware rule decide which images get labeled.
  • +7.7% on average over test-time adaptation across several medical datasets.
How it works
Problem
A model trained on one scanner drops in accuracy on another. Hospitals receive patients one after another, so the model must adapt online, with no access to the source data and no stored labels.
Approach
For each incoming batch the model predicts, then image pruning uses batch-norm statistics to keep only the most shifted images. An acquisition rule combines uncertainty, regional impurity, and spatial and feature diversity to pick small patches for the expert. The model updates once and moves on.
Practical fit
Expert labeling happens in the turnover time between patients, so it does not slow the clinical workflow.
Data
Three shifts: CHAOS MRI in-phase to out-of-phase, CHAOS T2-SPIR to DUKE, and prostate MRI from BMC to RUNMC.
+7.7%average gain over test-time adaptation
1 passper batch, no data stored
Tracking plant cells across 3D time-lapse microscopy
Demo coming soon

DEGAST3D: Learning Deformable 3D Graph Similarity to Track Plant Cells in Unregistered Time Lapse Images IEEE TCBB 2025

June 2021 to June 2022

A deformable 3D graph neural network that tracks densely packed plant cells in unregistered time-lapse microscopy.

  • +6.40% F1 over the previous best for tracking.
  • 3D registration and division detection: +14.78% F1 over 2D methods.
How it works
Problem
Plant tissue in 3D microscopy has many tightly packed cells, uneven growth, divisions in different planes, and noisy deep layers. The stacks are also unregistered between time points.
Approach
Cells are segmented in 3D, then each pair of time points is registered with RANSAC on point-cloud features. A graph neural network compares each cell’s local 3D neighbor graph to find its match. A separate 3D CNN checks mother and daughter cell shapes to detect divisions.
Data
Public confocal time-lapse stacks of the Arabidopsis shoot apical meristem, with 4 hours between time points.
+6.83%precision, cell matching
+5.96%recall, cell matching
+15.38%recall, division detection
Word-level uncertainty map over a 3D CT scan
Demo coming soon

Uncertainty Quantification for 3D Vision-Language Models

August 2026 to present

A training-free method that links each word of a generated CT report to the parts of the scan that support or contradict it.

  • Occludes blocks of a 3D ViT token grid to measure how each region changes each word.
  • Pools the evidence into word-level and scan-level uncertainty, then marks a small 3D region for a radiologist to review first.
  • Runs on 900+ CT volumes from AbdomenAtlas 3.0 with NV-Reason-CT on a SLURM GPU cluster.
How it works
Problem
A 3D vision-language model can write a fluent CT report that is wrong in places. Radiologists need to know which words to doubt and where to look.
Approach
Block-wise feature replacement (occlusion) on the 3D ViT token grid measures how each scan region supports or contradicts each generated word. An evidential framework pools this into word-level vacuity and dissonance, then into a scan-level uncertainty map.
Output
A budgeted 3D review region that points the radiologist at the most conflicting anatomy, shown as NIfTI overlays in 3D Slicer.
Evaluation
A label-free protocol with clinically grounded error labels.
900+CT volumes, AbdomenAtlas 3.0
0extra training needed
Grounded chest X-ray report generation
Demo coming soon

Vision-Language Models for Automated Radiology Report Generation

January 2026 to May 2026

Fine-tuned LLaVA-Med 1.5 on MIMIC-CXR to write chest X-ray reports tied to image evidence.

  • Multi-GPU training with supervised fine-tuning and preference optimization (DPO, GRPO).
  • Image-grounded reasoning reduces hallucinated findings and improves spatial accuracy.
How it works
Model
LLaVA-Med 1.5, a vision-language model for medical images.
Training
Supervised fine-tuning, then preference optimization with DPO and GRPO, on multiple GPUs.
Grounding
Generation is tied to image evidence, which cuts hallucinated findings and improves spatial accuracy over raw-image baselines.
Data
MIMIC-CXR chest X-rays and reports.
Music turned into a stick-figure dance and a realistic video frame
Demo coming soon

Synthesis of Style-Specific Dance by Learning Body Gestures from Music Multimedia Tools and Applications 2025

January 2019 to July 2020

A model that listens to music and generates a matching dance, first as a stick figure and then as realistic video.

  • A CNN-LSTM mixture density network predicts rhythm-aligned poses from the music.
  • A Pix2pixHD GAN renders each pose as a photo-realistic frame.
  • Learns Ballet, Rumba, Cha-Cha, Tango, and Waltz, and beats existing methods on standard metrics.
How it works
Problem
Dance moves fast, and body parts must stay coherent across frames. The model has to learn this from limited audio and motion data.
Approach
A short-time Fourier transform turns the music into features. A CNN and an LSTM read them along with recent poses, and a mixture density network predicts the next pose as a stick figure. A Pix2pixHD GAN with post-processing then renders a realistic dancer.
Styles
Ballet, Rumba, Cha-Cha, Tango, and Waltz.
Data
YouTube dance videos and motion-capture recordings.
Gait analysis pipeline: motion capture, keypoint detection, and a neural network
Demo coming soon

Gait Phase Classification from sEMG in Multiple Locomotion Mode Using Deep Learning ICICT 2024

Published 2024

A deep learning method that classifies gait phases from surface EMG (sEMG) signals across several locomotion modes.

  • Covers three locomotion modes: DNS, UPS, and WAK.
  • Built for robust, real-time locomotion analysis.

Multi-Human Tracking

2022 to 2023

ByteTrack association with person re-identification and gait analysis to keep identities across frames.

DAAFL: Delay Aware Asynchronous Federated Learning

2024 to 2025

A delay-aware framework that handles slow clients with buffer diversification and contribution-based weighting.

Transforming Social Media Analysis: TweetEval Benchmarking with Advanced Transformer Models ISMSIT 2024

2024

Benchmarked transformer models for social-media text classification on TweetEval across several language tasks. Paper

Location Agnostic Source-Free Domain Adaptive Learning to Predict Solar Power Generation IEEE ETFG 2023

2023

A source-free domain-adaptive model that predicts solar power generation in new regions without local labels. Paper

Publications

Newest first. Full list on Google Scholar.

  1. 2027
    LINGUAL: Language-INtegrated GUidance in Active Learning for Medical Image Segmentation WACV 2027 Md Shazid Islam, Shreyangshu Bera, Sudipta Paul, Amit K. Roy-Chowdhury
  2. 2025
    ODES: Online Domain Adaptation with Expert Guidance for Medical Image Segmentation MICCAI 2025 Md Shazid Islam, Sayak Nag, Arindam Dutta, Miraj Ahmed, Fahim Faisal Niloy, Shreyangshu Bera, Amit K. Roy-Chowdhury
  3. 2025
    Synthesis of Dance by Learning Body Gestures from Music Multimedia Tools and Applications Md Shazid Islam, SM Mahbubur Rahman
  4. 2025
    DEGAST3D: Learning Deformable 3D Graph Similarity to Track Plant Cells in Unregistered Time Lapse Images IEEE TCBB Md Shazid Islam, Arindam Dutta, Calvin-Khang Ta, Kevin Rodriguez, Christian Michael, Mark Alber, G. Venugopala Reddy, Amit K. Roy-Chowdhury
  5. 2024
    A ConvNeXt V2 Approach to Document Image Analysis: Enhancing High-Accuracy Classification IEEE CITDS Md Saiful Islam Sajol, ASM Jahid Hasan, Md Shazid Islam, Md Saydur Rahman
  6. 2024
    Transforming Social Media Analysis: TweetEval Benchmarking with Advanced Transformer Models ISMSIT Md Saiful Islam Sajol, ASM Jahid Hasan, Md Shazid Islam, Md Saydur Rahman
  7. 2024
    Gait Phase Classification from sEMG in Multiple Locomotion Mode Using Deep Learning ICICT London Md Sanzid Bin Hossain, Md Shazid Islam, Md Saad Ul Haque, Md Saydur Rahman
  8. 2023
    Location Agnostic Source-Free Domain Adaptive Learning to Predict Solar Power Generation IEEE ETFG Md Shazid Islam, ASM Jahid Hasan, Md Saydur Rahman, Jubair Yusuf, Md Saiful Islam Sajal, Farhana Akter Tumpa
  9. Review
    SALT: Spectral Instability based Active Learning for Tiny Object Detection Shreyangshu Bera, Md Shazid Islam, Saketh Bachu, B. S. Manjunath, Amit Roy-Chowdhury
  10. Review
    DAAFL: Delay Aware Asynchronous Federated Learning Md Shazid Islam, M Yashwanth, Xiangyu Chang, Anirban Chakraborty, Srikanth Krishnamurthy, Amit Roy-Chowdhury

Leadership and honors

  • Team lead, LLNL Data Science Challenge2026. Led a four-member team building an LLM-driven agentic workflow for defect detection in materials.
  • MICCAI NIH Registration Grant2025
  • Dean's Distinguished FellowshipUC Riverside
  • Dean's List and Merit ScholarshipsDepartment of EEE, BUET

Beyond research

Football Tactics Board app showing two teams on a pitch with drawing tools and an AI analyst panel

Football Tactics Board

An interactive tactics whiteboard. Moves play out in animated phases, the drawing tools follow real football rules, and a built-in AI analyst answers questions about the board.

World Cup 2026 predictor showing group tables and match picks

World Cup 2026 Predictor

Pick every group-stage result and the knockout bracket builds itself. It tracks the race for the best third-place teams and shows win, draw, and loss odds for each match. It runs fully in the browser.

Photography

I like taking pictures on trips, mostly coastlines, national parks, and cities. A few favorites are below. Each one opens its full album on Google Photos.