A Generative AI Framework for End-to-End Semantic Communication
Efficient medical image & video transmission through semantic communication
Overview
The rapid growth of wireless healthcare systems has increased the need to transmit medical images and videos efficiently and reliably. Traditional systems reproduce raw data bit-for-bit, but in medicine what matters most is preserving the clinically meaningful information. This project proposes a Generative-AI-based semantic communication framework that extracts, compresses, transmits and reconstructs the semantic content of medical images and videos instead of the full raw data — reducing bandwidth, improving robustness to channel noise, and preserving diagnostic relevance.
Semantic features (anatomical structures, abnormal regions, disease-related patterns) are extracted with Vision Transformers and Large Language Models, compressed by a hybrid (classical + deep-learning) compressor, and — after a noisy wireless channel — reconstructed using CLIP-guided prompt embedding and a diffusion model, maintaining semantic integrity and perceptual quality.
Methodology & Workflow
The framework moves compact semantic representations across the channel instead of raw pixels — from transmitter, through the noisy channel, to generative reconstruction at the receiver.
Medical Input
Image / Video / Text
Semantic Feature Extraction
ViT + LLM
Encoding & Compression
Hybrid compressor
Noisy Wireless Channel
Interference & distortion
CLIP + Prompt → Diffusion
Guided reconstruction
Semantic Reconstruction
Diagnostic output
Research Team
Sandaruwan W.P.P.C.
