A SwinBERT framework with temporal windowed cross attention for video captioning for visual language intelligence
Jun 1, 2026·
·
0 min read
Reshma DSouza
Dr. Snigdha Sen
Abstract
Video captioning lets computers understand visual content and describe it in natural language. Traditional CNN-RNN architectures struggle with long-range temporal dependencies and contextual relationships in video. This work uses a Swin Transformer to extract video features and a BERT decoder to generate captions, combining semantic features to produce fluent, context-appropriate descriptions. The system shows reasonable captioning performance on the MSVD and MSR-VTT benchmarks while remaining efficient enough for near real-time use, with applications in video summarization and assistive technologies.
Type
Publication
Discover Applied Sciences
Authors
Senior Assistant Professor, School of Computer Engineering
Dr. Snigdha Sen is a Senior Assistant Professor in the School of Computer
Engineering at Manipal Institute of Technology Bengaluru, MAHE, with around
16 years of teaching experience. Before joining Manipal she was an Associate
Professor at Global Academy of Technology, Bengaluru. She holds a PhD from
IIIT Allahabad, an M.Tech in Information Technology from Jadavpur University
and a B.E. from UIT, University of Burdwan. Her research spans
astroinformatics, deep learning, computer vision, explainable AI for space
science and big data analytics. She has published over 40 SCI- and
Scopus-indexed papers and is a life member of the Computer Society of India.