AI for Healthcare | Computer Vision | Multimodal Learning

Building AI Systems for Real-World Impact

Leveraging deep learning and multimodal data to build scalable, interpretable AI solutions for real-world healthcare challenges.

View Publications Get in Touch

20+ Citations

h-index: 1

Publications

Internships

20+

Citations

Latest Updates

News & Announcements

University at Buffalo, SUNY, Buffalo, NY, USA

spandey8@buffalo.edu

Ph.D. in Computer Science

Connect

About Me

I am a PhD student in Computer Science at the University at Buffalo, where I work at the intersection of computer vision, deep learning and multi-modal AI with a focus on healthcare applications. My research centers on building intelligent systems that leverage diverse data sources to improve patient care, clinical decision-making and workflow optimization, in collaboration with institutions such as Roswell Park Comprehensive Cancer Center. I have contributed to multiple peer-reviewed publications in leading venues, spanning areas such as multimodal learning, scientific visual question answering and biometric recognition. Alongside my research, I have served as a lead teaching assistant for data-intensive computing courses and gained industry experience at MathWorks and Wobot Intelligence, where I worked on computer vision systems and real-world AI deployments.

I am passionate about developing impactful AI solutions that bridge theory and practice, particularly in high-stakes domains like healthcare.

Education

Ph.D. in Computer Science

University at Buffalo, SUNY | 2023 - Present

Thesis: Multimodality, Explainability and Interpretability in Health AI

M.S. in Computer Science

University at Buffalo, SUNY | 2021 - 2023

B.Tech in Electronics Engineering

Kamla Nehru Institute of Technology | 2016 - 2020

Secondary School (PCM)

City Monterssori School | 2015

Research Interests

Multimodal AI Computer Vision Deep Learning & Representation Learning Explainable & Interpretable AI (XAI) AI for Healthcare & Clinical Decision Support Vision-Language Models (VLMs) & Multimodal LLMs

Professional Experience

Academic and Industry roles throughout my career.

Research Assistant

University at Buffalo, SUNY

Buffalo, NY, USA

Aug 2025 - Present Research and Development

• Exploring domain of Multi-Modality and Explainability in Health AI
• Funded by Roswell Park Comprehensive Cancer Center, Buffalo, NY
• Developing deep learning architectures using multi-modal data for patient care and clinical workflow optimization

Predoctoral Trainee

Roswell Park Comprehensive Cancer Center

Buffalo, NY, USA

May, 2025 - August, 2025 Research and Development

• Worked on project focused on using Multi-modal preoperative data to predict probability of Pulmonary Postoperative Complications
• Another Project focuses on automated generation of radiology reports using X-Ray and CT scans
• Paper accepted in P2P-CV @ WACV 2026

Teaching Assistant

University at Buffalo, SUNY

Buffalo, NY, USA

August, 2024 - May, 2025 Teaching

• Lead Teaching Assistant for CSE 587 (Data Intensive Computing)

Research Assistant

University at Buffalo, SUNY

Buffalo, NY, USA

May, 2024 - August, 2024 Research and Development

• Developed a portal to crawl health care resources such as NIH to gather grant and proposals
• Portal paraphrased the proposal information accurately, making them concise using Llama 3.1
• Funded by Office of the Vice-president for Health Sciences, University at Buffalo

Teaching Assistant

University at Buffalo, SUNY

Buffalo, NY, USA

August, 2023 - May, 2024 Teaching

• Lead Teaching Assistant for CSE 587 (Data Intensive Computing)

EDG Intern - Computer Vision and Deep Learning Team

The MathWorks Inc.

Natick, MA, USA

May, 2022 - August, 2022 Computer Vision and Deep Learning

• Worked on Optical Flow Method for cylindrical, sphere and plane model giving a clear perception on working of optical flow
• Added Rotated Rectangle support to Image, Video and Ground Truth Labeler apps for MATLAB 2022b release with unit and automation testing
• Applied DRY principle and inheritance to create new software in existing codebase
• Gained thorough knowledge of Sandbox and Perforce client

Research Assistant

University at Buffalo, SUNY

Buffalo, NY, USA

August, 2021 - May, 2023 Research and Development

• Completed projects focused on VQA in scientific informatics and got a paper accepted in ICDAR’23
• Worked on project involving object detection and scene text detection funded by NSF S&CC program

Vision Data Intern - Computer Vision team

Wobot Intelligence Pvt. Ltd.

Delhi, India

October, 2020 - January, 2021 Research and Development

• Curated 15 datasets, employing web crawlers and web scrapers to scrap around 5 websites
• Implemented 2 GUIs to Filter and annotate datasets in PyQt5 in Python
• Implemented a Deep Learning architecture with the aim to enforce COVID sanitation guidelines for 3 food and beverage vendors

Research Areas

Exploring the frontiers of computer science through interdisciplinary research.

Multimodal AI and Computer Vision

Developing multimodal AI systems that integrate vision, language, and structured data for real-world decision-making and healthcare applications.

Multimodal Learning & Vision-Language Models
Medical Imaging & Risk Prediction

Explainable & Interpretable AI (XAI)

Building transparent and interpretable AI systems to enable reliable and accountable decision-making in real-world applications.

Attribution Methods & Feature Importance Analysis
Interpretable Deep Learning & Post-hoc Explanation Methods

Deep Learning & Representation Learning

Designing deep learning models that learn robust and meaningful representations from complex, high-dimensional data for real-world applications.

Representation Learning & Feature Embeddings
Self-Supervised & Contrastive Learning

AI for Healthcare

Designing machine learning models that integrate heterogeneous healthcare data to support clinical decision-making and personalized medicine.

Multimodal Healthcare Data Integration
Risk Prediction & Clinical Decision Support

Publications

Peer-reviewed articles and conference proceedings

Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) Workshops 2026

LLM Augmented Intervenable Multimodal Adaptor for Post-operative Complication Prediction in Lung Cancer Surgery

We present MIRACLE, a deep learning architecture for prediction of risk of postoperative complications in lung cancer surgery by integrating preoperative clinical and radiological data. MIRACLE employs a hyperspherical embedding space fusion of heterogeneous inputs, enabling the extraction of robust, discriminative features from both structured clinical records and high-dimensional radiological images. To enhance transparency of prediction and clinical utility, we incorporate an interventional deep learning module in MIRACLE, that not only refines predictions but also provides interpretable and actionable insights, allowing domain experts to interactively adjust recommendations based on clinical expertise.

Authors: S. Pandey, B. Jawade, S. Setlur, V. Govindaraju, K. Seastedt

View Paper PDF

IEEE International Conference on Image Processing (ICIP) 2025

Ridgeformer: Multi-Stage Contrastive Training for Fine-Grained Cross-Domain Fingerprint Recognition

We propose a novel multi-stage transformer-based contactless fingerprint matching approach that first captures global spatial features and subsequently refines localized feature alignment across fingerprint samples. By employing a hierarchical feature extraction and matching pipeline, our method ensures fine-grained, cross-sample alignment while maintaining the robustness of global feature representation.

Authors: S. Pandey, B. Jawade and S. Setlur

View Paper PDF Cited by 1

AACR Special Conference in Cancer Research: Artificial Intelligence and Machine Learning 2025

Abstract A054: Multimodal LLM-Driven Intervention for Precision Risk Prediction in Lung Cancer Surgery Available

Development of computational pipelines for analyzing single-cell RNA sequencing data, enabling the discovery of rare cell populations in heterogeneous tissues.

Authors: S. Pandey, B. Jawade, S. Setlur, V. Govindaraju, K. Seastedt

View Abstract Cited by 1

International Conference on Document Analysis and Recognition (ICDAR) 2023

RealCQA: Scientific Chart Question Answering as a Test-Bed for First-Order Logic

We present a comprehensive study of chart visual question-answering(QA) task, to address the challenges faced in comprehending and extracting data from chart visualizations within documents.

Authors: S. Ahmed, B. Jawade, S. Pandey, S. Setlur, V. Govindaraju

View Paper PDF Cited by 21

Recent Blogs

Thoughts on popular research and technology

Research

Februrary 18, 2024 • 8 min read

Evolving domain of Parameter Efficient Fine-Tuning and increasing model capacity for free!!

Explaining the paper "Increasing Model Capacity For Free: A Simple Strategy for Fine-Tuning Large Models" published in ICLR, 2024 which proposes CapaBoost, a novel approach to address a key challenge in parameter-efficient fine-tuning (PEFT) methods like LoRA and Adapters.

Shubham Pandey

Research

January 24, 2024 • 9 min read

Revolutionizing Medical Image Segmentation: Unveiling MedSAM and the Future of Diagnosis

Explaining "Segment anything in medical images", which proposes MedSAM, a novel approach to medical image segmentation that leverages the power of the Segment Anything Model (SAM) to achieve state-of-the-art performance across a wide range of medical imaging modalities and tasks.

Shubham Pandey

Awards & Recognition

Honors and accolades received throughout my career.

Winner of INFORMS Ignite

2025

A fast-paced research pitch competition designed to foster collaboration and idea exchange among students across operations research, analytics, and data science.

Second Runner Up

2018

"Avishkar", annual Tech-Fest of MNNIT, Allahabad

Second Runner Up in North Region

2018

Quiz "Electron", National level Buiz quiz, National Thermal Power Corporation, India

National Level 8th Rank

2018

Quiz "Electron", National level Buiz quiz, National Thermal Power Corporation, India

First Runner up

2018

Hurdlemania, at Technex, annual Tech-Fest of IIT BHU

Get in Touch

Interested in collaboration or discussing research? I'd love to hear from you.

spandey8@buffalo.edu

Office

113V, Davis Hall, White Road
University at Buffalo (North Campus), SUNY
Buffalo, NY 14260

Message sent successfully!