Autonomous Image Quality and Hallucination Assessment for Virtual Staining and Digital Pathology

Introduction: : Histopathological staining is essential for diagnosing human diseases by clearly delineating specific tissue features. Recent advances in virtual tissue staining offer promising alternatives to traditional histochemical staining methods, drawing considerable interest due to several benefits, such as rapid tissue processing by eliminating tedious chemical staining steps, multiplexed staining capabilities, enhanced tissue preservation, and standardized color representation. However, despite these significant advantages, virtually stained images continue to face reliability concerns, particularly due to the potential presence of artifacts and hallucinations. Such hallucinations may erroneously introduce diagnostic features related to non-existent diseases or fictitious patient scenarios, thereby posing diagnostic challenges. To address these critical issues, we introduce AQuA (Autonomous Quality and Hallucination Assessment), a novel tool specifically designed to autonomously assess the quality of virtually stained images and effectively reduce risks associated with hallucinations. Additionally, AQuA's functionality extends beyond virtual staining to autonomous support quality assessment for conventional histochemically stained images within digital pathology, overperforming current existing techniques.

Materials and

Methods: : AQuA comprises lightweight AQuA-Net ensembles. Each AQuA-Net ensemble employs an iterative process between virtual and autofluorescence image domains, leveraging three-dimensional spatio-temporal data as input. By exploiting consistency across iterative cycles, this approach surpasses the traditional two-dimensional analyses typically used by existing quality assessment workflows and human evaluators. AQuA-Net leverages a pre-trained ResNet backbone to extract features from spatio-temporal image sequences. These features are subsequently processed through temporal convolutional and linear layers, resulting in a confidence score that reflects the likelihood of a given test image passing quality control. The final confidence score is obtained by averaging the outputs from multiple AQuA-Net ensembles. For training and testing purposes, we constructed a dataset of virtually stained images from well-converged models (negative class, good quality) and those from early-stopping models (positive class, hallucinations detected). These datasets, comprising images from lung and heart tissues, were rigorously separated to ensure distinct training and testing subsets, using different models and unique patient data to maintain the robustness and generalizability of the evaluation. Beyond early-stopped virtual staining models, AQuA was further evaluated by testing its inference on overfitted virtual staining models and histochemically stained images.

Results, Conclusions, and Discussions:: We first validated AQuA on virtually stained hematoxylin and eosin (H&E) images of human kidney and lung tissue. On the blind test set, AQuA achieved 99.8% accuracy for kidney and 97.8% for lung tissue sections in distinguishing hallucinated (positive) from high-quality (negative) images without the need to access histochemically stained reference images. In comparison, traditional supervised metrics reliant on histochemical reference images, e.g., mean square error (MSE), Pearson’s correlation coefficient (PCC), and peak signal-to-noise ratio (PSNR), performed significantly worse. To further benchmark the performance of AQuA, three board-certified pathologists were invited to score the same set of unseen VS images without access to histochemical references. AQuA outperformed human evaluators by achieving 100% accuracy in detecting subtle and realistic hallucinations produced by poorly trained models—features that may visually resemble standard histochemical staining but represent non-existent structures and could mislead diagnoses. For high-quality (negative) virtually stained images, AQuA demonstrated a 98.5% agreement with pathologist consensus for kidney and a 98.3% agreement for lung tissue sections. AQuA also proved resilient to unseen failure patterns arising from distribution shifts, achieving 100% accuracy in distinguishing high-quality virtually stained images from poorly generated ones by overfitting models. Beyond virtual staining, AQuA extended its quality assessment capability to histochemically stained images, achieving 100% accuracy in differentiating well- and poorly-stained samples—significantly outperforming existing methods such as TransMIL.

Acknowledgements and/or References (Optional)::