cyto‐Knet: An instance segmentation approach for multiple myeloma plasma cells using conditional kernels

Multiple myeloma disrupts normal blood cell production, requiring early detection due to the increased risk of bone metastases. Although various artificial intelligence (AI) methods have been developed to assist pathologists, they often lack comprehensive metrics to measure both detection and segmentation errors. This study presents a hybrid framework that combines deep learning and heuristic techniques to achieve accurate instance segmentation of individual plasma cells in Giemsa-stained cytology images. Our proposed network, called cyto-Knet, incorporates an innovative color-balancing method as a preprocessing step to standardize the appearance of cytological images. Our network leverages a 4-class segmentation strategy with conditional kernels to enhance segmentation performance and accuracy. Additionally, a marker-based watershed algorithm is employed in the postprocessing step to address the challenge of merged objects. Extensive validation at both pixel and object-based levels demonstrates superior performance compared with state-of-the-art techniques. Our method achieves pixel-based metrics (precision, recall, and F1-score) of approximately 0.90. The object-based evaluation reveals an average Dice coefficient of 0.9130 and an aggregated Jaccard index of 0.8237. Importantly, our solution is designed for integration into an end-to-end system for diagnosis support and can be easily extended to other applications.