Artificial Intelligence Reveals Features Associated with Breast Cancer Neoadjuvant Chemotherapy Responses from Multi-stain Histopathologic Images

By combining H&E and multiplex IHC (PD-L1, CD8+, and CD163+) images, artificial intelligence (AI)-based algorithms using automatic feature extraction methods can predict neoadjuvant chemotherapy (NAC) outcomes in breast cancer patients from pre-treatment biopsies.
Artificial Intelligence Reveals Features Associated with Breast Cancer Neoadjuvant Chemotherapy Responses from Multi-stain Histopathologic Images

Predicting clinical outcomes from pre-treatment histopathologic images brings tremendous impact. However, it remains as a big challenge due to the limited understanding and interpretation of tumor immune micro-environments. Comparing to the H&E-stained images, multiplexed immunohistochemistry (IHC) images can reveal multiple markers simultaneously from a single tissue section. In this study, we leveraged the multi-stain histopathologic images, proposed an automatic workflow for breast cancer pathological complete response (pCR) prediction from pre-neoadjuvant chemotherapy (pre-NAC) biopsies.

With our approach, IHC stained information including PD-L1, CD8+ T cells, and CD163+ macrophages are co-registered into H&E-stained tumor immune micro-environment, generate a combined feature set to predict the NAC response. In total, we extracted 36 interpretable and meaningful histopathological features, established three categories of quantitative features to characterize different cellular components – namely the “area ratio”, “proportion”, and “purity” – in our proposed pipeline, and formally designated our pipeline as “IMage-based Pathological REgistration and Segmentation Statistics”, or “IMPRESS” in short.

Sixty-two HER2+ and sixty-four TNBC female patients were included in our cohort to examine whether machine learning model using IMPRESS would be able to predict pCR for NAC. We found that the developed machine learning models utilize IMPRESS and clinical features can accurately predict the response to NAC in breast cancer patients (HER2+ AUC = 0.8975; TNBC AUC = 0.7674), and outperform the results learned by features that manually generated by pathologists. The developed approach was further externally validated in two independent cohort for HER2+ and TNBC subtypes, yielding AUC=0.90 for HER2+, and AUC=0.59 for TNBC. These results suggest pre-NAC IMPRESS feature and model can help predict post-NAC outcomes, especially for HER2+ subtype. Additionally, we comprehensively evaluated those automatically extracted features by feature importance analysis and residual cancer burden analysis. These results present promising insight into the tumor immune micro-environment of breast cancer NAC patients, prompting the need for multi-stained computational analysis before the NAC treatment.

Our study has also demonstrated the relationship between several tumor immune micro-environment features and pCR. Perhaps one of the most interesting findings is PD-L1 expression in pre-treatment tumor immune micro-environment, especially in TNBC cohort. It has been reported that the upregulation of PD-L1 is involved in various cellular processes in cancer cells as well as interactions between cancer cells and immune cells. It has been conflicting whether PD-L1 expression is a favorable or adverse prognostic factor for breast cancer patients’ survival. The conflicting conclusions may result from the differences in composition of cohorts, PD-L1 antibody clones, or assessment methods (most studies used manual assessment). In our study, PD-L1 in lymphocytes aggregated region was found to associate with a favorable response to NAC. Kong et al. [1] suggested that PD-L1 expression at different locations had different impact on survival in colorectal cancer (CRC) patients, and showed that total PD-L1 expression was a favorable prognostic marker. In our study, we observed similar behavior of high TIL and PD-L1 expression.

Furthermore, our data has also demonstrated that the most important IMPRESS features identified from the logistic regression model to predict pCR (such as CD8, CD163, and PD-L1 ratios in lymphocytes aggregated region, and CD8 proportion in lymphocytes aggregated region) also correlated with RCB, at least partially. The correlation analyses for IMPRESS features to themselves reveals the highly and densely correlated features, providing additional insights to morphologic and clinical features which are important for therapy response in breast cancers. The correlation analyses for IMPRESS features to residual cancer burden (RCB) found more significant features in HER2+ subtype (13 out of 36) than in TNBC subtype (3 out of 36), suggesting IMPRESS features may well characterize those residual tumors in HER2+ breast cancer patients.

Comparing to the classic clinical scores, AI can objectively evaluate slides. As further depicted in [2], AI helps discriminate high and low risk of relapse for early ER+HER2- breast cancer. These emerging AI tools can enable an early rule-out with a decent amount of the cases. However, as we shown from the TNBC cohort, the power of AI may be limited to several, not all, cancer subtypes. Thus, the prognosis power we presented on HER2+ and TNBC subtypes may further benefit the future research endeavors, including but not limited to power analysis and model selection.

Please sign in or register for FREE

If you are a registered user on Nature Portfolio Cancer Community, please sign in