I22 - Modeling Phenotypic Adaptation and Enzalutamide Resistance in Prostate Cancer Under Adaptive Therapy

Introduction: : Prostate cancer (PCa) is among the most prevalent cancers globally, typically treated with hormone therapy due to its dependence on androgen signaling. Despite initial efficacy, prolonged androgen deprivation often leads to therapeutic resistance, characterized by phenotypic shifts and genetic mutations, resulting in disease progression. While resistance mutations are well-documented, transient phenotypic adaptations that enable short-term persistence remain inadequately explored. Understanding these reversible mechanisms and their implications for clinical outcomes is essential for developing more effective therapeutic strategies.

Materials and

Methods: : We utilized the Soft Actor-Critic (SAC) algorithm within a Deep Reinforcement Learning (DRL) framework to optimize adaptive therapy (AT) for prostate cancer. This model incorporates stochastic differential equations to simulate interactions between sensitive and resistant tumor cell populations, responding dynamically to variable treatment intensities. SAC employs a continuous action space and entropy-driven exploration to balance effective tumor suppression with the minimization of drug exposure, enhancing treatment adaptability.

Model training involved comprehensive calibration and validation using both in vitro experimental data and longitudinal PSA measurements from clinical patient cohorts undergoing androgen deprivation therapies. The integration of patient-specific clinical data enabled robust validation and improved predictive accuracy regarding individual tumor dynamics and resistance trajectories, facilitating personalized adaptive therapy strategies.

Results, Conclusions, and Discussions:: Results
We developed a time-varied, mixed-effect generalized Lotka–Volterra (tM-GLV) model to simulate prostate cancer cell population dynamics under adaptive therapy. This model was rigorously calibrated using both experimental and patient-derived prostate-specific antigen (PSA) data to reflect clinically relevant tumor behavior. The simulations reproduced distinct patient response profiles frequently observed in clinical practice. First, a subset of patients—designated as responders—demonstrated an initial decline in PSA levels, followed by a gradual development of therapeutic resistance. Second, non-responders exhibited minimal PSA reduction, indicative of intrinsic resistance to treatment from the outset. Third, an intermediate group showed pronounced early PSA suppression but subsequently underwent rapid phenotypic shifts leading to resistance emergence. Beyond characterization of response patterns, we integrated reinforcement learning to optimize adaptive therapy schedules. These data-driven strategies significantly outperformed standard intermittent androgen deprivation therapy (IADT), achieving more effective control of both sensitive and resistant cell subpopulations while substantially reducing cumulative drug exposure (Figure 1).

Conclusion
This study demonstrates the effectiveness of combining stochastic modeling and reinforcement learning, validated by clinical PSA data, to optimize personalized adaptive therapies for advanced prostate cancer. Our computational approach successfully predicts patient-specific tumor dynamics and resistance patterns, offering substantial improvements over conventional treatment methods. By dynamically adjusting therapy based on individual tumor responses, this framework promises to enhance clinical outcomes and manage therapeutic resistance in prostate cancer patients.

Acknowledgements and/or References (Optional):: JTG is a CPRIT Scholar in Cancer Research and is supported by CPRIT grant (RR210080).