Talk to the Veterans Crisis Line now
U.S. flag
An official website of the United States government

Office of Research & Development

print icon sign up for VA Research updates

Computer Vision and Machine Learning in Precision Oncology (CoMPL)

Computer Vision and Machine Learning in Precision Oncology (CoMPL) is a VA hub established in October 2021 to develop an enterprise-wide community around the use of computer vision and machine learning (CVML) for precision oncology within the VA and nurture the development of novel concepts to solve pressing real-world clinical problems facing Veterans. CoMPL's objectives include building the computational infrastructure and tools to allow for expanding the scope and access to CVML resources within the VA; building a community to enable VA researchers to take advantage of these tools to develop their own CVML applications; and developing new companion diagnostic tools for risk assessment that can predict both the response and need for more or less aggressive therapy in prostate, lung, and other cancers.

Cancers & Diseases:

Brain, Prostate, Lung, Breast, Rectal, Oropharyngeal Head & Neck, Ovarian, Diabetic Macular Edema, Cardiac arrythmia

Areas of Research:

  • Predictive Companion Diagnostics
    • Vessel Tortuosity
    • Integrated radiomic/deep learning
    • Leakage distribution network and vasculature networks
    • 12-lead ECG CRT patient response
  • Prognosis:
    • Risk Stratification using Quantitative histomorphometric features

Contact Information:

Director: Anant Madabhushi, PhD
Administrative Director: Michael Gilkey, MS MBA (may change)

Computer Assisted Diagnosis and Detection

This work uses computers to locate important clinical information on routine images taken by clinicians during care and identifies information not visible to the naked eye.  Our researchers create software programs that generally take 6-10 features from these images to predict what the disease is and how a specific treatment will work for a patient.  Together with the clinical team, we are working to create tailored treatments for patients and provide custom care to Veterans.

Brain cancer: We developed an in-house gradient based radiomics descriptor to distinguish brain radiation necrosis grades and tumor recurrence. Our radiomics descriptor can distinguish different grades of Radiation Necrosis in a multi-institutional study.

brain tumor diagnostics

Lung cancer: Peritumoral regions in lung CT scans are used to differentiate adenocarcinomas from granulomas, achieving an AUC of 80% in differentiating adenocarcinomas from granulomas.

lung cancer diagnosis

Prostate cancer: Prostate MRI scans are used to extract radiomic features to differentiate prostate cancer from high grade non-malignant inflammation.

prostate cancer diagnosis

Biomarkers and Technology

Here we are using computers to look deeply at images both from radiology (CT, MRI, X-ray, etc.) and pathology (biopsies and tissue taken out of the body and sliced thinly so we can see through it) to find clinically important information like closeness of immune cells or twistedness of blood vessels around a tumor to create an actionable test, like a COVID test, to help inform what treatment a patient should get or whether a patient’s cancer will resist specific treatments like immunotherapy.  Though not yet approved from use in the hospital, our research has shown these tests can be more accurate than genetic tests which cost $3,000 or more.  They can also be combined with other test results to create even more accurate computer-based tests for how the clinical team might want to treat Veterans.

VaNgoGH: Vascular Network Organization via Hough Transform, helps unravel the architecture of tumor vasculature from CT and MRI scans, which has been shown to be predictive of response to chemotherapy in breast and locally advanced non-small cell lung cancer (NSCLC).

Vascular Network Organization via Hough Transform lung CT scan

SpaTIL: SpaTIL was developed to interrogate the spatial interaction of immune cells as a potential biomarker to predict recurrence prognosis, response, and overall survival. It was shown to be prognostic of recurrence in triple negative breast cancer (TNBC) and early-stage NSCLC. It also predicted responses and survival rates of lung cancer patients treated with immunotherapy (IO).

SpaTIL was developed to interrogate the spatial interaction of immune cells as a potential biomarker to predict recurrence prognosis (Breast cancer), response and overall survival (Lung cancer).

Deformation radiomics: Deformation radiomics is applied on Brain tumor MRIs to quantify mass effect and post-treatment tumor heterogeneity. These features significantly stratified short-term and long-term survivors.

tumor deformation

HistoQC: HistoQC was developed to account for artifacts and batch effects unintentionally introduced during pathology slide preparation and digitation. Compared to two pathologists, HistoQC, yielded an agreement of >95% suggesting its suitability as a Quality Control tool.

Tissue organization: Tissue organization captured by radiomics scans are used to develop a biomarker to predict response and treatment of Rectal Cancer.

spatial rectal cancer

Collagen Fiber Orientation: Collagen Fiber Orientation Disorder in Tumor associated Stroma (CFOD-TS) was shown to be prognostic of disease-free survival in a large multi-center clinical trial using ER+ breast cancer. Over-expression of CFOD-TS was found to be independently associated with lower likelihood of recurrence and could potentially serve as a prognostic marker of outcome for ER+ breast cancer.

Predictive Companion Diagnostics

We are also looking into tests that can let the clinical team know whether to increase or decrease treatment times or dosage, whether to withhold a certain part of the treatment, and if the treatment is working or not.  These tests work alongside standard treatments but give additional information for how patients should do during and after treatment at the individual level.  The goal of this work is to reduce unnecessary toxicity for Veterans for treatments that won’t work on their cancer.

Vessel Tortuosity: Immunotherapy can drastically improve survival in advanced lung cancer but only around 20% of patients respond to it. Current prognostic methods, including PDL-1 and TMP, have only limited accuracy. Utilizing changes in vessel tortuosity radiomics between pre-treatment and early post-treatment CT scans can predict survival in immunotherapy treated patients. Responders to IO showed a marked decrease in tortuosity, unlike non-responders showing no change or increase in tortuosity.

qvt lung cancer

Integrated radiomic/deep learning: Integrating radiomic methods and deep learning methodologies increases the accuracy in predicting response to chemotherapy in breast cancer. The integrated model includes significant features from both inside the tumor and its adjoining habitat.

chemo breast cancer flowchart

Leakage distribution network and vasculature networks: These two features are indicative of rebounders and non-rebounders in Anti-VEGF therapy in Diabetic Macular Edema (DME). At baseline FA scans, rebounders have a more tortuous vessel network and less chaotic leakage patterns when compared to non-rebounders.

12-lead ECG CRT patient response: Response to Cardiac Resynchronization therapy (CRT) can be evaluated by applying machine learning methods on 12-lead ECG waveforms, where CRT subgroups based on CRT outcome were identified.

Response to Cardiac Resynchronization therapy (CRT)


This last bit of work uses computers to separate patients, before treatment is given, into high risk and low risk groups based on the patient’s specific biology and a specific treatment.  By doing this, we can tailor treatment strategies that reduce toxicity and cost, and improve outcomes for Veterans. We are also using these strategies to predict whether treatments might cause other dangerous issues or adverse events, like lung inflammation after radiation therapy for lung cancer, so that the clinical team can closely monitor at-risk patients or tailor their treatment to avoid complications.

Quantitative histomorphometric features:

  • Nuclear shape and orientation are strongly and independently predictive of patient survival in ER+, LN- Breast Cancer.
  • Nuclear morphology can stratify patients with HPV+ oropharyngeal head and neck cancer into low- and high-risk groups.
  • Lumen morphology on prostatectomy tissue (shape and architecture) can stratify patients and predict biochemical recurrence.
  • Quantification of Tumor Infiltrating Lymphocytes is prognostic of survival in epithelial ovarian cancer patients.
breast cancer ovarian cancer

Questions about the R&D website? Email the Web Team

Any health information on this website is strictly for informational purposes and is not intended as medical advice. It should not be used to diagnose or treat any condition.