Client Profile:
The U.S. Department of Veterans Affairs (VA) is uniquely positioned to advance AI research and development, using its resources to enhance healthcare for veterans. The "black box" nature of many AI tools necessitates robust oversight and accountability systems to mitigate potential dangers, such as the risk of biased or inaccurate predictions that can harm veterans. The VA developed principles to ensure any developed AI systems foster veteran trust, comply with laws, and uphold the highest ethical standards, including privacy and civil rights.
Challenge:
The VA AI Trustworthy framework was created as we were already implementing an AI system and it was necessary for us to be one of the first to tactically follow the framework. Because of this, the challenge was to implement an AI system by adhering to the principles and requirements of the VA AI Trustworthy framework.
Solution:
Our team followed the VA Trustworthy AI framework on our developed AI solutions, pivoting when necessary, and created a tactical blueprint to cater to the trustworthy framework.
Implementation:
Purposeful Design: AI systems must be designed with a clear purpose, focused on enhancing veteran care and delivering measurable benefits. We considered using concrete metrics to measure the impact of our AI models, prioritizing veteran well-being and assessing their ability to inform decisions to seek care sooner than they otherwise would have been.
Effective & Safe: The AI models underwent rigorous testing to ensure that they were effective in improving decision-making processes and safe for use in healthcare. We created a model using majority voting to provide a decision and in cases where the model did not have enough confidence, we reverted to human decision-making.
Secure & Private: All data was stored with access controls and security measures to protect sensitive information and ensure data was handled responsibly. Patient-identifiable information was encrypted and all DICOM metadata was anonymized to reduce the risk of exposure. To enhance resilience against malicious attacks, we performed adversarial attacks on our classification models by creating small, intentionally crafted perturbations in images.
Fair & Equitable: To ensure fairness and equity in the AI systems, we collected data in a way to represent the diversity of the population and not systematically disadvantage any group. We took random samples of the dataset to help ensure a balanced representation of genders and races.
Transparent & Explainable: LIME (Local Interpretable Model-agnostic Explanations) was integrated to provide clear, understandable explanations of how decisions were made, fostering transparency and trust. LIME showed which features most influence AI model’s predictions and the impact of individual features. It perturbs the input data around the instance being explained, creating synthetic data points, and then observes how the model's predictions change with these perturbations. LIME showed that the model’s prediction of the disease is strongly influenced by specific features found. We determined some unimportant features contributed to the outcome and thus our models should be retrained with additional segmentation of the region of interest.
Accountable & Monitored: It is important to have continuous monitoring and accountability mechanisms to ensure that AI systems remain reliable, accurate, and aligned with the agency’s ethical standards. We implemented logging stored in a centralized way and only be accessed to those with authorized access. In the future, we plan to further implement continuously monitoring of key performance metrics and plan to provide providers knowledge about how our models provide decision support for clinical care.
Results:
As a result of our thorough approach, we created a repeatable and auditable process that closely follows the VA Trustworthy AI framework.
Conclusion:
Ensuring our AI models are implemented in a way that adheres to the Trustworthy AI framework represents a significant advancement in leveraging AI for public service. We piloted adherence to the practice of AI trustworthy guidance focused on purposefulness, safety, privacy, fairness, transparency, and accountability. We built a decision-support model that enhanced the quality of services for veterans and fosters trust and confidence. This approach serves as a model to implement AI in a manner that aligns with high ethical standards and legal requirements.
