Evaluation of an NHS Wales Generative Pretraining Transformer (GPT) AI Tool

Description

Generative AI is increasingly being explored as a transformative tool in various sectors, includinghealth and social care. To support its safe and effective adoption Life Sciences Hub Wales seeksto procure the services of a suitably capable and experienced provider (the successful supplier) toassess and evaluate the performance and impact of a GPT tool develop within NHS Wales.A GPT tool has been developed for use within NHS Wales to enhance AI-related skills,understanding, and operational efficiency within the workforce. The required evaluation servicesis intended to assess the impact of the developed GPT tool against the following:- AI Skills and Workforce Understanding: Evaluation of how effectively the tool improves theworkforce’s comprehension and readiness to use AI systems.- Efficacy and Usability: Measuring the tool’s practical effectiveness in meeting intendedobjectives, including relevance, accuracy, and ease of use.- Areas for Improvement: Collecting user feedback to identify gaps or barriers to adoption andinform potential enhancements for future iterations.