Interactive online case study repository containing examples of the use of AI assurance techniques

Description

Summary of the work At minimum: -Prioritised set of user stories describing user needs to be addressed. -Alpha prototype for interactive repository (e.g. wireframes) -Beta version of the interactive repository. -Summary of key findings from user-testing Alpha/Beta versions. -Documentation for CDEI ownership. -Work in a format which is simple to update/iterate Expected Contract Length 4 months Latest start date Monday 14 November 2022 Budget Range It is estimated that the work can be delivered within a budget of £75,000 plus VAT. Why the Work is Being Done As set out in the CDEI AI Assurance Roadmap and AI Assurance Guide, the benefits of Artificial Intelligence (AI) will only be realised if organisations, users, and citizens can trust AI systems and how they are used. AI assurance techniques provide the ‘infrastructure’ for assessing and communicating whether AI systems are trustworthy. CDEI’s AI Assurance Programme builds on the priority areas identified in the AI Assurance Roadmap. The Programme aims to help stimulate a thriving AI assurance ecosystem that enables greater responsible use of AI, and establishes the UK as a leader in trustworthy AI. There has been demand for practical guidance from industry actors in order to raise awareness about, and help them navigate through, the range of available techniques to drive the adoption of assurance practices in this space. This proposed online case study repository aims to deliver such information. Problem to Be Solved As part of the initial Discovery phase the CDEI is conducting stakeholder engagement activities to explore the barriers and enablers to the adoption of technical standards for AI assurance and AI assurance techniques. We have found that there is a need for practical guidance and examples to demonstrate why and how to conduct AI assurance. CDEI aims to develop an interactive case study repository aimed at industry executives responsible for procuring/deploying AI systems to provide guidance on the topic of AI assurance. The case study repository will build on existing guidance like the AI Assurance Guide and will be informed by insights into user needs identified during the initial CDEI-led Discovery phase. Using these insights the suppliers will develop an online interactive case study repository to support executives to use and refer to appropriate assurance techniques as they buy/deploy AI systems. This work roughly corresponds to the Alpha/Beta phases of Agile delivery. The suppliers will conduct user-testing of the repository to help refine and validate the deliverables. This repository will require in-built mechanisms for user feedback directly from the webpage for the public version of the repository to facilitate improvements/updates. Who Are the Users The target audience for this work - which is to be delivered via an interactive online case study repository - are industry executives responsible for buying and/or deploying AI systems as they are uniquely positioned to drive demand for AI assurance. As described in the CDEI AI Assurance Guide, executives need to demonstrate that the AI solutions they adopt meet applicable legal, ethical and technical criteria, assess and manage risks associated with the AI systems, and communicate their compliance and risk management to regulators, boards, frontline users and affected individuals. They also require clear and reliable evidence from developers and/or vendors that the AI systems they buy and deploy meet applicable legal, ethical and technical criteria to gain enough confidence to deploy these systems. Executives need to understand which AI assurance techniques (or combination of techniques) are most appropriate for their context, and the respective strengths and weaknesses of different approaches. They need to know how they should use them (or have confidence that they are being used by developers) to build confidence in the AI system. Work Already Done When this contract begins, the Discovery phase will be complete, providing insights into user needs. Earlier outputs include a report discussing the role of standards in AI assurance; the AI Assurance Guide; findings from ministerial roundtables with assurance providers/AI companies; workshop with TechUK members to understand the barriers to using AI assurance faced by industry and interviews and a survey to identify barriers to using AI assurance. CDEI is also working with TechUK to gather case studies for inclusion within the repository. These will form the basis of the online interactive case study repository. Existing Team The existing CDEI project team consists of a Senior Technology Advisor and one Policy Advisor, both of whom work within a wider team focussed on AI Assurance. The supplier will be expected to work collaboratively and closely with the existing team members, working towards shared objectives. Current Phase Alpha Skills & Experience • Demonstrable previous experience (minimum 3 years) in iteratively designing, developing, prototyping, and releasing online services which meet user needs and GDS Service Standards for content design, service design and accessibility • Demonstrable previous experience designing online interactive user-facing guides/tools/repositories. • Demonstrable experience (minimum 3 years) in UX and UI design, using user-centred design approaches to create responsive digital user experiences that meet accessibility legislation and best practice guidance • Demonstrable experience in successfully running digital projects through their life cycle including Discovery, Alpha, Beta, Live and Retirement phases. • Demonstrable experience with GDPR compliance and following ICO guidelines, especially those relevant to interactive websites. • Demonstrable experience of good planning, coordination, communication and project management skills (within the past 3 years) • Demonstrable experience of testing interactive online guides/services for accessibility and usability, using a range of devices (mobile/desktop), internet browsers and assistive technology used by their various audiences. • Demonstrate an understanding of, and implementing website metric software such as Google Analytics. Nice to Haves • Awareness and/or experience with AI assurance techniques • Awareness and/or experience of developing, testing or deploying AI based systems • Experience of bringing innovative ideas to business problems when designing digital solutions • Awareness and/or experience with developing/designing interactive tools/guides/repositories • Awareness and/or experience of working with industry executives and their decision making process Work Location Onsite working is not required and most meetings can be conducted online via Google Meet. However, the supplier is expected to attend occasional in-person meetings at our London office located at: 100 Parliament Street, London, SW1A 2BQ Working Arrangments Onsite working is not required as the work can be conducted remotely. Meetings will be conducted in-person or using Google Meet depending on the working location of staff and business needs. The CDEI team works flexibly and in a hybrid format with an average of two days in the office per week. This work will be collaborative and the supplier staff will be expected to attend regular meetings (at least one hour per week, which can be held virtually or in-person) with the CDEI team to review progress and align delivery with wider project and programme outputs. Security Clearance All suppliers must have BPSS as a minimum. No. of Suppliers to Evaluate 5 Proposal Criteria • Quality of technical solution • Approach and methodology • How the approach or solution meets user needs • How the approach or solution meets your organisation’s policy or goal • Estimated timeframes for the work • How they’ve identified risks and dependencies and offered approaches to manage them • Team structure • Value for money Cultural Fit Criteria • work as a team with our organisation and other suppliers • be transparent and collaborative when making decisions • take responsibility for their work • share knowledge and experience with other team members • can work with clients with ranging technical expertise Payment Approach Fixed price Assessment Method • Case study • Work history • Reference Evaluation Weighting Technical competence 70% Cultural fit 10% Price 20% Questions from Suppliers 1. Does the £75,000 budget cover all the work listed in the ‘summary of work’?:-Prioritised set of user stories describing user needs to be addressed.-Alpha prototype for interactive repository (e.g. wireframes)-Beta version of the interactive repository.-Summary of key findings from user-testing Alpha/Beta versions.-Documentation for CDEI ownership.-Work in a format which is simple to update/iteratePlease confirm. Yes, the £75,000 budget covers all of the work listed in the summary of work, although primarily the work to be completed by the supplier will be focussed on delivery of the Alpha and Beta phases of an interactive repository, published online for industry actors to access. CDEI have conducted a series of stakeholder engagement activities as part of the Discovery phase, the outcomes of which will inform this work. CDEI are also working with TechUK to collect case studies to populate the repository. CDEI are willing to consider a range of different approaches to delivering an interactive repository. 2. Is there an incumbent supplier who has delivered the Discovery? No, the Discovery phase of this work has been conducted by the CDEI AI assurance team. It consisted of a combination of literature review, ministerial roundtable meetings, a workshop with TechUK members, interviews, and a survey. These activities helped to explore the key barriers and enablers to AI assurance and better understand the needs of our target audience.