Data Science Testing Service for Digital Service Delivery

Description

Summary of the work A flexible team of multiskilled QAs (5-10) to work in fast-paced agile delivery teams, delivering Government Digital Services. The successful supplier will be able to demonstrate that the individuals they put forward for work have the essential experience, knowledge and technical skills set out in this proposal. Expected Contract Length 2 years, but may be shorter depending on progress to deliver the service. Latest start date Monday 1 April 2019 Budget Range The budget could range up to a maximum of £3.0m consisting of multiple SoWs, but that scope could expand or contract. DfE does not commit to any minimum spend at this point. A pricing template will be given at the proposal stage to those suppliers we shortlist. Why the Work is Being Done The ESFA Data Collections systems are used to collect financial, participation, attainment and building information from Academy Trusts, Local Authorities, FE and Apprenticeship Providers and Diocese to inform its funding allocations, payments, services and market oversight remit. The data is collected through bulk upload, online forms and document exchange platforms. As Data Collections systems are the source and starting point of Data Flow in Data Science, Functional and Non-Functional testing is required at the point of collection, data storage, data transformation & reporting to meet quality assurance requirements and high quality product delivery demanded by GDS assessments Problem to Be Solved During 2018 and 2019, The ESFA is upgrading its data collections software, consolidating its platforms and upgrading the software to improve the quality and timeliness of its data and to reduce the burden on its data suppliers. The ESFA is also consolidating its data storage and data management systems and automating data flows to reduce reliance on manual processes to transform, upload and manipulate data for its data scientists. We need to assure that data is processed correctly and data is reported correctly, poor output will impact users referenced above and risks finacial and reputational damage of ESFA and DfE. Who Are the Users As a provider of Primary/Secondary/Tertiary education, I need to submit financial returns to the DfE so that our financial status can be established for approved funding and the DfE can provide accurate reports on our Funding Allocations/ Achievement Rates/ Risk Assessment / Qualitative Reports. As a DfE civil servant, I need to support critical services and support the development and improvement of essential Data Collections systems so that our users above recieve the highest quality ESFA Systems. Early Market Engagement N/A Existing Team The supplier will be working with civil servants and multiple suppliers across various strands of work for delivery of the service. Typically within Agile delivery teams, working with Product Owners, Delivery Managers, Business Analysts, Developers, other Test Engineers and Test Analysts. Current Phase Not applicable Skills & Experience • At least 5 years demonstrable experience in Agile (preferably Scrum and/or Kanban) methodology to projects, to be applied in line with the Data Science Test Strategy • Be able to confidently talk about agile processes • Demonstrable experience of defining or breaking down user stories and acceptance criteria using test design techniques, thereby statically testing the solution at the time. • Contribute to defining and measuring ‘Done’, in order to actually determinewhat the definition of ‘Done’ actually means in terms of quality. • Demonstrable experience of helping developers in generating comprehensive unit tests and pair with them as appropriate. • Use quality characteristics such as functionality, reliability, usability, efficiency, maintainability and portability helps define the context of value and level and type of testing required to meet definition of done. • Demonstrable experience of using a risk based approach to testing. • Demonstrable experience of Evaluating Emergent Behaviour: Using exploratory testing techniques following the customer’s needs, finding defects not found in unit/integration testing but could easily cause the acceptance testing to fail. • Demonstrable experience of Test-Driven Development (TDD), Behaviour Driven Development (BDD) and Continuous Integration (CI) and some of the tools that are used. • At least 3 years’ experience of Test Automation in an Agile environment, including setting up and using frameworks which include Specflow / Cucumber and Selenium • Proven experience in API testing within Agile sprints • Proven experience of accessibility testing within Sprints and presenting finding back to the team made up of technical and non-technical members • Proven experience of Performance testing web based systems and presenting finding back to the team made up of technical and non-technical members • Produce Test Approaches and Sprint Test Plans collaboratively with the team and suppliers, owning the documentation and managing its completion in line with the Data Science Test Strategy • Demonstrable experience of Non-Functional requirements, definition and testing. • Demonstrable experience in exploratory testing techniques and knowing the difference between ad hoc testing and exploratory. • Confident in explaining decisions to stakeholders who don’t understand agile, user needs or user-centred design and delivery Nice to Haves • Experience of working on UK central government services • Strong understanding of the Government Digital Service Standard and Service Manual. • Able to contribute to areas beyond the usual boundaries of the discipline, for example helping the team understand the service’s legal and regulatory environment. • Knowledge of the Education and skills secto • Experience of open source and cloud technologies and their sourcing • Passed certified Agile Tester course • ISEB foundation and practitioner qualified. • Having worked on a Azure platform • Experienced in using the following tools, Sortsite, OWASP Zed Attack Proxy (ZAP), and visual studio. Work Location Education and Skills Funding Agency, Cheylesmore House, 5 Quinton Road, Coventry CV1 2WT, with occasional travel to other locations nationwide. Working Arrangments Onsite full-time 5 days a week co-located with the rest of the team at the ESFA offices in Coventry. Occasional travel to other ESFA (or partner organisation) offices may be required, subject to ESFA travel and subsistence policy. This programme uses agile methodology. Successful application of agile principles and techniques requires that the teams are co-located and able to work directly with colleagues representing digital delivery, operational and policy requirements. Security Clearance The successful supplier must be able to demonstrate that all proposed team members have been subject to Baseline Personnel Security Standards checks. However higher levels of Security Clearance maybe required dependant on work agreed in the Data Science Programme of work. Additional T&Cs Expenses must be pre-agreed and comply with DfE Travel and Subsistence Policy. Any expenses shall be submitted in line with DfE standard T&C policy. Primary work location stated in SoW will not attract expenses. No parking is available on site. No. of Suppliers to Evaluate 3 Proposal Criteria • Demonstrate/provide evidence of experience and knowledge of agile testing and blended skilled Quality Assurance testers (QA). • Demonstrate/provide evidence of experience and knowledge test automation within CI pipeline. • Demonstrate/provide evidence of experience and knowledge of exploratory testing. • Demonstrate/provide examples of how the supplier has worked with delivery teams in a QA role • Demonstrate/provide evidence of how the supplier contributes to building an effective multidisciplinary team • Demonstrate/provide evidence of the supplier’s approach to incorporating user needs and government needs into the product backlog and prioritises work effectively and efficiently • Demonstrate/provide evidence of how the supplier actively promotes understanding and acceptance of agile, user-centred delivery • Demonstrate/provide evidence of how the supplier has worked in a multi-team environment with multiple product owners collaborating to deliver a broader service Cultural Fit Criteria • Work as a team – Collaborate, establish good working relationships, generate team spirit • Show a commitment to ensuring a team understands the user need over simply delivering work • Innovate – Act on initiative, drive improvements, balance risks • Build capability – Encourage learning, share knowledge, address capability gaps, act on feedback • Demonstrate responsibility (for example, act like a leader, serve your team, provide solutions to problems, engage in external/internal environments, seek the best for the organisation beyond your team) • Focus on delivery – Focus on goals, set stretching targets, take responsibility for delivery • Approach to developing a "One Team" approach Payment Approach Capped time and materials Evaluation Weighting Technical competence 50% Cultural fit 20% Price 30% Questions from Suppliers 1. Is this requirement to provide skilled people to work under the direction of ESFA, as opposed to suppliers taking responsibility for achieving project outcomes? The requirements will be specified in Statement of Work (Request for Quote (RFQ)), the supplier will be responsible for the quality of work and delivery outcomes. 2. Is the work starting on 1st April commencing with a very specific phase of the government delivery lifecycle (i.e Discovery, Alpha, Beta) Alternatively, is the requirement to provide skilled people to work under ESFA's general direction? Within the Data Science programme there are a number of projects being worked on all at different stages. Some of this work is currently in progress. 3. How many roles will need to be deployed on 1st April? How many roles will be required within the first month? The service provider needs to advise on the team size following review of Statements of Work. 4. The requirements is "to work in fast paced Agile teams" how many teams are relevant to 1st April start date? Please also describe at a high level the focus of each of these teams? 4 Agile teams are involved to develop new and enhance new data collections and data products/cubes/reportsThe Data Science journey starts with - Data Collections with Online Forms and File Collectionsand moves through...- Data Storage (Data Lake / Data Warehouse)- Data Transformation ETL/ Modelling- Data Visualisation - Reports/Dashboards/PowerBI/OLAP Cubes- Statistical publicationsWith the range of projects ahead in Data Science for the next financial year, we will need support for future projects. 5. Can you please share a copy of the DfE Travel and Subsistence Policy, or explain the key elements? Expenses can only be claimed for meetings such as GDS assessments. Must be included in the Request for Quote (RFQ) and PO. Applied at the managers discretion. 6. Is this opportunity outside of IR35? Yes 7. Can we ask what criteria ESFA will be using to judge whether an evidence answer 'exceeded' requirements and therefore scores full marks (3)? Each question will be scored in line with the standard DOS methodology outlined on the CCS website. 8. ESFA are procuring multiple workstreams in parallel, will ESFA reconsider staggering the procurement schedule? Change in schedule is not planned, but we will inform you if this changes. 9. 1. Is it acceptable to have multi-location team set-up (nearshore and/or offshore along with UK ESFA offices )?2. For the dev/test activities will we be utilising the customer sponsored Cloud / DC environments3. For the dev/test activities will we be utilising the customer sponsored toolset or free tools4. For test data used for testing will it be sanitized / anonymization for use by customer/expert team outside test 1. As Agile teams are expected to be co-located, representatives of service are expected to be on site, unless otherwise stated on SOW.2. Yes - customer sponsored Cloud / DC environments3. Yes - customer sponsored toolset4. Service provider will need to supply representation with required Security Clearances, they will be involved with setting up Test Data, which may need to be annonimised. 10. Security Testing12) How many static and dynamic pages are in each application (on an average)?13) How many input fields does the dynamic pages have each application (on an average)?14) How many roles are available for each application (on an average)?15) Is the security testing to be done on a production environment or a controlled environment?16) Are there any devices in place that may impact the results of a security test such as a firewall, intrusion detection/prevention system, web application firewall, or load balances? We carry out CHECK Accedited ITHC Checks prior to deployments on new/major changes on Digital IT Services. Highlight risks at earlier stage, provider representatives will be embedded in Agile teams,providing Assurance on all aspects of testing. We have high turnaround of On-line forms and Data Collections being developed and evolving. From conception/sprints, representatives guide/advise on Security concerns, work with BAs, Developers, Architects to specify acceptance criteria for security. Using open source toolsets, complete tests to prove/meet acceptance criteria. 5-8 rolesSecurity testing done on a production like environment, where environments not available, advise on the best way forward.Yes 11. 6) What types of application(s) are within the scope of engagement (weband on-Premise)7) What type of architecture being used for web application (2 tier or 3tier or micro services-based architecture)8) What is the expected user load (normal and peak) for the application(s) under scope?9) Please share information on sprint release timelines currently beingfollowed. 6-8. To highlight risks at an earlier stage, we are looking for provider representatives that can be embedded in Agile teams, to provide Assurance on all aspects of testing. We have a high turnaround of On-line forms and Data Collections being developed and evolving. From conception through the sprints, we need representatives to guide and advise on affects to System Performance, need to work with team BAs, Developers and Architects to specify acceptance criteria. Using in house tools - VSTS and JMeter to complete tests to prove and meet the acceptance criteria. 12. Good morning- in which section are we able to insert our rate card, please. This will form part of the second stage of the evaluation process. 13. 1) Would the written proposal include response to the questions underthe section proposal criteria? Please share the email address of therecipient for written proposal.2) Would the pricing details to be provided during the next round ofevaluation (during presentation)?3) Do you expect Functional and Non-Functional testing skills in thesame QA resource?4) Is Non- Functional testing restricted to Performance and Securitytesting services only5) Please let us the know the following – – Number of projects/agile teams – Application details – Testing tools (automation, test & defect management, performance & load testing, application monitoring) YesYes at stage 2Yes where multiple representatives are being put forward, if one representative has lesser skills, we expect other representatives from supplier to fill.No - looking for strong awareness with other elements including Assesibility testing, Operational acceptance testing. ESFA predominantly uses Microsoft Products and services, Technology for Applications includes, bespoke applications coded in C#, supported by MS Azure infrastructure. MS Dynamics CRM, Achieve Forms (COTS), Oracle Policy Automation (COTS), MS SQL Server Data Management Tools.Application and Test Managment is on VSTS, Jira also used. Automation Frameworks are typically c#, specflow, Java frameworks also in place. 14. Will scoring from the evidence stage be taken through to final evaluation scoring? As opposed to being used solely for the purpose of shortlisting? Yes scoring will be taken through to the second stage of the evaluation process. 15. What stages in the procurement timeline does ESFA expect between 15th February and 1st April? In particular are there planned dates for proposal submission and presentation? Shortlisting will take place w/c 18th February, with evaluation panel scheduled for the 1st week of March. 16. Are there any test management tools currently in place that you would expect us to use? Can you share what they are if this is the case please? VSTS is used for Test Management, is some projects JIRA is used but is being phased out. 17. There is reference to the DfE Travel and subsistence policy and adhering to this policy. Can you share the policy with us please? Please see question 5 18. Do security clearance checks need to be in place before the bid is submitted or before the work commences? Can you help with the process for any team members who don't have this currently? Prior to the 1st April. The supplier is responsible for ensuring all contractors have passed BPSS checks prior to responding to a statement of work request 19. There is reference to the Data Science Test Strategy. Can you advise what this strategy is please? This is the Test Strategy that guides test teams and projects within Data Science’s programme of work.