A/B Test Programme Implementation, Support and Consultancy

Description

Summary of the work A specialist A/B test partner using Optimizely to help MAS improve conversion rate optimisation on our core website. This includes consultancy, test implementation and test reporting. Expected Contract Length 12 months with option of 12 month extension Latest start date Thursday 31 August 2017 Why the Work is Being Done MAS require an A/B test partner using Optimizely to help us improve conversion rate optimisation on our core site. Problem to Be Solved MAS does not currently have the in-house resource capacity to deliver the desired A/B test programme at the required speed. Who Are the Users Not applicable Work Already Done MAS have been running an A/B test programme for 20 months. Existing Team The partner would work in conjunction with the MAS digital team. Current Phase Not applicable Skills & Experience • Be experts in the creation and implementation of Optimizely tests • Be able to undertake front end development for more complex tests and QA within Optimizely • Consideration for mobile, ipad and desktop • Website accessibility standards • Be experts in creating new customer journey flows in Optimizely • Be experts in developing tests for SPA (Single Page Application) sites in Optimizely Nice to Haves Demonstrable competent account and contract management will be viewed with encouragement Work Location The Money Advice Service Holborn Centre 120 Holborn London EC1N 2TD Working Arrangments Must be able to attend monthly review meetings at MAS offices and potential adhoc request to attend at short notice. Typical day to day contact will be via online communication channels such as Slack, Trello and Basecamp. No. of Suppliers to Evaluate 3 Proposal Criteria • Organisational Structure and Core Business Relevance • Communications and Contract Management • Quality and Fulfilment of requirements • Information Technology • Understanding of Requirements • Project Team • Approach and Methodology Cultural Fit Criteria • Work as a team with our organisation and other suppliers • Be transparent and collaborative when making decisions • Have a no-blame culture and encourage people to learn from their mistakes • Take responsibility for their work • Share knowledge and experience with other team members • Challenges the status quo Payment Approach Time and materials Evaluation Weighting Technical competence 60% Cultural fit 10% Price 30% Questions from Suppliers 1. Has the existing A/B test programme been run in house or by an external supplier? Our A/B test programme started as a fully managed service 18 months ago with an external supplier reporting into an internal Product Manager, but we now utilise both in-house and external partner resource to design and implement tests. 2. Can you provide an indication of budget? We are looking on average for between 30 to 50 hours per month test implementation and support. 3. Does remote location mean UK only or can it be any international locations like India? We would only consider partners in the same time zone. 4. What analytics packages are currently being used? Onsite analytics services we use include:- Google Analytics 360- Decibel Insight (but under review)- Optimizely 5. What are the conversion points for success to be measured against? As we are a non-commercial site this can vary on a test by test basis.Typically we review success based on completion rate (did they complete a tools workflow) or interaction rate (did they interact with certain specific page elements on an article or other webpage). 6. Are KPIs set at a page-by-page level? If so what is the MAS sites core objective? Our corporate KPIs can be seen in chapter 6 of the attached MAS business plan.. KPIs 3.3, 4.1 and 4.3 are most relevant to digital https://masassets.blob.core.windows.net/cms/files/000/000/659/original/Money_Advice_Service_2017_2018_Business_Plan_FINAL_PUBLIC_EDITION.PDFWe don't have KPIs on a page by page level but we do continually monitor completion rates and engagement on tools and articles so we can identify opportunities to improve them to support the overall corporate KPIs. 7. Once tests are successful what is the production process hereafter? We either continue iterating variations within Optimizely or we raise it into the backlog for coded implementation onto the website with the dev team. 8. Are there any website personalisation campaigns currently running or in scope currently? Or is this brief specifically for MVT testing solely? Currently it is MVT testing but we are looking into Optimizely X for personalisation. 9. Do MAS carry out any research beyond site analytics and on-site satisfaction surveys? Yes- Whatusersdo- Lab testing- Guerilla testing- Focus groups- Connexity Hitwise- We also have our own in-house Insight and Evaluation team - they are not part of the digital function but often commission bespoke research 10. The brief talks about capacity issues from a production and technical competence perspective, so how do they see this breaking down between strategic planning, project management, test implementation and analysis? This could vary on a test by test basis20% strategic planning15% project management40% test implementation25% analysis 11. Can MAS provide any volumetrics for the current and projected service? The response to this question is based on the assumption that volumetrics refers to the number of hours of support required as opposed to anything to do with website usage/analytics.We'd be looking at an average 55 hours support per month across test implementation, support and consultancy to match the current arrangement.