Stotles logo
Closed

Consultancy for Azure Data Platform

Published

Description

Summary of the work Seeking proposals from interested, qualified and experienced professional consultancies who are knowledgeable in all aspects of Azure Data Engineering, Azure Infrastructure and with experience of technical authoring, to assist our in house data management team in delivering an enhanced Secure Data Environment (SDE). Expected Contract Length 3 Months Latest start date Monday 6 February 2023 Budget Range £270,000 -£300,000 (excluding VAT) for 3 month delivery of services Why the Work is Being Done NECS works across the UK and beyond to help health and social care customers meet their strategic and operational challenges, to improve outcomes and increase efficiency. NECS’ Business Information Services Directorate (BIS) covers several services including ICT, Data Management, and Business Intelligence and Analytics. We deliver a wide range of services to a wide range of customers across the health, care, voluntary and university sectors including developing, designing, maintaining and supporting a range of web-based applications, sites and cloud-based data access platforms. NECS has developed a Secure Data environment (SDE) to support our customers in accessing and analysing health and care data. We need to accelerate this work to make a number of technical enhancements to our SDE by the end of March 23 to align with recent national guidance. Problem to Be Solved NECS is seeking a qualified and experienced professional consultancy partner who are knowledgeable in all aspects of Azure Data Engineering, Azure Infrastructure and with experience of technical authoring, to assist us in delivering an enhanced SDE. NECS requires a partner to provide technical expertise for the enhancement of a Secure Data Environment including : 1. Data Modelling, to support standardisation. 2. Enhancement of a data catalogue user interface. 3. Development of an airlock 4. Remodeling the existing data architecture. 5. Development of enhanced logging and monitoring We wish to accelerate the development of our SDE. The platform is made up of a number of Azure based virtual desktops including data hosted within SQL databases. Tooling to support analytics such as R and python are also included within each workspace with stringent role based access and firewalls in place to maintain the security of the environment. We are looking for a combination of cloud architects, data engineers, desktop engineers and software developers to support us. Who Are the Users Analytical and Data Science professionals within Health and Care, University Researchers need to query a project specific standardised dataset within an SDE and be able to interrogate a meta data catalogue. SDE Data Engineer - needs to be able to access detailed logging and monitoring to understand and user activity. Existing Team We have a team of 50+ data engineers and application development roles. This current team includes up to 10 individuals with a skill mix such as product managers, delivery managers, scrum masters, business analysts, data engineers, and cloud specialists. Current Phase Beta Skills & Experience • Data Catalogue: Front-end software development (react) - Provide a standardised self-service platform for organisations to browse datasets and associated metadata - 5 Points • Cloud infra-architect - Ensure separation between the code development area of the SDE and the data analysis area within the workspace. - 5 Points • Software development (.net + Python) – Ensure all code brought into the environment passes through an airlock. - 5 Points • Cloud data architect- Transition existing architecture to make use of enhanced technologies. - 5 Points • Cloud data architect- Transition existing architecture to make use of enhanced technologies. - 5 Points • Microsoft Gold Partner - 5 Points • On shore resource only - 5 Points • Able to mobilise resource immediately upon award - 5 Points • Ability to provide a standardised self-service platform for organisations to browse datasets and associated metadata - 5 Points • Ability to align data in the platform to common standards to allow for interoperability with other SDEs and for code to be shared between analysts. - 5 Points • Abilit to ensure all code brought into the environment passes through an airlock. - 5 Points • Ability to ensure separation between the code development area of the SDE and the  data analysis area within the workspace. - 5 Points • Ability to transition existing architecture to make use of enhanced technologies. - 5 Points Nice to Haves Previous health and care experience - 3 Points Work Location The work is expected to be performed remotely from any location but will be co-ordinated by staff in Durham and Sheffield. Working Arrangments Working patterns will generally be Monday to Friday between 08:30 – 17:30. We will work with the partner flexibly through a range of video calls, emails and face to face meetings as required. Day to day delivery will be governed by agile delivery techniques. These will be via the scrum approach. Peer reviews by technical leads will also occur regularly to ensure fit to best practice and engineering standards. No. of Suppliers to Evaluate 5 Proposal Criteria • Technical Solution - 24 Points • Approach and methodology and how you would work with the CSU - 24 Points • How the approach or solution meets user needs - 24 Points • How the approach or solution meets our goal - 24 Points • Estimated timeframes for the work - 24 Points • How you’ve identified risks and dependencies and offered approaches to manage them - 24 Points • Team structure and completeness of expertise (including CVs) - 24 Points • Any expectations of the CSU to support delivery - 24 Points • Value for money - 24 Points Cultural Fit Criteria • Demonstrate ability to work as a team and collaborate effectively with our organisation and other suppliers. • Provide details on your approach, principles and/or practices in how you share knowledge and experience • Demonstrate how you will work as a team with our organisation/team and other suppliers where appropriate • Please provide case studies demonstrating your experience in supporting organisations to develop bespoke work packages • Provide information on your social value as an organisation with emphasis on three key areas 1. Economic 2. Social 3. Social Environmental Payment Approach Fixed price Assessment Method Presentation Evaluation Weighting Technical competence 45% Cultural fit 10% Price 45% Questions from Suppliers 1. The Essential requirement: Cloud data architect- Transition existing architecture to make use of enhanced technologies. – 5 PointsThis is included twice. Can you please confirm that you are expecting two separate examples? This is a duplication of essential requirement. This should have read the below requirement which is an essential requirement.Cloud infra-architect - Ensure separation between the code development area of the SDE and the data analysis area within the workspace. - 5 Points 2. Is being a Microsoft Gold Partner an essential prerequisite? This prerequisite is now a 'nice to have requirement and not an 'essential' requirement. This will now have a score of 3 points. 3. Can you provide more information on the existing data architecture and infrastructure currently in place for the SDE? The SDE is built upon Microsoft Azure. Further details would be provided upon award but the design is similar to those described by https://www.turing.ac.uk/research/research-projects/data-safe-havens-cloud or https://microsoft.github.io/AzureTRE/ 4. Can you provide more information on the data sets and associated metadata that will be included in the SDE, and how they will be standardized? The data catalogue will contain datasets covering existing health and care datasets. Metadata from other catalogues or public sources i.e gov.uk can be ingested via an API. 5. Microsoft changed their accreditation programme to a new designation model. We would have passed Gold last year, but the programme closed to new applicants. Will you consider accepting submissions from orgs that have the capability of a gold partner, but not the ‘badge’? Yes, submissions from organisations that have the "capabilities" of a gold partner would be welcome 6. What are the specific technical enhancements that need to be made to the SDE, and how do they align with recent national guidance? Data Modelling – Work is required to support standardisation and remodelling of data to an OMOP standard to allow interoperability. Data catalogue - An MVP meta data catalogue exists developed using open source software. We want to develop the user interface making it easier to navigate for users.Restricting data access – We need to develop the existing airlock to provide additional controls for the ingress and egress of data and code. Remodelling existing architecture – We are looking for support to transition from existing architecture to make use of enhanced technologies. 7. Can you provide more details on the role-based access and firewalls currently in place for the SDE, and how they will be integrated with the proposed enhancements? Please see response to question 2. Roles and rules can be modified to incorporate those enhancements, but they must continue to align with the existing SDE principles. 8. "DOS Stage 1 closes 1st Feb, then NECS will evaluate before going to Stage 2 Shortlist of 5. How is the latest start date of Monday 6 February 2023 possible? Similarly, completion is put as end Mar for what is budgeted as 3 months of work. Can you please explain how these deadlines are achievable OR if there is flexibility within these?" evaluations stage 1 submissions on 2 February, successful suppliers invited to stage 2 submission on the 2nd February. Timeline below.We anticipate mobilisation commence on 13 February or as quickly thereafter. Willing to work with the successful supplier on delivery timescales and feasibility including incremental achievement of milestonesTimeline:Procurement Live - 18 JanuaryStage 1 submission - 1 FebruaryReview Stage 1 - 2 FebruaryInvite to Stage 2 - 2 FebruaryStage 2 proposal submission - 8 FebruaryReview Stage 2 - 9 FebruaryPresentation - 10 FebruaryContract Award - 13 FebruaryGo Live - 13 February 9. Per the DOS buyers’ guide https://www.gov.uk/guidance/how-to-shortlist-digital-outcomes-and-specialists-suppliers#proof-of-skills-and-experience-1, suppliers are required to provide examples to show they have the experience and skills you are looking for. Some of the criteria are written as requirements for the supplier to deliver on your project, e.g. criterion 2. Could you please clarify that the supplier should provide an example of a similar deliverable in another organisation? Please can you re-word this question as it is not fully understood. Thanks 10. Could you clarify the timetable for the procurement? The opportunity closes on 1 February, after which shortlisting will take place. Then shortlisted suppliers will need to complete an RFP and deliver a presentation, then be evaluated on the second stage, plus contract award activities. The start date given is 6 February, is this correct? The opportunity closes on the 1st. The presentation will be focused on the technical solution and proposal that forms part of your submission. This is also an opportunity for us to ask any questions on your proposal that we may have. TimelineProcurement Live - 18 JanuaryStage 1 submission - 1 FebruaryReview Stage 1 - 2 FebruaryInvite to Stage 2 - 2 FebruaryStage 2 proposal submission - 8 FebruaryReview Stage 2 - 9 FebruaryPresentation - 10 FebruaryContract Award - 13 FebruaryGo Live - 13 February 11. Will you be providing a more detailed description of each of the required roles, and their scope and responsibilities in the next phase of this procurement? We will review initial responses and undertake a further discussion with supplier as part of their presentation. 12. Could you provide some clarity around the stated timelines for this procurement please? Closing date: 1st Latest start date: 6th February Would an ITT be issued to those suppliers successful at stage 1 of the DOS submission and can this process (inc potential presentations)be completed to meet the proposed latest start date of 6th February? The opportunity close on 1st February. NECS will evaluate shortlisted suppliers on 2nd February and invite those successful to stage 2 submission on 2nd February. TimelineProcurement Live - 18 JanuaryStage 1 submission - 1 FebruaryReview Stage 1 - 2 FebruaryInvite to Stage 2 - 2 FebruaryStage 2 proposal submission - 8 FebruaryReview Stage 2 - 9 FebruaryPresentation - 10 FebruaryContract Award - 13 FebruaryGo Live - 13 February 13. The opportunity notice states a 3 month contract length however technical enhancements to the SDE are to be delivered by the end of March 23. Is the contract and work expected to continue beyond the end of March? We are willing to work with the successful supplier on delivery timescales and feasibility including incremental achievement of milestones or deliverables. 14. Is NECS seeking a single supplier to fulfil this consultancy contract? NECS wishes to contract with one supplier to support this work 15. In the essential skills it gives 5 points for onshore resources. Would the Trust allow resources that are allocated outside of the UK, specifically Poland, to work on this project remotely? No, NECS requires resource to be located within the UK. 16. Please can you confirm that your preferred start date is definitely the 6th February given the submission deadline at Stage 1 is the 1st February and we expect there will be second stage to this procurement? The opportunity closes on 1st February. NECS will evaluate shortlisted suppliers on 2nd February and invite those successful to a stage 2 submission on the 2 February. TimelineProcurement Live - 18 JanuaryStage 1 submission - 1 FebruaryReview Stage 1 - 2 FebruaryInvite to Stage 2 - 2 FebruaryStage 2 proposal submission - 8 FebruaryReview Stage 2 - 9 FebruaryPresentation - 10 FebruaryContract Award - 13 FebruaryGo Live - 13 February 17. Please can you confirm which roles you required to be mobilised by the 6th February/your preferred start date? We would work with the successful supplier to agree the priorities for mobilisation but expect the required skillsets to be made available from the outset. 18. In DOS framework, there are stage 1 (100 words answers) and stage 2 ( detail answers by further shortlisted suppliers only). considering timeline, do all suppliers need to submit stage 1 and stage 2 together ? No, suppliers will submit a stage 1 document for review and then be invited to submit a stage 2 application following review 19. Please can you provide a timeline indicating the dates on which you intend to notify suppliers they have been shortlisted, the submission deadline for Stage 2, and the date of award etc.? The opportunity closes on 1st February. NECS will evaluate shortlisted suppliers on 2nd February and invite those successful to a stage 2 submission on the 2 February. Timeline:Procurement Live - 18 JanuaryStage 1 submission - 1 FebruaryReview Stage 1 - 2 FebruaryInvite to Stage 2 - 2 FebruaryStage 2 proposal submission - 8 FebruaryReview Stage 2 - 9 FebruaryPresentation - 10 FebruaryContract Award - 13 FebruaryGo Live - 13 February 20. Please can you confirm whether Stage 1 scores will be carried over and used as part of the evaluation at Stage 2? The first stage is for shortlisting based on essential requirements, the scores will be carried over for the second stage for the essential criteria. 21. Regarding the response criteria at Stage 2, please can you confirm that the Presentation and each of the 9 points listed in the Proposal Criteria section will be equally weighted within the 45% allocated to Technical Competence within the evaluation? Yes the technical competence section will be waited equally between the criteria. 22. Please could you confirm what scoring methodologies will be applied at Stage 1 and Stage 2? The first stage is for shortlisting based on essential requirements, only suppliers with all essential requirements will be put through to the second stage. A written proposal will then be required which will evaluated using the criteria stated with weighting as stated. 23. Could you please clarify what is meant by an ‘airlock’? The airlock places technical and process controls over the data and code that can enter and leave the environment, allowing opportunities for human and/or automated review. Eg this can be seen in https://re-docs.genomicsengland.co.uk/airlock/ and https://microsoft.github.io/AzureTRE/azure-tre-overview/airlock/ 24. Please can you clarify who carried out the Discovery and Alpha? Inital phases of development were carried out by the NECS team 25. Is it possible to see the outputs of the discovery and alpha phases please? Details of the current phases will be shared with the successful supplier 26. Can you please define the term ‘airlock’. Please see response to question 25. The airlock places technical and process controls over the data and code that can enter and leave the environment, allowing opportunities for human and/or automated review 27. Would an off-shore team managed by an on-shore lead suffice? No, NECS requires resource to be located within the UK. 28. The submission date for stage one of the procurement, assuming there is a stage two for the five shortlisted suppliers, is the 1st Feb however you have a start date of the 6th of Feb which does not allow much time for stage two. Is the start date correct? What are your expectations of the successful bidder in terms of timescales? What compelling events if any are there that are driving the timescales? Shortlisted suppliers will be notified on the 2nd February and will be required to submit their proposal by the 8th February. TimelineProcurement Live - 18 JanuaryStage 1 submission - 1 FebruaryReview Stage 1 - 2 FebruaryInvite to Stage 2 - 2 FebruaryStage 2 proposal submission - 8 FebruaryReview Stage 2 - 9 FebruaryPresentation - 10 FebruaryContract Award - 13 FebruaryGo Live - 13 February 29. Please describe the support you have received from suppliers to date in the development of your SDE. For example, who, what, and when? Thank you. The initial phases of development were conducted by the NECS team. 30. Please can you provide some more context behind the development of your SDE in terms of who it is intended for, i.e. is it to be used regionally or as your suggest UK-wide? The SDE needs to be flexible and to be able to support users across a wide geography. 31. Given the examples provided on their current architecture, are there any key paradigm changes that the client are looking for in the new solution? We are not looking to make a key paradigm change but we are looking to work with a partner who would be able to support us or recommend such changes in future. 32. Please provide an understanding of how the current SDE/TRE is deployed in your environment. Is this integrated into DevOps, and /or do you intend to build on the existing environment or replace your current Alpha with the ‘renewed’ version of the SDE? NECS intends to continue developing our existing environment. 33. Please could you provide an understanding of how you intend to manage the project / programme. E.g. Will you provide a SCRUM Lead, Agile DevOps Environment, etc. Are you intending that this be managed through an Agile Method or other means? NECS works with agile methodology. We would expect the supplier to manage the work packages agreed. NECS will support the supplier with any blockers. 34. To what extent has the existing environment been tested, and do you have a structured approach to testing? Would you be able to briefly summarise this? Will this approach to testing be carried forward to the advanced builds as described in the tender? We would expect the new developments to be modular in nature and for the supplier to have responsibility for initial testing. 35. To what extent have you developed any current requirements details for the advanced builds / differential builds to the Microsoft Azure SDE/TRE environment, or are you anticipating these will be designed alongside yourselves prior to build / deployment? We have high level requirements and would want to work with the supplier to enhance these. 36. 1)Which other suppliers have provided services related to this data platform in the last 12 months? 2)Are there any incumbent supplier provided related services currently? 3)Could you please explain how the budget has been calculated and is there any supporting documentation to help us understand your budget scoping? 4)Would this service fall inside or outside of IR35? 5)Do you have any security vetting or significant onboarding processes that might limit time available to work on the project? "1. and 2. Please see response to question 233. The budget has been set based on available resource for this project4. This service would fall inside IR 355. There are no enhanced security vetting processes that would be required. " 37. Could authority share national guidance as stated to be followed for enhancing the SDE. Policy guidelines were shared https://www.gov.uk/government/publications/secure-data-environment-policy-guidelines/secure-data-environment-for-nhs-health-and-social-care-data-policy-guidelines but more detailed technical specifications will be shared with the successful supplier 38. What is the product/technology used for Data Catalogue ? The data catalogue is based on an open source tool, DataHub. Further details will be provided to the successful supplier. 39. What roles from NECS will be available to this engagement? NECS will work flexibly with the provider to deliver the required outcome. NECS will provide access to our Technical Lead when required. The supplier will be managed by a project manager and a delivery manger on a day to day basis. 40. "As per the CQ, the question E5 is updated as: Cloud infra-architect – Ensure separation between the code development area of the SDE and the data analysis area within the workspaceHowever, this is a duplication of question E2. Request you to confirm if this is correct." ''Cloud infra-architect - Ensure separation between the code development area of the SDE and the data analysis area within the workspace. - 5 Points'' is only one criteria and only has 5 points assigned and is just a duplication. 41. With reference to the below essential criteria “Cloud data architect- Transition existing architecture to make use of enhanced technologies” – could you please expand further on what your specific requirements and the technologies you envisage / are using? Thanks We intend to retain Azure and while we have a high level plan but would want to work with the successful supplier to explore potential options 42. 1.Please confirm if you are expecting these questions to be answered in the standard DOS format – using a single of example for each answer of a similar requirement covering the situation, work carried out and result? Many of these questions do not lend themselves to this answer structure.2.Airlock questions: There are two questions that address “...code brought into the environment passes through an airlock.” – is this deliberate and do these two questions require separate / different answers?3.Data catalogue: Azure provides a standardised platform in Purview. Would NECS consider Purview rather than a a React-based solution? 1. Yes please use the standard DOS format2. These are the same thing so only one answer required.3. NECS has previously evaluated purview and it didn’t meet requirements"

Timeline

Publish date

a year ago

Close date

a year ago

Buyer information

Explore contracts and tenders relating to NHS North of England Commissioning Support Unit

Go to buyer profile
To save this opportunity, sign up to Stotles for free.
Save in app
  • Looking glass on top of a file iconTender tracking

    Access a feed of government opportunities tailored to you, in one view. Receive email alerts and integrate with your CRM to stay up-to-date.

  • ID card iconProactive prospecting

    Get ahead of competitors by reaching out to key decision-makers within buying organisations directly.

  • Open folder icon360° account briefings

    Create in-depth briefings on buyer organisations based on their historical & upcoming procurement activity.

  • Teamwork iconCollaboration tools

    Streamline sales workflows with team collaboration and communication features, and integrate with your favourite sales tools.

Stop chasing tenders, start getting ahead.

Create your free feed

Explore other contracts published by NHS North of England Commissioning Support Unit

Explore more open tenders, recent contract awards and upcoming contract expiries published by NHS North of England Commissioning Support Unit.

Explore more suppliers to NHS North of England Commissioning Support Unit

Sign up