From: DHSCAssessment date: 25/01/2024Reassessment date: 02/05/2024Stage: Alpha reassessmentResult: Met at reassessmentService provider: UKHSA
Service descriptionThe National Poisons Information Service (NPIS) is a 24/7 phoneline that healthcare practitioners can call for bespoke advice on suspected poisoning.
The Product Data Centre (PDC) is one of an ecosystem of tools and services which support the Specialists in Poison Information (SPIs) who answer the phoneline. The PDC holds information on products and their ingredients, supplied by industry.
The focus of this discovery and alpha is the database of over 300,000 household and industrial products and their corresponding safety data sheets which are used by SPIs to identify a product and its ingredients during calls and to support the creation of healthcare guidance.
Service usersService users are Specialists in Poison Information (SPIs) who input data, create guidance and take calls.
Industry professionals who either wish to, or are mandated to, (in Northern Ireland) supply product ingredient information.
Report contents Understand users and their needs Solve a whole problem for users Provide a joined-up experience across all channels Make the service simple to use Make sure everyone can use the service Have a multidisciplinary team Use agile ways of working Iterate and improve frequently Create a secure service which protects users’ privacy Define what success looks like and publish performance data Choose the right tools and technology Make new source code open Use and contribute to open standards, common components and patterns Operate a reliable service 1. Understand users and their needs DecisionThe service met point 1 of the Standard.
What the team has done wellThe panel was impressed that:
the team has worked to understand who their users are using a mix of appropriate methods to explore user needs. They have a good knowledge of the needs of Specialists in Poison Information (SPIs) in particular the team has created user archetypes and refined these during alpha the team has logged, in detail, user needs for each type of user, including needs for users with dyslexia the team worked closely together in User Research (UR) sessions and design sessions and UR feedback was incorporated into design work What the team needs to exploreBefore their next assessment, the team needs to:
fill gaps in knowledge around industry users who don’t currently engage with the service, users with English as a second language and users of all types with access needs keep refining and iterating the user archetypes throughout beta. It was surprising that only a single archetype for Industry users was found and this should be explored further with a broad range of industry users include in the log where evidence for each user need can be found and ensure that needs for users with other access needs are also logged. Given the shift in personnel for this project, it may be valuable to include some of the assumptions that were not fully validated during alpha and the team include some optional aspects based on this. They should ensure that anything that was not fully validated during alpha is explored further in beta 2. Solve a whole problem for users DecisionThe service met point 2 of the Standard.
What the team has done wellThe panel was impressed that:
the team has focused on how to ensure the PDC meets its operational demands as the volume of work increases ,by up 500 submissions per month from 2019/20, and how to maximise the use of SPIs time on non-administrative tasks the team has undertaken a discovery and alpha and researched, tested and iterated a workable prototype within 4 months the team has prototyped a digital solution which much improves the efficiency of the current information submission, processing and retrieval process the team has streamlined the user journey for SPIs: from having to spend 30% of their time manually transferring data from email to receiving it directly into a secure database the team has improved the user experience: for example, by making data easier and faster to find and date for SPIs the team has removed the risk of file conversion errors and time wasted troubleshooting the team has supported the industry user journey by making submissions traceable What the team needs to exploreBefore their next assessment, the team needs to:
consider whether to solve a whole problem for users, the scope of the project should now be widened. The problem statement given in discovery is: "the PDC does not enable the NPIS to work efficiently or effectively meet the needs of the service’s users”. While this alpha addresses the issue of inefficient data submission and the quick identification of data, there are other potential needs of the users of this service that remain unexplored. For example, how the accuracy of the information supplied is verified, or what happens when a product search returns no information and how that gap is then closed 3. Provide a joined-up experience across all channels DecisionThe service did not meet point 3 of the Standard.
The panel was not able to assess point 3 properly for the following reasons:
little evidence of service design thinking to attempt to capture and understand how this part of the service interacts with the wider ecosystem of products and tools that are accessed by SPIs and doctors to deliver the overall service key user journey for information providers begins with marketing campaigns to make them aware of the change in the service from emailing information to account creation and upload. No content for the campaigns was shown and the panel couldn’t see the start of the journey there was mention of an assisted digital path which involved posting CD Roms or memory sticks. The panel needs to understand better how this use of offline channels is communicated to users and the risks involved.Reassessment Decision The service did meet point 3 of the Standard.What the team has done well The panel was impressed that: the team has utilised service design thinking and techniques to investigate and test assumptions around the interoperability of the service within the existing ecosystem, and have iterated the MVP to reflect their findings the team has engaged with wider comms teams to develop onboarding comms, and have been testing and iterating the content the team has identified an assisted digital route for the service
4. Make the service simple to use DecisionThe service met point 4 of the Standard.
What the team has done wellThe panel was impressed that:
the team has a dedicated content designer on the service team the team has worked hard to collaborate with policy teams and wider subject matter experts (SMEs) to attempt to reduce content and jargon in the service while still meeting legal needs the team has been creative in their thinking and responded to research insights, including images of the products to help users try to understand the context of what they are discussing is a nice design idea and is just one example of this creativity the team has iterated designs in a short space of time and were willing to test design hypotheses. For example the ideation around chatbots which were then not included in the final designs What the team needs to exploreBefore their next assessment, the team needs to:
justify that deviations from gov.uk design patterns are necessary explore what quality assurance can be built into the service and interaction design to help ensure safety and scalability of the system, given the nature of the subject matter and the potential health consequences. This may require input from other teams and departments and iterations should be backed by user research to validate design ideation do more research to understand why FAQs are needed and look to address these challenges through broader service, content and interaction design. If evidence emerges that there is a specific need for FAQs this should be documented and thorough in its reasoning 5. Make sure everyone can use the service DecisionThe service did not meet point 5 of the Standard.
What the team has done wellThe panel was impressed that:
the team broadly used known components and patterns while also adhering to a principle of writing in plain English the team has tried to keep information capture to a minimum viable amount to help all users engage with the service. A good example of this is the qualitative work done in balancing industry professionals needs with SPIs needs to reduce effort and cognitive load What the team needs to exploreBefore moving into private beta the team needs to:
produce a plan to test the service on a far wider set of users. Numbers of usability testing with the prototype were generally low. So far, there has not been much exploration of users with different needs, to help validate design decisions to date validate the assumption that all users of the service will have high digital literacy. While efforts were made to design the service in an accessible way by following good design practice this is an area which requires further exploration plan more robust testing of the content, specifically language levels. The team acknowledged that research indicated a significant portion of users of this service may not speak English or have English as a second language. Usability testing was carried out with a single native German speaker, this does not feel comprehensive enough given context of the users and the nature of the subject matter to validate the robustness of the end-to-end service experience for the volume of non-native speakers likely to be supplying the informationReassessment Decision The service met point 5 of the Standard. What the team has done well The panel was impressed that: the team has developed an ambitious plan for testing with a wide range of users prior to private beta the team has undertaken some user research with individuals with English as a second language (ESL), albeit proxy users What the team needs to explore Before their next assessment, the team needs to: conduct significantly more user research with users with low digital literacy, ESL, accessibility needs. Having the plan is great but you must stick to it to ensure the service is meeting user needs and to support team success at beta assessment strive to include individuals with accessibility needs, ESL and low digital literacy in every round of user research where possible. Your proposed plan appears to give a relatively short period of time to focusing on these individuals, they should feature throughout all your research and be a constant consideration in recruitment amend your research plan considering pre-election period inform the DHSC digital assurance team to help with designing a more realistic plan if you feel that you cannot deliver against your user research plan in the timeframe stated ensure that you are aware of, and working to confirm or disprove, your riskiest assumptions about your user group and the service you are creating ensure that the service you are creating caters to the needs of a broad range of users. It may appear that the user group is highly digitally literate at present, but this is not a certainty, nor will it necessarily be the case for all future users. The service should be designed for those with the most complex needs with regards to literacy, ESL, accessibility. If the service meets the needs of these users, it will meet the needs of the average user keep in mind that an individual's self-assessment of their digital literacy is not an accurate measure of their actual ability make use of recruitment approaches beyond agencies, particularly if this route is delaying your research. There was a mention of using charities to recruit individuals with accessibility needs, this can often be a very fruitful approach and would be worth exploring. Ensure that you can show the panel at the beta assessment that you are doing all you can to recruit appropriate participants
6. Have a multidisciplinary team DecisionThe service met point 6 of the Standard.
What the team has done wellThe panel was impressed that:
the team has had a full multidisciplinary service team in place throughout the Discovery and Alpha What the team needs to exploreBefore their next assessment, the team needs to:
look to reduce the reliance on external contractors put more robust knowledge transfer protocols in place if the team is likely to remain external for some time 7. Use agile ways of working DecisionThe service met point 7 of the Standard.
What the team has done wellThe panel was impressed that:
the team has clearly planned and executed the Discovery and Alpha using agile ways of working the team has a strong ‘test, learn and iterate’ culture the team works collaboratively and in the open the team used strict scoping and prioritisation to achieve an impressive pace on this discovery and alpha What the team needs to exploreBefore their next assessment, the team needs to:
consider the logic behind the agile delivery phases and consider whether the conclusion of this Alpha presents an opportunity to rescope before entering the beta phase. For example, is there an opportunity to do a further Discovery into how this service interfaces with the wider service ecosystem, such as Toxbase, and whether multiple separate services-within-a service are required? Or even to raise issues around the policy behind this service 8. Iterate and improve frequently DecisionThe service met point 8 of the Standard.
What the team has done wellThe panel was impressed that:
the team tested and iterated designs multiple times at pace. Speed of progress was impressive design iterations were clearly informed by the research and testing done, many examples of which are given in this report What the team needs to exploreBefore their next assessment, the team needs to:
ensure that information informing design iterations and technical decisions has been captured consistently, fully documented and in a format which meets public sector standards, for example much information is currently held on externally administered Miro boards. Team might explore using hypothesis logs or confluence or word documents to ensure the rationale for decisions made while prototyping are clear and accessible 9. Create a secure service which protects users’ privacy DecisionThe service met point 9 of the Standard.
What the team has done wellThe panel was impressed that:
the team presented a high-level technical design and have thought about the platform security the team has made sure the service does not store any PII or personal sensitive data that may be used for reporting purpose the team has ensured industry users are required to register before they are allowed to submit content the team is using Azure DNS and Azure AD B2C to manage industry user authentication and access the team is using and improving existing backend system that SPIs use to advice HCPs the team will perform penetration testing the cloud solution for web portal will detect and manage any potential BOT/Hack/DDOS attacks What the team needs to exploreBefore their next assessment, the team needs to:
explore how they might authenticate an industry user before granting access to the web portal mitigate against potential BOT attack on email channel define requirements for cadence or level of change to trigger penetration testing 10. Define what success looks like and publish performance data DecisionThe service met point 10 of the Standard.
What the team has done wellThe panel was impressed that:
the team has a comprehensive set of KPIs for the service which include both qualitative and quantitative measures the team has agreed KPIs for both key user groups, SPIs and Industry professionals, as well as PDC management the team’s PDC management KPIs will allow the business to assess value for money the team has made it so KPIs will be measured via user research in many cases, ensuring ongoing user feedback will be conducted and surfaced to management and stakeholders What the team needs to exploreBefore their next assessment, the team needs to:
consider how to solve the problem of being unable to measure the accuracy of information given or its completeness as this is fundamental to the value of the database use the performance framework and their own leading indicators or metrics in private beta to measure some key design or product decisions against hypotheses and then inform future iterations 11. Choose the right tools and technology DecisionThe service met point 11 of the Standard.
What the team has done wellThe panel was impressed that:
the team has explored various tools and technologies for both the web portal and back end supporting SPIs the team has made use of Azure stack to support both the web portal the back end underpinning the content the team has made use of Azure AI language to support search engine optimisation the team’s web portal design adheres to Gov.UK Accessibility standards What the team needs to exploreBefore their next assessment, the team needs to:
test the use of Azure AI Language to ensure outputs are consistently useable by SPIs provide a roadmap to retire the use of the legacy email channel for receiving and processing information received from industry ensure any technical design precludes use of CD ROM, DVD, USB etc to receive and process information from suppliers and manufacturers, which was mentioned as a possible assisted digital route 12. Make new source code open DecisionThe service met point 12 of the Standard.
What the team has done wellThe panel was impressed that:
the team will make all source code open What the team needs to exploreBefore their next assessment, the team needs to:
publish documentation with examples to ensure third parties can use the available code publish contact details for third parties to raise queries and/or potential code issues they discover 13. Use and contribute to open standards, common components and patterns DecisionThe service met point 13 of the Standard.
What the team has done wellThe panel was impressed that:
the team is processing data received in i6Z standard format What the team needs to exploreBefore their next assessment, the team needs to:
explore adoption of i6Z format in PDF and develop a business case for the change. The team are bound by regulation and change to data received in PDF file format, may require law change. The team should present the business case for any potential regulation changes required 14. Operate a reliable service DecisionThe service met point 14 of the Standard.
What the team has done wellThe panel was impressed with:
the resilience of the proposed Azure cloud-based design the teams use of an email channel alongside new web portal, which supports service reliability the real time feedback loop to manage potential errors when data is submitted via web portal What the team needs to exploreBefore their next assessment, the team needs to:
define SLAs for expected response time for SPIs users design an audit and alerting process to detect and prevent internal and external threats explore how they can manage potentially duplicate content coming though the same, or multiple, channels ensure the web portal design adheres to Gov.UK Building a resilient frontend using progressive enhancement. seen at 16:39, 19 September in Digital health and social care.