UKAS response over ISO 15189 member concerns
We have been in contact with a couple of laboratory managers who have perceived an inequity in UKAS inspection standards. They have noted that their labs have been visited by different assessors who do not agree with the previous assessment and have, on occasion, applied findings to systems that were put in place to address previous findings. There is also the perception that the same assessor may apply different findings across labs where the issue is believed to be identical. These perceptions are obviously unhelpful for our members, who value the role of accreditation in providing assurance to patients, commissioners and stakeholders - not least, because of the time and costs in responding to assessor's findings.
In order to address this, the IBMS raised its concerns and asked for some clarity on UKAS's approach to ensuring consistency of assessment in different lab settings and between assessors reviewing the same laboratory. The response we received was as follows:
Ensuring UKAS Assessment Consistency
Following completion of the transition to ISO 15189:2012 from CPA Standards, it is a good time for UKAS to reflect on the project as a whole, as well as the feedback received from laboratories.
UKAS regularly receives queries on how consistency of assessors is assured. To meet the requirements of a competent Accreditation Body, UKAS must change the technical assessors on an assessment team on a regular basis. This introduction of a new assessor, and inevitably a different style to a previous assessor can give the perception of inconsistency. However, the consistency between two assessors shouldn’t be confused with uniformity and one assessor will ask a different set of questions to another. A difference in styles between assessors can be beneficial to UKAS from an objectivity perspective but also for the laboratory too. It would be expected though, that different assessors would reach the same conclusions with regards to conformity with requirements given the same set of facts. If this isn’t the case, UKAS would recommend that you refer the issue to your Assessment Manager at the earliest opportunity.
When findings/non-conformities are raised, the assessment team will look at how effective the “fix” is at the next assessment because how laboratories monitor the effectiveness of any improvement actions is a very important part of any assessment. Often the “fix” initially proposed needs to be further refined or changed in order to fully meet the laboratory’s needs – in which case this will be assessed in context of the justification behind the amended action. In all cases it would be expected that the laboratory is monitoring the effectiveness of the actions and their implementation to avoid the need for further findings to be raised.
Across UKAS, ensuring assessor and assessment consistency is an issue that is key to delivering a good service to customers. To manage this, all assessors and Assessment Managers are monitored on an annual basis and there is a full evaluation of an assessor’s performance every three years. This takes into account feedback from laboratories and Assessment Managers, the reports written, findings raised, turnaround times and CPD.
It follows that UKAS relies on the feedback from laboratories in order that there can be focussed, ongoing training and development of assessors. Whilst UKAS monitors assessors annually and evaluates every three years, UKAS would encourage laboratory staff to provide feedback to their Assessment Manager regarding any concerns they may have regarding assessor performance and consistency. In doing so, issues can be captured and followed up quickly. This can be escalated through our six weekly team meetings where assessment performance is discussed and can inform the agenda for our annual, discipline-specific assessor workshops.
Assessment consistency is a very important issue for UKAS and to continually improve and resolve issues, UKAS relies on the feedback from our customers. This can be provided directly to your Assessment Manager or, if you have any problems contacting your Assessment Manager, directly to ben.courtney@ukas.com.