Once the data has been collected at the medical center and entered into the ACS NSQIP central server, the next step of the process begins.

Validation, IRR Audits & Formatting

QCMetrix has developed a number of automated programs to monitor the quality of incoming preoperative, intraoperative and postoperative surgical data. These programs check first to see if the data is complete and that all the required variables have been captured and are in the appropriate format. The second test verifies that all the data meet the ACS NSQIP’s inclusion criteria. A third program looks for cases to exclude based on characteristics that fall outside the accepted protocols. Cases that fail any of the validation tests are sent back to the participating hospital for correction.

QCMetrix also performs Inter-Rater Reliability (IRR) audits to test for consistent case handling across the surgical clinical nurse reviewers. Cases are pulled on a random basis and examined using a methodology developed at the University of Colorado Health Outcomes (COHO) Program.

Validated data is then scrubbed (e.g., duplicates and extraneous characters are removed, field entries are set in consistent formats, etc.) and prepared for import into sophisticated SAS statistical analysis applications at the COHO.

O/E Ratios

The team at the COHO, led by Dr. William Henderson, analyzes all the incoming data through a number of statistical models. On an annual basis (and increasing in frequency), each participating hospital receives a report showing its O/E ratios in relationship to those of the other institutions in the system and providing comparative workload and patient risk data.

An O/E ratio is a mathematical construct accurately showing the risk-adjusted outcome for a specific site. The numerator, ‘O’, represents the total number of observed events, postoperative occurrences (deaths or complications), and the denominator, ‘E’, represents the number of events that are expected on the basis of the compendium of the preoperative risk factors prevalent in that population. An O/E ratio significantly less than 1 means that the site is performing better than expected, while a ratio significantly greater than 1 indicates an excess of adverse outcomes.

Interpretation & Feedback

Once a site receives its annual O/E ratios, they normally need help interpreting the data and planning a course of action. QCMetrix has created a highly effective consulting practice to provide further insight and analysis into the COHO reports and help the medical center pinpoint specific areas for improvement.