Recently in The New York Times, an article (http://tiny.cc/ZC2eI) cited a new report that raises questions about the integrity of the data included in the Major Crimes Report for New York City during the 1990s. The report raises questions about whether, and how, pressure to lower the Crime Rate lead to periodic manipulation of the data by some NYC Police precinct captains.
This is not the first check on the integrity of the data. An earlier report by New York University professor Dennis C. Smith contained this assessment: “We conclude, as did the state comptroller five years ago, that the city and department officials, and the public can be reasonably assured that the NYPD data are accurate, complete and reliable.”
So there are differences in findings and opinions about whether the NYC crime data was manipulated. However, there is little or no debate about whether the experience of crime in NYC has changed significantly for the better since the implementation of COMSTAT by Bill Bratton back in the 1990s – crime in New York City has plunged, and NYC has become one of the safest large cities in the nation. Similar efforts have been successfully implemented in law enforcement agencies across the country with similar results: using performance information can make communities much safer.
Data Integrity is Worth Fighting For
The stories from NYC point out the importance of data integrity. Data integrity goes to the quality of decisions made on the basis of that data – and to the integrity of the government organization reporting it.
I remember when in 2007 Toby Futrell, former City Manager for Austin, TX, won praise when she fired the Director of the Austin Convention Center for falsifying customer satisfaction data. (You can see newspaper coverage of that by clicking here.)
When Metro Nashville Davidson County, TN, started Managing for Results in 2002, then Finance Director David Manning insisted that the Internal Auditor certify the data before a performance measure could be included in their performance-based budget. And from the beginning, the Maricopa County, AZ, Internal Audit shop checked the integrity of performance information, and their efforts led to their receiving an Award for Excellence in Government Finance from the Government Finance Officers Association (GFOA).
Solutions to Better Ensure Data Integrity
If you’re collecting, analyzing, managing, and reporting performance data, the integrity of your performance data matters. There are several steps you and your organization can take to help ensure the performance data you receive and use is accurate and consistent:
** Set yourself up for success by clearly and consistently documenting the definitions of your performance measures and the people who are responsible for collecting, reviewing and reporting the data. Don’t depend on someone to remember how to do it – get it documented, so that anyone can step in and replicate the measurement, if necessary. Best practice performance information management systems like Weidner’s MFR Live software may support this documentation and role definition.
** Emphasize with all staff that service delivery decisions and performance reports are only as good as the data supporting them. Ensure they understand that this is not an exercise in submitting numbers by a deadline, but is how your organization does business.
** Use Internal Auditors or other resources to periodically check the data. Whether a periodic sampling review or a more rigorous assessment, a systematic approach to data quality will yield significant benefits.
** Publicly report on data integrity alongside performance reports. In Metro Nashville, the level of performance data certification was closely watched by department leaders.
No comments:
Post a Comment