The FBI’s Crime Data: What Happens When States Don’t Fully Report (#GotBitcoin?)
Bureau’s decades-old method to account for missing data could use an upgrade, some experts say. The FBI’s Crime Data: What Happens When States Don’t Fully Report
In any given year, more than 18,000 U.S. police agencies are asked to submit crime data to the FBI. But some don’t provide complete information or, in some cases, any information at all.
When that happens, the Federal Bureau of Investigation uses crude estimates to account for the missing data. Those figures are then used to generate “Crime in the United States,” an annual tally of violent and property crimes that is a quality-of-life measure as well as a gauge of criminal justice policies and spending.
In most instances, the estimates, though rough, don’t stray substantially from the submitted numbers.
In its latest report, the FBI didn’t adjust the counts of violent crime for 11 states at all. But it inflated Indiana’s numbers by 9.9%, West Virginia’s by 13% and Mississippi’s by 68%, raising that state’s count to an estimated 8,526, up from a reported 5,084.
Homicides, generally considered the most reliable of the FBI’s crime statistics, showed similar discrepancies. The numbers for 28 states were not adjusted, but Mississippi’s count, the worst of the bunch, was increased by 56%, to an estimated 245 murders from a reported 157.
“What we need are error bars on this, but we don’t have that,” said Jeff Asher, a crime analyst based in New Orleans who argues the agency should publish margins of error with its numbers.
It’s impossible to know how far off the FBI’s adjustments are, and experts offered mixed opinions about the significance of the potential discrepancies.
Robert Weisberg, co-director of Stanford University’s Criminal Justice Center, and James Alan Fox, a criminologist at Northeastern University, said the numbers are close enough to evaluate trends over multiple years.
“The problem is looking one year to the next,” Dr. Fox said. “Don’t get hung up on the year-to-year changes.”
The FBI’s most recent report showed murders and violent crimes declined less than 1% from 2016 to 2017.
“That’s just noise,” said Richard Berk, a criminologist at the University of Pennsylvania, suggesting the change, with or without estimates, was insignificant.
Other experts believe, at minimum, the FBI should use a more sophisticated system for generating estimates.
Two decades ago, the Bureau of Justice Statistics, a unit of the Justice Department, published a 78-page paper critiquing the FBI’s procedure and recommending ways to improve it.
“So far, nothing has come of it,” said Michael D. Maltz, the criminologist who wrote the paper and is now a researcher at Ohio State University’s Criminal Justice Research Center.
The FBI did not respond to requests for information, but Dr. Maltz, who with a grant from the American Statistical Association has developed an alternative way to estimate the missing data, described the bureau’s procedure:
When a jurisdiction provides data for three or more months, the FBI estimates the total by multiplying the reported number of crimes by 12/N, where N is the number of months for which reports exist.
Using that formula, an agency that reports four months of crimes would be estimated to have 12/4, or three times the number of crimes it reported.
Because crimes are not evenly distributed throughout the year, that method could lead to over- or underestimates, depending on whether the reported data cover months that are historically lower in crime (winter) or those that are historically higher in crime (summer).
When a jurisdiction provides no data, the FBI estimates the number of crimes based on rates for similarly sized areas in the same state that provided all 12 months of data.
Using that measure, if a jurisdiction with a population of 150,000 reports two months or less of crime data while jurisdictions in the state with a similar population report 620.2 crimes per 100,000 people, the agency that didn’t fully report will be estimated to have had 930.3 crimes (620.2 x 150,000 / 100,000).
A better approach, according to Alicia Carriquiry, a statistician at Iowa State University, would take advantage of all of available data, not only across agencies but across time.
The FBI’s less rigorous method was developed in the 1960s out of necessity when a single year of crime data was stored on seven or eight large reels of computer tape.
“Longitudinal imputation of missing data was truly impossible,” Dr. Maltz said. “To fill in the gaps, the FBI used data that was most readily available to them—data on the same tape, either from the agency itself, inflating the available count to a 12-month estimate, or using similar agencies’ data.”
Now, all the data collected since 1960 will fit on a single thumb drive.
And that, he said, is something you can longitudinally count on.
FBI Moves To Fix Critical Flaw In Its Crime Reporting System
In response to an investigation by Newsy, Reveal from the Center for Investigative Reporting and ProPublica, the bureau says it has expedited a process expected to change reporting rules and require police to disclose cases they classify as unfounded.
The FBI will fast-track a fix to address flaws in its uniform crime report and is expected to change reporting rules to encourage more transparency about the outcomes of investigations by local law enforcement agencies, following a yearlong investigation by Newsy, Reveal from the Center For Investigative Reporting and ProPublica.
The investigation uncovered a major flaw in the FBI’s next generation crime reporting system, the National Incident-Based Reporting System (NIBRS). The new system does not track cases police classify as “unfounded,” a category for when police say the victim is lying or the reported crime did not occur.
In our November investigation, we found that the FBI reports zero unfounded cases for thousands of agencies using the new system, although records from those agencies show they classify many cases this way.
For example, the Prince William County Police Department in Virginia showed no unfounded cases in the FBI crime statistics for 2016. However, internal department records show that Prince William County police classified nearly 40 percent of all rape cases as unfounded that year.
“You have found something that needs to be corrected,” said Col. Edwin C. Roessler Jr., chairman of the FBI’s NIBRS transition task force, and chief of police in Fairfax County, Virginia. “This is a crisis, an emergency.”
Roessler said following the news report, he reached out to senior FBI leadership and received a commitment from the bureau last week that the FBI would move swiftly and bypass its typically lengthy requirements for advance notice to consider major policy changes. It’s unclear how exactly the system will change to reflect rape cases classified as unfounded. Roessler said one possibility being discussed is to require police agencies to not only disclose the cases they classify as unfounded, but also the reasons they’ve done so.
“Because of the importance of the ‘unfounded data’ issue, this topic has been inserted into the next round and will go before the semi-annual board in June,” said FBI spokesman Stephen Fischer.
“This is lightning fast compared to how this usually works,” said Erica Smith, chief of the incident-based reporting unit for the Department of Justice’s Bureau of Justice Statistics.
When reporters first brought the flaw to Smith’s attention, she called dropping unfounded cases from the data collection unacceptable. She has since had multiple calls with FBI officials urging they address the issue and investigate whether misleading information is being released to the public.
At a meeting of the FBI Advisory Policy Board this week in New Orleans, Col. Doug Middleton, the chair of the UCR subcommittee, said that the FBI is assessing system changes in the “near future” to correct the misleading zeros currently entered into UCR data for agencies that report to NIBRS. This spring, when the board’s working groups meet, a paper will be presented to discuss changes to the NIBRS data collection to “better reflect the resolution of crimes coming to the attention of law enforcement.”
“Some attention has been brought to unfounding and clearing offense data,” Middleton told the board. “The FBI acknowledges these concerns as more agencies move to NIBRS data collection and they are going to move to address this issue.”
Mitch Beemer, the incoming president of the Association of State UCR Programs and the incident-based reporting manager for the Kansas Bureau of Investigation, told Newsy after the meeting that he thinks his members would support a change to track unfounded cases in the NIBRS data.
“We have a concern as an association about any gaps in data, any gaps in criminal justice data,” Beemer said. “It would help us give one more check of our local agencies to see how often it might be occurring.”
Law enforcement agencies have for decades been criticized for misusing the unfounded designation, leading to scandals in Baltimore, Philadelphia, New Orleans and elsewhere. An earlier FBI and Bureau of Justice Statistics joint task force cited a scandal in Chicago and explicitly recommended that unfounded cases continue to be tracked under the NIBRS system — neither the BJS nor FBI provided an explanation for why the recommendations were not followed when NIBRS was originally implemented in 1989.
Currently, about 40 percent of police agencies across the nation have transitioned to using NIBRS and are affected by the flaw. All police agencies are expected to switch to NIBRS by Jan. 1, 2021.
The FBI is expected to work over the next several months on drafting recommendations to be voted on in June at the next meeting of the full Advisory Policy Board. If approved, the recommendations go to the FBI director for implementation.
The FBI’s Crime Data,The FBI’s Crime Data,The FBI’s Crime Data,The FBI’s Crime Data,The FBI’s Crime Data,The FBI’s Crime Data,The FBI’s Crime Data,The FBI’s Crime Data,
Your questions and comments are greatly appreciated.
Monty H. & Carolyn A.Go back