The FBI’s Crime Data: What Happens When States Don’t Fully Report (#GotBitcoin?)
Bureau’s decades-old method to account for missing data could use an upgrade, some experts say. The FBI’s Crime Data: What Happens When States Don’t Fully Report
In any given year, more than 18,000 U.S. police agencies are asked to submit crime data to the FBI. But some don’t provide complete information or, in some cases, any information at all.
When that happens, the Federal Bureau of Investigation uses crude estimates to account for the missing data. Those figures are then used to generate “Crime in the United States,” an annual tally of violent and property crimes that is a quality-of-life measure as well as a gauge of criminal justice policies and spending.
In most instances, the estimates, though rough, don’t stray substantially from the submitted numbers.
In its latest report, the FBI didn’t adjust the counts of violent crime for 11 states at all. But it inflated Indiana’s numbers by 9.9%, West Virginia’s by 13% and Mississippi’s by 68%, raising that state’s count to an estimated 8,526, up from a reported 5,084.
Homicides, generally considered the most reliable of the FBI’s crime statistics, showed similar discrepancies. The numbers for 28 states were not adjusted, but Mississippi’s count, the worst of the bunch, was increased by 56%, to an estimated 245 murders from a reported 157.
“What we need are error bars on this, but we don’t have that,” said Jeff Asher, a crime analyst based in New Orleans who argues the agency should publish margins of error with its numbers.
It’s impossible to know how far off the FBI’s adjustments are, and experts offered mixed opinions about the significance of the potential discrepancies.
Robert Weisberg, co-director of Stanford University’s Criminal Justice Center, and James Alan Fox, a criminologist at Northeastern University, said the numbers are close enough to evaluate trends over multiple years.
“The problem is looking one year to the next,” Dr. Fox said. “Don’t get hung up on the year-to-year changes.”
The FBI’s most recent report showed murders and violent crimes declined less than 1% from 2016 to 2017.
“That’s just noise,” said Richard Berk, a criminologist at the University of Pennsylvania, suggesting the change, with or without estimates, was insignificant.
Other experts believe, at minimum, the FBI should use a more sophisticated system for generating estimates.
Two decades ago, the Bureau of Justice Statistics, a unit of the Justice Department, published a 78-page paper critiquing the FBI’s procedure and recommending ways to improve it.
“So far, nothing has come of it,” said Michael D. Maltz, the criminologist who wrote the paper and is now a researcher at Ohio State University’s Criminal Justice Research Center.
The FBI did not respond to requests for information, but Dr. Maltz, who with a grant from the American Statistical Association has developed an alternative way to estimate the missing data, described the bureau’s procedure:
When a jurisdiction provides data for three or more months, the FBI estimates the total by multiplying the reported number of crimes by 12/N, where N is the number of months for which reports exist.
Using that formula, an agency that reports four months of crimes would be estimated to have 12/4, or three times the number of crimes it reported.
Because crimes are not evenly distributed throughout the year, that method could lead to over- or underestimates, depending on whether the reported data cover months that are historically lower in crime (winter) or those that are historically higher in crime (summer).
When a jurisdiction provides no data, the FBI estimates the number of crimes based on rates for similarly sized areas in the same state that provided all 12 months of data.
Using that measure, if a jurisdiction with a population of 150,000 reports two months or less of crime data while jurisdictions in the state with a similar population report 620.2 crimes per 100,000 people, the agency that didn’t fully report will be estimated to have had 930.3 crimes (620.2 x 150,000 / 100,000).
A better approach, according to Alicia Carriquiry, a statistician at Iowa State University, would take advantage of all of available data, not only across agencies but across time.
The FBI’s less rigorous method was developed in the 1960s out of necessity when a single year of crime data was stored on seven or eight large reels of computer tape.
“Longitudinal imputation of missing data was truly impossible,” Dr. Maltz said. “To fill in the gaps, the FBI used data that was most readily available to them—data on the same tape, either from the agency itself, inflating the available count to a 12-month estimate, or using similar agencies’ data.”
Now, all the data collected since 1960 will fit on a single thumb drive.
And that, he said, is something you can longitudinally count on.
Your questions and comments are greatly appreciated.
Monty H. & Carolyn A.Go back