Congress responded to the 2001 strikes and anthrax-tainted letters sent to lawmakers by putting much more money toward emergency preparedness. State health departments typically get tens of millions of dollars per year to prepare for bioterrorism; it was in the hundreds of thousands before Sept. 11.
The money came with a catch: Washington had to set criteria to evaluate how well the dollars were spent. That assignment fell to the Centers for Disease Control and Prevention, which has struggled with the task.
"We're not able to demonstrate accountability," said Craig Thomas, chief of the CDC office that evaluates and monitors public health departments. "It's not just accountability to the CDC. It's accountability to your community. It's accountability to your local stakeholders and the people who fund you as well."
Thomas was speaking to public health leaders at a recent conference in Washington. His candid assessment does not mean local departments have squandered the money. Indeed, health officials say the departments are much better able to respond to major threats than they were five years to 10 years ago.
It is, however, an acknowledgment the CDC relies on anecdotal evidence to demonstrate the improvement. Congress demanded hard, statistical evidence.
"The difficulty comes down to, how do you measure (improvement), how do you quantify that, so you have something you can track over time, something you can use to identify gaps that have to be filled," said the CDC's Dr. Richard Besser. He oversees the Office for Terrorism Preparedness and Emergency Response.
The government began awarding money for bioterrorism preparedness in 1999, sending $40.7 million to the states. In 2002, the total jumped to $950 million. That is about one-quarter of what the U.S. spends each year on bioterrorism and emergency preparedness - not counting the money for preventing a pandemic.
The government also has increased spending on research at the National Institutes of Health and for improving the capabilities of hospitals and first responders.
Health departments used federal grants to stock up on antivirals, buy needles and syringes, and hire more doctors and nurses. One of the most important upgrades came in disease surveillance.
In Michigan, emergency rooms workers plug the symptoms of some 6 million visitors each year into a huge computer database. A spike in vomiting may indicate that a certain food product - spinach, for example- has been tainted with e-coli.
"Sometimes it's nothing. Somtimes, the Super Bowl happened and you had more headaches the next day. Sometimes it's worth looking into and they can discover a new outbreak," said Dr. JoLynn Montgomery, director of the Center for Public Health Preparedness at the University of Michigan. The challenge for the CDC is how to measure Michigan's improved ability to respond to bioterrorism or other such health threats.
An initial list of 100 benchmarks, drafted in 2003, has shrunk to the 23 that were used last year.
States are asked such things as how long it takes:
â€¢To get a "knowledgeable public health professional" to respond to an urgent call.
â€¢To ship a specimen to a laboratory
â€¢To begin an epidemiological investigation of an event that may be of urgent health consequences.
CDC officials point to two reasons for the lag in developing the measures.
For one, the CDC is a scientific organization. It makes recommendations based on scientific data, but such data does not exist when it comes to showing which steps taken by health officials would bring about the best result during a particular emergency.
Also, the agency had difficulty getting health departments to agree about what the government should measure.
"Every health department is different, so where one may have strengths and they feel very confident in measuring something, another may have that as a weakness and feel less confident," said Donna Knutson, a senior adviser at the CDC. State officials who attended the health officials' conference say many measurements are still unclear.
"I don't think they're asking things that are measurable," said Kimberly Allan of the Virginia Department of Health.
said at the recent meeting of public health officials. "The right questions are not being asked." Allan said the questions were too broad and hard to answer. But she said she did not have suggestions for improving them. "Everything is so slippery and vague," added Karen Brady, preparedness coordinator for Tennessee's Health Department. "I could answer the questions five different ways." The CDC's Besser agreed that the current measures are not perfect. "But it's absolutely essential that we move forward and start measuring, and then as we get more experience, we'll continue to refine and improve them," he said. Dr. Paul Jarris, executive director of the Association of State and Territorial Health Officials, said the CDC measures are critical to improving the work of health departments. "We need to work as rapidly as possible toward the best measures we can come up with," Jarris said. "The most important reason to have good measures is to continually improve your performance and it creates transparency."