Consider the record. Washington didn't predict the fall of the shah in Iran, or the end of the Cold War, or the disintegration of the Soviet Union. Nor did our pundits (whether left, right or center) predict the war between Vietnam and China after the U.S. lost more than 50,000 service members in the region to prevent the spread of monolithic communism, or, for that matter, China's turn toward becoming a capitalistic and trading giant. There are innumerable other examples in Latin America, Africa and the Middle East.
Everything is fine as long as nothing much is happening. But when something happens, especially if it's sudden or revolutionary, we usually don't see it coming. Since World War II at least, Washington's success rate at predicting change in countries with which we have a hostile relationship is close to zero.
Why this failure when most of the events mentioned were years in the making? In part, it's that we don't look at the right things. Our intelligence apparatus is better at counting missiles than at following the intricate politics of a central committee or overhearing the street talk in Tehran — or Baghdad. But even good intelligence on those matters would have had a hard time making it into the Washington debate.