During the trip, Rumsfeld dismissed talk of a scheduled pullout of U.S. troops, saying "I don't talk deadlines."
The defense secretary said that the possibility of withdrawing some U.S. troops will depend on the stability of the new Iraqi government.
There are currently 129,000 U.S. troops serving in Iraq. President George W. Bush has come under pressure to reduce that number by several thousand by the end of the year.
Rumsfeld urged Iraqis towards political reconciliation, saying security could be established only through genuine reconciliation between Sunnis and Shiites:
"There are those who would like to see a civil war in this country," Rumsfeld said. "They know they can't win on the battlefield. They only way they can win, they believe, is to create anarchy and to cause the country to become a failed state so that they can then pick up the pieces and impose their will and establish a caliphate here."
The top U.S. commander in Iraq, General George Casey, told Rumsfeld that death squads are fueling the conflict in Iraq.
Casey said both Sunni and Shi'ite armed groups seem to be specifically targeting civilians from the other community.