US officials are scrambling to end a dispute between Ankara and Baghdad that threatens to derail a carefully developed plan to retake Mosul from the Islamic State group.

Officials fear a war of words between Turkey and Iraq's central government could jeopardize a fragile pact that would keep rival sectarian and ethnic militias out of the northern Iraqi city's center.

"There is a lot of tension and the public rhetoric has gotten a bit out of hand," a senior administration official told AFP. "It's an extremely troubling scenario."

Defense Secretary Ash Carter will visit Turkey on Friday to make sure the plan stays on track.

The long-awaited battle to retake Iraq's second city began on Monday and will probably take months to complete.

"We are trying to make sure that those tensions are not going to create such a chaotic situation that the military success would simply be overshadowed," the official said.

A dizzying array of ethnic and sectarian groups are already jockeying for influence "the day after" the Islamic State (IS) group is ousted.

The United States has brokered a deal to keep powerful Shiite and Kurdish militias on the outskirts of the majority Sunni Arab city, hoping to avoid more of the sectarian bloodletting that has plagued Iraq since a US-led invasion toppled Saddam Hussein.

Amnesty International has accused Shiite paramilitaries of carrying out war crimes during the recapture of Tikrit from the IS group, including the torture and execution of thousands of civilians.

The Iraqi military will take the lead in Mosul.

But Turkish President Recep Tayyip Erdogan appears to have jettisoned the tacit agreement, saying it was unthinkable for Ankara or its allies to stay on the sidelines.

"We will be in the operation and we will be at the table," he said in a televised speech. "Our brothers are there and our relatives are there. It is out of the question that we are not involved."

Turkey is concerned about the influence of anti-Ankara Kurdish militia and Iranian-backed Iraqi Shiite militias known as the Popular Mobilization Forces (PMF).

Around 700 Turkish troops are already deployed in Bashiqa, in northern Iraq, and Ankara backs a number of Sunni and Turkomen militias active inside the country.

Iraq's Prime Minister Haider al-Abadi responded angrily, calling for Turkey to withdraw troops and stop violating Iraqi sovereignty.

A senior US administration official warned that, in the worst case, the war of words could provoke clashes between Turkish and Iraqi government forces as well as a factional free-for-all that would see Shiite militias pile into the city.

"This is part of what we are telling the Turkish government," the official said. "The more their rhetoric escalates, the more their actions -- if they were to take actions -- the more it's going to give both pretext and justification for the PMF to demand a more active role."

"Unilateral actions, in the worst circumstance, could lead to direct clashes between Iraqis and Turks and between the Shia militia and Turks."

But keeping the Iraqi military at the tip of the spear will be essential if President Barack Obama's administration is to avoid repeating past failures in Iraq.

"Yes, it's a campaign to defeat ISIL militarily, but the president's goal by the end of the administration is to put ISIL on the path to lasting defeat," the senior administration official said, using an alternate acronym for the Islamic State group.

"That meant taking steps to ensure that in the aftermath of whatever military liberation of territory or defeat of ISIL that we wouldn't be in a position where the level of destruction, marginalization, alienation, either from a humanitarian point of view or from a governance point of view, that you would just be planting the seeds of the next generation of extremist or terrorist group."

More than a decade of war in Iraq has seen one Sunni insurgent group defeated, only to be replaced by another.

There are also fears in Washington that the spat with Erdogan could weaken Abadi, who is already in a difficult position at home.

The moderate US-backed Shiite leader faces a stiff challenge from hardliners, including former president Nuri al-Maliki, whom the White House has blamed for many of Iraq's sectarian woes.

"The more you see this spiral between Turkey and Baghdad, the harder it is going to be for Abadi to resist calls by his Shia base," the US official said.

"If Abadi appears too weak in the face of Turkey, then he is going provide too much of an opportunity for some of his rivals to pick up the mantle and be more nationalistic."