An immigration summit prompted rebel Tory MPs to demand strong action David Cameron was urged to 'shut the door' to EU migrants during talks Former Cabinet Minister Peter Lilley is claimed to be among the rebels Supporters claimed that hysteria over immigration is being exaggerated By Simon Walters and Glen Owen and Brendan Carlin for The Mail on Sunday David Cameron's attempt to recover from his defeat to Ukip in the Rochester by-election was hit last night by a Downing Street row over whether Britain is "full" of immigrants. An "immigration summit" led to an angry clash between the PM's allies and rebels demanding a tougher response to Nigel Farage. Former Cabinet Minister Peter Lilley is said to have claimed the UK must shut the door to EU migrants, arguing: "Britain is full." David Cameron, pictured, has attempted to put the defeat in the Rochester by-election behind him He was backed by senior MPs on the Number 10 policy board, but branded "shrill" by one of the PM's aides. Jo Johnson, brother of Boris and head of the policy unit working on the party's Election manifesto, also challenged his claim. And pro-Cameron MP Margot James praised "hard-working Poles and Bulgarians," arguing the Tories "must not dance to Ukip's tune." The details have emerged as the Prime Minister faces criticism from two leading MPs in the wake of Thursday's defeat at the hands of Ukip defector Mark Reckless. Charles Walker, vice-chairman of the Tory 1922 Committee, calls on Mr Cameron in today's Mail on Sunday to kick out renegade MPs who have flirted with Ukip. And former Shadow Home Secretary David Davis says he must close "the yawning gap between Britain's political leadership and the people they serve" or lose the Election. The summit was held on Tuesday to prepare for a crunch speech on immigration which Mr Cameron is expected to give within days - which MPs hope will include a vote-winning set of migration curbs. Former government minister Peter Lilley, pictured, told the meeting that in his opinion, Britain is full According to one account, the row at Number 10 flared after Mr Lilley demanded a so-called "emergency brake" on migrants from the EU. Most people believe Britain is full - and they are right. We are already a nation of more than 60 million. We cannot take any more and have to shut the door," he said. Mr Johnson appeared to suggest immigration fears are exaggerated. He reportedly said that "95 per cent of Britain is undeveloped." One insider said: "It sounded as though Johnson was saying there is plenty of room for more immigrants." Mr Lilley, who yesterday attended a conference of the Eurosceptic Bruges Group with Mr Reckless, was called "shrill" by Mr Cameron's immigration special adviser, Daniel Korski, who once worked for the former EU High Commissioner, Labour's Baroness Cathy Ashton. Others rallied behind Mr Lilley. Lancashire MP Jake Berry said Mr Cameron must not be "outflanked" by Ed Miliband. The Labour leader last week pledged to deny dole to new EU immigrants for two years. Many Tory MPs want an even longer ban. Margot James, pictured, warned the meeting that the Conservative Party should not march to Ukip's tune A well-placed source said: "It was pretty lively. It was Jo, Margot and Korski versus Lilley and the rest. Mr Johnson, 42, has argued for stronger ties with India. Some pundits say he is brainier than brother Boris and could beat him to the Tory leadership. A former banker, he is married to Amelia Gentleman, who writes for the Guardian newspaper. Millionairess entrepreneur Ms James, 57, is the first openly lesbian Conservative MP. She gave those at the meeting a written statement, which said: "There is a danger that the anti-immigration and anti-EU minority tail is wagging the majority British dog. There is a clear case that needs to be made for immigration. Their role in wealth creation should not be underestimated. The day people are put off by the constant negative rhetoric about people coming to this country, and stop coming, is the day we will have far more to worry about. She is said to have highlighted a recent poll which showed that more than two thirds of the public would not vote for Ukip in any circumstance. Mr Lilley, 71, a hardliner on welfare and immigration, once wrote a pamphlet entitled "Immigration, too much of a good thing." In 1992 he called for an end to the "something for nothing society" via a parody of the Lord High Executioner's "Little List" song from Gilbert and Sullivan's Mikado. A source said: "Jo Johnson and Korski are part of the cosy Downing St chumocracy. They just don't get ordinary people's concerns about immigration. Jo Johnson, left, head of David Cameron's policy unit called for calm claiming that fears over immigration are being 'exaggerated'. However, one Tory branded Johnson as part of 'the cosy Downing St chumocracy' But a Government aide said there was concern at the tone of some of the comments, adding: "Some of them sounded more like Nigel Farage than Conservative MPs. They were coming out with crude stuff like "we've got to stop Poles stealing the jobs of our voters."" The Prime Minister has discussed with aides the logistics of imposing a "brake" on individual EU countries when, for example, migrants reach a specified proportion of the UK's population. He has been advised by the Foreign Office that as the treaties stand, any brake could only be triggered in the event of an emergency such as a natural disaster. In his article, Mr Walker says his party "put aside its moral compass" during the Rochester by-election by "banging on" about immigration. "We fought the wrong campaign against Mark Reckless and have paid the price," he writes. In recent weeks and months we have allowed the terms of the debate to be framed and set by the pernicious, dripping, corrosive cynicism of Ukip. And in his article, below, Mr Davis says: "The Rochester result is not some bolt out of the blue. Last night, Mr Johnson declined to comment. But a Number 10 spokesman said: "This is not a fair and accurate representation of the meeting." When contacted, Mr Lilley also declined to comment on the discussions. Ms James and Mr Berry could not be reached for comment.