David Cameron accused of losing control of Conservative party David Cameron, UK prime minister, has been accused of "losing control" of his Conservative party, amid growing acrimony over his policy on Europe, gay marriage and his general leadership style. With two years remaining until the next British election, Mr Cameron's party has reopened its old divisions over Europe which have helped to destroy previous prime ministers including Margaret Thatcher and John Major. Mr Cameron is seen by some on the Tory right as being insufficiently robust towards Europe, while his promotion of gay marriage - the subject of a House of Commons debate this week - has infuriated many in the party. Tory MPs mutter that Mr Cameron is out of touch and running an elitist government with a coterie of friends, an impression given dangerous new credence by claims that an unnamed ally had described Tory activists as "mad, swivel-eyed loons." The quote, carried in several newspapers, was vehemently denied by its assumed author, Tory co-chairman Lord Feldman, a former university friend and tennis partner of Mr Cameron. Lord Howe, Tory former foreign secretary, said on Sunday that Mr Cameron had made a serious error by raising the prospect of a referendum in 2017 - if he wins the next election - in a Europe speech in January. "The prime minister has opened a Pandora's box politically and seems to be losing control of his party in the process," Lord Howe wrote in the Observer. "The ratchet effect of Euroscepticism has now gone so far that the Conservative leadership is in effect running scared of its own backbenchers," he added. Lord Howe, who served in Thatcher's government in the 1980s, is one of the few remaining pro-Europeans in Mr Cameron's party. The Conservatives are divided on a question of tactics towards Europe and the degree of their scepticism towards the EU. Last week two cabinet ministers - education secretary Michael Gove and defence secretary Philip Hammond - said they would vote to leave the EU if there were a vote tomorrow, although they hoped Mr Cameron could negotiate acceptable new terms allowing them to ultimately vote to stay in. Mr Cameron was furious with the remarks. He believes the party should stop agonising about Europe and concentrate on the core issues facing the country - including focusing on early signs of a possible economic recovery. William Hague, foreign secretary, told the Sunday Telegraph that all ministers were now expected to be speculating on how they might vote if an EU referendum were held immediately: "The rest of the cabinet will not be answering hypothetical questions." Mr Cameron had hoped that the offer of an EU referendum in the next parliament would be enough to satisfy his party's Eurosceptics and head off the rise of the populist anti-Brussels UK Independence party. Instead his pledge of a referendum has stirred up Tory activists many of whom do not want to wait until 2017 for more decisive action on Europe. The ratchet effect of Euroscepticism has now gone so far that the Conservative leadership is in effect running scared of its own backbenchers Lord Howe More than 30 local Tory chairmen and former chairman will deliver a letter to Mr Cameron on Monday warning that his "failure to listen" to the concerns of grassroots members was causing many to quit. Last week 114 Conservative MPs criticised Mr Cameron for failing to include a bill in this year's legislative package to enshrine in law an EU referendum in the next parliament. As many as 200 Tory MPs, including ministers, could vote against the gay marriage proposal during two days of debate on the issue starting in the House of Commons on Monday. Downing St let it be known last Friday that it was "dismayed" at the comments of Mr Hammond, who criticised Mr Cameron's handling of the same-sex marriage issue.