Obama, Senate head back to Washington as fiscal cliff deadline looms NEW: Senate leader Reid questions whether enough time remains for an agreement Without a deal, taxes go up and spending gets slashed in the new year A broad deficit reduction plan seems out of reach for now An impasse over taxes remains the major obstacle Washington (CNN) -- Let the endgames begin. After a Christmas holiday, President Barack Obama was headed back to Washington from Hawaii and the U.S. Senate reconvened Thursday as the deadline approached for going over the fiscal cliff of automatic tax hikes and spending cuts. However, the House remained on Christmas break with members warned they could be called back on 48 hours' notice if needed. Hopes for a so-called grand bargain that would address the nation's chronic federal deficits and debt appeared dashed for now, with four days remaining to reach agreement on a smaller plan that would avoid the harshest effects of the fiscal cliff's tax increases and slashed spending. With House Republicans declaring themselves unable to find a solution, the focus was on the Democratic majority in the Senate to come up with a way forward that could pass the House and get signed into law by Obama. The White House said Thursday that the president spoke with all four congressional leaders before leaving Hawaii on Wednesday. In the Senate, Majority Leader Harry Reid of Nevada expressed doubt that enough time remained to reach an agreement, especially with the House at least 48 hours from coming back. "I don't know, timewise, how it can happen now," Reid said in opening the Senate's first session back from holiday. Possible scenarios include a short-term deal now, setting up continued negotiations next year when Obama and a new Congress that convenes in January confront a need to raise the federal debt ceiling and approve further spending to keep the government funded. Another possibility would be a short-term deal reached after January 1 that would change the political calculus by having legislators vote for cutting the higher tax rates from the fiscal cliff -- a much more palatable exercise than the current debate over allowing top rates to increase. Otherwise, economists warn the full impact of the fiscal cliff could spark another recession. Retiring Republican Rep. Steve LaTourette of Ohio told CNN on Thursday that such an outcome would be entirely due to politics. "Nobody is willing to pull the trigger" on an agreement because "everybody wants to play the blame game," he said. This blame game is about to put us over the edge. The principal dispute continues to be over taxes, specifically the demand by Obama and Democrats to extend most of the tax cuts passed under President George W. Bush while allowing higher rates of the 1990s to return on top income brackets. Republicans oppose any kind of increase in tax rates, and House Speaker John Boehner suffered the political indignity last week of offering a compromise -- a $1 million threshold for the higher rates to kick in -- that his colleagues refused to support because it raised taxes and had no chance of passing the Senate. Reid lambasted Boehner in his remarks Thursday, saying the speaker wanted to wait until after the new House chooses him as speaker on January 3 before proceeding with a compromise that would pass the chamber today. Boehner was "more concerned about his speakership than putting the country on firm financial footing," Reid charged. Hayworth also made clear to CNN that a limited agreement was the most to expect for now, saying: "I don't think we're going to get the big plan in the next six days." A statement Wednesday by Boehner's leadership team said the Senate must act first on proposals already passed by the House but rejected by Senate leaders and Obama. "If the Senate will not approve and send them to the president to be signed into law in their current form, they must be amended and returned to the House," the leadership statement said. Once this has occurred, the House will then consider whether to accept the bills as amended, or to send them back to the Senate with additional amendments. The House will take this action on whatever the Senate can pass, but the Senate first must act. Obama and Democrats have leverage, based on the president's reelection last month and Democratic gains in the House and Senate in the new Congress. A senior Senate Democratic source told CNN on Wednesday that Reid has made clear in private conversations that he will need assurance that any plan can pass both the Senate and the House before he will bring it up. "It is to nobody's advantage to have a failed Senate vote at this point," the source said on condition of not being identified. This will be the last train we will have, and there is no sense in it leaving the station before we have assurance it will get through. Remaining questions include whether enough Republicans will support a compromise acceptable to Democrats, and whether Senate GOP leader Mitch McConnell will allow a simple majority vote to take up and pass any proposal, or stick to the filibuster level of 60%. "We believe very strongly a reasonable package can get majorities in both houses," a senior White House official said. The only thing that would prevent it is if Senator McConnell and Speaker Boehner don't cooperate. Some Senate Democrats have discussed holding off on bringing up a proposal until the final days of 2012 to increase pressure on Republicans to support avoiding higher taxes on everyone due to the fiscal cliff While the focus now is on a possible agreement in coming days or weeks, anti-tax crusader Grover Norquist told CNN this week that the nation should gird for long-range battle. He predicted "a regular fight" when Congress needs to authorize more government spending and raise the federal debt ceiling in coming months. "There the Republicans have a lot of clout because they can say we'll let you run the government for the next month, but you've got to make these reforms," he said. On Wednesday, Treasury Secretary Timothy Geithner informed Congress that the government would reach its borrowing limit at the end of the year -- in five days' time -- but could take steps to create what he called "headroom" for two months or so. However, Geithner said uncertainty about the fiscal cliff negotiations and possible changes to the deficit situation made it difficult to predict precisely how long the government's steps to ease the situation would last. Reid and other Senate Democrats say House Republicans must accept that an agreement will require support from legislators in both parties, rather than a GOP majority in the House pushing through a measure on its own. Boehner and Republicans complain that the Senate has refused to take up any proposals they have passed in the past two years. CNN's Brianna Keilar, Ted Barrett and Kevin Liptak contributed to this report.