Second term mostly drama for Obama Scandals and controversies have weighed down the start of President Barack Obama's second term. President Obama slammed by controversies, criticism from the right and left The bitter political divide of his first term remains, and may be getting worse Even the first lady got heckled -- at a Democratic fundraiser, no less History shows that second terms can be tougher for presidents Washington (CNN) -- History shows that second terms in the White House can be much tougher than first ones, and that is proving true so far for President Barack Obama. Less than five months in, Obama and his administration appear knocked off balance by a barrage of controversies and criticisms exacerbating the bitter political battles that marked his first four years in office. He's under fire from the right and left, accused by some of conspiratorial machinations to grab even more power than the leader of the free world legally holds. Headlines are dominated by scandals such as the Internal Revenue Service targeting of conservative groups and classified leaks that disclosed details of the vast data mining and surveillance apparatus created after the 2001 terrorist attacks. Meanwhile, Republicans and some Democrats say his attorney general should resign over various issues including secret subpoenas of journalist phone records. Conservative Democrat: Holder should think about quitting Even the first lady got heckled -- at a Democratic fundraiser, no less. While the issue was gay rights, the incident showed how Obama supporters also were frustrated by what they consider to be a lack of sufficient progress on progressive issues they expect the president to champion. CNN exclusive video: First lady confronts heckler Second-term blues Second-term blues have been the norm for presidents in recent decades, with Ronald Reagan facing the Iran-Contra scandal in the 1980s and Bill Clinton getting impeached over the Monica Lewinskly affair in the 1990s. To columnist and CNN contributor John Avlon, the latest Washington scandals "have put the president off balance," with the administration on defense instead of driving the agenda. "The choice will be in how the administration tries to deal with it," Avlon said. If it's in denial and acts like these events are occurring outside its purview or control, that will be a big problem. Obama finds himself mired in topics and disputes far from his second-term agenda, in part because he maintained some of the national security policies of his predecessor that were adopted in a nation traumatized by 9/11, noted CNN Senior Political Analyst Gloria Borger. "This is a president now who's dealing with issues he never thought he was going to have to deal with," Borger said Monday, referring to drone strikes, government surveillance and classified leaks. But when you continue some of the policies of George W. Bush, you're going to have some of the same questions that are raised about them. How should Obama respond? Avlon and Borger agreed that Obama must be proactive in dealing with the newly revealed details about how the government has access to phone records and Internet activity as tools in fighting terrorism. Last week's leak of classified documents on the U.S. intelligence programs forced Obama to try to reassure the nation by declaring "nobody is listening to your telephone calls." However, the president who campaigned in 2007 by criticizing what he called his predecessor's claim of a "false choice" between civil liberties and national security following the 2001 al Qaeda attacks now argued that such a choice was inevitable. 12 moments in national security for Obama's team Borger said Obama now needs to bolster his administration's explanation so far -- that Congress and federal judges have sufficient oversight to prevent abuses of the information available. "I think he ought to get out there, speak with the American public, say not only why this kind of surveillance is defensible, but why he thinks it is essential in a post-9/11 world and lay it out there for the American people within the strictures of what he can say given the classified nature of the program," she said. CNN Chief National Correspondent John King noted that despite Obama's insistence his administration struck a proper balance between national security and domestic surveillance, critics from ranging from libertarian GOP Sen. Rand Paul of Kentucky to liberal Democratic Sens. Mark Udall of Colorado and Ron Wyden of Oregon are calling for more details and debate. "If the president doesn't try to get ahead of it, guess what, he'll get dragged along with it," King said. Other issues on the table The controversies come as the administration and Congress wrestle with other volatile issues, including a bipartisan push for immigration reform and a debate over whether to arm Syrian rebels. Avlon and King agreed that Obama needs to move forward on issues where progress can occur, and both cited the Senate immigration bill that would offer a path to legal status for millions of immigrants living illegally in the United States. Conservative Republicans fiercely oppose the measure, but more mainstream GOP legislators consider it vital to the party's hopes of building support among the nation's fastest-growing demographic -- the Hispanic vote. In last year's presidential election, exit polls showed Obama trounced GOP nominee Mitt Romney among Hispanic voters. "The fact that immigration reform looks possible is significant," Avlon said, saying passage of a bill would be a "significant legacy accomplishment" for Obama and also a boost for Congress at a time of extremely low public approval ratings. Republicans led by Sen. Jeff Sessions of Alabama consider the immigration measure drafted by a bipartisan Senate "Gang of Eight" little more than amnesty for undocumented immigrants. They demand stricter border enforcement provisions before providing legal status of the estimated 11 million immigrants in the country illegally. Meanwhile, the Obama administration is looking at possible options this week to increase support to Syrian rebels after resisting persistent calls for months to provide weapons. "There is a recognition that unless we provide help, a lot of help, the situation is going to be very, very terrible," one senior administration official said. The official described a "somber mood" among Syria hands throughout the administration, who remain frustrated at inaction by the Obama administration in the face of a deterioration on the ground in Syria and spillover to neighboring states like Lebanon. GOP scrutinizes Justice Department Republicans are quick to seize on any foreign policy objection with the Obama administration as an opening to attack former Secretary of State Hillary Clinton, who is considered the leading possible Democratic presidential contender in 2016. In the House, the Oversight and Government Reform Committee has demanded more information on erroneous talking points by the administration four days after the terrorist attack last September on a diplomatic outpost in Benghazi, Libya, that killed four Americans, including Ambassador Christopher Stevens. Rep. Darrell Issa of California, the panel's GOP chairman, also is a leading critic of Attorney General Eric Holder over the botched "Fast and Furious" gun walking program and other issues. Holder now faces further scrutiny over the Justice Department's secret subpoenas of phone records for Associated Press journalists last year as part of an investigation of a classified leak. He has told legislators he recused himself from that case and never would prosecute a journalist for doing investigative reporting. Republicans question if Holder lied to Congress when he told a congressional hearing last month that he had no knowledge of any possible prosecution of journalists linked to classified leaks. Avlon said the various issues provide grist for Obama critics now, but the major factor in determining public opinion on a presidency will be the economy, which is moving in the right direction for the White House. "Teflon presidents are Teflon in part because of the economy," he said, rejecting accusations by some conservatives that Obama's troubles amount to scandals that could lead to impeachment. "The impulse to simply throw out Nixon narratives as a poster boy for the second-term curse is silly," Avlon added. This ain't Watergate and calls for impeachment reflect hyperpartisan wishes rather than reality. CNN's Elise Labott and Jill Dougherty contributed to this report.