'Angry' Obama announces IRS leader's ouster in scandal NEW: "There is a strong and immediate need to restore the public's trust," Steve Miller says President Obama says "Americans have a right to be angry ... and I am angry" IRS identifies two "rogue" employees, congressional source says House Speaker John Boehner: "My question is who's going to jail?" Washington (CNN) -- President Barack Obama vowed Wednesday to hold accountable those at the Internal Revenue Service involved in the targeting of conservative groups applying for federal tax-exempt status, beginning with the resignation of the agency's acting commissioner. In a brief statement delivered to reporters at the East Room of the White House, the president announced that Treasury Secretary Jack Lew requested -- and accepted -- the resignation of acting IRS Commissioner Steve Miller. The president said the "misconduct" detailed in the IRS Inspector General's report released Tuesday over the singling out of conservative groups is "inexcusable." "Americans have a right to be angry about, and I'm angry about it," Obama said. The president said new safeguards will be put in place so that "this doesn't happen again." In an internal message to IRS employees obtained by CNN, Miller said he would be stepping down as commissioner in early June. "This has been an incredibly difficult time for the IRS given the events of the past few days, and there is a strong and immediate need to restore public trust in the nation's tax agency," Miller wrote. I believe the Service will benefit from having a new Acting Commissioner in place during this challenging period. News of the resignation followed revelations that the IRS has identified two "rogue" employees in the agency's Cincinnati office as being principally responsible for the "overly aggressive" handling of requests by conservative groups for tax-exempt status, a congressional source told CNN. Miller said the staffers have already been disciplined, according to another source familiar with Miller's discussions with congressional investigators. The second source said Miller emphasized that the problem with IRS handling of tax-exempt status for tea party groups was not limited to these two employees. Miller met with Senate Finance Committee Chairman Max Baucus of Montana on Tuesday to discuss an appearance before Congress. Asked in a Senate hallway about his meeting with Miller, Baucus told CNN, "I did not learn as much from the meeting as I would have liked." "I told him that it was in his best interest to be totally cooperative -- that it's often the coverup that causes more problems than the original malfeasance," the senator said. And just to be totally straight with me and everybody, and he said he would. Meanwhile, Republican congressional leaders on Wednesday accused Obama's administration of potentially criminal behavior in the handling of requests for tax-exempt status from conservative groups. Senate Republican Leader Mitch McConnell suggested criminal behavior had occurred, saying that the "very serious" allegations involve "an effort to bring the power of the federal government to bear on those the administration disagreed with in the middle of a heated national election." "It actually could be, could be criminal and we are determined to get the answers," McConnell said. House Speaker John Boehner was more definitive, declaring that "my question is, who's going to jail over this scandal?" He told reporters that "clearly someone violated the law" in what an IRS inspector general's report described as delayed processing of applications by groups associated with the political right wing. Attorney General Eric Holder, who ordered a criminal investigation into the situation, said Wednesday at a congressional hearing that the investigation will look at conduct of IRS offices nationwide. "The facts will take us where ever they take us," he said. While the allegations originated in the Cincinnati office, the Justice Department inquiry is based out of Washington, Holder said. The comments came as all 45 Senate Republicans sent the White House a letter that called for the administration to "comply with all requests related to congressional inquiries without any delay" involving the controversy. The letter called the scandal "yet another completely inexcusable attempt to chill the speech of political opponents and those who would question their government, consistent with a broader pattern of intimidation by arms of your administration to silence political dissent." Meanwhile, GOP Sen. John Thune of South Dakota called Wednesday for the acting commissioner of the IRS to step down. The clearly coordinated attacks were part of a GOP effort to increase pressure on the Obama administration over the controversy, one of three potential scandals that has the White House on the defensive less than four months into the president's second term. Read the IRS IG report According to the report by the agency's inspector general released Tuesday, the IRS developed and followed a faulty policy to determine whether the applicants were engaged in political activities, which would disqualify the groups from receiving tax-exempt status. The controversial move began in early 2010 and continued for more than 18 months, the report said, declaring that "the IRS used inappropriate criteria that identified for review Tea Party and other organizations applying for tax-exempt status based upon their names or policy positions instead of indications of potential political campaign intervention." In a statement released late Tuesday, Obama called the report findings "intolerable and inexcusable." The federal government must conduct itself in a way that's worthy of the public's trust, and that's especially true for the IRS. The IRS must apply the law in a fair and impartial way, and its employees must act with utmost integrity. This report shows that some of its employees failed that test," the president said. Obama also said he has directed Treasury Secretary Jack Lew "to hold those responsible for these failures accountable." However, law professor Jonathan Turley of George Washington University said it was unlikely that anyone would end up facing criminal charges. "It's a violation of federal law, but rarely do people go to jail for it," Turley told CNN on Wednesday. More often, criminal charges come from federal officials lying to cover up wrongdoing, rather than from following orders, he said. "The most likely conclusion is no one would be charged, if you look at history," Turley added. IRS officials, according to the report, did not consult anyone beyond the agency about the development of the additional screening criteria. They believed that the criteria they came up with were a screening shortcut meant to help with the influx of applications, the report said. Will the scandal affect 2014 elections? The controversial actions began after the 2010 Supreme Court ruling in the Citizens United case that greatly expanded the ability of corporations, unions and other organizations to participate in election spending, though not through direct contributions to candidates or parties. After the ruling, the number of politically oriented groups seeking tax exempt status as social welfare organizations under section 501 (c) (4) of the federal tax code increased greatly at a time when the federal government, including the IRS, was dealing with austerity measures that reduced or stagnated personnel and resources. The IRS' top watchdog found that the criteria used to flag potential political applications resulted in substantial delays and the request of unnecessary information from the groups. Among the criteria used by IRS officials to flag applications was a "Be On the Look Out" list, which was discontinued in 2012, according to the report. The criteria included: Whether "Tea Party," "Patriots" or "9/12 Project" was referenced in the case file. Whether the issues outlined in the application included government spending, government debt or taxes. Whether there was advocating or lobbying to "make America a better place to live." Whether a statement in the case file criticized how the country is being run. Whether it advocated education about the Constitution and the Bill of Rights. The investigation by the Treasury inspector general for tax administration was initiated after congressional complaints began to surface in the media in 2012 that the IRS was targeting conservative groups and holding up applications. "Whether the inappropriate criterion was shorthand for all potential political cases or not, developing and using criteria that focuses on organization names and policy positions instead of the activities permitted under the Treasury regulations does not promote public confidence that tax-exempt laws are being adhered to impartially," the report said. The IRS welcomed the report, saying that it agreed that aspects of its original approach in handling the influx of tax-exempt applications was inappropriate. The IRS is required by law to determine if organizations are engaging in a legally permissible level of political activity. Centralizing these cases was necessary to achieve consistent treatment," it said in a statement. In a written response included in the report, the IRS commissioner of the Tax Exempt and Government Entities Division said there was no criminal behavior behind the actions of the agents, but rather inefficient management. "We believe the front-line career employees that made the decisions acted out of a desire for efficiency and not out of any political and partisan viewpoint," the commissioner wrote. The report's findings indicate that of the 298 cases reviewed by the IRS inspector general as potential political cases not eligible for tax exempt status: 72 contained the name "tea party," 11 contained "9/12" and 13 contained the word "patriots," according to the report. There were 202 cases that did not contain any such reference. Of those applications still open for review, 160 cases were open from 206 days to more than three years -- through two election cycles. Among the recommendations made by the Treasury inspector general: The IRS must better document reasons why applications are chosen for review, develop a process to track requests for assistance, develop and provide training to employees before each election cycle and immediately resolve outstanding cases. The report also called on Treasury to develop guidelines to explain social welfare activity -- the primary factor in obtaining tax-exempt status. The Republican-led House Ways and Means Committee, which oversees the IRS, will hold a hearing on Friday. Miller, the acting IRS commissioner, and the Treasury inspector general investigating the complaints, J. Russell George, are scheduled to testify. CNN's Kevin Liptak, Kevin Bohn, Jake Tapper, Drew Griffin, David Fitzpatrick and Ted Barrett contributed to this report.