Secret Service director denies culture of misconduct NEW: Senators differ with the Secret Service director on the scope of the problem Director Sullivan says agents involved in the Colombia prostitution scandal were "reckless" Sullivan says no security breach occurred The incident occurred before President Obama arrived for a regional summit Washington (CNN) -- Secret Service Director Mark Sullivan told a Senate committee Wednesday that agents involved in last month's prostitution scandal in Colombia were "reckless" and did "dumb things," but he rejected accusations that a culture condoning such misconduct exists in the agency. In his first appearance before Congress since the incident that embarrassed the nearly 150-year-old agency and raised security concerns, Sullivan said an investigation of allegations of a similar incident in El Salvador turned up no wrongdoing. "This is not a cultural issue, this is not a systemic issue," Sullivan insisted, arguing that similar misconduct hasn't occurred on thousands of other overseas trips by Secret Service agents. On this particular trip, we had individuals who made very bad decisions. However, the Senate Homeland Security Committee's ranking members suggested the problem evolved from a culture of similar misconduct within the agency that needed to be addressed. "I continue to believe that the problem is broader than you believe it to be," Sen. Susan Collins, R-Maine, told Sullivan at the end of Wednesday's hearing. She later told reporters that she was disturbed by Sullivan's repeated insistence that the Colombia scandal, in which at least a dozen agents drank heavily and consorted with prostitutes, was an isolated incident. Collins and committee chairman Sen. Joe Lieberman, an independent from Connecticut who caucuses with the Democrats, called an announcement at the hearing that the Department of Homeland Security's acting inspector general will launch an independent investigation of the Colombia incident "a big change" and "significant." Previously, the acting inspector general, Charles Edwards, was going to review the internal investigation by the Secret Service. The hearing was the first by a congressional committee on the events in Cartagena shortly before President Barack Obama arrived to attend the Summit of the Americas. A morning-after dispute between an agent and a woman over payment led to a dozen Secret Service members being sent home and the resulting media coverage and investigations. "If one of the agents had not argued with one of the women about how much he owed her, the world would not have known this sordid story," Lieberman said in his opening statement Wednesday. He added it was hard to believe that on one night, the 12 agents involved "suddenly and spontaneously did something they and other agents never had done before." Lieberman and Collins provided new details of the incident, describing how the Secret Service agents went out on the town in groups of two or three to four different strip clubs and night clubs and returned to their hotel with foreign women who were signed in as overnight guests. Collins said the circumstances suggest that "different rules apply on the road." When Collins noted the agents used their own names in signing in the women to their rooms, indicating they had no fear of being caught, Sullivan responded that "between the alcohol and I don't know, the environment, these individuals did some dumb things," adding that he did not believe they acted because they thought their behavior would be tolerated. "When many of these people were interviewed, I don't think they could explain why they exhibited the behavior they did," he said, adding that "there is no excuse for that type of behavior from a conduct perspective and a national security perspective. That type of behavior was reckless. Sullivan also revealed that two of the Secret Service agents who initially said they would resign over the scandal now are seeking to challenge their ouster. He said the agency will seek to revoke the security clearances of the two, which would effectively prevent them from continuing to work for the Secret Service. Under respectful but persistent questioning by the panel members, Sullivan said the misconduct never compromised the security plan for the Obama trip. "I am deeply disappointed, and I apologize for the conduct of these employees and the distraction that it has caused," Sullivan said, later adding: "I want to ensure that this type of conduct that occurred in Cartagena is never repeated." The military is investigating the alleged involvement of 12 service members, but has yet to announce any results. They willingly made themselves potential targets not only for intelligence or security services, but also for groups like" drug cartels. Sullivan noted in his opening statement that "at the time the misconduct occurred, none of the individuals involved in misconduct had received any specific protective information, sensitive security documents, firearms, radios or other security related equipment in their hotel rooms." He also outlined previously reported steps being taken to prevent future instances, including enhanced supervision on such assignments and a reinforced code of conduct. Sullivan also encouraged agents to blow the whistle on any misconducted by peers that they encounter. In addition, Sullivan referenced allegations that Secret Service personnel were involved in similar misconduct in March 2011 in San Salvador, El Salvador. Sullivan said Wednesday that the owner of the business was interviewed and provided a sworn written statement saying "he had no knowledge or any other information that any Secret Service personnel had been to his business or information about misconduct by Secret Service personnel." CNN's Ted Barrett, Kate Bolduan, Ashley Hayes and Dana Bash contributed to this report.