ABC News" Reena Ninan, Christine Romo, Pierre Thomas and Mary Bruce report: Prostitution is legal in Colombia and the women who work at the Pley Club, located in the Bosque neighborhood of Cartagena, live on the site of the club. The club, located on a dusty street in the industrial Bosque neighborhood, has a rough exterior but boasts plush "pley rooms," according to its website. The Secret Service personnel were in Cartagena on assignment preparing for the president's visit for Summit of the Americas last weekend. After allegations surfaced that some may have had women or prostitutes in their rooms, 11 members were sent back to the United States. The scandal widened to include members of the military team that was in Cartagena assisting the Secret Service advance team. Sources told ABC News military officials now suspect 10 or more soldiers may have solicited sex from prostitutes. "If violations are determined to have been the case, then these individuals will be held accountable," Defense Secretary Leon Panetta said today. Gen. Martin Dempsey, the chairman of the Joint Chiefs of Staff, said today he is embarrassed by the conduct of the military service members and that "we let the boss down," referring to President Obama, the commander in chief of the military. The accused had arrived a week before the president to prepare security for his visit. By Wednesday, officials at the Hotel Caribe were irritated with the agents" allegedly heavy drinking and loud partying. Local women were coming back to the hotel with agents, sources said. The alleged behavior might have remained secret if it were not for an angry prostitute, who government sources said refused to leave their room after a dispute with at least one of the Secret Service personnel over the bill. "There was a dispute the next morning when one of the women did not leave the room," Rep. Peter King said. Police came and she refused to leave until she was paid for her services. The Secret Service personnel were ordered back to Washington. They have been suspended pending the investigation, and their security clearances have been revoked. Sources say the Secret Service will be giving lie detector tests to agents who were present the night the prostitutes were allegedly in the agents" hotel rooms, and asking the hotel for its security tapes to see if it can determine exactly who was involved.