Focus in widening scandal shifts to 'flirtatious' e-mails Defense Secretary Leon Panetta says he has "continued confidence" in Gen. John Allen A defense official says Allen is being investigated due to "flirtatious" e-mails with Jill Kelley Allen and ex-CIA chief David Petraeus wrote letters supporting Jill Kelley's sister Petraeus, who resigned while acknowledging an affair, could testify on Capitol Hill this week (CNN) -- The complicated web entangling an ex-CIA director, his mistress, a top military leader and a woman he allegedly flirted with got no less confusing Wednesday, though the U.S. defense secretary insisted facts will emerge. U.S. Secretary of Defense Leon Panetta defended his request to withhold Marine Gen. John Allen's nomination to become NATO's supreme allied commander, pending an investigation into Allen's communications with a Florida woman. The Petraeus scandal: Who is Jill Kelley? The move was "a prudent measure until we can determine what the facts are, and we will," Panetta told reporters Wednesday. No one should leap to any conclusions. He added that the general "certainly has my continued confidence to lead our forces." Read more: How pillow talk could be security threat According to the Defense Department, Allen is under investigation for what one defense official referred to as "flirtatious" e-mail messages with Jill Kelley -- the woman whose complaints about anonymous, harassing e-mails led to the discovery of CIA Director David Petraeus' affair with a woman later identified as his biographer. Petraeus resigned Friday after acknowledging the affair. Allen will retain his post as the commander of U.S. and allied forces in Afghanistan, pending Senate confirmation of a successor, according to the Defense Department. That vote is due Thursday, Senate officials said. Read more: The Petraeus scandal: What we know Allen has denied wrongdoing, a senior defense official said. The investigation into Allen is in its early stages, but authorities are looking into some 20,000 to 30,000 pages of documents, a defense official told CNN. It is not clear how many of those might involve inappropriate communications. A U.S. official told CNN on Tuesday that many of the documents could be innocuous, involving routine business that Allen and Kelley were both involved in at the U.S. Central Command. In his duties at CENTCOM, Gen. Allen conducted a lot of legitimate business. She (Kelley) did a lot of work with CENTCOM, including Wounded Warriors and such," the official said. It could be that 29,900 of the documents are legitimate business, and the few remaining raise a few eyebrows. Read more: Timeline of the Petraeus affair Sources familiar with Kelley have said the relationship between the two was not sexual, but one said that the e-mails were flirtatious. The fact that Allen remains in command in Afghanistan suggests that there is no criminal issue, another U.S. official told CNN. But the official said the Defense Department's inspector general, an agency watchdog, could still find evidence of criminal conduct. Read more: Tough choice for Obama on Petraeus' successor Widespread media reports have described Kelley as a liaison at MacDill Air Force Base in Tampa, Florida, where the U.S. Central Command is headquartered. A Central Command spokesman said she is a volunteer with no official position. Both Allen and Petraeus appear to know Kelley's sister, Natalie Khawam. The men wrote letters in support of the sister in a custody battle, court records show. One of the sources familiar with Kelley said she first mentioned the alleged harassment in a casual conversation with an FBI agent she knew socially. She did not seek him out specifically for action on the matter, but he was happy for the help, the source said. The source added that Kelley did not know at first that the e-mails led to Petraeus. Kelley, 37, and her husband have released a statement saying they have been friends with Petraeus and his family for more than five years and asked for privacy. Read more: Gen. John Allen: A Marine of distinction A senior official close to Allen says it was Allen who received an anonymous e-mail about Kelley, and tipped Kelley off that someone was threatening her. "There is no affair," the senior official said. She is a bored rich socialite involved with every single senior commander at CENTCOM, because she worked as an honorary ambassador. Meanwhile, the FBI continues to look into the Petraeus affair amid congressional calls for an inquiry into why leaders were not notified of that matter sooner. Amanpour: CIA v. FBI in Petraeus scandal FBI agents were at Broadwell's home in Charlotte, North Carolina, late Monday, said local FBI spokeswoman Shelley Lynch. A source told CNN national security contributor Fran Townsend that Broadwell was acting as Petraeus' archivist, and that the FBI went to the house to look for any documents she might have. It was not clear whether any of the material was classified, the source said. Read more: How Petraeus changed the U.S. military The Internet has been abuzz with video of a speech by Broadwell in which she suggested the Libya attack on September 11 was targeting a secret prison at the Benghazi consulate annex, raising unverified concerns about possible security leaks. Read more: Petraeus' other seduction A senior intelligence official disputed that notion. These detention claims are categorically not true. Nobody was ever held at the annex before, during, or after the attacks," the official said. Acting CIA chief has been here before Petraeus was scheduled to testify on Capitol Hill this week about the Benghazi attack at closed-door hearings. Senate Intelligence Committee Chairman Dianne Feinstein, D-California, said Tuesday that she hopes to bring Petraeus before the panel as early as Friday. CNN's Joe Johns, Gloria Borger, Barbara Starr, Suzanne Kelly, Carol Cratty, Candy Crowley, Dana Bash, Pam Benson and Ted Barrett contributed to this report.