(CNN)No prostitutes. No ifs, ands or buts -- and yes, that includes when and where prostitution is perfectly legal. That was the message Friday from Attorney General Eric Holder to members of the U.S. Justice Department, which includes the FBI, U.S. Marshals Service, Bureau of Alcohol, Tobacco, Firearms and Explosives and other agencies. "The solicitation of prostitution threatens the core mission of the department," Holder wrote in a memo to all personnel in the department he heads. "... Regardless of whether prostitution is legal or tolerated in a particular jurisdiction, soliciting prostitutes creates a greater demand for human trafficking and a consequent increase in the number of minor and adult persons trafficked into commercial sex slavery." Holder doesn't mention specific cases of federal agents and prostitution in his memo. Nor is he dictating a new policy; the attorney general said only that he wanted "to reiterate to all department personnel, including attorneys and law enforcement officers, that they are prohibited from soliciting, procuring or accepting commercial sex." Agents behaving badly overseas . The directive comes a few weeks after a Justice Department inspector general report found DEA agents in foreign postings attended sex parties with prostitutes paid for by drug cartels, among other indiscretions. That report, by department Inspector General Michael Horowitz, cited light punishments and poor handling of sexual misconduct cases at DEA and other Justice Department agencies. Justice Department employees don't have a monopoly on such stories: In 2012, a group of agents and officers in the Secret Service -- which is part of the Department of Homeland Security -- and officers sent to Colombia ahead of President Barack Obama were relieved of duty and returned home amid allegations of misconduct that involved prostitution. That prostitute visit was arranged for by a DEA agent stationed in Colombia, according to Horowitz's office. If someone from the ATF, FBI, Federal Bureau of Prisons or a federal prosecutor is caught with a prostitute they'll be suspended or fired, according to Holder's memo. "This rule applies at all times during an individual's employment, including while off duty or on personal leave."
Attorney General Holder reiterates Justice Department policy on prostitutes .
Soliciting prostitutes is banned, even in places where it's legal, Holder says .
His memo comes weeks after a report involving DEA agents and prostitutes .