--- license: apache-2.0 language: - en metrics: - accuracy - bleu - perplexity tags: - ai - agents - rl - reinforcement - learning --- # ISOPro: Pro Tools for Intelligent Simulation Orchestration for Large Language Models ISOPRO is a powerful and flexible Python package designed for creating, managing, and analyzing simulations involving Large Language Models (LLMs). It provides a comprehensive suite of tools for reinforcement learning, conversation simulations, adversarial testing, custom environment creation, and advanced orchestration of multi-agent systems. ## Features - **Custom Environment Creation**: Easily create and manage custom simulation environments for LLMs - **Conversation Simulation**: Simulate and analyze conversations with AI agents using various user personas - **Adversarial Testing**: Conduct adversarial simulations to test the robustness of LLM-based systems - **Reinforcement Learning**: Implement and experiment with RL algorithms in LLM contexts - **Workflow Automation**: Learn and replicate UI workflows from video demonstrations - **Car Environment Simulation**: Train and evaluate RL agents in driving scenarios - **Utility Functions**: Analyze simulation results, calculate LLM metrics, and more - **Flexible Integration**: Works with popular LLM platforms like OpenAI's GPT models, Claude (Anthropic), and Hugging Face models - **Orchestration Simulation**: Manage and execute complex multi-agent simulations with different execution modes ## Installation You can install isopro using pip: ```bash pip install isopro ``` For workflow simulation features, ensure you have the required dependencies: ```bash pip install opencv-python numpy torch stable-baselines3 gymnasium tqdm ``` If you plan to use Claude capabilities: ```bash export ANTHROPIC_API_KEY=your_api_key_here ``` ## Examples To explore IsoPro examples, visit https://github.com/iso-ai/isopro_examples. ## Usage ### Adversarial Simulation Test the robustness of AI models against adversarial attacks. ```python from isopro.adversarial_simulation import AdversarialSimulator, AdversarialEnvironment from isopro.agents.ai_agent import AI_Agent import anthropic class ClaudeAgent(AI_Agent): def __init__(self, name): super().__init__(name) self.client = anthropic.Anthropic(api_key=os.getenv("ANTHROPIC_API_KEY")) def run(self, input_data): response = self.client.messages.create( model="claude-3-opus-20240229", max_tokens=100, messages=[{"role": "user", "content": input_data['text']}] ) return response.content[0].text # Create the AdversarialEnvironment adv_env = AdversarialEnvironment( agent_wrapper=ClaudeAgent("Claude Agent"), num_adversarial_agents=2, attack_types=["textbugger", "deepwordbug"], attack_targets=["input", "output"] ) # Set up the adversarial simulator simulator = AdversarialSimulator(adv_env) # Run the simulation input_data = ["What is the capital of France?", "How does photosynthesis work?"] simulation_results = simulator.run_simulation(input_data, num_steps=1) ``` ### Conversation Simulation Simulate conversations between an AI assistant and various user personas. ```python from isopro.conversation_simulation.conversation_simulator import ConversationSimulator # Initialize the ConversationSimulator simulator = ConversationSimulator( ai_prompt="You are an AI assistant created to be helpful, harmless, and honest. You are a customer service agent for a tech company. Respond politely and professionally." ) # Run a simulation with a predefined persona conversation_history = simulator.run_simulation("upset", num_turns=3) # Run a simulation with a custom persona custom_persona = { "name": "Techie Customer", "characteristics": ["tech-savvy", "impatient", "detail-oriented"], "message_templates": [ "I've tried rebooting my device, but the error persists. Can you help?", "What's the latest update on the cloud service outage?", "I need specifics on the API rate limits for the enterprise plan." ] } custom_conversation = simulator.run_custom_simulation(**custom_persona, num_turns=3) ``` ### Workflow Simulation Automate UI workflows by learning from video demonstrations. ```python from isopro.workflow_simulation import WorkflowAutomation, AgentConfig # Basic workflow automation automation = WorkflowAutomation( video="path/to/workflow.mp4", config="config.json", output="output_dir", logs="logs_dir" ) automation.run() # Advanced configuration agent_config = AgentConfig( learning_rate=3e-4, pretrain_epochs=10, use_demonstration=True, use_reasoning=True ) simulator = WorkflowSimulator( video_path="path/to/video.mp4", agent_config=agent_config, viz_config=visualization_config, validation_config=validation_config, output_dir="output" ) training_results = simulator.train_agents() evaluation_results = simulator.evaluate_agents() ``` ### Car Reinforcement Learning Train and evaluate RL agents in driving scenarios. ```python from isopro.car_simulation import CarRLEnvironment, LLMCarRLWrapper, CarVisualization # Create the car environment with LLM integration env = CarRLEnvironment() llm_env = LLMCarRLWrapper(env) # Initialize visualization viz = CarVisualization(env) # Train and visualize observation = llm_env.reset() for step in range(1000): action = llm_env.get_action(observation) observation, reward, done, info = llm_env.step(action) viz.render(observation) if done: observation = llm_env.reset() ``` ### Reinforcement Learning with LLM Integrate Large Language Models with reinforcement learning environments. ```python import gymnasium as gym from isopro.rl.rl_agent import RLAgent from isopro.rl.rl_environment import LLMRLEnvironment from stable_baselines3 import PPO from isopro.rl.llm_cartpole_wrapper import LLMCartPoleWrapper agent_prompt = """You are an AI trained to play the CartPole game. Your goal is to balance a pole on a moving cart for as long as possible. You will receive observations about the cart's position, velocity, pole angle, and angular velocity. Based on these, you should decide whether to move the cart left or right.""" env = LLMCartPoleWrapper(agent_prompt, llm_call_limit=100, api_key=os.getenv("ANTHROPIC_API_KEY")) rl_agent = RLAgent("LLM_CartPole_Agent", env, algorithm='PPO') # Train the model model.learn(total_timesteps=2) # Test the model obs, _ = env.reset() for _ in range(1000): action, _ = model.predict(obs, deterministic=True) obs, reward, done, _, _ = env.step(action) if done: obs, _ = env.reset() ``` ### AI Orchestration Orchestrate multiple AI agents to work together on complex tasks. ```python from isopro.orchestration_simulation import OrchestrationEnv from isopro.orchestration_simulation.components import LLaMAAgent, AnalysisAgent, WritingAgent from isopro.orchestration_simulation.evaluator import Evaluator # Create the orchestration environment env = OrchestrationEnv() # Add agents to the environment env.add_component(LLaMAAgent("Research", "conduct thorough research on the impact of artificial intelligence on job markets")) env.add_component(AnalysisAgent("Analysis")) env.add_component(WritingAgent("Writing")) # Define the task task = "Prepare a comprehensive report on the impact of artificial intelligence on job markets in the next decade." # Run simulations in different modes modes = ['parallel', 'sequence', 'node'] results = {} for mode in modes: result = env.run_simulation(mode=mode, input_data={'task': task, 'run_order': 'first'}) results[mode] = result # Evaluate the results evaluator = Evaluator() best_mode = evaluator.evaluate(results) print(f"The best execution mode for this task was: {best_mode}") ``` ## Documentation For more detailed information on each module and its usage, please refer to the [full documentation](https://isopro.readthedocs.io). ## Examples The [isopro examples](https://github.com/iso-ai/isopro_examples) repository contains Jupyter notebooks with detailed examples: - `adversarial_example.ipynb`: Demonstrates adversarial testing of language models - `conversation_simulation_example.ipynb`: Shows how to simulate conversations with various user personas - `workflow_automation_example.ipynb`: Illustrates automated UI workflow learning - `car_rl_example.ipynb`: Demonstrates car environment training scenarios - `run_cartpole_example.ipynb`: Illustrates the integration of LLMs with reinforcement learning - `orchestrator_example.ipynb`: Provides a tutorial on using the AI orchestration capabilities ## Troubleshooting IsoPro 0.1.6 may have some issues downloading dependencies. If you encounter any, simply run `pip install isopro stable-baselines3 gymnasium isozero iso-adverse tiktoken` to resolve any missing package errors. ## Contributing We welcome contributions! Please see our [Contributing Guide](CONTRIBUTING.md) for more details. ## License This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details. ## Support If you encounter any problems or have any questions, please [open an issue](https://github.com/iso-ai/isopro/issues) on our GitHub repository. ## Citation If you use ISOPRO in your research, please cite it as follows: ``` @software{isopro2024, author = {Jazmia Henry}, title = {ISOPRO: Intelligent Simulation Orchestration for Large Language Models}, year = {2024}, publisher = {GitHub}, journal = {GitHub repository}, howpublished = {\url{https://github.com/iso-ai/isopro}} } ``` ## Contact For questions or support, please open an issue on our [GitHub issue tracker](https://github.com/iso-ai/isopro/issues).