metadata

license: apache-2.0
language:
  - en
metrics:
  - accuracy
  - bleu
  - perplexity
tags:
  - ai
  - agents
  - rl
  - reinforcement
  - learning

ISOPro: Pro Tools for Intelligent Simulation Orchestration for Large Language Models

ISOPRO is a powerful and flexible Python package designed for creating, managing, and analyzing simulations involving Large Language Models (LLMs). It provides a comprehensive suite of tools for reinforcement learning, conversation simulations, adversarial testing, custom environment creation, and advanced orchestration of multi-agent systems.

Features

Custom Environment Creation: Easily create and manage custom simulation environments for LLMs
Conversation Simulation: Simulate and analyze conversations with AI agents using various user personas
Adversarial Testing: Conduct adversarial simulations to test the robustness of LLM-based systems
Reinforcement Learning: Implement and experiment with RL algorithms in LLM contexts
Workflow Automation: Learn and replicate UI workflows from video demonstrations
Car Environment Simulation: Train and evaluate RL agents in driving scenarios
Utility Functions: Analyze simulation results, calculate LLM metrics, and more
Flexible Integration: Works with popular LLM platforms like OpenAI's GPT models, Claude (Anthropic), and Hugging Face models
Orchestration Simulation: Manage and execute complex multi-agent simulations with different execution modes

Installation

You can install isopro using pip:

pip install isopro

For workflow simulation features, ensure you have the required dependencies:

pip install opencv-python numpy torch stable-baselines3 gymnasium tqdm

If you plan to use Claude capabilities:

export ANTHROPIC_API_KEY=your_api_key_here

Examples

To explore IsoPro examples, visit https://github.com/iso-ai/isopro_examples.

Usage

Adversarial Simulation

Test the robustness of AI models against adversarial attacks.

from isopro.adversarial_simulation import AdversarialSimulator, AdversarialEnvironment
from isopro.agents.ai_agent import AI_Agent
import anthropic

class ClaudeAgent(AI_Agent):
    def __init__(self, name):
        super().__init__(name)
        self.client = anthropic.Anthropic(api_key=os.getenv("ANTHROPIC_API_KEY"))

    def run(self, input_data):
        response = self.client.messages.create(
            model="claude-3-opus-20240229",
            max_tokens=100,
            messages=[{"role": "user", "content": input_data['text']}]
        )
        return response.content[0].text

# Create the AdversarialEnvironment
adv_env = AdversarialEnvironment(
    agent_wrapper=ClaudeAgent("Claude Agent"),
    num_adversarial_agents=2,
    attack_types=["textbugger", "deepwordbug"],
    attack_targets=["input", "output"]
)

# Set up the adversarial simulator
simulator = AdversarialSimulator(adv_env)

# Run the simulation
input_data = ["What is the capital of France?", "How does photosynthesis work?"]
simulation_results = simulator.run_simulation(input_data, num_steps=1)

Conversation Simulation

Simulate conversations between an AI assistant and various user personas.

from isopro.conversation_simulation.conversation_simulator import ConversationSimulator

# Initialize the ConversationSimulator
simulator = ConversationSimulator(
    ai_prompt="You are an AI assistant created to be helpful, harmless, and honest. You are a customer service agent for a tech company. Respond politely and professionally."
)

# Run a simulation with a predefined persona
conversation_history = simulator.run_simulation("upset", num_turns=3)

# Run a simulation with a custom persona
custom_persona = {
    "name": "Techie Customer",
    "characteristics": ["tech-savvy", "impatient", "detail-oriented"],
    "message_templates": [
        "I've tried rebooting my device, but the error persists. Can you help?",
        "What's the latest update on the cloud service outage?",
        "I need specifics on the API rate limits for the enterprise plan."
    ]
}

custom_conversation = simulator.run_custom_simulation(**custom_persona, num_turns=3)

Workflow Simulation

Automate UI workflows by learning from video demonstrations.

from isopro.workflow_simulation import WorkflowAutomation, AgentConfig

# Basic workflow automation
automation = WorkflowAutomation(
    video="path/to/workflow.mp4",
    config="config.json",
    output="output_dir",
    logs="logs_dir"
)
automation.run()

# Advanced configuration
agent_config = AgentConfig(
    learning_rate=3e-4,
    pretrain_epochs=10,
    use_demonstration=True,
    use_reasoning=True
)

simulator = WorkflowSimulator(
    video_path="path/to/video.mp4",
    agent_config=agent_config,
    viz_config=visualization_config,
    validation_config=validation_config,
    output_dir="output"
)

training_results = simulator.train_agents()
evaluation_results = simulator.evaluate_agents()

Car Reinforcement Learning

Train and evaluate RL agents in driving scenarios.

from isopro.car_simulation import CarRLEnvironment, LLMCarRLWrapper, CarVisualization

# Create the car environment with LLM integration
env = CarRLEnvironment()
llm_env = LLMCarRLWrapper(env)

# Initialize visualization
viz = CarVisualization(env)

# Train and visualize
observation = llm_env.reset()
for step in range(1000):
    action = llm_env.get_action(observation)
    observation, reward, done, info = llm_env.step(action)
    viz.render(observation)
    
    if done:
        observation = llm_env.reset()

Reinforcement Learning with LLM

Integrate Large Language Models with reinforcement learning environments.

import gymnasium as gym
from isopro.rl.rl_agent import RLAgent
from isopro.rl.rl_environment import LLMRLEnvironment
from stable_baselines3 import PPO
from isopro.rl.llm_cartpole_wrapper import LLMCartPoleWrapper

agent_prompt = """You are an AI trained to play the CartPole game. 
Your goal is to balance a pole on a moving cart for as long as possible. 
You will receive observations about the cart's position, velocity, pole angle, and angular velocity. 
Based on these, you should decide whether to move the cart left or right."""

env = LLMCartPoleWrapper(agent_prompt, llm_call_limit=100, api_key=os.getenv("ANTHROPIC_API_KEY"))
rl_agent = RLAgent("LLM_CartPole_Agent", env, algorithm='PPO')

# Train the model
model.learn(total_timesteps=2)

# Test the model
obs, _ = env.reset()
for _ in range(1000):
    action, _ = model.predict(obs, deterministic=True)
    obs, reward, done, _, _ = env.step(action)
    if done:
        obs, _ = env.reset()

AI Orchestration

Orchestrate multiple AI agents to work together on complex tasks.

from isopro.orchestration_simulation import OrchestrationEnv
from isopro.orchestration_simulation.components import LLaMAAgent, AnalysisAgent, WritingAgent
from isopro.orchestration_simulation.evaluator import Evaluator

# Create the orchestration environment
env = OrchestrationEnv()

# Add agents to the environment
env.add_component(LLaMAAgent("Research", "conduct thorough research on the impact of artificial intelligence on job markets"))
env.add_component(AnalysisAgent("Analysis"))
env.add_component(WritingAgent("Writing"))

# Define the task
task = "Prepare a comprehensive report on the impact of artificial intelligence on job markets in the next decade."

# Run simulations in different modes
modes = ['parallel', 'sequence', 'node']
results = {}

for mode in modes:
    result = env.run_simulation(mode=mode, input_data={'task': task, 'run_order': 'first'})
    results[mode] = result

# Evaluate the results
evaluator = Evaluator()
best_mode = evaluator.evaluate(results)
print(f"The best execution mode for this task was: {best_mode}")

Documentation

For more detailed information on each module and its usage, please refer to the full documentation.

Examples

The isopro examples repository contains Jupyter notebooks with detailed examples:

adversarial_example.ipynb: Demonstrates adversarial testing of language models
conversation_simulation_example.ipynb: Shows how to simulate conversations with various user personas
workflow_automation_example.ipynb: Illustrates automated UI workflow learning
car_rl_example.ipynb: Demonstrates car environment training scenarios
run_cartpole_example.ipynb: Illustrates the integration of LLMs with reinforcement learning
orchestrator_example.ipynb: Provides a tutorial on using the AI orchestration capabilities

Troubleshooting

IsoPro 0.1.6 may have some issues downloading dependencies. If you encounter any, simply run

pip install isopro stable-baselines3 gymnasium isozero iso-adverse tiktoken

to resolve any missing package errors.

Contributing

We welcome contributions! Please see our Contributing Guide for more details.

License

This project is licensed under the MIT License - see the LICENSE file for details.

Support

If you encounter any problems or have any questions, please open an issue on our GitHub repository.

Citation

If you use ISOPRO in your research, please cite it as follows:

@software{isopro2024,
  author = {Jazmia Henry},
  title = {ISOPRO: Intelligent Simulation Orchestration for Large Language Models},
  year = {2024},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/iso-ai/isopro}}
}

Contact

For questions or support, please open an issue on our GitHub issue tracker.