<jupyter_start><jupyter_text>"Reinforcement Learning - Hill Climbing"> "Reinforcement Learning Balancing Cartpole with Hill Climbing Policy"- author: <NAME>- toc: true - badges: true- comments: true- image: images/hill.jpg- categories: [ jupyter,Reinforcement-Learning,CartPole,Hill-climbing]<jupyter_code>import gym
import numpy as np<jupyter_output><empty_output><jupyter_text>Hill Climbing Algorithm    function HILL-CLIMBING(problem) returns a state that is a local maximum     current ← problem.INITIAL-STATE     loop do       neighbor ← a highest-valued successor of current       if VALUE(neighbour) ≤ VALUE(current) then return current       current ← neighbor<jupyter_code>class HillClimbingPolicy:
    def __init__(self,env,init_weight=None):
        self.env = env
        self.parameters = np.random.random(env.observation_space.shape)# parameters
        if init_weight is not None:
            self.parameters = init_weight
        self.noise = 0.001 # Noise value
        self.noise_factor = 3 # Noise factor
        self.best_reward = -1 # Best reward till now
        self.max_steps = 300 # Max steps in each episode
    
    def get_action(self,observation):
        # get action for current observation
        action = self.parameters @ observation
        if action > 0:
            action = 1
        else:
            action = 0
        
        return action

    
    def simulate(self,params):
        observation = self.env.reset()
        reward_sum = 0
        
        for step in range(self.max_steps):
            action = params @ observation
            if action > 0:
                action = 1
            else:
                action = 0

            observation, reward, done, info = env.step(action)
            reward_sum += reward
            if done:
                break
                
        return reward_sum
        
        
    def fit(self,):
        new_params = self.parameters + (self.noise * np.random.random(env.observation_space.shape))
        new_reward = self.simulate(new_params)
        
        # if new reward is more than the current best we update parameters else we increse the noise
        if new_reward > self.best_reward:
            self.best_reward = new_reward
            self.parameters = new_params
            self.noise = self.noise / self.noise_factor
            
        else:
            self.noise = self.noise * self.noise_factor
            
        return new_reward<jupyter_output><empty_output><jupyter_text>Training<jupyter_code>env = gym.make("CartPole-v0")
hill_climbing = HillClimbingPolicy(env)
for episode in range(10000):
    new_reward = hill_climbing.fit()
    print(f"Reward episode {episode} : {new_reward}")<jupyter_output>Reward episode 0 : 57.0
Reward episode 1 : 95.0
Reward episode 2 : 129.0
Reward episode 3 : 104.0
Reward episode 4 : 51.0
Reward episode 5 : 78.0
Reward episode 6 : 57.0
Reward episode 7 : 84.0
Reward episode 8 : 116.0
Reward episode 9 : 104.0
Reward episode 10 : 74.0
Reward episode 11 : 153.0
Reward episode 12 : 114.0
Reward episode 13 : 86.0
Reward episode 14 : 97.0
Reward episode 15 : 57.0
Reward episode 16 : 10.0
Reward episode 17 : 88.0
Reward episode 18 : 26.0
Reward episode 19 : 200.0
Reward episode 20 : 200.0
Reward episode 21 : 88.0
Reward episode 22 : 71.0
Reward episode 23 : 9.0
Reward episode 24 : 170.0
Reward episode 25 : 30.0
Reward episode 26 : 200.0
Reward episode 27 : 158.0
Reward episode 28 : 64.0
Reward episode 29 : 10.0
Reward episode 30 : 200.0
Reward episode 31 : 10.0
Reward episode 32 : 10.0
Reward episode 33 : 98.0
Reward episode 34 : 23.0
Reward episode 35 : 9.0
Reward episode 36 : 200.0
Reward episode 37 : 83.0
Reward episode 38 : 114.0
Reward episode 39 : 46.[...]<jupyter_text>Test With Final Saved weights<jupyter_code>render = True
filename = "./hill_climbimg_params_final.npy"
parameters = np.load(filename)
env = gym.make("CartPole-v0")
hill_climbing = HillClimbingPolicy(env, parameters)
while True:
    observation = env.reset()
    reward_sum = 0
    done = False

    while not done:
        action = hill_climbing.get_action(observation)
        observation, reward, done, info = env.step(action)
        if render :
            env.render()
        reward_sum += reward
    if reward_sum >= 200:
        print("Goal Reached!!",reward_sum)
        break<jupyter_output>Goal Reached!! 200.0
{'fraction_non_alphanumeric': 0.0831424644983967, 'fraction_numerical': 0.05313788364635822, 'mean_word_length': 2.553295362082994, 'pattern_counts': {'":': 0, '<': 16, '<?xml version=': 0, '>': 21, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '531681', 'license': 'Apache-2.0', 'n_tokens_mistral': 1558, 'n_tokens_neox': 1347, 'n_words': 410, 'path': '_notebooks/2020-10-09-Hill Climbing.ipynb', 'repo_name': 'isbhargav/portfolio', 'size': '4371'}