<jupyter_start><jupyter_text>OGTC Hackathon: Data analysis from Tesla vehicle* by: Elon's Little Helpers    * <NAME>    * <NAME>    * <NAME>    * <NAME>    * <NAME>  Problem Setting * By reverse-engineering the Tesla API, Intelligent Plant was able to download the data historian of the usage of a Tesla car (three months). * $\approx182$ variables can be obtained. *  More information of these variable can be found [here](https://tesla-api.timdorr.com) (unofficial). * Some information that can be obtained:    * Screen status    * Battery charge    * Odometer data    * Heating    * Windshields    * Fan    * And much more!  Gestalt tool API* By querying $*car*$, one can visualise the different variables: ![Fig 1.](figs/fig1.png) * You can plot different variables to see the performance across time: ![Fig. 2.](figs/fig2.png) * The spike-down is a moment where no data was recorded. ![Fig. 3.](figs/fig3.png) * All data that falls above 0 is considered screen on (i.e. car on)* 8 is a mode called "dog mode", which monitors an animal inside the car!  Data Analysis **Installing the necessary packages**<jupyter_code>%pip install pandas matplotlib<jupyter_output><empty_output><jupyter_text>**Importing the necessary packages**<jupyter_code>import intelligent_plant.app_store_client as app_store_client
import intelligent_plant.utility as utility
from os.path import expanduser
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
from math import *<jupyter_output><empty_output><jupyter_text>**Getting the data from the repository**<jupyter_code>app_store = app_store_client.AppStoreClient(open(expanduser("~/.access_token"), "r").read())
data_core = app_store.get_data_core_client()
app_store.get_data_core_client()
app_store.get_user_info()
data_core.get_tags("FCBB05262EADC0B147746EE6DFB2B3EA5C272C33C2C5E3FE8F473D85529461CA.Edge Historian")
tags = {
    "FCBB05262EADC0B147746EE6DFB2B3EA5C272C33C2C5E3FE8F473D85529461CA.Edge Historian": [
        "StevesCar..response.vehicle_state.odometer",
        "StevesCar..response.charge_state.battery_level",
        "StevesCar..response.climate_state.outside_temp",
        "StevesCar..response.vehicle_state.center_display_state",
        "StevesCar..response.drive_state.speed",
        "StevesCar..response.climate_state.seat_heater_right"
    ]
}<jupyter_output><empty_output><jupyter_text>**Data filtering*** All interpolated data from 10 days ago in 3-minute intervals.<jupyter_code>resp = data_core.get_processed_data(tags, "*-10d", "*", "3m", "interp")<jupyter_output><empty_output><jupyter_text>**Plotting odometer data**<jupyter_code>df = utility.query_result_to_data_frame(resp)
plt.plot(df["StevesCar..response.vehicle_state.odometer"])
plt.xlabel('Data Index')
plt.ylabel('Odometer miles')
plt.show()<jupyter_output><empty_output><jupyter_text>**Creation of a *new* differential dataset**<jupyter_code>import pandas as pd
type(df.loc[0].TimeStamp)
df.loc[1].TimeStamp-df.loc[0].TimeStamp
df_new=[]
for i in range(1,len(df)-1):
    df_new.append(df.loc[i]-df.loc[i-1])
df_new =pd.DataFrame(df_new)<jupyter_output><empty_output><jupyter_text>* This dataset could be used for time series analysis purposes. **Distance travelled**<jupyter_code>plt.plot(df_new["StevesCar..response.vehicle_state.odometer"])
plt.ylabel('Distance travelled in time intervaled (miles)')
plt.xlabel('Data Index')<jupyter_output><empty_output><jupyter_text>* By plotting the differential of the odometer, we can see the distance travelled (in miles) for every entry. **Plotting the differential data on the battery level**<jupyter_code>plt.plot(df_new["StevesCar..response.charge_state.battery_level"])
plt.ylabel('Change in Battery % in time interval')
plt.xlabel('Data Index')<jupyter_output><empty_output><jupyter_text>* Negative entries occur when battery discharges and vice versa. **Plotting efficiency*** We created our own efficiency metric as the ratio between battery level differential and odometer differential.<jupyter_code>df_new['Efficiency'] = df_new["StevesCar..response.charge_state.battery_level"]/df_new["StevesCar..response.vehicle_state.odometer"]
plt.plot(df_new['Efficiency'], 'bo')
plt.ylim([-1, 0])
plt.ylabel('Battery % drop per mile')
plt.xlabel('Data Index') 
plt.show()<jupyter_output><empty_output><jupyter_text>* Data seems to be random as for each entry, we obtain a very spread distribution.* We only want to see efficiency from 0 to -1, as others would be considered outliers    * Very huge or very little efficiencies don't make sense, as you cannot have more battery level than actual mileage. **Another view of the data**<jupyter_code>df_new.replace(inf, 0)
df_new.replace(-inf, 0)
plt.plot(df_new['Efficiency'], 'bo')<jupyter_output><empty_output><jupyter_text>* Here we can see the presence of outliers. **Histogram of efficiency metric**<jupyter_code>eff=np.array(df_new["Efficiency"])
sh=np.array(df_new["StevesCar..response.climate_state.seat_heater_right"])
# Limit the efficiency between 0 and -3
eff2= eff[np.where(eff>-3)]
eff3 = eff2[np.where(eff2<0)]
plt.hist(eff3,bins=20)
plt.xlabel('Battery % drop per mile')
plt.ylabel('Number')
plt.show()<jupyter_output>/usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py:7: RuntimeWarning: invalid value encountered in greater
  import sys<jupyter_text>**Efficiency vs other variables*** Here we compare our efficiency metric against other variables, such as speed or outside temp.* Purpose: Try to find correlations.* Speed may show some correlation, outside temperature does not.<jupyter_code>temp = np.array(df["StevesCar..response.climate_state.outside_temp"])[0:-2]
speed = np.array(df["StevesCar..response.drive_state.speed"])[0:-2]

temp2 = temp[np.where(eff>-3)]
temp3 = temp2[np.where(eff2<0)]
speed2 = speed[np.where(eff>-3)]
speed3 = speed2[np.where(eff2<0)]

plt.plot(speed3, eff3, 'bo')
plt.xlim([0, 60])
plt.ylabel('Battery % drop per mile')
plt.xlabel('Speed mph')
plt.show()

plt.plot(temp3, eff3, 'ro')
plt.ylabel('Battery % drop per mile')
plt.xlabel('Outside Temp degC')
plt.show()<jupyter_output>/usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py:4: RuntimeWarning: invalid value encountered in greater
  after removing the cwd from sys.path.
/usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py:6: RuntimeWarning: invalid value encountered in greater<jupyter_text>**Measuring the effect of heating the driver's seat*** We were also able to isolate this measurement and compare efficiency w.r.t. right seat heated/unheated.* There does not seem to be a change!<jupyter_code>sh = np.array(df["StevesCar..response.climate_state.seat_heater_right"])[0:-2]

eff_without_sh = eff[np.where(sh==0)]
eff_with_sh = eff[np.where(sh>0)]

print(len(eff_without_sh))
print(len(eff_with_sh))

eff2_without= eff_without_sh[np.where(eff_without_sh>-2)]
eff3_without = eff2_without[np.where(eff2_without<0)]

speed_without = speed[np.where(sh==0)]
speed2_without = speed_without[np.where(eff_without_sh>-2)]
speed3_without = speed2_without[np.where(eff2_without<0)]

eff2_with= eff_with_sh[np.where(eff_with_sh>-2)]
eff3_with = eff2_with[np.where(eff2_with<0)]

speed_with = speed[np.where(sh>0)]
speed2_with = speed_with[np.where(eff_with_sh>-2)]
speed3_with = speed2_with[np.where(eff2_with<0)]

plt.plot(speed3_without, eff3_without, 'bo', label='Without Seat Heater')
plt.plot(speed3_with, eff3_with, 'ro',label='With Seat Heater')
plt.xlim([0, 60])
plt.xlabel('Speed mph')
plt.ylabel('Battery % drop per mile')
plt.legend()
plt.show()<jupyter_output>2796
1922
{'fraction_non_alphanumeric': 0.12703885426336028, 'fraction_numerical': 0.025062988993502188, 'mean_word_length': 6.459940652818991, 'pattern_counts': {'":': 1, '<': 62, '<?xml version=': 0, '>': 64, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '13', 'dir': 'jupyter-structured-clean-dedup', 'id': '8204', 'license': 'Unlicense', 'n_tokens_mistral': 2743, 'n_tokens_neox': 2565, 'n_words': 601, 'path': 'ELH_Presentation+Code.ipynb', 'repo_name': 'StarlingProjects/Ellons_Little_Helpers', 'size': '7591'}