task_categories:reinforcement-learning task_categories:robotics annotations_creators:experts-generated language:en license:cc-by-sa-4.0 self-driving robotics navigation region:us
---
license: cc-by-sa-4.0
task_categories:
  - reinforcement-learning
  - robotics
language:
  - en
annotations_creators:
  - experts-generated  
tags:
  - self-driving
  - robotics navigation
pretty_name: FrodoBots (1K)
dataset_info:
  features:
    - name: speed
      dtype: float32
    - name: angular
      dtype: float32
    - name: rpm_1
      dtype: int32
    - name: rpm_2
      dtype: int32
    - name: rpm_3
      dtype: int32
    - name: rpm_4
      dtype: int32
    - name: timestamp
      dtype: float64
---

## Dataset Description

- **Homepage:** https://www.frodobots.com/
- **Hours of tele-operation:** ~1,000 Hrs
- **Dataset Size:** 900+ GB 
- **Point of Contact:** michael.cho@frodobots.com


# FrodoBots 1K Dataset

The **FrodoBots 1K Dataset** is a diverse collection of camera footage, GPS, IMU, audio recordings & control data collected from close to 1,000 hours of tele-operated sidewalk robot driving in 10+ cities (during internal game testing).

- If you're interested in testing out your AI models on your own FrodoBots, we can buy your own unit(s) from our [**online shop**](https://shop.frodobots.com/) (from US$699 per unit).

- If you're interested in testing out your AI models on our existing fleet of FrodoBots (in Singapore, London, Madrid, etc), feel free to contact [Michael Cho](https://twitter.com/micoolcho) to gain access to our SDK.

- If you're interested in playing the game (ie. remotely driving a FrodoBot), you may [**join our Discord to gain access**](https://discord.gg/frodobots) (all you need is a game controller & browser).

> "We deeply appreciate the collaboration of the BlurIt team, who have been instrumental in helping us blur faces and license plates in our videos. Their dedication to pushing the boundaries of AI innovation and research is truly commendable and inspiring. If you're looking to protect privacy by redacting faces or license plates in your images or videos, their website is a valuable resource. Don't hesitate to explore their services to ensure your content remains both informative and anonymous. https://blurit.io/en

| ![Left to right in Stanford.gif](https://cdn-uploads.huggingface.co/production/uploads/63be791624e75d1efdf6ce54/UI7pts68FozvOEbIFHPef.gif) | 
|:--:| 
| *FrodoBots being tested in Stanford University campus* |

## Dataset Summary

There are 7 types of data that are associated with a typical FrodoBots drive, as follows:

1. **Control data**: This includes information about the robot's control inputs captured at a frequency of 10Hz. 

2. **GPS data**: This includes latitude, longitude, and timestamp information for each data point collected during the robot drives at a frequency of 1Hz. 

3. **IMU (Inertial Measurement Unit) data**: This provides 9-degree of freedom sensor data, including acceleration (captured at 100Hz), gyroscope (captured at 1Hz), and compass ride information (captured at 1Hz), along with timestamp data. 

4. **Rear camera video**: This offers video footage captured by the robot's rear-facing camera at 20 frames per second (FPS) with a resolution of 480x480. 

5. **Front camera video**: This consists of video footage captured by the robot's front-facing camera at 20 FPS with a resolution of 800x600. 

6. **Microphone**: This includes audio recordings captured by the robot's microphone, with a sample rate of 48000Hz. 

7. **Speaker**: This incorporates audio recordings of the robot's speaker output (ie. gamer's microphone), also with a sample rate of 48000Hz.

In total, there were more than 4,000 individual driving sessions recorded, with majority of these drives under 10 minutes long, although there were also drives lasting more than an hour.

| ![Histogram of Session Duration.png](https://cdn-uploads.huggingface.co/production/uploads/63be791624e75d1efdf6ce54/vXaVIWxqaBy2x6N9jJnrD.png) | 
|:--:| 
| *Histogram of Session Duration (minutes)* |

These drives were done with FrodoBots in over 10 cities, registering close to 1,000 hours of footages, with Liuzhou being the city with longest cumulative hours of drives.

| ![Cumulative sessions hours by city.png](https://cdn-uploads.huggingface.co/production/uploads/63be791624e75d1efdf6ce54/iAH1bhnc4olXhpbdJHDGv.png)| 
|:--:| 
| *Cumulative sessions hours by city* |

Out of these 1,000 hours of recorded drive, roughly 312 hours have registered speed above 0 Km/Hr. Most of these other duration were either stationary or short pauses during a regular game drives.

| ![Distribution of robot speed.png](https://cdn-uploads.huggingface.co/production/uploads/63be791624e75d1efdf6ce54/e_dh5xDedWbnhlkhJX6O9.png)| 
|:--:| 
| *Distribution of robot speed (excluding zero speed)* |

## About FrodoBots

[**FrodoBots**](https://www.frodobots.com/) is an open-world video driving game where gamers remotely control sidewalk robots to complete missions in different cities (here's a video about the [robot](https://youtu.be/WZy1u7ds1wQ)).

The game objective is to complete the pre-defined navigation missions in single-player mode (see [video](https://www.youtube.com/@frodobots)) or gather digital items in the multiplayer arena mode (see [video](https://youtu.be/2E3lve-Xvak)). 

| ![Shopify_Madrid.gif](https://cdn-uploads.huggingface.co/production/uploads/63be791624e75d1efdf6ce54/EOm26XBMsdvq96qLZxbri.gif) | 
|:--:| 
| *Testing in Madrid* |

| ![Shopify_London.gif](https://cdn-uploads.huggingface.co/production/uploads/63be791624e75d1efdf6ce54/LOuEmUpIJFpVbbd-vNI2u.gif) | 
|:--:| 
| *Testing in London* |

| ![Shopify_Stockholm.gif](https://cdn-uploads.huggingface.co/production/uploads/63be791624e75d1efdf6ce54/qis6jk9p0jMmdwxS9HlGO.gif) | 
|:--:| 
| *Testing in Stockholm* |

| ![Shopify_Wuhan.gif](https://cdn-uploads.huggingface.co/production/uploads/63be791624e75d1efdf6ce54/j96Zl9MxhNG3p3GorF0N3.gif) | 
|:--:| 
| *Testing in Wuhan* |

| ![Shopify_Liuzhou.gif](https://cdn-uploads.huggingface.co/production/uploads/63be791624e75d1efdf6ce54/Dis2qJvK91C4ZH5vuzKhU.gif) | 
|:--:| 
| *Testing in Liuzhou* |

| ![Shopify_Berlin.gif](https://cdn-uploads.huggingface.co/production/uploads/63be791624e75d1efdf6ce54/ewrQwk7qzgfge1aoIHEuV.gif) | 
|:--:| 
| *Testing in Berlin* |

| ![Game controller view.gif](https://cdn-uploads.huggingface.co/production/uploads/63be791624e75d1efdf6ce54/tTLeGQdRHmp36zZ_9ECHV.gif) | 
|:--:| 
| *Game Controller + Browser = Control FrodoBots Anywhere* |

| ![Chatting with people.gif](https://cdn-uploads.huggingface.co/production/uploads/63be791624e75d1efdf6ce54/M3xpC4UlbDRKj-2NhhaiR.gif) | 
|:--:| 
| *Chatting with locals via built-in microphone/speaker* |

| ![Turning on the spot.gif](https://cdn-uploads.huggingface.co/production/uploads/63be791624e75d1efdf6ce54/dxDB6r5jFkUWK198Pt7yV.gif) | 
|:--:| 
| *Zero turning radius = Easy maneuvering* |

| ![Night Driving.gif](https://cdn-uploads.huggingface.co/production/uploads/63be791624e75d1efdf6ce54/sDjijKb0YSUQoe4WWt89d.gif) | 
|:--:| 
| *Night driving test in Palo Alto* |

| ![Driving through rain.gif](https://cdn-uploads.huggingface.co/production/uploads/63be791624e75d1efdf6ce54/EBGM9HicSYMFwVr8RVyiz.gif) | 
|:--:| 
| *Driving through rain* |

| ![Road Crossing in University Ave Palo Alto.gif](https://cdn-uploads.huggingface.co/production/uploads/63be791624e75d1efdf6ce54/rulSOo_unmUEmG9Y_-_Wt.gif) | 
|:--:| 
| *Road crossing in Palo Alto* |

## Motivations for open-sourcing the dataset

The team behind FrodoBots is focused on building an open-world video gaming experience using real-life robots (we call it "robotic gaming"). A by-product of gamers playing the game in real-life is the accompanying dataset that's generated. 

By sharing this dataset with the research community, we hope to see new innovations that can be tested (via our SDK) directly on our fleet of FrodoBots, and then ultimately deployed back into our production environment in order to create a more fun and safer game.  

## **Download**
> **_We are releasing the dataset progressively as we complete the video redaction tasks in the coming weeks. We expect the entire dataset to be made fully available by Oct 2023._**

Download FrodoBots dataset using the link in this [csv file](https://frodobots-1k.s3.ap-southeast-1.amazonaws.com/frodobots-1k_dataset.csv). 

## **Helper code**
We've provided a helpercode.ipynb file that will hopefully serve as a quick-start for researchers to play around with the dataset. 

## **About our partners**

BlurIt

BlurIt is a leading solution for data anonymization. A state-of-the-art AI solution that excels at blurring sensitive details, especially faces and license plates in images and videos. This is vital when personal identifiers are in visual content. BlurIt handles data privacy so that companies can focus on their core business. Our partnership with BlurIt emphasizes the growing need to balance data use with privacy. This partnership underscores our commitment to responsible data handling and privacy protection.

## **Contributions**

The team at FrodoBots Lab created this dataset, including [Michael Cho](https://twitter.com/micoolcho), Sam Cho, Aaron Tung, Niresh Dravin & Jiamin Ho. 