size_categories:10K<n<100K license:cc-by-4.0 region:us
---
dataset_info:
  features:
  - name: fname
    dtype: string
  - name: path
    dtype: string
  - name: label
    struct:
    - name: attributes
      struct:
      - name: timeofday
        dtype: string
      - name: weather
        dtype: string
    - name: labels
      list:
      - name: attributes
        struct:
        - name: drivingConditions
          dtype: string
        - name: laneChange
          dtype: string
        - name: occluded
          dtype: bool
      - name: box2d
        struct:
        - name: x1
          dtype: int64
        - name: x2
          dtype: int64
        - name: y1
          dtype: int64
        - name: y2
          dtype: int64
      - name: category
        dtype: string
      - name: id
        dtype: int64
      - name: manualAttributes
        dtype: bool
      - name: manualShape
        dtype: bool
      - name: poly2d
        list:
        - name: closed
          dtype: bool
        - name: filled
          dtype: bool
        - name: vertices
          sequence:
            sequence: int64
    - name: name
      dtype: string
  - name: img
    dtype: image
  splits:
  - name: train
    num_bytes: 1088252764.96
    num_examples: 25136
  download_size: 1025635407
  dataset_size: 1088252764.96
license: cc-by-4.0
size_categories:
- 10K<n<100K
---
# Dataset Card for sama-drives-california
![Alt text](https://sama-documentation-assets.s3.amazonaws.com/sama-drives-california/samples/samples.png "Samples")

## Dataset Description

- **Homepage:** www.sama.com
- **Point of Contact:** datasets@samasource.org

### Dataset Summary

This is an object detection dataset (bounding boxes and polygons) of **25 136 frames** (848x480 pixels) taken by a dashboard video camera of a car driving in California. 
The frames were captured at 1 FPS, and hence the entire footage covers over 7 hours of driving. 
All but 110 frames contain at least one annotated object (25 026) of interest. 

## Dataset Structure
### Data Instances
The dataset is saved according to the `bdd100k` format described [here](https://doc.bdd100k.com/format.html#segmentation-formats) (no affiliation with Sama).
Frames are named according to the original video they are from, along with the sequence index in that video (1-indexed): **videoNumber-frameIndex.jpg** \
(e.g., 099-002.jpg for the second frame of the 99th video)

`label:id`s are used to denote unique objects, such as a specific vehicle, throughout an entire video, but not across videos. 
The first digits of a `label:id` denote what video it is from (e.g., the `id` 53002 comes from video 53).

Frames were taken from videos that were recorded in a continuous sequence without any time gap in between videos. However, some videos were not included \
in the final dataset either because they contained sensitive information or because they were part of a long sequence when the car was parked and facing a scene of no interest.

The labelling format and different classes supported are described in the section Data Fields below. 

Sample annotation:
```json
{
  "name": "001-019.jpg",
  "attributes": {"weather": "Sunny", "timeofday": "Day"},
  "labels":
  [
    {"category": "Drivable Space", "attributes": {"occluded": true}, "manualShape": true, "manualAttributes": true, "id": 1001, "poly2d": [{"vertices": [[369, 296], [370, 276], [389, 277], [432, 278], [494, 279], [504, 266], [563, 262], [590, 270], [656, 271], [705, 276], [776, 270], [847, 274], [847, 337], [847, 419], [766, 408], [681, 402], [626, 400], [550, 393], [507, 391], [426, 390], [321, 387], [242, 394], [206, 402], [170, 402], [135, 399], [72, 405], [29, 413], [0, 418], [0, 259], [66, 259], [91, 267], [154, 265], [126, 280], [145, 288], [188, 284], [155, 265], [187, 265], [225, 263], [309, 260], [301, 271], [345, 272], [370, 276], [369, 296], [306, 300], [225, 300], [226, 312], [309, 334], [416, 353], [552, 373], [635, 375], [669, 365], [666, 343], [654, 338], [542, 313]], "closed": true, "filled": true}], "box2d": {"x1": 0, "y1": 259, "x2": 847, "y2": 419}},
    {"category": "Vehicles | Truck", "attributes": {"occluded": true}, "manualShape": true, "manualAttributes": true, "id": 1041, "poly2d": [{"vertices": [[708, 247], [692, 247], [688, 251], [687, 258], [687, 265], [709, 265], [714, 265], [713, 255]], "closed": true, "filled": true}], "box2d": {"x1": 687, "y1": 247, "x2": 714, "y2": 265}},
    {"category": "Vehicles | Truck", "attributes": {"occluded": true}, "manualShape": true, "manualAttributes": true, "id": 1043, "poly2d": [{"vertices": [[468, 238], [486, 251], [494, 253], [500, 257], [507, 258], [515, 262], [527, 267], [530, 278], [531, 293], [503, 300], [482, 299], [425, 291], [426, 296], [415, 298], [409, 291], [391, 288], [390, 299], [375, 300], [369, 289], [353, 284], [354, 254], [409, 256], [424, 238]], "closed": true, "filled": true}], "box2d": {"x1": 353, "y1": 238, "x2": 531, "y2": 300}},
    {"category": "Vehicles | Car", "attributes": {"occluded": true}, "manualShape": true, "manualAttributes": true, "id": 1044, "poly2d": [{"vertices": [[560, 256], [539, 253], [541, 257], [553, 264], [561, 271], [563, 288], [568, 288], [584, 290], [596, 288], [599, 277], [595, 271], [589, 267], [577, 264], [570, 260]], "closed": true, "filled": true}], "box2d": {"x1": 539, "y1": 253, "x2": 599, "y2": 290}},
    {"category": "Vehicles | Car", "attributes": {"occluded": true}, "manualShape": true, "manualAttributes": true, "id": 1045, "poly2d": [{"vertices": [[507, 246], [499, 247], [495, 248], [506, 255], [523, 262], [526, 270], [532, 281], [530, 295], [547, 296], [565, 294], [562, 271], [551, 261], [537, 254], [519, 251]], "closed": true, "filled": true}], "box2d": {"x1": 495, "y1": 246, "x2": 565, "y2": 296}},
    {"category": "Vehicles | Car", "attributes": {"occluded": false, "drivingConditions": "Light Traffic"}, "manualShape": true, "manualAttributes": true, "id": 1046, "poly2d": [{"vertices": [[30, 249], [14, 249], [9, 256], [8, 262], [10, 271], [13, 271], [13, 269], [24, 269], [24, 271], [30, 271], [32, 268], [36, 268], [38, 271], [41, 269], [41, 263], [40, 256], [37, 252], [34, 250]], "closed": true, "filled": true}], "box2d": {"x1": 8, "y1": 249, "x2": 41, "y2": 271}}
  ]
}
```

### Data Fields

Each frame contains a label for `timeofday` and `weather`. `Dusk`, `Dawn` and `Twilight` all fall in the same `timeofday` category.

|      timeofday      | weather |
|:--------------------|:--------|
| Day                 | Sunny   |
| Night               | Cloudy  |
| Dusk/Dawn/Twilight  | Rainy   |
|                     | Snowy   |
|                     | Other   |

Bounding boxes are provided for all objects as `box2d`. 
`Vehicles`, `People` and `Areas` are also identified with closed `Polygons` of the type `poly2d`.
`Lanes` are available as `Lines`, that are denoted as open `Polygons` of the type `poly2d`. 
`Traffic Lights` and `Traffic Signs` are only available as `Bounding Boxes`.

| Vehicles (Polygons)   |  People (Polygons)    |  Areas (Polygons)  |  Lanes (Lines)    |  Traffic (Bounding Boxes) |
|:----------------------|:----------------------|:-------------------|:------------------|:--------------------------|
| Car                   |  Pedestrians          |  Drivable Space    |  Current Lane     |  Traffic Lights           |
| Truck                 |                       |                    |  Alternate Lane   |  Traffic Signs            |
| Van                   |                       |                    |  Opposite Lane    |                           |
| SUV                   |                       |                    |                   |                           |
| Bus                   |                       |                    |                   |                           |
| Other LV              |                       |                    |                   |                           |
| Bicycles              |                       |                    |                   |                           |
| Motorbikes            |                       |                    |                   |                           |


The objects above can each be `occluded` (true) or not (false).

`Vehicles` also have a label called `drivingConditions` that denotes the amount of vehicle traffic they are facing. 
Note that this label is not always present.

| drivingConditions (for Vehicles)    |
|:------------------------------------|
| Light Traffic                       |
| Moderate Traffic                    |
| Heavy Traffic                       |


`Lanes` also contain a laneChange label. Note that this label is not always present.
|  laneChange (for Lanes)    |    
|:---------------------------|
|  Current                   |
|  Alternate                 | 
|  Opposite                  |



### Visualize Dataset
To visualize the dataset on the [FiftyOne](https://docs.voxel51.com/) app, download and unzip the following [zip file](https://sama-documentation-assets.s3.amazonaws.com/sama-drives-california/zipped/sama-drives-california.zip) (2.3GB). 

```python
import fiftyone as fo

# <dataset_dir>/
#    labels.json
#    data/
#        001-001.jpg
#        001-002.jpg
#        ...

name = "sama-drives-california"
dataset_dir = "/path/to/dataset"

# Create the dataset
dataset = fo.Dataset.from_dir(
    dataset_dir=dataset_dir,
    dataset_type=fo.types.BDDDataset,
    name=name
)
```

### Dataset in Video Format
This dataset is also available as a video dataset with [FiftyOne](https://docs.voxel51.com/) style label format. You can download a zipped file of the dataset (videos and fiftyone labels) [here](https://sama-documentation-assets.s3.amazonaws.com/sama-drives-california/zipped/sama-drives-california-videos.zip) (1.1GB).

```python
import fiftyone as fo

# <video_dataset_dir>/
#    frames.json
#    metadata.json
#    samples.json
#    data/
#        001.mp4
#        002.mp4
#        ...

name = "sama-drives-california-videos"
dataset_dir = "/path/to/videos-dataset"

# Create the dataset
dataset = fo.Dataset.from_dir(
    dataset_dir=dataset_dir,
    dataset_type=fo.types.FiftyOneDataset,
    name=name
)
```

### Annotations
The dataset was annotated by a team of Sama Associates.
They were instructed to annotate all objects of the classes described in the section *Data Fields* above with the following details:
* Ignore objects under 10 pixels in width or height.
* Annotate with a pixel tolerance of 2 pixels.
* For motorized vehicles, include the mirrors but do not include the antennas.
* For bicycles, include the cyclist.
* For motorbikes, include the rider.
* For traffic lights, place the bounding box around the light fixture but not the pole.
* For traffic signs, do not include the pole or structure.

### Personal and Sensitive Information
All personal and sensitive information has been removed. Vehicle license plates and faces are blurred.

### Other Known Limitations

Objects of interest that were smaller than 10 pixels in width or height were not annotated.

### Licensing Information

(CC BY 4.0) [https://creativecommons.org/licenses/by/4.0/]