File size: 2,350 Bytes
31fc7e1 744ba87 31fc7e1 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 |
# CLIP-Based Break Dance Move Classifier
A deep learning model for classifying break dance moves using CLIP (Contrastive Language-Image Pre-Training) embeddings. The model is fine-tuned on break dance videos to classify different power moves including windmills, halos, swipes, and baby mills.
## Features
- Video-based classification using CLIP embeddings
- Multi-frame temporal analysis
- Configurable frame sampling and data augmentation
- Real-time inference using Cog
- Misclassification analysis tools
- Hyperparameter tuning support
## Setup
```bash
# Install dependencies
pip install -r requirements.txt
# Install Cog (if not already installed)
curl -o /usr/local/bin/cog -L https://github.com/replicate/cog/releases/latest/download/cog_`uname -s`_`uname -m`
chmod +x /usr/local/bin/cog
```
## Cog
build the image
```bash
cog build --separate-weights
```
push the image
```bash
cog push
```
## Training
```bash
# Run training with default configuration
python scripts/train.py
# Run hyperparameter tuning
python scripts/hyperparameter_tuning.py
```
## Inference
```bash
# Using Cog for inference
cog predict -i video=@path/to/your/video.mp4
# Using standard Python script
python scripts/inference.py --video path/to/your/video.mp4
```
## Analysis
```bash
# Generate misclassification report
python scripts/visualization/miscalculations_report.py
# Visualize model performance
python scripts/visualization/visualize.py
```
## Project Structure
```
clip/
├── src/ # Source code
│ ├── data/ # Dataset and data processing
│ ├── models/ # Model architecture
│ └── utils/ # Utility functions
├── scripts/ # Training and inference scripts
│ └── visualization/ # Visualization tools
├── config/ # Configuration files
├── runs/ # Training runs and checkpoints
├── cog.yaml # Cog configuration
└── requirements.txt # Python dependencies
```
## Model Architecture
- Base: CLIP ViT-Large/14
- Custom temporal pooling layer
- Fine-tuned vision encoder (last 3 layers)
- Output: 4-class classifier
## License
[Your License Here]
## Citation
If you use this model in your research, please cite:
```bibtex
[Your Citation Here]
``` |