whaohan commited on
Commit
160cf28
β€’
1 Parent(s): ada4b81

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +14 -111
README.md CHANGED
@@ -1,111 +1,14 @@
1
- # Scaling Mesh Generation via Compressive Tokenization
2
-
3
- ### [Project Page](https://whaohan.github.io/bpt) | [Paper](https://arxiv.org/abs/2411.07025) | [Weight](https://huggingface.co/whaohan/bpt/tree/main)
4
-
5
-
6
- ## πŸ“‘ Open-source Plan
7
-
8
- - [x] Inference conditioned on point cloud
9
- - [x] Checkpoints
10
- - [x] Evaluation metrics
11
- - [ ] Inference conditioned on images
12
- - [ ] Training
13
-
14
-
15
- ## **Abstract**
16
- <p align="center">
17
- <img src="./assets/teaser.png" height=450>
18
- </p>
19
-
20
- We propose a compressive yet effective mesh representation, Blocked and Patchified Tokenization (BPT), facilitating the generation of meshes exceeding 8k faces. BPT compresses mesh sequences by employing block-wise indexing and patch aggregation, reducing their length by approximately 75% compared to the original sequences. This compression milestone unlocks the potential to utilize mesh data with significantly more faces, thereby enhancing detail richness and improving generation robustness. Empowered with the BPT, we have built a foundation mesh generative model training on scaled mesh data to support flexible control for point clouds and images. Our model demonstrates the capability to generate meshes with intricate details and accurate topology, achieving SoTA performance on mesh generation and reaching the level for direct product usage.
21
-
22
- ## πŸŽ‰ **Blocked and Patchified Tokenization (BPT)**
23
-
24
- <p align="center">
25
- <img src="assets/BPT.png" height=300>
26
- </p>
27
-
28
-
29
- ## Get Started
30
-
31
- #### Begin by cloning the repository:
32
-
33
- ```shell
34
- git clone https://github.com/whaohan/bpt.git
35
- cd bpt
36
- ```
37
-
38
- #### Installation Guide for Linux
39
-
40
-
41
- Install the packages in `requirements.txt`. The code is tested under CUDA version 12.1 and python 3.9.
42
-
43
- ```bash
44
- conda create -n bpt python=3.9
45
- conda activate bpt
46
- pip install torch==2.1.2 torchvision==0.16.2 --index-url https://download.pytorch.org/whl/cu121
47
- pip install -r requirements.txt
48
- ```
49
-
50
-
51
- #### Download Pretrained Models
52
-
53
- The models are available at [huggingface](https://huggingface.co/whaohan/bpt/tree/main).
54
- Currently, we resealse a lite version of model with the point-encoder finetuned from [Michelangelo](https://github.com/NeuralCarver/Michelangelo).
55
-
56
- To download the model, first install the huggingface-cli. (Detailed instructions are available [here](https://huggingface.co/docs/huggingface_hub/guides/cli).)
57
-
58
- ```shell
59
- python3 -m pip install "huggingface_hub[cli]"
60
- ```
61
-
62
- Then download the model using the following commands:
63
-
64
- ```shell
65
- mkdir weights
66
- huggingface-cli download whaohan/bpt --local-dir ./weights
67
- ```
68
-
69
- #### Inference conditioned on point clouds
70
- For text to 3d generation, we supports bilingual Chinese and English, you can use the following command to inference.
71
- ```python
72
- python main.py \
73
- --config 'config/BPT-open-8k-8-16.yaml' \
74
- --model_path /path/to/model/ckpt \
75
- --output_path output/ \
76
- --batch_size 1 \
77
- --temperature 0.5 \
78
- --input_type mesh \
79
- --input_dir /path/to/your/dense/meshes
80
- ```
81
- It requires ~12GB VRAM to run with fp16 precision. It takes averagely 2mins to generate a single mesh.
82
-
83
-
84
- #### Evaluation
85
-
86
- ```bash
87
- python metrics.py \
88
- --input_dir /path/to/dense/meshes \
89
- --output_dir /path/to/output/meshes
90
- ```
91
-
92
- ### Acknowledgement
93
-
94
- - [MeshGPT](https://github.com/lucidrains/meshgpt-pytorch)
95
- - [PivotMesh](https://github.com/whaohan/pivotmesh)
96
- - [Michelangelo](https://github.com/NeuralCarver/Michelangelo)
97
- - [MeshAnything](https://github.com/buaacyw/MeshAnythingV2/)
98
- - [MeshXL](https://github.com/OpenMeshLab/MeshXL/)
99
-
100
-
101
- ## Citation
102
-
103
- If you found this repository helpful, please cite our report:
104
- ```bibtex
105
- @article{weng2024scaling,
106
- title={Scaling Mesh Generation via Compressive Tokenization},
107
- author={Haohan Weng and Zibo Zhao and Biwen Lei and Xianghui Yang and Jian Liu and Zeqiang Lai and Zhuo Chen and Yuhong Liu and Jie Jiang and Chunchao Guo and Tong Zhang and Shenghua Gao and C. L. Philip Chen},
108
- journal={arXiv preprint arXiv:2411.07025},
109
- year={2024}
110
- }
111
- ```
 
1
+ ---
2
+ title: Scaling Mesh Generation via Compressive Tokenization
3
+ emoji: πŸš€
4
+ colorFrom: blue
5
+ colorTo: green
6
+ sdk: gradio
7
+ sdk_version: 5.6.0
8
+ app_file: app.py
9
+ pinned: false
10
+ license: mit
11
+ short_description: demo for BPT
12
+ ---
13
+
14
+ Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference