<jupyter_start><jupyter_text>Facial MaskWe provide the facial mask in UV space to extract the facial part from the full head TU-model. Here we show how to extract the facial region using the mask. Firstly, make sure the environment and sample data have been prepared following [README-toolkit](https://github.com/zhuhao-nju/facescape/blob/master/toolkit/README.md).  (1) For TU models, Bilinear model v1.0/1.2/1.3The TU model, bilinear model ver1.0/1.2/1.3 use the [facial_mask_v10.png](https://github.com/zhuhao-nju/facescape_debug/blob/master/toolkit/predef/facial_mask_v10.png). The 'facial_mask_v10' is not strictly symmetric. Here is a demo:<jupyter_code>import trimesh, cv2, PIL, os, PIL.ImageOps
import numpy as np
import src.renderer as renderer
import src.camera as camera
import src.utility as util

# read model
model = trimesh.load_mesh("../samples/sample_tu_model/1_neutral.obj", process = False)

# rotate around y-axis
model.vertices = util.rotate_verts_y(model.vertices, 30)

# set material
model.visual.material.diffuse = np.array([255, 255, 255, 255], dtype=np.uint8)

# set K Rt
K = np.array([[400, 0 , 100],
              [0, 400, 100],
              [0, 0, 1]])
Rt = np.array([[1, 0 , 0, 0],
               [0, -1, 0, 0],
               [0, 0, -1, 600]])

# render img
_, rend_img = renderer.render_cvcam(model, K, Rt, scale=1.0, 
                                    rend_size=(200, 200), flat_shading=True)

# read facial seg
facial_seg = PIL.ImageOps.invert(PIL.Image.open("./predef/facial_mask_v10.png"))

# render mask
model.visual.material.image = facial_seg
_, mask = renderer.render_cvcam(model, K, Rt, scale=1.0, 
                                rend_size=(200, 200), flat_shading=True)

# apply mask
rend_img_m = rend_img.copy()
rend_img_m[mask>200] = 255

util.show_img_arr(np.concatenate((rend_img, rend_img_m), 1), bgr_mode = True)<jupyter_output><empty_output><jupyter_text>left: The original full head; right: the extracted facial region.  (2) For Bilinear model v1.6 and latter versionsBilinear model v1.6 and later versions use the 'facial_mask_v16'.  [facial_mask_v16.png](https://github.com/zhuhao-nju/facescape_debug/blob/master/toolkit/predef/facial_mask_v16.png) is strictly symmetric in X-axis. Here is the demo:<jupyter_code>import trimesh, cv2, PIL, os, PIL.ImageOps
import numpy as np
import src.renderer as renderer
import src.camera as camera
import src.utility as util

# read model
model = trimesh.load_mesh("../samples/bm_v1.6_sample.obj", process = False)

# rotate around y-axis
model.vertices = util.rotate_verts_y(model.vertices, 30)

# set material
model.visual.material.diffuse = np.array([255, 255, 255, 255], dtype=np.uint8)

# set K Rt
K = np.array([[400, 0 , 100],
              [0, 400, 100],
              [0, 0, 1]])
Rt = np.array([[1, 0 , 0, 0],
               [0, -1, 0, 0],
               [0, 0, -1, 600]])

# render img
_, rend_img = renderer.render_cvcam(model, K, Rt, scale=1.0, 
                                    rend_size=(200, 200), flat_shading=False)

# read facial seg
facial_mask = PIL.ImageOps.invert(PIL.Image.open("./predef/facial_mask_v16.png"))

# render mask
model.visual.material.image = facial_mask
_, mask = renderer.render_cvcam(model, K, Rt, scale=1.0, 
                                rend_size=(200, 200), flat_shading=True)

# apply mask
rend_img_m = rend_img.copy()
rend_img_m[mask>200] = 255

util.show_img_arr(np.concatenate((rend_img, rend_img_m), 1), bgr_mode = True)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1290414878397711, 'fraction_numerical': 0.050643776824034335, 'mean_word_length': 3.4535031847133757, 'pattern_counts': {'":': 0, '<': 9, '<?xml version=': 0, '>': 11, 'https://': 3, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '541166', 'license': 'MIT', 'n_tokens_mistral': 1403, 'n_tokens_neox': 1256, 'n_words': 314, 'path': 'toolkit/demo_mask.ipynb', 'repo_name': 'lelechen63/idinvert_pytorch', 'size': '3495'}