update files

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+assets/demoImage.png filter=lfs diff=lfs merge=lfs -text

Dockerfile

@@ -0,0 +1,34 @@
+FROM ubuntu:22.04
+
+# apt换源，安装pip
+RUN echo "==> 换成阿里源，并更新..."  && \
+    sed -i s@/archive.ubuntu.com/@/mirrors.aliyun.com/@g /etc/apt/sources.list  && \
+    sed -i s@/security.ubuntu.com/@/mirrors.aliyun.com/@g /etc/apt/sources.list  && \
+    apt-get clean  && \
+    apt-get update
+
+# 安装python3.10
+RUN apt-get install -y python3 curl && \
+    curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py  && \
+    python3 get-pip.py && \
+    pip3 install -U pip && \
+    pip3 config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple
+
+# 安装ffmpeg等库
+RUN apt-get install libpython3.10-dev ffmpeg libgl1-mesa-glx libglib2.0-0 cmake -y && \
+    pip3 install --no-cache-dir cmake
+
+WORKDIR /app
+
+COPY . .
+
+RUN pip3 install -r requirements.txt
+
+RUN echo "==> Clean up..."  && \
+    rm -rf ~/.cache/pip
+
+# 指定工作目录
+
+EXPOSE 8080
+
+ENTRYPOINT ["python3", "deploy_api.py"]

README.md

@@ -1,13 +1,146 @@
+<div align="center">
+<h1>HivisionIDPhoto</h1>
+
+[English](README_EN.md) / 中文
+
+[![GitHub](https://img.shields.io/static/v1?label=GitHub&message=GitHub&color=black)](https://github.com/xiaolin199912/HivisionIDPhotos)
+[![SwanHub Demo](https://img.shields.io/static/v1?label=在线体验&message=SwanHub%20Demo&color=blue)](https://swanhub.co/ZeYiLin/HivisionIDPhotos/demo)
+[![zhihu](https://img.shields.io/static/v1?label=知乎&message=知乎&color=blue)](https://zhuanlan.zhihu.com/p/638254028)
+
+<img src="assets/demoImage.png" width=900>
+</div>
+
+
+# 🤩项目更新
+- 在线体验: [![SwanHub Demo](https://img.shields.io/static/v1?label=Demo&message=SwanHub%20Demo&color=blue)](https://swanhub.co/ZeYiLin/HivisionIDPhotos/demo)
+- 2023.12.1: 更新**API部署（基于fastapi）**
+- 2023.6.20: 更新**预设尺寸菜单**
+- 2023.6.19: 更新**排版照**
+- 2023.6.13: 更新**中心渐变色**
+- 2023.6.11: 更新**上下渐变色**
+- 2023.6.8: 更新**自定义尺寸**
+- 2023.6.4: 更新**自定义底色、人脸检测Bug通知**
+- 2023.5.10: 更新**不改尺寸只换底**
+
+# Overview
+
+> 🚀谢谢你对我们的工作感兴趣。您可能还想查看我们在图像领域的其他成果，欢迎来信:[email protected].
+
+HivisionIDPhoto旨在开发一种实用的证件照智能制作算法。
+
+它利用一套完善的模型工作流程，实现对多种用户拍照场景的识别、抠图与证件照生成。
+
+
+**HivisionIDPhoto可以做到:**
+
+1. 轻量级抠图
+2. 根据不同尺寸规格生成不同的标准证件照、六寸排版照
+3. 美颜（waiting）
+4. 智能换正装（waiting）
+
+<div align="center">
+<img src="assets/gradio-image.jpeg" width=900>
+</div>
+
+
 ---
-title: HivisionIDPhotos
-emoji: 🐢
-colorFrom: blue
-colorTo: indigo
-sdk: gradio
-sdk_version: 4.42.0
-app_file: app.py
-pinned: false
-license: apache-2.0
----
 
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+如果HivisionIDPhoto对你有帮助，请star这个repo或推荐给你的朋友，解决证件照应急制作问题！
+
+<br>
+
+# 🔧环境安装与依赖
+
+- Python >= 3.7（项目主要测试在python 3.10）
+- onnxruntime
+- OpenCV
+- Option: Linux, Windows, MacOS
+
+**1. 克隆项目**
+
+```bash
+git clone https://github.com/Zeyi-Lin/HivisionIDPhotos.git
+cd  HivisionIDPhotos
+```
+
+**2. 安装依赖环境**
+
+```bash
+pip install -r requirements.txt
+```
+
+**3. 下载权重文件**
+
+在我们的[Release](https://github.com/Zeyi-Lin/HivisionIDPhotos/releases/tag/pretrained-model)下载权重文件`hivision_modnet.onnx`，存到根目录下。
+
+<br>
+
+# 运行Gradio Demo
+
+```bash
+python app.py
+```
+
+运行程序将生成一个本地Web页面，在页面中可完成证件照的操作与交互。
+
+<br>
+
+# 部署API服务
+
+```
+python deploy_api.py
+```
+
+
+**请求API服务（Python）**
+
+用Python给服务发送请求：
+
+证件照制作（输入1张照片，获得1张标准证件照和1张高清证件照的4通道透明png）：
+
+```bash
+python requests_api.py -u http://127.0.0.1:8080 -i test.jpg -o ./idphoto.png -s '(413,295)'
+```
+
+增加底色（输入1张4通道透明png，获得1张增加了底色的图像）：
+
+```bash
+python requests_api.py -u http://127.0.0.1:8080 -t add_background -i ./idphoto.png -o ./idhoto_ab.jpg  -c '(0,0,0)'
+```
+
+得到六寸排版照（输入1张3通道照片，获得1张六寸排版照）：
+
+```bash
+python requests_api.py -u http://127.0.0.1:8080 -t generate_layout_photos -i ./idhoto_ab.jpg -o ./idhoto_layout.jpg  -s '(413,295)'
+```
+
+<br>
+
+# 🐳Docker部署
+
+在确保将模型权重文件[hivision_modnet.onnx](https://github.com/Zeyi-Lin/HivisionIDPhotos/releases/tag/pretrained-model)放到根目录下后，在根目录执行：
+
+```bash
+docker build -t hivision_idphotos .
+```
+
+等待镜像封装完毕后，运行以下指令，即可开启API服务：
+
+```bash
+docker run -p 8080:8080 hivision_idphotos
+```
+
+<br>
+
+
+# 引用项目
+
+1. MTCNN: https://github.com/ipazc/mtcnn
+2. ModNet: https://github.com/ZHKKKe/MODNet
+
+<br>
+
+
+# 📧联系我们
+
+如果您有任何问题，请发邮件至 [email protected]

README_EN.md

@@ -0,0 +1,155 @@
+<div align="center">
+<h1>HivisionIDPhoto</h1>
+
+
+English / [中文](README.md)
+
+[![GitHub](https://img.shields.io/static/v1?label=Github&message=GitHub&color=black)](https://github.com/xiaolin199912/HivisionIDPhotos)
+[![SwanHub Demo](https://swanhub.co/git/repo/SwanHub%2FAuto-README/file/preview?ref=main&path=swanhub.svg)](https://swanhub.co/ZeYiLin/HivisionIDPhotos/demo)
+[![zhihu](https://img.shields.io/static/v1?label=知乎&message=zhihu&color=blue)](https://zhuanlan.zhihu.com/p/638254028)
+
+
+<img src="assets/demoImage.png" width=900>
+</div>
+
+</div>
+
+
+# 🤩Project Update
+
+- Online Demo: [![SwanHub Demo](https://swanhub.co/git/repo/SwanHub%2FAuto-README/file/preview?ref=main&path=swanhub.svg)](https://swanhub.co/ZeYiLin/HivisionIDPhotos/demo)
+- 2023.12.1: Update **API deployment (based on fastapi)**
+- 2023.6.20: Update **Preset size menu**
+- 2023.6.19: Update **Layout photos**
+- 2023.6.13: Update **Center gradient color**
+- 2023.6.11: Update **Top and bottom gradient color**
+- 2023.6.8: Update **Custom size**
+- 2023.6.4: Update **Custom background color, face detection bug notification**
+- 2023.5.10: Update **Change the background without changing the size**
+
+
+<br>
+
+# Overview
+
+> 🚀Thank you for your interest in our work. You may also want to check out our other achievements in the field of image processing. Please feel free to contact us at [email protected].
+
+HivisionIDPhoto aims to develop a practical intelligent algorithm for producing ID photos. It uses a complete set of model workflows to recognize various user photo scenarios, perform image segmentation, and generate ID photos. 
+
+**HivisionIDPhoto can:**
+
+1. Perform lightweight image segmentation
+2. Generate standard ID photos and six-inch layout photos according to different size specifications
+3. Provide beauty features (waiting)
+4. Provide intelligent formal wear replacement (waiting)
+
+<div align="center">
+<img src="assets/gradio-image.jpeg" width=900>
+</div>
+
+
+---
+
+If HivisionIDPhoto is helpful to you, please star this repo or recommend it to your friends to solve the problem of emergency ID photo production!
+
+<br>
+
+# 🔧Environment Dependencies and Installation
+
+- Python >= 3.7（The main test of the project is in Python 3.10.）
+- onnxruntime
+- OpenCV
+- Option: Linux, Windows, MacOS
+
+### Installation
+
+1. Clone repo
+
+```bash
+git clone https://github.com/Zeyi-Lin/HivisionIDPhotos.git
+cd  HivisionIDPhotos
+```
+
+2. Install dependent packages
+
+```
+pip install -r requirements.txt
+```
+
+**3. Download Pretrain file**
+
+Download the weight file `hivision_modnet.onnx` from our [Release](https://github.com/Zeyi-Lin/HivisionIDPhotos/releases/tag/pretrained-model) and save it to the root directory.
+
+<br>
+
+# Gradio Demo
+
+```bash
+python app.py
+```
+
+Running the program will generate a local web page, where operations and interactions with ID photos can be completed.
+
+<br>
+
+# Deploy API service
+
+```
+python deploy_api.py
+```
+
+**Request API service (Python)**
+
+Use Python to send a request to the service:
+
+ID photo production (input 1 photo, get 1 standard ID photo and 1 high-definition ID photo 4-channel transparent png):
+
+```bash
+python requests_api.py -u http://127.0.0.1:8080 -i test.jpg -o ./idphoto.png -s '(413,295)'
+```
+
+Add background color (input 1 4-channel transparent png, get 1 image with added background color):
+
+```bash
+python requests_api.py -u http://127.0.0.1:8080 -t add_background -i ./idphoto.png -o ./idhoto_ab.jpg -c '(0,0,0)'
+```
+
+Get a six-inch layout photo (input a 3-channel photo, get a six-inch layout photo):
+
+```bash
+python requests_api.py -u http://127.0.0.1:8080 -t generate_layout_photos -i ./idhoto_ab.jpg -o ./idhoto_layout.jpg -s '(413,295)'
+```
+
+<br>
+
+# 🐳Docker deployment
+
+After ensuring that the model weight file [hivision_modnet.onnx](https://github.com/Zeyi-Lin/HivisionIDPhotos/releases/tag/pretrained-model) is placed in the root directory, execute in the root directory:
+
+```bash
+docker build -t hivision_idphotos .
+```
+
+After the image is packaged, run the following command to start the API service:
+
+```bash
+docker run -p 8080:8080 hivision_idphotos
+```
+
+<br>
+
+# Reference Projects
+
+1. MTCNN: https://github.com/ipazc/mtcnn
+2. ModNet: https://github.com/ZHKKKe/MODNet
+
+
+<br>
+
+# 📧Contact 
+
+If you have any questions, please email [email protected]
+
+
+Copyright © 2023, ZeYiLin. All Rights Reserved.
+

beautyPlugin/GrindSkin.py

@@ -0,0 +1,43 @@
+"""
+@author: cuny
+@file: GrindSkin.py
+@time: 2022/7/2 14:44
+@description: 
+磨皮算法
+"""
+import cv2
+import numpy as np
+
+
+def grindSkin(src, grindDegree: int = 3, detailDegree: int = 1, strength: int = 9):
+    """
+    Dest =(Src * (100 - Opacity) + (Src + 2 * GaussBlur(EPFFilter(Src) - Src)) * Opacity) /100
+    人像磨皮方案，后续会考虑使用一些皮肤区域检测算法来实现仅皮肤区域磨皮，增加算法的精细程度——或者使用人脸关键点
+    https://www.cnblogs.com/Imageshop/p/4709710.html
+    Args:
+        src: 原图
+        grindDegree: 磨皮程度调节参数
+        detailDegree: 细节程度调节参数
+        strength: 融合程度，作为磨皮强度（0 - 10）
+
+    Returns:
+        磨皮后的图像
+    """
+    if strength <= 0:
+        return src
+    dst = src.copy()
+    opacity = min(10., strength) / 10.
+    dx = grindDegree * 5  # 双边滤波参数之一
+    fc = grindDegree * 12.5  # 双边滤波参数之一
+    temp1 = cv2.bilateralFilter(src[:, :, :3], dx, fc, fc)
+    temp2 = cv2.subtract(temp1, src[:, :, :3])
+    temp3 = cv2.GaussianBlur(temp2, (2 * detailDegree - 1, 2 * detailDegree - 1), 0)
+    temp4 = cv2.add(cv2.add(temp3, temp3), src[:, :, :3])
+    dst[:, :, :3] = cv2.addWeighted(temp4, opacity, src[:, :, :3], 1 - opacity, 0.0)
+    return dst
+
+
+if __name__ == "__main__":
+    input_image = cv2.imread("test_image/7.jpg")
+    output_image = grindSkin(src=input_image)
+    cv2.imwrite("grindSkinCompare.png", np.hstack((input_image, output_image)))

beautyPlugin/MakeBeautiful.py

@@ -0,0 +1,45 @@
+"""
+@author: cuny
+@file: MakeBeautiful.py
+@time: 2022/7/7 20:23
+@description: 
+美颜工具集合文件，作为暴露在外的插件接口
+"""
+from .GrindSkin import grindSkin
+from .MakeWhiter import MakeWhiter
+from .ThinFace import thinFace
+import numpy as np
+
+
+def makeBeautiful(input_image: np.ndarray,
+                  landmark,
+                  thinStrength: int,
+                  thinPlace: int,
+                  grindStrength: int,
+                  whiterStrength: int
+                  ) -> np.ndarray:
+    """
+    美颜工具的接口函数，用于实现美颜效果
+    Args:
+        input_image: 输入的图像
+        landmark: 瘦脸需要的人脸关键点信息，为fd68返回的第二个参数
+        thinStrength: 瘦脸强度，为0-10（如果更高其实也没什么问题），当强度为0或者更低时，则不瘦脸
+        thinPlace: 选择瘦脸区域，为0-2之间的值，越大瘦脸的点越靠下
+        grindStrength: 磨皮强度，为0-10（如果更高其实也没什么问题），当强度为0或者更低时，则不磨皮
+        whiterStrength: 美白强度，为0-10（如果更高其实也没什么问题），当强度为0或者更低时，则不美白
+    Returns:
+        output_image 输出图像
+    """
+    try:
+        _, _, _ = input_image.shape
+    except ValueError:
+        raise TypeError("输入图像必须为3通道或者4通道!")
+    # 三通道或者四通道图像
+    # 首先进行瘦脸
+    input_image = thinFace(input_image, landmark, place=thinPlace, strength=thinStrength)
+    # 其次进行磨皮
+    input_image = grindSkin(src=input_image, strength=grindStrength)
+    # 最后进行美白
+    makeWhiter = MakeWhiter()
+    input_image = makeWhiter.run(input_image, strength=whiterStrength)
+    return input_image

beautyPlugin/MakeWhiter.py

@@ -0,0 +1,108 @@
+"""
+@author: cuny
+@file: MakeWhiter.py
+@time: 2022/7/2 14:28
+@description: 
+美白算法
+"""
+import os
+import cv2
+import math
+import numpy as np
+local_path = os.path.dirname(__file__)
+
+
+class MakeWhiter(object):
+    class __LutWhite:
+        """
+        美白的内部类
+        """
+
+        def __init__(self, lut):
+            cube64rows = 8
+            cube64size = 64
+            cube256size = 256
+            cubeScale = int(cube256size / cube64size)  # 4
+
+            reshapeLut = np.zeros((cube256size, cube256size, cube256size, 3))
+            for i in range(cube64size):
+                tmp = math.floor(i / cube64rows)
+                cx = int((i - tmp * cube64rows) * cube64size)
+                cy = int(tmp * cube64size)
+                cube64 = lut[cy:cy + cube64size, cx:cx + cube64size]  # cube64 in lut(512*512 (512=8*64))
+                _rows, _cols, _ = cube64.shape
+                if _rows == 0 or _cols == 0:
+                    continue
+                cube256 = cv2.resize(cube64, (cube256size, cube256size))
+                i = i * cubeScale
+                for k in range(cubeScale):
+                    reshapeLut[i + k] = cube256
+            self.lut = reshapeLut
+
+        def imageInLut(self, src):
+            arr = src.copy()
+            bs = arr[:, :, 0]
+            gs = arr[:, :, 1]
+            rs = arr[:, :, 2]
+            arr[:, :] = self.lut[bs, gs, rs]
+            return arr
+
+    def __init__(self, lutImage: np.ndarray = None):
+        self.__lutWhiten = None
+        if lutImage is not None:
+            self.__lutWhiten = self.__LutWhite(lutImage)
+
+    def setLut(self, lutImage: np.ndarray):
+        self.__lutWhiten = self.__LutWhite(lutImage)
+
+    @staticmethod
+    def generate_identify_color_matrix(size: int = 512, channel: int = 3) -> np.ndarray:
+        """
+        用于生成一张初始的查找表
+        Args:
+            size: 查找表尺寸，默认为512
+            channel: 查找表通道数，默认为3
+
+        Returns:
+            返回生成的查找表图像
+        """
+        img = np.zeros((size, size, channel), dtype=np.uint8)
+        for by in range(size // 64):
+            for bx in range(size // 64):
+                for g in range(64):
+                    for r in range(64):
+                        x = r + bx * 64
+                        y = g + by * 64
+                        img[y][x][0] = int(r * 255.0 / 63.0 + 0.5)
+                        img[y][x][1] = int(g * 255.0 / 63.0 + 0.5)
+                        img[y][x][2] = int((bx + by * 8.0) * 255.0 / 63.0 + 0.5)
+        return cv2.cvtColor(img, cv2.COLOR_RGB2BGR).clip(0, 255).astype('uint8')
+
+    def run(self, src: np.ndarray, strength: int) -> np.ndarray:
+        """
+        美白图像
+        Args:
+            src: 原图
+            strength: 美白强度，0 - 10
+        Returns:
+            美白后的图像
+        """
+        dst = src.copy()
+        strength = min(10, int(strength)) / 10.
+        if strength <= 0:
+            return dst
+        self.setLut(cv2.imread(f"{local_path}/lut_image/3.png", -1))
+        _, _, c = src.shape
+        img = self.__lutWhiten.imageInLut(src[:, :, :3])
+        dst[:, :, :3] = cv2.addWeighted(src[:, :, :3], 1 - strength, img, strength, 0)
+        return dst
+
+
+if __name__ == "__main__":
+    # makeLut = MakeWhiter()
+    # cv2.imwrite("lutOrigin.png", makeLut.generate_identify_color_matrix())
+    input_image = cv2.imread("test_image/7.jpg", -1)
+    lut_image = cv2.imread("lut_image/3.png")
+    makeWhiter = MakeWhiter(lut_image)
+    output_image = makeWhiter.run(input_image, 10)
+    cv2.imwrite("makeWhiterCompare.png", np.hstack((input_image, output_image)))

beautyPlugin/ThinFace.py

@@ -0,0 +1,267 @@
+"""
+@author: cuny
+@file: ThinFace.py
+@time: 2022/7/2 15:50
+@description:
+瘦脸算法，用到了图像局部平移法
+先使用人脸关键点检测，然后再使用图像局部平移法
+需要注意的是，这部分不会包含dlib人脸关键点检测，因为考虑到模型载入的问题
+"""
+import cv2
+import math
+import numpy as np
+
+
+class TranslationWarp(object):
+    """
+    本类包含瘦脸算法，由于瘦脸算法包含了很多个版本，所以以类的方式呈现
+    前两个算法没什么好讲的，网上资料很多
+    第三个采用numpy内部的自定义函数处理，在处理速度上有一些提升
+    最后采用cv2.map算法，处理速度大幅度提升
+    """
+
+    # 瘦脸
+    @staticmethod
+    def localTranslationWarp(srcImg, startX, startY, endX, endY, radius):
+        # 双线性插值法
+        def BilinearInsert(src, ux, uy):
+            w, h, c = src.shape
+            if c == 3:
+                x1 = int(ux)
+                x2 = x1 + 1
+                y1 = int(uy)
+                y2 = y1 + 1
+                part1 = src[y1, x1].astype(np.float64) * (float(x2) - ux) * (float(y2) - uy)
+                part2 = src[y1, x2].astype(np.float64) * (ux - float(x1)) * (float(y2) - uy)
+                part3 = src[y2, x1].astype(np.float64) * (float(x2) - ux) * (uy - float(y1))
+                part4 = src[y2, x2].astype(np.float64) * (ux - float(x1)) * (uy - float(y1))
+                insertValue = part1 + part2 + part3 + part4
+                return insertValue.astype(np.int8)
+
+        ddradius = float(radius * radius)  # 圆的半径
+        copyImg = srcImg.copy()  # copy后的图像矩阵
+        # 计算公式中的|m-c|^2
+        ddmc = (endX - startX) * (endX - startX) + (endY - startY) * (endY - startY)
+        H, W, C = srcImg.shape  # 获取图像的形状
+        for i in range(W):
+            for j in range(H):
+                # # 计算该点是否在形变圆的范围之内
+                # # 优化，第一步，直接判断是会在（startX,startY)的矩阵框中
+                if math.fabs(i - startX) > radius and math.fabs(j - startY) > radius:
+                    continue
+                distance = (i - startX) * (i - startX) + (j - startY) * (j - startY)
+                if distance < ddradius:
+                    # 计算出（i,j）坐标的原坐标
+                    # 计算公式中右边平方号里的部分
+                    ratio = (ddradius - distance) / (ddradius - distance + ddmc)
+                    ratio = ratio * ratio
+                    # 映射原位置
+                    UX = i - ratio * (endX - startX)
+                    UY = j - ratio * (endY - startY)
+
+                    # 根据双线性插值法得到UX，UY的值
+                    # start_ = time.time()
+                    value = BilinearInsert(srcImg, UX, UY)
+                    # print(f"双线性插值耗时;{time.time() - start_}")
+                    # 改变当前 i ，j的值
+                    copyImg[j, i] = value
+        return copyImg
+
+    # 瘦脸pro1, 限制了for循环的遍历次数
+    @staticmethod
+    def localTranslationWarpLimitFor(srcImg, startP: np.matrix, endP: np.matrix, radius: float):
+        startX, startY = startP[0, 0], startP[0, 1]
+        endX, endY = endP[0, 0], endP[0, 1]
+
+        # 双线性插值法
+        def BilinearInsert(src, ux, uy):
+            w, h, c = src.shape
+            if c == 3:
+                x1 = int(ux)
+                x2 = x1 + 1
+                y1 = int(uy)
+                y2 = y1 + 1
+                part1 = src[y1, x1].astype(np.float64) * (float(x2) - ux) * (float(y2) - uy)
+                part2 = src[y1, x2].astype(np.float64) * (ux - float(x1)) * (float(y2) - uy)
+                part3 = src[y2, x1].astype(np.float64) * (float(x2) - ux) * (uy - float(y1))
+                part4 = src[y2, x2].astype(np.float64) * (ux - float(x1)) * (uy - float(y1))
+                insertValue = part1 + part2 + part3 + part4
+                return insertValue.astype(np.int8)
+
+        ddradius = float(radius * radius)  # 圆的半径
+        copyImg = srcImg.copy()  # copy后的图像矩阵
+        # 计算公式中的|m-c|^2
+        ddmc = (endX - startX) ** 2 + (endY - startY) ** 2
+        # 计算正方形的左上角起始点
+        startTX, startTY = (startX - math.floor(radius + 1), startY - math.floor((radius + 1)))
+        # 计算正方形的右下角的结束点
+        endTX, endTY = (startX + math.floor(radius + 1), startY + math.floor((radius + 1)))
+        # 剪切srcImg
+        srcImg = srcImg[startTY: endTY + 1, startTX: endTX + 1, :]
+        # db.cv_show(srcImg)
+        # 裁剪后的图像相当于在x,y都减少了startX - math.floor(radius + 1)
+        # 原本的endX, endY在切后的坐标点
+        endX, endY = (endX - startX + math.floor(radius + 1), endY - startY + math.floor(radius + 1))
+        # 原本的startX, startY剪切后的坐标点
+        startX, startY = (math.floor(radius + 1), math.floor(radius + 1))
+        H, W, C = srcImg.shape  # 获取图像的形状
+        for i in range(W):
+            for j in range(H):
+                # 计算该点是否在形变圆的范围之内
+                # 优化，第一步，直接判断是会在（startX,startY)的矩阵框中
+                # if math.fabs(i - startX) > radius and math.fabs(j - startY) > radius:
+                #     continue
+                distance = (i - startX) * (i - startX) + (j - startY) * (j - startY)
+                if distance < ddradius:
+                    # 计算出（i,j）坐标的原坐标
+                    # 计算公式中右边平方号里的部分
+                    ratio = (ddradius - distance) / (ddradius - distance + ddmc)
+                    ratio = ratio * ratio
+                    # 映射原位置
+                    UX = i - ratio * (endX - startX)
+                    UY = j - ratio * (endY - startY)
+
+                    # 根据双线性插值法得到UX，UY的值
+                    # start_ = time.time()
+                    value = BilinearInsert(srcImg, UX, UY)
+                    # print(f"双线性插值耗时;{time.time() - start_}")
+                    # 改变当前 i ，j的值
+                    copyImg[j + startTY, i + startTX] = value
+        return copyImg
+
+    # # 瘦脸pro2,采用了numpy自定义函数做处理
+    # def localTranslationWarpNumpy(self, srcImg, startP: np.matrix, endP: np.matrix, radius: float):
+    #     startX , startY = startP[0, 0], startP[0, 1]
+    #     endX, endY = endP[0, 0], endP[0, 1]
+    #     ddradius = float(radius * radius)  # 圆的半径
+    #     copyImg = srcImg.copy()  # copy后的图像矩阵
+    #     # 计算公式中的|m-c|^2
+    #     ddmc = (endX - startX)**2 + (endY - startY)**2
+    #     # 计算正方形的左上角起始点
+    #     startTX, startTY = (startX - math.floor(radius + 1), startY - math.floor((radius + 1)))
+    #     # 计算正方形的右下角的结束点
+    #     endTX, endTY = (startX + math.floor(radius + 1), startY + math.floor((radius + 1)))
+    #     # 剪切srcImg
+    #     self.thinImage = srcImg[startTY : endTY + 1, startTX : endTX + 1, :]
+    #     # s = self.thinImage
+    #     # db.cv_show(srcImg)
+    #     # 裁剪后的图像相当于在x,y都减少了startX - math.floor(radius + 1)
+    #     # 原本的endX, endY在切后的坐标点
+    #     endX, endY = (endX - startX + math.floor(radius + 1), endY - startY + math.floor(radius + 1))
+    #     # 原本的startX, startY剪切后的坐标点
+    #     startX ,startY = (math.floor(radius + 1), math.floor(radius + 1))
+    #     H, W, C = self.thinImage.shape  # 获取图像的形状
+    #     index_m = np.arange(H * W).reshape((H, W))
+    #     triangle_ufunc = np.frompyfunc(self.process, 9, 3)
+    #     # start_ = time.time()
+    #     finalImgB, finalImgG, finalImgR = triangle_ufunc(index_m, self, W, ddradius, ddmc, startX, startY, endX, endY)
+    #     finaleImg = np.dstack((finalImgB, finalImgG, finalImgR)).astype(np.uint8)
+    #     finaleImg = np.fliplr(np.rot90(finaleImg, -1))
+    #     copyImg[startTY: endTY + 1, startTX: endTX + 1, :] = finaleImg
+    #     # print(f"图像处理耗时;{time.time() - start_}")
+    #     # db.cv_show(copyImg)
+    #     return copyImg
+
+    # 瘦脸pro3,采用opencv内置函数
+    @staticmethod
+    def localTranslationWarpFastWithStrength(srcImg, startP: np.matrix, endP: np.matrix, radius, strength: float = 100.):
+        """
+        采用opencv内置函数
+        Args:
+            srcImg: 源图像
+            startP: 起点位置
+            endP: 终点位置
+            radius: 处理半径
+            strength: 瘦脸强度，一般取100以上
+
+        Returns:
+
+        """
+        startX, startY = startP[0, 0], startP[0, 1]
+        endX, endY = endP[0, 0], endP[0, 1]
+        ddradius = float(radius * radius)
+        # copyImg = np.zeros(srcImg.shape, np.uint8)
+        # copyImg = srcImg.copy()
+
+        maskImg = np.zeros(srcImg.shape[:2], np.uint8)
+        cv2.circle(maskImg, (startX, startY), math.ceil(radius), (255, 255, 255), -1)
+
+        K0 = 100 / strength
+
+        # 计算公式中的|m-c|^2
+        ddmc_x = (endX - startX) * (endX - startX)
+        ddmc_y = (endY - startY) * (endY - startY)
+        H, W, C = srcImg.shape
+
+        mapX = np.vstack([np.arange(W).astype(np.float32).reshape(1, -1)] * H)
+        mapY = np.hstack([np.arange(H).astype(np.float32).reshape(-1, 1)] * W)
+
+        distance_x = (mapX - startX) * (mapX - startX)
+        distance_y = (mapY - startY) * (mapY - startY)
+        distance = distance_x + distance_y
+        K1 = np.sqrt(distance)
+        ratio_x = (ddradius - distance_x) / (ddradius - distance_x + K0 * ddmc_x)
+        ratio_y = (ddradius - distance_y) / (ddradius - distance_y + K0 * ddmc_y)
+        ratio_x = ratio_x * ratio_x
+        ratio_y = ratio_y * ratio_y
+
+        UX = mapX - ratio_x * (endX - startX) * (1 - K1 / radius)
+        UY = mapY - ratio_y * (endY - startY) * (1 - K1 / radius)
+
+        np.copyto(UX, mapX, where=maskImg == 0)
+        np.copyto(UY, mapY, where=maskImg == 0)
+        UX = UX.astype(np.float32)
+        UY = UY.astype(np.float32)
+        copyImg = cv2.remap(srcImg, UX, UY, interpolation=cv2.INTER_LINEAR)
+        return copyImg
+
+
+def thinFace(src, landmark, place: int = 0, strength=30.):
+    """
+    瘦脸程序接口，输入人脸关键点信息和强度，即可实现瘦脸
+    注意处理四通道图像
+    Args:
+        src: 原图
+        landmark: 关键点信息
+        place: 选择瘦脸区域，为0-4之间的值
+        strength: 瘦脸强度，输入值在0-10之间，如果小于或者等于0，则不瘦脸
+
+    Returns:
+        瘦脸后的图像
+    """
+    strength = min(100., strength * 10.)
+    if strength <= 0.:
+        return src
+    # 也可以设置瘦脸区域
+    place = max(0, min(4, int(place)))
+    left_landmark = landmark[4 + place]
+    left_landmark_down = landmark[6 + place]
+    right_landmark = landmark[13 + place]
+    right_landmark_down = landmark[15 + place]
+    endPt = landmark[58]
+    # 计算第4个点到第6个点的距离作为瘦脸距离
+    r_left = math.sqrt(
+        (left_landmark[0, 0] - left_landmark_down[0, 0]) ** 2 +
+        (left_landmark[0, 1] - left_landmark_down[0, 1]) ** 2
+    )
+
+    # 计算第14个点到第16个点的距离作为瘦脸距离
+    r_right = math.sqrt((right_landmark[0, 0] - right_landmark_down[0, 0]) ** 2 +
+                        (right_landmark[0, 1] - right_landmark_down[0, 1]) ** 2)
+    # 瘦左边脸
+    thin_image = TranslationWarp.localTranslationWarpFastWithStrength(src, left_landmark[0], endPt[0], r_left, strength)
+    # 瘦右边脸
+    thin_image = TranslationWarp.localTranslationWarpFastWithStrength(thin_image, right_landmark[0], endPt[0], r_right, strength)
+    return thin_image
+
+
+if __name__ == "__main__":
+    import os
+    from hycv.FaceDetection68.faceDetection68 import FaceDetection68
+    local_file = os.path.dirname(__file__)
+    PREDICTOR_PATH = f"{local_file}/weights/shape_predictor_68_face_landmarks.dat"  # 关键点检测模型路径
+    fd68 = FaceDetection68(model_path=PREDICTOR_PATH)
+    input_image = cv2.imread("test_image/4.jpg", -1)
+    _, landmark_, _ = fd68.facePoints(input_image)
+    output_image = thinFace(input_image, landmark_, strength=30.2)
+    cv2.imwrite("thinFaceCompare.png", np.hstack((input_image, output_image)))

beautyPlugin/__init__.py

@@ -0,0 +1,4 @@
+from .MakeBeautiful import makeBeautiful
+
+
+

deploy_api.py

@@ -0,0 +1,138 @@
+from fastapi import FastAPI, UploadFile, Form
+import onnxruntime
+from src.face_judgement_align import IDphotos_create
+from src.layoutCreate import generate_layout_photo, generate_layout_image
+from hivisionai.hycv.vision import add_background
+import base64
+import numpy as np
+import cv2
+import ast
+
+app = FastAPI()
+
+
+# 将图像转换为Base64编码
+
+def numpy_2_base64(img: np.ndarray):
+    retval, buffer = cv2.imencode('.png', img)
+    base64_image = base64.b64encode(buffer).decode('utf-8')
+
+    return base64_image
+
+
+# 证件照智能制作接口
+@app.post("/idphoto")
+async def idphoto_inference(input_image: UploadFile,
+                            size: str = Form(...),
+                            head_measure_ratio=0.2,
+                            head_height_ratio=0.45,
+                            top_distance_max=0.12,
+                            top_distance_min=0.10):
+    image_bytes = await input_image.read()
+    nparr = np.frombuffer(image_bytes, np.uint8)
+    img = cv2.imdecode(nparr, cv2.IMREAD_COLOR)
+
+    # 将字符串转为元组
+    size = ast.literal_eval(size)
+
+    result_image_hd, result_image_standard, typography_arr, typography_rotate, \
+        _, _, _, _, status = IDphotos_create(img,
+                                             size=size,
+                                             head_measure_ratio=head_measure_ratio,
+                                             head_height_ratio=head_height_ratio,
+                                             align=False,
+                                             beauty=False,
+                                             fd68=None,
+                                             human_sess=sess,
+                                             IS_DEBUG=False,
+                                             top_distance_max=top_distance_max,
+                                             top_distance_min=top_distance_min)
+
+    # 如果检测到人脸数量不等于1（照片无人脸 or 多人脸）
+    if status == 0:
+        result_messgae = {
+            "status": False
+        }
+
+    # 如果检测到人脸数量等于1, 则返回标准证和高清照结果（png 4通道图像）
+    else:
+        result_messgae = {
+            "status": True,
+            "img_output_standard": numpy_2_base64(result_image_standard),
+            "img_output_standard_hd": numpy_2_base64(result_image_hd),
+        }
+
+    return result_messgae
+
+
+# 透明图像添加纯色背景接口
+@app.post("/add_background")
+async def photo_add_background(input_image: UploadFile,
+                               color: str = Form(...)):
+    
+    # 读取图像
+    image_bytes = await input_image.read()
+    nparr = np.frombuffer(image_bytes, np.uint8)
+    img = cv2.imdecode(nparr, cv2.IMREAD_UNCHANGED)
+
+    # 将字符串转为元组
+    color = ast.literal_eval(color)
+    # 将元祖的0和2号数字交换
+    color = (color[2], color[1], color[0])
+
+    # try:
+    result_messgae = {
+        "status": True,
+        "image": numpy_2_base64(add_background(img, bgr=color)),
+    }
+
+    # except Exception as e:
+    #     print(e)
+    #     result_messgae = {
+    #         "status": False,
+    #         "error": e
+    #     }
+
+    return result_messgae
+
+
+# 六寸排版照生成接口
+@app.post("/generate_layout_photos")
+async def generate_layout_photos(input_image: UploadFile, size: str = Form(...)):
+    try:
+        image_bytes = await input_image.read()
+        nparr = np.frombuffer(image_bytes, np.uint8)
+        img = cv2.imdecode(nparr, cv2.IMREAD_COLOR)
+
+        size = ast.literal_eval(size)
+
+        typography_arr, typography_rotate = generate_layout_photo(input_height=size[0],
+                                                                  input_width=size[1])
+
+        result_layout_image = generate_layout_image(img, typography_arr,
+                                                    typography_rotate,
+                                                    height=size[0],
+                                                    width=size[1])
+
+        result_messgae = {
+            "status": True,
+            "image": numpy_2_base64(result_layout_image),
+        }
+
+    except Exception as e:
+        result_messgae = {
+            "status": False,
+        }
+
+    return result_messgae
+
+
+if __name__ == "__main__":
+    import uvicorn
+
+    # 加载权重文件
+    HY_HUMAN_MATTING_WEIGHTS_PATH = "./hivision_modnet.onnx"
+    sess = onnxruntime.InferenceSession(HY_HUMAN_MATTING_WEIGHTS_PATH)
+
+    # 在8080端口运行推理服务
+    uvicorn.run(app, host="0.0.0.0", port=8080)

hivision_modnet.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7e0cb9a2a841b426dd0daf1a788ec398dab059bc039041d62b15636c0783bc56
+size 25888609

hivisionai/app.py

@@ -0,0 +1,452 @@
+# -*- coding: utf-8 -*-
+
+"""
+@Time     : 2022/8/27 14:17
+@Author   : cuny
+@File     : app.py
+@Software : PyCharm
+@Introduce: 
+查看包版本等一系列操作
+"""
+import os
+import sys
+import json
+import shutil
+import zipfile
+import requests
+from argparse import ArgumentParser
+from importlib.metadata import version
+try:  # 加上这个try的原因在于本地环境和云函数端的import形式有所不同
+    from qcloud_cos import CosConfig
+    from qcloud_cos import CosS3Client
+except ImportError:
+    try:
+        from qcloud_cos_v5 import CosConfig
+        from qcloud_cos_v5 import CosS3Client
+        from qcloud_cos.cos_exception import CosServiceError
+    except ImportError:
+        raise ImportError("请下载腾讯云COS相关代码包:pip install cos-python-sdk-v5")
+
+
+class HivisionaiParams(object):
+    """
+    定义一些基本常量
+    """
+    # 文件所在路径
+    # 包名称
+    package_name = "HY-sdk"
+    # 腾讯云相关变量
+    region = "ap-beijing"
+    zip_key = "HY-sdk/"  # zip存储的云端文件夹路径，这里改了publish.yml也需要更改
+    # 云端用户配置，如果在cloud_config_save不存在，就需要下载此文件
+    user_url = "https://hy-sdk-config-1305323352.cos.ap-beijing.myqcloud.com/sdk-user/user_config.json"
+    bucket = "cloud-public-static-1306602019"
+    # 压缩包类型
+    file_format = ".zip"
+    # 下载路径（.hivisionai文件夹路径）
+    download_path = os.path.expandvars('$HOME')
+    # zip文件、zip解压缩文件的存放路径
+    save_folder = f"{os.path.expandvars('$HOME')}/.hivisionai/sdk"
+    # 腾讯云配置文件存放路径
+    cloud_config_save = f"{os.path.expandvars('$HOME')}/.hivisionai/user_config.json"
+    # 项目路径
+    hivisionai_path = os.path.dirname(os.path.dirname(__file__))
+    # 使用hivisionai的路径
+    getcwd = os.getcwd()
+    # HY-func的依赖配置
+    # 每个依赖会包含三个参数，保存路径（save_path，相对于HY_func的路径）、下载url（url）
+    functionDependence = {
+        "configs":  [
+            # --------- 配置文件部分
+            # _lib
+            {
+                "url": "https://hy-sdk-config-1305323352.cos.ap-beijing.myqcloud.com/hy-func/_lib/config/aliyun-human-matting-api.json",
+                "save_path": "_lib/config/aliyun-human-matting-api.json"
+            },
+            {
+                "url": "https://hy-sdk-config-1305323352.cos.ap-beijing.myqcloud.com/hy-func/_lib/config/megvii-face-plus-api.json",
+                "save_path": "_lib/config/megvii-face-plus-api.json"
+            },
+            {
+                "url": "https://hy-sdk-config-1305323352.cos.ap-beijing.myqcloud.com/hy-func/_lib/config/volcano-face-change-api.json",
+                "save_path": "_lib/config/volcano-face-change-api.json"
+            },
+            # _service
+            {
+                "url": "https://hy-sdk-config-1305323352.cos.ap-beijing.myqcloud.com/hy-func/_service/config/func_error_conf.json",
+                "save_path": "_service/utils/config/func_error_conf.json"
+            },
+            {
+                "url": "https://hy-sdk-config-1305323352.cos.ap-beijing.myqcloud.com/hy-func/_service/config/service_config.json",
+                "save_path": "_service/utils/config/service_config.json"
+            },
+            # --------- 模型部分
+            # 模型部分存储在Notion文档当中
+            # https://www.notion.so/HY-func-cc6cc41ba6e94b36b8fa5f5d67d1683f
+        ],
+        "weights": "https://www.notion.so/HY-func-cc6cc41ba6e94b36b8fa5f5d67d1683f"
+    }
+
+
+class HivisionaiUtils(object):
+    """
+    本类为一些基本工具类，包含代码复用相关内容
+    """
+    @staticmethod
+    def get_client():
+        """获取cos客户端对象"""
+        def get_secret():
+            # 首先判断cloud_config_save下是否存在
+            if not os.path.exists(HivisionaiParams.cloud_config_save):
+                print("Downloading user_config...")
+                resp = requests.get(HivisionaiParams.user_url)
+                open(HivisionaiParams.cloud_config_save, "wb").write(resp.content)
+            config = json.load(open(HivisionaiParams.cloud_config_save, "r"))
+            return config["secret_id"], config["secret_key"]
+        # todo 接入HY-Auth-Sync
+        secret_id, secret_key = get_secret()
+        return CosS3Client(CosConfig(Region=HivisionaiParams.region, Secret_id=secret_id, Secret_key=secret_key))
+
+    def get_all_versions(self):
+        """获取云端的所有版本号"""
+        def getAllVersion_base():
+            """
+            返回cos存储桶内部的某个文件夹的内部名称
+            ps:如果需要修改默认的存储桶配置，请在代码运行的时候加入代码 s.bucket = 存储桶名称 （s是对象实例）
+            返回的内容存储在response["Content"]，不过返回的数据大小是有限制的，具体内容还是请看官方文档。
+            Returns:
+                [版本列表]
+            """
+            resp = client.list_objects(
+                Bucket=HivisionaiParams.bucket,
+                Prefix=HivisionaiParams.zip_key,
+                Marker=marker
+            )
+            versions_list.extend([x["Key"].split("/")[-1].split(HivisionaiParams.file_format)[0] for x in resp["Contents"] if int(x["Size"]) > 0])
+            if resp['IsTruncated'] == 'false':  # 接下来没有数据了,就退出
+                return ""
+            else:
+                return resp['NextMarker']
+        client = self.get_client()
+        marker = ""
+        versions_list = []
+        while True:  # 轮询
+            try:
+                marker = getAllVersion_base()
+            except KeyError as e:
+                print(e)
+                raise
+            if len(marker) == 0:  # 没有数据了
+                break
+        return versions_list
+
+    def get_newest_version(self):
+        """获取最新的版本号"""
+        versions_list = self.get_all_versions()
+        # reverse=True，降序
+        versions_list.sort(key=lambda x: int(x.split(".")[-1]), reverse=True)
+        versions_list.sort(key=lambda x: int(x.split(".")[-2]), reverse=True)
+        versions_list.sort(key=lambda x: int(x.split(".")[-3]), reverse=True)
+        return versions_list[0]
+
+    def download_version(self, v):
+        """
+        在存储桶中下载文件，将下载好的文件解压至本地
+        Args:
+            v: 版本号，x.x.x
+
+        Returns:
+            None
+        """
+        file_name = v + HivisionaiParams.file_format
+        client = self.get_client()
+        print(f"Download to {HivisionaiParams.save_folder}...")
+        try:
+            resp = client.get_object(HivisionaiParams.bucket, HivisionaiParams.zip_key + "/" + file_name)
+            contents = resp["Body"].get_raw_stream().read()
+        except CosServiceError:
+            print(f"[{file_name}.zip] does not exist, please check your version!")
+            sys.exit()
+        if not os.path.exists(HivisionaiParams.save_folder):
+            os.makedirs(HivisionaiParams.save_folder)
+        open(os.path.join(HivisionaiParams.save_folder, file_name), "wb").write(contents)
+        print("Download success!")
+
+    @staticmethod
+    def download_dependence(path=None):
+        """
+        一键下载HY-sdk所需要的所有依赖，需要注意的是，本方法必须在运行pip install之后使用（运行完pip install之后才会出现hivisionai文件夹）
+        Args:
+            path: 文件路径，精确到hivisionai文件夹的上一个目录，如果为None，则默认下载到python环境下hivisionai安装的目录
+
+        Returns:
+            下载相应内容到指定位置
+        """
+        # print("指定的下载路径：", path)  # 此时在path路径下必然存在一个hivisionai文件夹
+        # print("系统安装的hivisionai库的路径:", HivisionaiParams.hivisionai_path)
+        print("Dependence downloading...")
+        if path is None:
+            path = HivisionaiParams.hivisionai_path
+        # ----------------下载mtcnn模型文件
+        mtcnn_path = os.path.join(path, "hivisionai/hycv/mtcnn_onnx/weights")
+        base_url = "https://linimages.oss-cn-beijing.aliyuncs.com/"
+        onnx_files = ["pnet.onnx",  "rnet.onnx", "onet.onnx"]
+        print(f"Downloading mtcnn model in {mtcnn_path}")
+        if not os.path.exists(mtcnn_path):
+            os.mkdir(mtcnn_path)
+        for onnx_file in onnx_files:
+            if not os.path.exists(os.path.join(mtcnn_path, onnx_file)):
+                # download onnx model
+                onnx_url = base_url + onnx_file
+                print("Downloading Onnx Model in:", onnx_url)
+                r = requests.get(onnx_url, stream=True)
+                if r.status_code == 200:
+                    open(os.path.join(mtcnn_path, onnx_file), 'wb').write(r.content)  # 将内容写入文件
+                    print(f"Download finished -- {onnx_file}")
+                del r
+        # ----------------
+        print("Dependence download finished...")
+
+
+class HivisionaiApps(object):
+    """
+    本类为app对外暴露的接口，为了代码规整性，这里使用类来对暴露接口进行调整
+    """
+    @staticmethod
+    def show_cloud_version():
+        """查看在cos中的所有HY-sdk版本"""
+        print("Connect to COS...")
+        versions_list = hivisionai_utils.get_all_versions()
+        # reverse=True，降序
+        versions_list.sort(key=lambda x: int(x.split(".")[-1]), reverse=True)
+        versions_list.sort(key=lambda x: int(x.split(".")[-2]), reverse=True)
+        versions_list.sort(key=lambda x: int(x.split(".")[-3]), reverse=True)
+        if len(versions_list) == 0:
+            print("There is no version currently, please release it first!")
+            sys.exit()
+        versions = "The currently existing versions (Keep 10): \n"
+        for i, v in enumerate(versions_list):
+            versions += str(v) + " "
+            if i == 9:
+                break
+        print(versions)
+
+    @staticmethod
+    def upgrade(v: str, enforce: bool = False, save_cached: bool = False):
+        """
+        自动升级HY-sdk到指定版本
+        Args:
+            v: 指定的版本号，格式为x.x.x
+            enforce: 是否需要强制执行更新命令
+            save_cached: 是否保存下载的wheel文件，默认为否
+        Returns:
+            None
+        """
+        def check_format():
+            # noinspection PyBroadException
+            try:
+                major, minor, patch = v.split(".")
+                int(major)
+                int(minor)
+                int(patch)
+            except Exception as e:
+                print(f"Illegal version number!\n{e}")
+            pass
+        print("Upgrading, please wait a moment...")
+        if v == "-1":
+            v = hivisionai_utils.get_newest_version()
+        # 检查format的格式
+        check_format()
+        if v == version(HivisionaiParams.package_name) and not enforce:
+            print(f"Current version: {v} already exists, skip installation.")
+            sys.exit()
+        hivisionai_utils.download_version(v)
+        # 下载完毕（下载至save_folder），解压文件
+        target_zip = os.path.join(HivisionaiParams.save_folder, f"{v}.zip")
+        assert zipfile.is_zipfile(target_zip), "Decompression failed, and the target was not a zip file."
+        new_dir = target_zip.replace('.zip', '')  # 解压的文件名
+        if os.path.exists(new_dir):  # 判断文件夹是否存在
+            shutil.rmtree(new_dir)
+        os.mkdir(new_dir)  # 新建文件夹
+        f = zipfile.ZipFile(target_zip)
+        f.extractall(new_dir)  # 提取zip文件
+        print("Decompressed, begin to install...")
+        os.system(f'pip3 install {os.path.join(new_dir, "**.whl")}')
+        # 开始自动下载必要的模型依赖
+        hivisionai_utils.download_dependence()
+        # 安装完毕，如果save_cached为真，删除"$HOME/.hivisionai/sdk"内部的所有文件元素
+        if save_cached is True:
+            os.system(f'rm -rf {HivisionaiParams.save_folder}/**')
+
+    @staticmethod
+    def export(path):
+        """
+        输出最新版本的文件到命令运行的path目录
+        Args:
+            path: 用户输入的路径
+
+        Returns:
+            输出最新的hivisionai到path目录
+        """
+        # print(f"当前路径: {os.path.join(HivisionaiParams.getcwd, path)}")
+        # print(f"文件路径: {os.path.dirname(__file__)}")
+        export_path = os.path.join(HivisionaiParams.getcwd, path)
+        # 判断输出路径存不存在，如果不存在，就报错
+        assert os.path.exists(export_path), f"{export_path} dose not Exists!"
+        v = hivisionai_utils.get_newest_version()
+        # 下载文件到.hivisionai/sdk当中
+        hivisionai_utils.download_version(v)
+        # 下载完毕（下载至save_folder），解压文件
+        target_zip = os.path.join(HivisionaiParams.save_folder, f"{v}.zip")
+        assert zipfile.is_zipfile(target_zip), "Decompression failed, and the target was not a zip file."
+        new_dir = os.path.basename(target_zip.replace('.zip', ''))  # 解压的文件名
+        new_dir = os.path.join(export_path, new_dir)  # 解压的文件路径
+        if os.path.exists(new_dir):  # 判断文件夹是否存在
+            shutil.rmtree(new_dir)
+        os.mkdir(new_dir)  # 新建文件夹
+        f = zipfile.ZipFile(target_zip)
+        f.extractall(new_dir)  # 提取zip文件
+        print("Decompressed, begin to export...")
+        # 强制删除bin/hivisionai和hivisionai/以及HY_sdk-**
+        bin_path = os.path.join(export_path, "bin")
+        hivisionai_path = os.path.join(export_path, "hivisionai")
+        sdk_path = os.path.join(export_path, "HY_sdk-**")
+        os.system(f"rm -rf {bin_path} {hivisionai_path} {sdk_path}")
+        # 删除完毕，开始export
+        os.system(f'pip3 install {os.path.join(new_dir, "**.whl")} -t {export_path}')
+        hivisionai_utils.download_dependence(export_path)
+        # 将下载下来的文件夹删除
+        os.system(f'rm -rf {target_zip} && rm -rf {new_dir}')
+        print("Done.")
+
+    @staticmethod
+    def hy_func_init(force):
+        """
+        在HY-func目录下使用hivisionai --init，可以自动将需要的依赖下载到指定位置
+        不过对于比较大的模型——修复模型而言，需要手动下载
+        Args:
+            force: 如果force为True，则会强制重新下载所有的内容，包括修复模型这种比较大的模型
+        Returns:
+            程序执行完毕，会将一些必要的依赖也下载完毕
+        """
+        cwd = HivisionaiParams.getcwd
+        # 判断当前文件夹是否是HY-func
+        dirName = os.path.basename(cwd)
+        assert dirName == "HY-func", "请在正确的文件目录下初始化HY-func!"
+        # 需要下载的内容会存放在HivisionaiParams的functionDependence���量下
+        functionDependence = HivisionaiParams.functionDependence
+        # 下载配置文件
+        configs = functionDependence["configs"]
+        print("正在下载配置文件...")
+        for config in configs:
+            if not force and os.path.exists(config['save_path']):
+                print(f"[pass]: {os.path.basename(config['url'])}")
+                continue
+            print(f"[Download]: {config['url']}")
+            resp = requests.get(config['url'])
+            # json文件存储在text区域，但是其他的不一定
+            open(os.path.join(cwd, config['save_path']), 'w').write(resp.text)
+        # 其他文件，提示访问notion文档
+        print(f"[NOTICE]: 一切准备就绪，请访问下面的文档下载剩下的模型文件:\n{functionDependence['weights']}")
+
+    @staticmethod
+    def hy_func_deploy(functionName: str = None, functionPath: str = None):
+        """
+        在HY-func目录下使用此命令，并且随附功能函数的名称，就可以将HY-func的部署版放到桌面上
+        但是需要注意的是，本方式不适合修复功能使用，修复功能依旧需要手动制作镜像
+        Args:
+            functionName: 功能函数名称
+            functionPath: 需要注册的HY-func路径
+
+        Returns:
+            程序执行完毕，桌面会出现一个同名文件夹
+        """
+        # 为了代码撰写的方便，这里仅仅把模型文件删除，其余配置文件保留
+        # 为了实现在任意位置输入hivisionai --deploy funcName都能成功，在使用前需要在.hivisionai/user_config.json中注册
+        # print(functionName, functionPath)
+        if functionPath is not None:
+            # 更新/添加路径
+            # functionPath为相对于使用路径的路径
+            assert os.path.basename(functionPath) == "HY-func", "所指向路径非HY-func!"
+            func_path = os.path.join(HivisionaiParams.getcwd, functionPath)
+            assert os.path.join(func_path), f"路径不存在: {func_path}"
+            # functionPath的路径写到user_config当中
+            user_config = json.load(open(HivisionaiParams.cloud_config_save, 'rb'))
+            user_config["func_path"] = func_path
+            open(HivisionaiParams.cloud_config_save, 'w').write(json.dumps(user_config))
+            print("HY-func全局路径保存成功!")
+        try:
+            user_config = json.load(open(HivisionaiParams.cloud_config_save, 'rb'))
+            func_path = user_config['func_path']
+        except KeyError:
+            return print("请先使用-p命令注册全局HY-func路径!")
+        # 此时func_path必然存在
+        # print(os.listdir(func_path))
+        assert functionName in os.listdir(func_path), functionName + "功能不存在!"
+        func_path_deploy = os.path.join(func_path, functionName)
+        # 开始复制文件到指定目录
+        # 我们默认移动到Desktop目录下，如果没有此目录，需要先创建一个
+        target_dir = os.path.join(HivisionaiParams.download_path, "Desktop")
+        assert os.path.exists(target_dir), target_dir + "文件路径不存在,你需要先创建一下!"
+        # 开始移动
+        target_dir = os.path.join(target_dir, functionName)
+        print("正在复制需要部署的文件...")
+        os.system(f"rm -rf {target_dir}")
+        os.system(f'cp -rf {func_path_deploy} {target_dir}')
+        os.system(f"cp -rf {os.path.join(func_path, '_lib')} {target_dir}")
+        os.system(f"cp -rf {os.path.join(func_path, '_service')} {target_dir}")
+        # 生成最新的hivisionai
+        print("正在生成hivisionai代码包...")
+        os.system(f'hivisionai -t {target_dir}')
+        # 移动完毕，删除模型文件
+        print("移动完毕，正在删除不需要的文件...")
+        # 模型文件
+        os.system(f"rm -rf {os.path.join(target_dir, '_lib', 'weights', '**')}")
+        # hivisionai生成时的多余文件
+        os.system(f"rm -rf {os.path.join(target_dir, 'bin')} {os.path.join(target_dir, 'HY_sdk**')}")
+        print("部署文件生成成功，你可以开始部署了!")
+
+
+hivisionai_utils = HivisionaiUtils()
+
+
+def entry_point():
+    parser = ArgumentParser()
+    # 查看版本号
+    parser.add_argument("-v", "--version", action="store_true", help="View the current HY-sdk version, which does not represent the final cloud version.")
+    # 自动更新
+    parser.add_argument("-u", "--upgrade", nargs='?', const="-1", type=str, help="Automatically update HY-sdk to the latest version")
+    # 查找云端的HY-sdk版本
+    parser.add_argument("-l", "--list", action="store_true", help="Find HY-sdk versions of the cloud, and keep up to ten")
+    # 下载云端的版本到本地路径
+    parser.add_argument("-t", "--export", nargs='?', const="./", help="Add a path parameter to automatically download the latest version of sdk to this path. If there are no parameters, the default is the current path")
+    # 强制更新附带参数，当一个功能需要强制执行一遍的时候，需要附带此参数
+    parser.add_argument("-f", "--force", action="store_true", help="Enforcement of other functions, execution of a single parameter is meaningless")
+    # 初始化HY-func
+    parser.add_argument("--init", action="store_true", help="Initialization HY-func")
+    # 部署HY-func
+    parser.add_argument("-d", "--deploy", nargs='?', const="-1", type=str, help="Deploy HY-func")
+    # 涉及注册一些自定义内容的时候，需要附带此参数，并写上自定义内容
+    parser.add_argument("-p", "--param", nargs='?', const="-1", type=str, help="When registering some custom content, you need to attach this parameter and write the custom content.")
+    args = parser.parse_args()
+    if args.version:
+        print(version(HivisionaiParams.package_name))
+        sys.exit()
+    if args.upgrade:
+        HivisionaiApps.upgrade(args.upgrade, args.force)
+        sys.exit()
+    if args.list:
+        HivisionaiApps.show_cloud_version()
+        sys.exit()
+    if args.export:
+        HivisionaiApps.export(args.export)
+        sys.exit()
+    if args.init:
+        HivisionaiApps.hy_func_init(args.force)
+        sys.exit()
+    if args.deploy:
+        HivisionaiApps.hy_func_deploy(args.deploy, args.param)
+
+
+if __name__ == "__main__":
+    entry_point()

hivisionai/hyService/cloudService.py

@@ -0,0 +1,406 @@
+"""
+焕影小程序功能服务端的基本工具函数,以类的形式封装
+"""
+try:  # 加上这个try的原因在于本地环境和云函数端的import形式有所不同
+    from qcloud_cos import CosConfig
+    from qcloud_cos import CosS3Client
+except ImportError:
+    try:
+        from qcloud_cos_v5 import CosConfig
+        from qcloud_cos_v5 import CosS3Client
+    except ImportError:
+        raise ImportError("请下载腾讯云COS相关代码包:pip install cos-python-sdk-v5")
+import requests
+import datetime
+import json
+from .error import ProcessError
+import os
+local_path_ = os.path.dirname(__file__)
+
+
+class GetConfig(object):
+    @staticmethod
+    def hy_sdk_client(Id:str, Key:str):
+        # 从cos中寻找文件
+        REGION: str = 'ap-beijing'
+        TOKEN = None
+        SCHEME: str = 'https'
+        BUCKET: str = 'hy-sdk-config-1305323352'
+        client_config = CosConfig(Region=REGION,
+                                  SecretId=Id,
+                                  SecretKey=Key,
+                                  Token=TOKEN,
+                                  Scheme=SCHEME)
+        return CosS3Client(client_config), BUCKET
+
+    def load_json(self, path:str, default_download=False):
+        try:
+            if os.path.isdir(path):
+                raise ProcessError("请输入具体的配置文件路径,而非文件夹!")
+            if default_download is True:
+                print(f"\033[34m 默认强制重新下载配置文件...\033[0m")
+                raise FileNotFoundError
+            with open(path) as f:
+                config = json.load(f)
+                return config
+        except FileNotFoundError:
+            dir_name = os.path.dirname(path)
+            try:
+                os.makedirs(dir_name)
+            except FileExistsError:
+                pass
+            base_name = os.path.basename(path)
+            print(f"\033[34m 正在从COS中下载配置文件...\033[0m")
+            print(f"\033[31m 请注意,接下来会在{dir_name}路径下生成文件{base_name}...\033[0m")
+            Id = input("请输入SecretId:")
+            Key = input("请输入SecretKey:")
+            client, bucket = self.hy_sdk_client(Id, Key)
+            data_bytes = client.get_object(Bucket=bucket,Key=base_name)["Body"].get_raw_stream().read()
+            data = json.loads(data_bytes.decode("utf-8"))
+            # data["SecretId"] = Id  # 未来可以把这个加上
+            # data["SecretKey"] = Key
+            with open(path, "w") as f:
+                data_str = json.dumps(data, ensure_ascii=False)
+                # 如果 ensure_ascii 是 true (即默认值),输出保证将所有输入的非 ASCII 字符转义。
+                # 如果 ensure_ascii 是 false，这些字符会原样输出。
+                f.write(data_str)
+                f.close()
+            print(f"\033[32m 配置文件保存成功\033[0m")
+            return data
+        except json.decoder.JSONDecodeError:
+            print(f"\033[31m WARNING: 配置文件为空!\033[0m")
+            return {}
+
+    def load_file(self, cloud_path:str, local_path:str):
+        """
+        从COS中下载文件到本地,本函数将会被默认执行的,在使用的时候建议加一些限制.
+        :param cloud_path: 云端的文件路径
+        :param local_path: 将云端文件保存在本地的路径
+        """
+        if os.path.isdir(cloud_path):
+            raise ProcessError("请输入具体的云端文件路径,而非文件夹!")
+        if os.path.isdir(local_path):
+            raise ProcessError("请输入具体的本地文件路径,而非文件夹!")
+        dir_name = os.path.dirname(local_path)
+        base_name = os.path.basename(local_path)
+        try:
+            os.makedirs(dir_name)
+        except FileExistsError:
+            pass
+        cloud_name = os.path.basename(cloud_path)
+        print(f"\033[31m 请注意,接下来会在{dir_name}路径下生成文件{base_name}\033[0m")
+        Id = input("请输入SecretId:")
+        Key = input("请输入SecretKey:")
+        client, bucket = self.hy_sdk_client(Id, Key)
+        print(f"\033[34m 正在从COS中下载文件: {cloud_name}, 此过程可能耗费一些时间...\033[0m")
+        data_bytes = client.get_object(Bucket=bucket,Key=cloud_path)["Body"].get_raw_stream().read()
+        # data["SecretId"] = Id  # 未来可以把这个加上
+        # data["SecretKey"] = Key
+        with open(local_path, "wb") as f:
+            # 如果 ensure_ascii 是 true (即默认值),输出保证将所有输入的非 ASCII 字符转义。
+            # 如果 ensure_ascii 是 false，这些字符会原样输出。
+            f.write(data_bytes)
+            f.close()
+        print(f"\033[32m 文件保存成功\033[0m")
+
+
+class CosConf(GetConfig):
+    """
+    从安全的角度出发,将一些默认配置文件上传至COS中,接下来使用COS和它的子类的时候,在第一次使用时需要输入Cuny给的id和key
+    用于连接cos存储桶,下载配置文��.
+    当然,在service_default_download = False的时候,如果在运行路径下已经有conf/service_config.json文件了,
+    那么就不用再次下载了,也不用输入id和key
+    事实上这只需要运行一次,因为配置文件将会被下载至源码文件夹中
+    如果要自定义路径,请在继承的子类中编写__init__函数,将service_path定向到指定路径
+    """
+    def __init__(self) -> None:
+        # 下面这些参数是类的共享参数
+        self.__SECRET_ID: str = None  # 服务的id
+        self.__SECRET_KEY: str = None  # 服务的key
+        self.__REGION: str = None  # 服务的存储桶地区
+        self.__TOKEN: str = None  # 服务的token,目前一直是None
+        self.__SCHEME: str = None  # 服务的访问协议,默认实际上是https
+        self.__BUCKET: str = None  # 服务的存储桶
+        self.__SERVICE_CONFIG: dict = None  # 服务的配置文件
+        self.service_path: str = f"{local_path_}/conf/service_config.json"
+        # 配置文件路径,默认是函数运行的路径下的conf文件夹
+        self.service_default_download = False  # 是否在每次访问配置的时候都重新下载文件
+
+    @property
+    def service_config(self):
+        if self.__SERVICE_CONFIG is None or self.service_default_download is True:
+            self.__SERVICE_CONFIG = self.load_json(self.service_path, self.service_default_download)
+        return self.__SERVICE_CONFIG
+
+    @property
+    def client(self):
+        client_config = CosConfig(Region=self.region,
+                                  SecretId=self.secret_id,
+                                  SecretKey=self.secret_key,
+                                  Token=self.token,
+                                  Scheme=self.scheme)
+        return CosS3Client(client_config)
+
+    def get_key(self, key:str):
+        try:
+            data = self.service_config[key]
+            if data == "None":
+                return None
+            else:
+                return data
+        except KeyError:
+            print(f"\033[31m没有对应键值{key},默认返回None\033[0m")
+            return None
+
+    @property
+    def secret_id(self):
+        if self.__SECRET_ID is None:
+            self.__SECRET_ID = self.get_key("SECRET_ID")
+        return self.__SECRET_ID
+
+    @secret_id.setter
+    def secret_id(self, value:str):
+        self.__SECRET_ID = value
+
+    @property
+    def secret_key(self):
+        if self.__SECRET_KEY is None:
+            self.__SECRET_KEY = self.get_key("SECRET_KEY")
+        return self.__SECRET_KEY
+
+    @secret_key.setter
+    def secret_key(self, value:str):
+        self.__SECRET_KEY = value
+
+    @property
+    def region(self):
+        if self.__REGION is None:
+            self.__REGION = self.get_key("REGION")
+        return self.__REGION
+
+    @region.setter
+    def region(self, value:str):
+        self.__REGION = value
+
+    @property
+    def token(self):
+        # if self.__TOKEN is None:
+        #     self.__TOKEN = self.get_key("TOKEN")
+        # 这里可以注释掉
+        return self.__TOKEN
+
+    @token.setter
+    def token(self, value:str):
+        self.__TOKEN= value
+
+    @property
+    def scheme(self):
+        if self.__SCHEME is None:
+            self.__SCHEME = self.get_key("SCHEME")
+        return self.__SCHEME
+
+    @scheme.setter
+    def scheme(self, value:str):
+        self.__SCHEME = value
+
+    @property
+    def bucket(self):
+        if self.__BUCKET is None:
+            self.__BUCKET = self.get_key("BUCKET")
+        return self.__BUCKET
+
+    @bucket.setter
+    def bucket(self, value):
+        self.__BUCKET = value
+
+    def downloadFile_COS(self, key, bucket:str=None, if_read:bool=False):
+        """
+        从COS下载对象(二进制数据), 如果下载失败就返回None
+        """
+        CosBucket = self.bucket if bucket is None else bucket
+        try:
+            # 将本类的Debug继承给抛弃了
+            # self.debug_print(f"Download from {CosBucket}", font_color="blue")
+            obj = self.client.get_object(
+                Bucket=CosBucket,
+                Key=key
+            )
+            if if_read is True:
+                data = obj["Body"].get_raw_stream().read()  # byte
+                return data
+            else:
+                return obj
+        except Exception as e:
+            print(f"\033[31m下载失败! 错误描述:{e}\033[0m")
+            return None
+
+    def showFileList_COS_base(self, key, bucket, marker:str=""):
+        """
+        返回cos存储桶内部的某个文件夹的内部名称
+        :param key: cos云端的存储路径
+        :param bucket: cos存储桶名称，如果没指定名称（None）就会寻找默认的存储桶
+        :param marker: 标记,用于记录上次查询到哪里了
+        ps:如果需要修改默认的存储桶配置，请在代码运行的时候加入代码 s.bucket = 存储桶名称 （s是对象实例）
+        返回的内容存储在response["Content"]，不过返回的��据大小是有限制的，具体内容还是请看官方文档。
+        """
+        response = self.client.list_objects(
+            Bucket=bucket,
+            Prefix=key,
+            Marker=marker
+        )
+        return response
+
+    def showFileList_COS(self, key, bucket:str=None)->list:
+        """
+        实现查询存储桶中所有对象的操作，因为cos的sdk有返回数据包大小的限制，所以我们需要进行一定的改动
+        """
+        marker = ""
+        file_list = []
+        CosBucket = self.bucket if bucket is None else bucket
+        while True:  # 轮询
+            response = self.showFileList_COS_base(key, CosBucket, marker)
+            try:
+                file_list.extend(response["Contents"])
+            except KeyError as e:
+                print(e)
+                raise
+            if response['IsTruncated'] == 'false':  # 接下来没有数据了,就退出
+                break
+            marker = response['NextMarker']
+        return file_list
+
+    def uploadFile_COS(self, buffer, key, bucket:str=None):
+        """
+        从COS上传数据,需要注意的是必须得是二进制文件
+        """
+        CosBucket = self.bucket if bucket is None else bucket
+        try:
+            self.client.put_object(
+                Bucket=CosBucket,
+                Body=buffer,
+                Key=key
+            )
+            return True
+        except Exception as e:
+            print(e)
+            return False
+
+
+class FuncDiary(CosConf):
+    filter_dict = {"60a5e13da00e6e0001fd53c8": "Cuny",
+                   "612c290f3a9af4000170faad": "守望平凡",
+                   "614de96e1259260001506d6c": "林泽毅-焕影一新"}
+
+    def __init__(self, func_name: str, uid: str, error_conf_path: str = f"{local_path_}/conf/func_error_conf.json"):
+        """
+        日志类的实例化
+        Args:
+            func_name: 功能名称，影响了日志投递的路径
+        """
+        super().__init__()
+        # 配置文件路径,默认是函数运行的路径下的conf文件夹
+        self.service_path: str = os.path.join(os.path.dirname(error_conf_path), "service_config.json")
+        self.error_dict = self.load_json(path=error_conf_path)
+        self.__up: str = f"wx/invokeFunction_c/{datetime.datetime.now().strftime('%Y/%m/%d/%H')}/{func_name}/"
+        self.func_name: str = func_name
+        # 下面这个属性是的日志名称的前缀
+        self.__start_time = datetime.datetime.now().timestamp()
+        h_point = datetime.datetime.strptime(datetime.datetime.now().strftime('%Y/%m/%d/%H'), '%Y/%m/%d/%H')
+        h_point_timestamp = h_point.timestamp()
+        self.__prefix = int(self.__start_time - h_point_timestamp).__str__() + "_"
+        self.__uid = uid
+        self.__diary = None
+
+    def __str__(self):
+        return f"<{self.func_name}> DIARY for {self.__uid}"
+
+    @property
+    def content(self):
+        return self.__diary
+
+    @content.setter
+    def content(self, value: str):
+        if not isinstance(value, dict):
+            raise TypeError("content 只能是字典！")
+        if "status" in value:
+            raise KeyError("status字段已被默认占用，请在日志信息中更换字段名称!")
+        if self.__diary is None:
+            self.__diary = value
+        else:
+            raise PermissionError("为了减小日志对整体代码的影响，<content>只能被覆写一次！")
+
+    def uploadDiary_COS(self, status_id: str, suffix: str = "", bucket: str = "hy-hcy-data-logs-1306602019"):
+        if self.__diary is None:
+            self.__diary = {"status": self.error_dict[status_id]}
+        if status_id == "0000":
+            self.__up += f"True/{self.__uid}/"
+        else:
+            self.__up += f"False/{self.__uid}/"
+        interval = int(10 * (datetime.datetime.now().timestamp() - self.__start_time))
+        prefix = self.__prefix + status_id + "_" + interval.__str__()
+        self.__diary["status"] = self.error_dict[status_id]
+        name = prefix + "_" + suffix if len(suffix) != 0 else prefix
+        self.uploadFile_COS(buffer=json.dumps(self.__diary), key=self.__up + name, bucket=bucket)
+        print(f"{self}上传成功.")
+
+
+class ResponseWebSocket(CosConf):
+    # 网关推送地址
+    __HOST:str = None
+    @property
+    def sendBackHost(self):
+        if self.__HOST is None:
+            self.__HOST = self.get_key("HOST")
+        return self.__HOST
+
+    @sendBackHost.setter
+    def sendBackHost(self, value):
+        self.__HOST = value
+
+    def sendMsg_toWebSocket(self, message,connectionID:str = None):
+        if connectionID is not None:
+            retmsg = {'websocket': {}}
+            retmsg['websocket']['action'] = "data send"
+            retmsg['websocket']['secConnectionID'] = connectionID
+            retmsg['websocket']['dataType'] = 'text'
+            retmsg['websocket']['data'] = json.dumps(message)
+            requests.post(self.sendBackHost, json=retmsg)
+            print("send success!")
+        else:
+            pass
+
+    @staticmethod
+    def create_Msg(status, msg):
+        """
+        本方法用于创建一个用于发送到WebSocket客户端的数据
+        输入的信息部分,需要有如下几个参数:
+        1. id,固定为"return-result"
+        2. status,如果输入为1则status=true, 如果输入为-1则status=false
+        3. obj_key, 图片的云端路径, 这是输入的msg本身自带的
+        """
+        msg['status'] = "false" if status == -1 else 'true'  # 其实最好还是用bool
+        msg['id'] = "async-back-msg"
+        msg['type'] = "funcType"
+        msg["format"] = "imageType"
+        return msg
+
+
+# 功能服务类
+class Service(ResponseWebSocket):
+    """
+    服务的主函数,封装了cos上传/下载功能以及与api网关的一键通讯
+    将类的实例变成一个可被调用的对象,在服务运行的时候,只需要运行该对象即可
+    当然,因为是类,所以支持继承和修改
+    """
+    @classmethod
+    def process(cls, *args, **kwargs):
+        """
+        处理函数,在使用的时候请将之重构
+        """
+        pass
+
+    @classmethod
+    def __call__(cls, *args, **kwargs):
+        pass
+
+

hivisionai/hyService/dbTools.py

@@ -0,0 +1,337 @@
+import os
+import pymongo
+import datetime
+import time
+from .cloudService import GetConfig
+local_path = os.path.dirname(__file__)
+
+
+class DBUtils(GetConfig):
+    """
+    从安全的角度出发,将一些默认配置文件上传至COS中,接下来使用COS和它的子类的时候,在第一次使用时需要输入Cuny给的id和key
+    用于连接数据库等对象
+    当然,在db_default_download = False的时候,如果在运行路径下已经有配置文件了,
+    那么就不用再次下载了,也不用输入id和key
+    事实上这只需要运行一次,因为配置文件将会被下载至源码文件夹中
+    如果要自定义路径,请在继承的子类中编写__init__函数,将service_path定向到指定路径
+    """
+    __BASE_DIR: dict = None
+    __PARAMS_DIR: dict = None
+    db_base_path: str = f"{local_path}/conf/base_config.json"
+    db_params_path: str = f"{local_path}/conf/params.json"
+    db_default_download: bool = False
+
+    @property
+    def base_config(self):
+        if self.__BASE_DIR is None:
+            self.__BASE_DIR = self.load_json(self.db_base_path, self.db_default_download)
+        return self.__BASE_DIR
+
+    @property
+    def db_config(self):
+        return self.base_config["database_config"]
+
+    @property
+    def params_config(self):
+        if self.__PARAMS_DIR is None:
+            self.__PARAMS_DIR = self.load_json(self.db_params_path, self.db_default_download)
+        return self.__PARAMS_DIR
+
+    @property
+    def size_dir(self):
+        return self.params_config["size_config"]
+
+    @property
+    def func_dir(self):
+        return self.params_config["func_config"]
+
+    @property
+    def wx_config(self):
+        return self.base_config["wx_config"]
+
+    def get_dbClient(self):
+        return pymongo.MongoClient(self.db_config["connect_url"])
+
+    @staticmethod
+    def get_time(yyyymmdd=None, delta_date=0):
+        """
+        给出当前的时间
+        :param yyyymmdd: 以yyyymmdd给出的日期时间
+        :param delta_date: 获取减去delta_day后的时间，默认为0就是当天
+        时间格式:yyyy_mm_dd
+        """
+        if yyyymmdd is None:
+            now_time = (datetime.datetime.now() - datetime.timedelta(delta_date)).strftime("%Y-%m-%d")
+            return now_time
+        # 输入了yyyymmdd的数据和delta_date,通过这两个数据返回距离yyyymmdd   delta_date天的时间
+        pre_time = datetime.datetime(int(yyyymmdd[0:4]), int(yyyymmdd[4:6]), int(yyyymmdd[6:8]))
+        return (pre_time - datetime.timedelta(delta_date)).strftime("%Y-%m-%d")
+
+    # 获得时间戳
+    def get_timestamp(self, date_time:str=None) -> int:
+        """
+        输入的日期形式为:"2021-11-29 16:39:45.999"
+        真正必须输入的是前十个字符,及精确到日期,后面的时间可以不输入,不输入则默认置零
+        """
+        def standardDateTime(dt:str) -> str:
+            """
+            规范化时间字符串
+            """
+            if len(dt) < 10:
+                raise ValueError("你必须至少输入准确到天的日期!比如:2021-11-29")
+            elif len(dt) == 10:
+                return dt + " 00:00:00.0"
+            else:
+                try:
+                    date, time = dt.split(" ")
+                except ValueError:
+                    raise ValueError("你只能也必须在日期与具体时间之间增加一个空格,其他地方不能出现空格!")
+                while len(time) < 10:
+                    if len(time) in (2, 5):
+                        time += ":"
+                    elif len(time) == 8:
+                        time += "."
+                    else:
+                        time += "0"
+                return date + " " + time
+        if date_time is None:
+            # 默认返回当前时间(str), date_time精确到毫秒
+            date_time = datetime.datetime.now()
+        # 转换成时间戳
+        else:
+            date_time = standardDateTime(dt=date_time)
+            date_time = datetime.datetime.strptime(date_time, "%Y-%m-%d %H:%M:%S.%f")
+        timestamp_ms = int(time.mktime(date_time.timetuple()) * 1000.0 + date_time.microsecond / 1000.0)
+        return timestamp_ms
+
+    @staticmethod
+    def get_standardTime(yyyy_mm_dd: str):
+        return yyyy_mm_dd[0:4] + yyyy_mm_dd[5:7] + yyyy_mm_dd[8:10]
+
+    def find_oneDay_data(self, db_name: str, collection_name: str, date: str = None) -> dict:
+        """
+        获取指定天数的数据，如果date is None，就自动寻找距今最近的有数据的那一天的数据
+        """
+        df = None  # 应该被返回的数据
+        collection = self.get_dbClient()[db_name][collection_name]
+        if date is None:  # 自动寻找前几天的数据,最多三十天
+            for delta_date in range(1, 31):
+                date_yyyymmdd = self.get_standardTime(self.get_time(delta_date=delta_date))
+                filter_ = {"date": date_yyyymmdd}
+                df = collection.find_one(filter=filter_)
+                if df is not None:
+                    del df["_id"]
+                    break
+        else:
+            filter_ = {"date": date}
+            df = collection.find_one(filter=filter_)
+            if df is not None:
+                del df["_id"]
+        return df
+
+    def find_daysData_byPeriod(self, date_period: tuple, db_name: str, col_name: str):
+        # 给出一个指定的范围日期，返回相应的数据(日期的两头都会被寻找)
+        # 这个函数我们默认数据库中的数据是连续的，即不会出现在 20211221 到 20211229 之间有一天没有数据的情况
+        if len(date_period) != 2:
+            raise ValueError("date_period数据结构：(开始日期，截止日期)")
+        start, end = date_period  # yyyymmdd
+        delta_date = int(end) - int(start)
+        if delta_date < 0:
+            raise ValueError("传入的日期有误！")
+        collection = self.get_dbClient()[db_name][col_name]
+        date = start
+        while int(date) <= int(end):
+            yield collection.find_one(filter={"date": date})
+            date = self.get_standardTime(self.get_time(date, -1))
+
+    @staticmethod
+    def find_biggest_valueDict(dict_: dict):
+        # 寻找字典中数值最大的字段，要求输入的字典的字段值全为数字
+        while len(dict_) > 0:
+            max_value = 0
+            p = None
+            for key in dict_:
+                if dict_[key] > max_value:
+                    p = key
+                    max_value = dict_[key]
+            yield p, max_value
+            del dict_[p]
+
+    def copy_andAdd_dict(self, dict_base, dict_):
+        # 深度拷贝字典，将后者赋值给前者
+        # 如果后者的键名在前者已经存在，则直接相加。这就要求两者的数据是数值型
+        for key in dict_:
+            if key not in dict_base:
+                dict_base[key] = dict_[key]
+            else:
+                if isinstance(dict_[key], int) or isinstance(dict_[key], float):
+                    dict_base[key] = round(dict_[key] + dict_base[key], 2)
+                else:
+                    dict_base[key] = self.copy_andAdd_dict(dict_base[key], dict_[key])
+        return dict_base
+
+    @staticmethod
+    def compare_data(dict1: dict, dict2: dict, suffix: str, save: int, **kwargs):
+        """
+        有两个字典,并且通过kwargs会传输一个新的字典,根据字典中的键值我们进行比对,处理成相应的数据格式
+        并且在dict1中,生成一个新的键值,为kwargs中的元素+suffix
+        save:保留几位小数
+        """
+        new_dict = dict1.copy()
+        for key in kwargs:
+            try:
+                if kwargs[key] not in dict2 or int(dict2[kwargs[key]]) == -1 or float(dict1[kwargs[key]]) <= 0.0:
+                    # 数据不存在
+                    data_new = 5002
+                else:
+                    try:
+                        data_new = round(
+                            ((float(dict1[kwargs[key]]) - float(dict2[kwargs[key]])) / float(dict2[kwargs[key]])) * 100
+                            , save)
+                    except ZeroDivisionError:
+                        data_new = 5002
+                    if data_new == 0.0:
+                        data_new = 0
+            except TypeError as e:
+                print(e)
+                data_new = 5002  # 如果没有之前的数据,默认返回0
+            new_dict[kwargs[key] + suffix] = data_new
+        return new_dict
+
+    @staticmethod
+    def sum_dictList_byKey(dictList: list, **kwargs) -> dict:
+        """
+        有一个列表,列表中的元素为字典,并且所有字典都有一个键值为key的字段,字段值为数字
+        我们将每一个字典的key字段提取后相加,得到该字段值之和.
+        """
+        sum_num = {}
+        if kwargs is None:
+            raise ImportError("Please input at least ONE key")
+        for key in kwargs:
+            sum_num[kwargs[key]] = 0
+        for dict_ in dictList:
+            if not isinstance(dict_, dict):
+                raise TypeError("object is not DICT!")
+            for key in kwargs:
+                sum_num[kwargs[key]] += dict_[kwargs[key]]
+        return sum_num
+
+    @staticmethod
+    def sum_2ListDict(list_dict1: list, list_dict2: list, key_name, data_name):
+        """
+        有两个列表,列表内的元素为字典,我们根据key所对应的键值寻找列表中键值相同的两个元素,将他们的data对应的键值相加
+        生成新的列表字典(其余键值被删除)
+        key仅在一个列表中存在,则直接加入新的列表字典
+        """
+        sum_list = []
+
+        def find_sameKey(kn, key_, ld: list) -> int:
+            for dic_ in ld:
+                if dic_[kn] == key_:
+                    post_ = ld.index(dic_)
+                    return post_
+            return -1
+
+        for dic in list_dict1:
+            key = dic[key_name]  # 键名
+            post = find_sameKey(key_name, key, list_dict2)  # 在list2中寻找相同的位置
+            data = dic[data_name] + list_dict2[post][data_name] if post != -1 else dic[data_name]
+            sum_list.append({key_name: key, data_name: data})
+        return sum_list
+
+    @staticmethod
+    def find_biggest_dictList(dictList: list, key: str = "key", data: str = "value"):
+        """
+        有一个列表，里面每一个元素都是一个字典
+        这些字典有一些共通性质，那就是里面都有一个key键名和一个data键名，后者的键值必须是数字
+        我们根据data键值的大小进行生成，每一次返回列表中data键值最大的数和它的key键值
+        """
+        while len(dictList) > 0:
+            point = 0
+            biggest_num = int(dictList[0][data])
+            biggest_key = dictList[0][key]
+            for i in range(len(dictList)):
+                num = int(dictList[i][data])
+                if num > biggest_num:
+                    point = i
+                    biggest_num = int(dictList[i][data])
+                    biggest_key = dictList[i][key]
+            yield str(biggest_key), biggest_num
+            del dictList[point]
+
+    def get_share_data(self, date_yyyymmdd: str):
+        # 获得用户界面情况
+        visitPage = self.find_oneDay_data(date=date_yyyymmdd,
+                                          db_name="cuny-user-analysis",
+                                          collection_name="daily-userVisitPage")
+        if visitPage is not None:
+            # 这一部分没有得到数据是可以容忍的.不用抛出模态框错误
+            # 获得昨日用户分享情况
+            sum_num = self.sum_dictList_byKey(dictList=visitPage["data_list"],
+                                              key1="page_share_pv",
+                                              key2="page_share_uv")
+        else:
+            # 此时将分享次数等置为-1
+            sum_num = {"page_share_pv": -1, "page_share_uv": -1}
+        return sum_num
+
+    @staticmethod
+    def compare_date(date1_yyyymmdd: str, date2_yyyymmdd: str):
+        # 如果date1是date2的昨天，那么就返回True
+        date1 = int(date1_yyyymmdd)
+        date2 = int(date2_yyyymmdd)
+        return True if date2 - date1 == 1 else False
+
+    def change_time(self, date_yyyymmdd: str, mode: int):
+        # 将yyyymmdd的数据分开为相应的数据形式
+        if mode == 1:
+            if self.compare_date(date_yyyymmdd, self.get_standardTime(self.get_time(delta_date=0))) is False:
+                return date_yyyymmdd[0:4] + "年" + date_yyyymmdd[4:6] + "月" + date_yyyymmdd[6:8] + "日"
+            else:
+                return "昨日"
+        elif mode == 2:
+            date = date_yyyymmdd[0:4] + "." + date_yyyymmdd[4:6] + "." + date_yyyymmdd[6:8]
+            if self.compare_date(date_yyyymmdd, self.get_standardTime(self.get_time(delta_date=0))) is True:
+                return date + "~" + date + " | 昨日"
+            else:
+                return date + "~" + date
+
+    @staticmethod
+    def changeList_dict2List_list(dl: list, order: list):
+        """
+        列表内是一个个字典,本函数将字典拆解,以order的形式排列键值为列表
+        考虑到一些格式的问题,这里我采用生成器的形式封装
+        """
+        for dic in dl:
+            # dic是列表内的字典元素
+            tmp = []
+            for key_name in order:
+                key = dic[key_name]
+                tmp.append(key)
+            yield tmp
+
+    def dict_mapping(self, dict_name: str, id_: str):
+        """
+        进行字典映射，输入字典名称和键名，返回具体的键值
+        如果不存在，则原路返回键名
+        """
+        try:
+            return getattr(self, dict_name)[id_]
+        except KeyError:
+            return id_
+        except AttributeError:
+            print(f"[WARNING]: 本对象内部不存在{dict_name}!")
+            return id_
+
+    @staticmethod
+    def dictAddKey(dic: dict, dic_tmp: dict, **kwargs):
+        """
+        往字典中加入参数，可迭代
+        """
+        for key in kwargs:
+            dic[key] = dic_tmp[key]
+        return dic
+
+
+if __name__ == "__main__":
+    dbu = DBUtils()

hivisionai/hyService/error.py

@@ -0,0 +1,20 @@
+"""
+@author: cuny
+@fileName: error.py
+@create_time: 2022/03/10 下午3:14
+@introduce:
+保存一些定义的错误类型
+"""
+class ProcessError(Exception):
+    def __init__(self, err):
+        super().__init__(err)
+        self.err = err
+    def __str__(self):
+        return self.err
+
+class WrongImageType(TypeError):
+    def __init__(self, err):
+        super().__init__(err)
+        self.err = err
+    def __str__(self):
+        return self.err

hivisionai/hyService/serviceTest.py

@@ -0,0 +1,34 @@
+"""
+用于测试云端或者本地服务的运行是否成功
+"""
+import requests
+import functools
+import cv2
+import time
+
+def httpPostTest(url, msg:dict):
+    """
+    以post请求访问api,携带msg(dict)信息
+    """
+    re = requests.post(url=url, json=msg)
+    print(re.text)
+    return re
+
+
+def localTestImageFunc(path):
+    """
+    在本地端测试算法,需要注意的是本装饰器只支持测试和图像相关算法
+    path代表测试图像的路径,其余参数请写入被装饰的函数中,并且只支持标签形式输入
+    被测试的函数的第一个输入参数必须为图像矩阵(以cv2读入)
+    """
+    def decorator(func):
+        @functools.wraps(func)
+        def wrapper(**kwargs):
+            start = time.time()
+            image = cv2.imread(path)
+            image_out = func(image) if len(kwargs) == 0 else func(image, kwargs)
+            print("END.\n处理时间(不计算加载模型时间){}秒:".format(round(time.time()-start, 2)))
+            cv2.imshow("test", image_out)
+            cv2.waitKey(0)
+        return wrapper
+    return decorator

hivisionai/hyService/utils.py

@@ -0,0 +1,92 @@
+"""
+@author: cuny
+@fileName: utils.py
+@create_time: 2021/12/29 下午1:29
+@introduce:
+焕影服务的一些工具函数,涉及两类:
+1. 开发debug时候的工具函数
+2. 初始化COS配置时的工具函数
+"""
+import cv2
+from .error import WrongImageType
+import numpy as np
+
+class Debug(object):
+    color_dir:dict = {
+        "red":"31m",
+        "green":"32m",
+        "yellow":"33m",
+        "blue":"34m",
+        "common":"38m"
+    }  # 颜色值
+    __DEBUG:bool = True
+
+    @property
+    def debug(self):
+        return self.__DEBUG
+
+    @debug.setter
+    def debug(self, value):
+        if not isinstance(value, bool):
+            raise TypeError("你必须设定debug的值为bool的True或者False")
+        print(f"设置debug为: {value}")
+        self.__DEBUG = value
+
+    def debug_print(self, text, **kwargs):
+        if self.debug is True:
+            key = self.color_dir["common"] if "font_color" not in kwargs else self.color_dir[kwargs["font_color"]]
+            print(f"\033[{key}{text}\033[0m")
+
+    @staticmethod
+    def resize_image_esp(input_image, esp=2000):
+        """
+        输入：
+        input_path：numpy图片
+        esp：限制的最大边长
+        """
+        # resize函数=>可以让原图压缩到最大边为esp的尺寸(不改变比例)
+        width = input_image.shape[0]
+        length = input_image.shape[1]
+        max_num = max(width, length)
+
+        if max_num > esp:
+            print("Image resizing...")
+            if width == max_num:
+                length = int((esp / width) * length)
+                width = esp
+
+            else:
+                width = int((esp / length) * width)
+                length = esp
+            print(length, width)
+            im_resize = cv2.resize(input_image, (length, width), interpolation=cv2.INTER_AREA)
+            return im_resize
+        else:
+            return input_image
+
+    def cv_show(self, *args, **kwargs):
+        def check_images(img):
+            # 判断是否是矩阵类型
+            if not isinstance(img, np.ndarray):
+                raise WrongImageType("输入的图像必须是 np.ndarray 类型!")
+        if self.debug is True:
+            size = 500 if "size" not in kwargs else kwargs["size"]  # 默认缩放尺寸为最大边500像素点
+            if len(args) == 0:
+                raise ProcessError("你必须传入若干图像信息!")
+            flag = False
+            base = None
+            for image in args:
+                check_images(image)
+                if flag is False:
+                    image = self.resize_image_esp(image, size)
+                    h, w = image.shape[0], image.shape[1]
+                    flag = (w, h)
+                    base = image
+                else:
+                    image = cv2.resize(image, flag)
+                    base = np.hstack((base, image))
+            title = "cv_show" if "winname" not in kwargs else kwargs["winname"]
+            cv2.imshow(title, base)
+            cv2.waitKey(0)
+        else:
+            pass

hivisionai/hyTrain/APIs.py

@@ -0,0 +1,197 @@
+import requests, os
+import json
+import hashlib, base64, hmac
+import sys
+import oss2
+from aliyunsdkimageseg.request.v20191230.SegmentBodyRequest import SegmentBodyRequest
+from aliyunsdkimageseg.request.v20191230.SegmentSkinRequest import SegmentSkinRequest
+from aliyunsdkfacebody.request.v20191230.DetectFaceRequest import DetectFaceRequest
+from aliyunsdkcore.client import AcsClient
+
+# 头像抠图参数配置
+def params_of_head(photo_base64, photo_type):
+    print ('测试头像抠图接口 ...')
+    host = 'https://person.market.alicloudapi.com'
+    uri = '/segment/person/headrgba' # 头像抠图返回透明PNG图
+    # uri = '/segment/person/head'   # 头像抠图返回alpha图
+    # uri = '/segment/person/headborder' # 头像抠图返回带白边的透明PNG图
+    return host, uri, {
+        'photo': photo_base64,
+        'type': photo_type,
+        'face_required': 0, # 可选，检测是否必须带有人脸才进行抠图处理，0为检测，1为不检测，默认为0
+        'border_ratio': 0.3, # 可选，仅带白边接口可用，
+                             # 在头像边缘增加白边（或者其他颜色）宽度，取值为0-0.5，
+                             # 这个宽度是相对于图片宽度和高度最大值的比例，
+                             # 比如原图尺寸为640x480，border_ratio为0.2，
+                             # 则添加的白边的宽度为：max(640,480) * 0.2 = 96个像素
+        'margin_color': '#ff0000' # 可选，仅带白边接口可用，
+                                  # 在头像边缘增加边框的颜色，默认为白色
+
+    }
+
+# 头像抠图API
+def wanxing_get_head_api(file_name='/home/parallels/Desktop/change_cloth/input_image/03.jpg',
+                     output_path="./head.png",
+                     app_key='204014294',
+                     secret="pI2uo7AhCFjnaZWYrCCAEjmsZJbK6vzy",
+                     stage='RELEASE'):
+    info = sys.version_info
+    if info[0] < 3:
+        is_python3 = False
+    else:
+        is_python3 = True
+
+    with open(file_name, 'rb') as fp:
+        photo_base64 = base64.b64encode(fp.read())
+    if is_python3:
+        photo_base64 = photo_base64.decode('utf8')
+
+    _, photo_type = os.path.splitext(file_name)
+    photo_type = photo_type.lstrip('.')
+    # print(photo_type)
+    # print(photo_base64)
+
+    # host, uri, body_json = params_of_portrait_matting(photo_base64, photo_type)
+    # host, uri, body_json = params_of_object_matting(photo_base64)
+    # host, uri, body_json = params_of_idphoto(photo_base64, photo_type)
+    host, uri, body_json = params_of_head(photo_base64, photo_type)
+    # host, uri,  body_json = params_of_crop(photo_base64)
+    api = host + uri
+
+    body = json.dumps(body_json)
+    md5lib = hashlib.md5()
+    if is_python3:
+        md5lib.update(body.encode('utf8'))
+    else:
+        md5lib.update(body)
+    body_md5 = md5lib.digest()
+    body_md5 = base64.b64encode(body_md5)
+    if is_python3:
+        body_md5 = body_md5.decode('utf8')
+
+    method = 'POST'
+    accept = 'application/json'
+    content_type = 'application/octet-stream; charset=utf-8'
+    date_str = ''
+    headers = ''
+
+    string_to_sign = method + '\n' \
+                    + accept + '\n' \
+                    + body_md5 + '\n' \
+                    + content_type + '\n' \
+                    + date_str + '\n' \
+                    + headers \
+                    + uri
+    if is_python3:
+        signed = hmac.new(secret.encode('utf8'),
+                          string_to_sign.encode('utf8'),
+                          digestmod=hashlib.sha256).digest()
+    else:
+        signed = hmac.new(secret, string_to_sign, digestmod=hashlib.sha256).digest()
+    signed = base64.b64encode(signed)
+    if is_python3:
+        signed = signed.decode('utf8')
+
+    headers = {
+        'Accept': accept,
+        'Content-MD5': body_md5,
+        'Content-Type': content_type,
+        'X-Ca-Key': app_key,
+        'X-Ca-Stage': stage,
+        'X-Ca-Signature': signed
+    }
+    #print signed
+
+
+    resp = requests.post(api, data=body, headers=headers)
+    # for u,v in resp.headers.items():
+    #     print(u+": " + v)
+    try:
+        res = resp.content
+        res = json.loads(res)
+        # print ('res:', res)
+        if str(res['status']) == '0':
+            # print ('成功!')
+            file_object = requests.get(res["data"]["result"])
+            # print(file_object)
+            with open(output_path, 'wb') as local_file:
+                local_file.write(file_object.content)
+
+            # image = cv2.imread("./test_head.png", -1)
+            # return image
+        else:
+            pass
+            # print ('失败!')
+    except:
+        print('failed parse:', resp)
+
+# 阿里云抠图API
+def aliyun_human_matting_api(input_path, output_path, type="human"):
+    auth = oss2.Auth('LTAI5tP2NxdzSFfpKYxZFCuJ', 'VzbGdUbRawuMAitekP3ORfrw0i3NEX')
+    bucket = oss2.Bucket(auth, 'https://oss-cn-shanghai.aliyuncs.com', 'huanying-api')
+    key = os.path.basename(input_path)
+    origin_image = input_path
+    try:
+        bucket.put_object_from_file(key, origin_image, headers={"Connection":"close"})
+    except Exception as e:
+        print(e)
+
+    url = bucket.sign_url('GET', key, 10 * 60)
+    client = AcsClient('LTAI5tP2NxdzSFfpKYxZFCuJ', 'VzbGdUbRawuMAitekP3ORfrw0i3NEX', 'cn-shanghai')
+    if type == "human":
+        request = SegmentBodyRequest()
+    elif type == "skin":
+        request = SegmentSkinRequest()
+    request.set_accept_format('json')
+    request.set_ImageURL(url)
+
+    try:
+        response = client.do_action_with_exception(request)
+        response_dict = eval(str(response, encoding='utf-8'))
+        if type == "human":
+            output_url = response_dict['Data']['ImageURL']
+        elif type == "skin":
+            output_url = response_dict['Data']['Elements'][0]['URL']
+        file_object = requests.get(output_url)
+        with open(output_path, 'wb') as local_file:
+            local_file.write(file_object.content)
+            bucket.delete_object(key)
+    except Exception as e:
+        print(e)
+        response = client.do_action_with_exception(request)
+        response_dict = eval(str(response, encoding='utf-8'))
+        print(response_dict)
+        output_url = response_dict['Data']['ImageURL']
+        file_object = requests.get(output_url)
+        with open(output_path, 'wb') as local_file:
+            local_file.write(file_object.content)
+            bucket.delete_object(key)
+
+# 阿里云人脸检测API
+def aliyun_face_detect_api(input_path, type="human"):
+    auth = oss2.Auth('LTAI5tP2NxdzSFfpKYxZFCuJ', 'VzbGdUbRawuMAitekP3ORfrw0i3NEX')
+    bucket = oss2.Bucket(auth, 'https://oss-cn-shanghai.aliyuncs.com', 'huanying-api')
+    key = os.path.basename(input_path)
+    origin_image = input_path
+    try:
+        bucket.put_object_from_file(key, origin_image, headers={"Connection":"close"})
+    except Exception as e:
+        print(e)
+
+    url = bucket.sign_url('GET', key, 10 * 60)
+    client = AcsClient('LTAI5tP2NxdzSFfpKYxZFCuJ', 'VzbGdUbRawuMAitekP3ORfrw0i3NEX', 'cn-shanghai')
+    if type == "human":
+        request = DetectFaceRequest()
+    request.set_accept_format('json')
+    request.set_ImageURL(url)
+    try:
+        response = client.do_action_with_exception(request)
+        response_json = json.loads(str(response, encoding='utf-8'))
+        print(response_json["Data"]["PoseList"][-1])
+        bucket.delete_object(key)
+        return response_json["Data"]["PoseList"][-1]
+    except Exception as e:
+        print(e)
+
+if __name__ == "__main__":
+    wanxing_get_head_api()

hivisionai/hyTrain/DataProcessing.py

@@ -0,0 +1,37 @@
+import cv2
+import random
+from scipy.ndimage import grey_erosion, grey_dilation
+import numpy as np
+from glob import glob
+import random
+
+
+def make_a_and_trimaps(input_image, resize=(512, 512)):
+    image = cv2.resize(input_image, resize)
+    b, g, r, a = cv2.split(image)
+
+    a_scale_resize = a / 255
+    trimap = (a_scale_resize >= 0.95).astype("float32")
+    not_bg = (a_scale_resize > 0).astype("float32")
+    d_size = a.shape[0] // 256 * random.randint(10, 20)
+    e_size = a.shape[0] // 256 * random.randint(10, 20)
+    trimap[np.where((grey_dilation(not_bg, size=(d_size, d_size))
+                    - grey_erosion(trimap, size=(e_size, e_size))) != 0)] = 0.5
+
+    return a, trimap*255
+
+
+def get_filedir_filelist(input_path):
+    return glob(input_path+"/*")
+
+
+def extChange(filedir, ext="png"):
+    ext_origin = str(filedir).split(".")[-1]
+    return filedir.replace(ext_origin, ext)
+
+def random_image_crop(input_image:np.array, crop_size=(512,512)):
+    height, width = input_image.shape[0], input_image.shape[1]
+    crop_height, crop_width = crop_size[0], crop_size[1]
+    x = random.randint(0, width-crop_width)
+    y = random.randint(0, height-crop_height)
+    return input_image[y:y+crop_height, x:x+crop_width]

hivisionai/hycv/FaceDetection68/__init__.py

@@ -0,0 +1,8 @@
+"""
+@author: cuny
+@fileName: __init__.py
+@create_time: 2022/01/03 下午9:39
+@introduce:
+人脸68关键点检测sdk的__init__包,实际上是对dlib的封装
+"""
+from .faceDetection68 import FaceDetection68, PoseEstimator68

hivisionai/hycv/FaceDetection68/faceDetection68.py

@@ -0,0 +1,443 @@
+"""
+@author: cuny
+@fileName: faceDetection68.py
+@create_time: 2022/01/03 下午10:20
+@introduce:
+人脸68关键点检测主文件,以类的形式封装
+"""
+from hivisionai.hyService.cloudService import GetConfig
+import os
+import cv2
+import dlib
+import numpy as np
+local_file = os.path.dirname(__file__)
+PREDICTOR_PATH = f"{local_file}/weights/shape_predictor_68_face_landmarks.dat"  # 关键点检测模型路径
+MODULE3D_PATH = f"{local_file}/weights/68_points_3D_model.txt"  # 3d的68点配置文件路径
+
+# 定义一个人脸检测错误的错误类
+class FaceError(Exception):
+    def __init__(self, err):
+        super().__init__(err)
+        self.err = err
+    def __str__(self):
+        return self.err
+
+class FaceConfig68(object):
+    face_area:list = None  # 一些其他的参数,在本类中实际没啥用
+    FACE_POINTS = list(range(17, 68))  # 人脸轮廓点索引
+    MOUTH_POINTS = list(range(48, 61))  # 嘴巴点索引
+    RIGHT_BROW_POINTS = list(range(17, 22))  # 右眉毛索引
+    LEFT_BROW_POINTS = list(range(22, 27))  # 左眉毛索引
+    RIGHT_EYE_POINTS = list(range(36, 42))  # 右眼索引
+    LEFT_EYE_POINTS = list(range(42, 48))  # 左眼索引
+    NOSE_POINTS = list(range(27, 35))  # 鼻子索引
+    JAW_POINTS = list(range(0, 17))  # 下巴索引
+    LEFT_FACE = list(range(42, 48)) + list(range(22, 27))  # 左半边脸索引
+    RIGHT_FACE = list(range(36, 42)) + list(range(17, 22))  # 右半边脸索引
+    JAW_END = 17  # 下巴结束点
+    FACE_START = 0  # 人脸识别开始
+    FACE_END = 68  # 人脸识别结束
+    # 下面这个是整张脸的mark点,可以用:
+    # for group in self.OVERLAY_POINTS:
+    #     cv2.fillConvexPoly(face_mask, cv2.convexHull(dst_matrix[group]), (255, 255, 255))
+    # 来形成人脸蒙版
+    OVERLAY_POINTS = [
+        JAW_POINTS,
+        LEFT_FACE,
+        RIGHT_FACE
+    ]
+
+class FaceDetection68(FaceConfig68):
+    """
+    人脸68关键点检测主类,当然使用的是dlib开源包
+    """
+    def __init__(self, model_path:str=None, default_download:bool=False, *args, **kwargs):
+        # 初始化,检查并下载模型
+        self.model_path = PREDICTOR_PATH if model_path is None else model_path
+        if not os.path.exists(self.model_path) or default_download:  # 下载配置
+            gc = GetConfig()
+            gc.load_file(cloud_path="weights/shape_predictor_68_face_landmarks.dat",
+                         local_path=self.model_path)
+        self.__detector = None
+        self.__predictor = None
+
+    @property
+    def detector(self):
+        if self.__detector is None:
+            self.__detector = dlib.get_frontal_face_detector()  # 获取人脸分类器
+        return self.__detector
+    @property
+    def predictor(self):
+        if self.__predictor is None:
+            self.__predictor = dlib.shape_predictor(self.model_path)  # 输入模型,构建特征提取器
+        return self.__predictor
+
+    @staticmethod
+    def draw_face(img:np.ndarray, dets:dlib.rectangles, *args, **kwargs):
+        # 画人脸检测框, 为了一些兼容操作我没有设置默认显示,可以在运行完本函数后将返回值进行self.cv_show()
+        tmp = img.copy()
+        for face in dets:
+            # 左上角(x1,y1)，右下角(x2,y2)
+            x1, y1, x2, y2 = face.left(), face.top(), face.right(), face.bottom()
+            # print(x1, y1, x2, y2)
+            cv2.rectangle(tmp, (x1, y1), (x2, y2), (0, 255, 0), 2)
+        return tmp
+
+    @staticmethod
+    def draw_points(img:np.ndarray, landmarks:np.matrix, if_num:int=False, *args, **kwargs):
+        """
+        画人脸关键点, 为了一些兼容操作我没有设置默认显示,可以在运行完本函数后将返回值进行self.cv_show()
+        :param img: 输入的是人脸检测的图,必须是3通道或者灰度图
+        :param if_num: 是否在画关键点的同时画上编号
+        :param landmarks: 输入的关键点矩阵信息
+        """
+        tmp = img.copy()
+        h, w, c = tmp.shape
+        r = int(h / 100) - 2 if h > w else int(w / 100) - 2
+        for idx, point in enumerate(landmarks):
+            # 68点的坐标
+            pos = (point[0, 0], point[0, 1])
+            # 利用cv2.circle给每个特征点画一个圈，共68个
+            cv2.circle(tmp, pos, r, color=(0, 0, 255), thickness=-1)  # bgr
+            if if_num is True:
+                # 利用cv2.putText输出1-68
+                font = cv2.FONT_HERSHEY_SIMPLEX
+                cv2.putText(tmp, str(idx + 1), pos, font, 0.8, (0, 0, 255), 1, cv2.LINE_AA)
+        return tmp
+
+    @staticmethod
+    def resize_image_esp(input_image_, esp=2000):
+        """
+        输入：
+        input_path：numpy图片
+        esp：限制的最大边长
+        """
+        # resize函数=>可以让原图压缩到最大边为esp的尺寸(不改变比例)
+        width = input_image_.shape[0]
+
+        length = input_image_.shape[1]
+        max_num = max(width, length)
+
+        if max_num > esp:
+            print("Image resizing...")
+            if width == max_num:
+                length = int((esp / width) * length)
+                width = esp
+
+            else:
+                width = int((esp / length) * width)
+                length = esp
+            print(length, width)
+            im_resize = cv2.resize(input_image_, (length, width), interpolation=cv2.INTER_AREA)
+            return im_resize
+        else:
+            return input_image_
+
+    def facesPoints(self, img:np.ndarray, esp:int=None, det_num:int=1,*args, **kwargs):
+        """
+        :param img: 输入的是人脸检测的图,必须是3通道或者灰度图
+        :param esp: 如果输入了具体数值,会将图片的最大边长缩放至esp,另一边等比例缩放
+        :param det_num: 人脸检测的迭代次数, 采样次数越多,越有利于检测到更多的人脸
+        :return
+        返回人脸检测框对象dets, 人脸关键点矩阵列表(列表中每个元素为一个人脸的关键点矩阵), 人脸关键点元组列表(列表中每个元素为一个人脸的关键点列表)
+        """
+        # win = dlib.image_window()
+        # win.clear_overlay()
+        # win.set_image(img)
+        # dlib的人脸检测装置
+        if esp is not None:
+            img = self.resize_image_esp(input_image_=img, esp=esp)
+        dets = self.detector(img, det_num)
+        # self.draw_face(img, dets)
+        # font_color = "green" if len(dets) == 1 else "red"
+        # dg.debug_print("Number of faces detected: {}".format(len(dets)), font_color=font_color)
+        landmarkList = []
+        pointsList = []
+        for d in dets:
+            shape = self.predictor(img, d)
+            landmark = np.matrix([[p.x, p.y] for p in shape.parts()])
+            landmarkList.append(landmark)
+            point_list = []
+            for p in landmark.tolist():
+                point_list.append((p[0], p[1]))
+            pointsList.append(point_list)
+        # dg.debug_print("Key point detection SUCCESS.", font_color="green")
+        return dets, landmarkList, pointsList
+
+    def facePoints(self, img:np.ndarray, esp:int=None, det_num:int=1, *args, **kwargs):
+        """
+        本函数与facesPoints大致类似,主要区别在于本函数默认只能返回一个人脸关键点参数
+        """
+        # win = dlib.image_window()
+        # win.clear_overlay()
+        # win.set_image(img)
+        # dlib的人脸检测装置, 参数1表示对图片进行上采样一次，采样次数越多,越有利于检测到更多的人脸
+        if esp is not None:
+            img = self.resize_image_esp(input_image_=img, esp=esp)
+        dets = self.detector(img, det_num)
+        # self.draw_face(img, dets)
+        font_color = "green" if len(dets) == 1 else "red"
+        # dg.debug_print("Number of faces detected: {}".format(len(dets)), font_color=font_color)
+        if font_color=="red":
+            # 本检测函数必然只能检测出一张人脸
+            raise FaceError("Face detection error!!!")
+        d = dets[0]  # 唯一人脸
+        shape = self.predictor(img, d)
+        landmark = np.matrix([[p.x, p.y] for p in shape.parts()])
+        # print("face_landmark:", landmark)  # 打印关键点矩阵
+        # shape = predictor(img, )
+        # dlib.hit_enter_to_continue()
+        # 返回关键点矩阵,关键点,
+        point_list = []
+        for p in landmark.tolist():
+            point_list.append((p[0], p[1]))
+        # dg.debug_print("Key point detection SUCCESS.", font_color="green")
+        # 最后的一个返回参数只会被计算一次，用于标明脸部框的位置
+        # [人脸框左上角纵坐标（top），左上角横坐标（left），人脸框宽度（width），人脸框高度（height）]
+        return dets, landmark, point_list
+
+class PoseEstimator68(object):
+    """
+    Estimate head pose according to the facial landmarks
+    本类将实现但输入图的人脸姿态检测
+    """
+    def __init__(self, img:np.ndarray, params_path:str=None, default_download:bool=False):
+        self.params_path = MODULE3D_PATH if params_path is None else params_path
+        if not os.path.exists(self.params_path) or default_download:
+            gc = GetConfig()
+            gc.load_file(cloud_path="weights/68_points_3D_model.txt",
+                         local_path=self.params_path)
+        h, w, c = img.shape
+        self.size = (h, w)
+        # 3D model points.
+        self.model_points = np.array([
+            (0.0, 0.0, 0.0),             # Nose tip
+            (0.0, -330.0, -65.0),        # Chin
+            (-225.0, 170.0, -135.0),     # Left eye left corner
+            (225.0, 170.0, -135.0),      # Right eye right corner
+            (-150.0, -150.0, -125.0),    # Mouth left corner
+            (150.0, -150.0, -125.0)      # Mouth right corner
+        ]) / 4.5
+        self.model_points_68 = self._get_full_model_points()
+
+        # Camera internals
+        self.focal_length = self.size[1]
+        self.camera_center = (self.size[1] / 2, self.size[0] / 2)
+        self.camera_matrix = np.array(
+            [[self.focal_length, 0, self.camera_center[0]],
+             [0, self.focal_length, self.camera_center[1]],
+             [0, 0, 1]], dtype="double")
+
+        # Assuming no lens distortion
+        self.dist_coeefs = np.zeros((4, 1))
+
+        # Rotation vector and translation vector
+        self.r_vec = np.array([[0.01891013], [0.08560084], [-3.14392813]])
+        self.t_vec = np.array(
+            [[-14.97821226], [-10.62040383], [-2053.03596872]])
+        # self.r_vec = None
+        # self.t_vec = None
+
+    def _get_full_model_points(self):
+        """Get all 68 3D model points from file"""
+        raw_value = []
+        with open(self.params_path) as file:
+            for line in file:
+                raw_value.append(line)
+        model_points = np.array(raw_value, dtype=np.float32)
+        model_points = np.reshape(model_points, (3, -1)).T
+
+        # Transform the model into a front view.
+        # model_points[:, 0] *= -1
+        model_points[:, 1] *= -1
+        model_points[:, 2] *= -1
+        return model_points
+
+    def show_3d_model(self):
+        from matplotlib import pyplot
+        from mpl_toolkits.mplot3d import Axes3D
+        fig = pyplot.figure()
+        ax = Axes3D(fig)
+
+        x = self.model_points_68[:, 0]
+        y = self.model_points_68[:, 1]
+        z = self.model_points_68[:, 2]
+
+        ax.scatter(x, y, z)
+        ax.axis('auto')
+        pyplot.xlabel('x')
+        pyplot.ylabel('y')
+        pyplot.show()
+
+    def solve_pose(self, image_points):
+        """
+        Solve pose from image points
+        Return (rotation_vector, translation_vector) as pose.
+        """
+        assert image_points.shape[0] == self.model_points_68.shape[0], "3D points and 2D points should be of same number."
+        (_, rotation_vector, translation_vector) = cv2.solvePnP(
+            self.model_points, image_points, self.camera_matrix, self.dist_coeefs)
+
+        # (success, rotation_vector, translation_vector) = cv2.solvePnP(
+        #     self.model_points,
+        #     image_points,
+        #     self.camera_matrix,
+        #     self.dist_coeefs,
+        #     rvec=self.r_vec,
+        #     tvec=self.t_vec,
+        #     useExtrinsicGuess=True)
+        return rotation_vector, translation_vector
+
+    def solve_pose_by_68_points(self, image_points):
+        """
+        Solve pose from all the 68 image points
+        Return (rotation_vector, translation_vector) as pose.
+        """
+        if self.r_vec is None:
+            (_, rotation_vector, translation_vector) = cv2.solvePnP(
+                self.model_points_68, image_points, self.camera_matrix, self.dist_coeefs)
+            self.r_vec = rotation_vector
+            self.t_vec = translation_vector
+
+        (_, rotation_vector, translation_vector) = cv2.solvePnP(
+            self.model_points_68,
+            image_points,
+            self.camera_matrix,
+            self.dist_coeefs,
+            rvec=self.r_vec,
+            tvec=self.t_vec,
+            useExtrinsicGuess=True)
+
+        return rotation_vector, translation_vector
+
+    # def draw_annotation_box(self, image, rotation_vector, translation_vector, color=(255, 255, 255), line_width=2):
+    #     """Draw a 3D box as annotation of pose"""
+    #     point_3d = []
+    #     rear_size = 75
+    #     rear_depth = 0
+    #     point_3d.append((-rear_size, -rear_size, rear_depth))
+    #     point_3d.append((-rear_size, rear_size, rear_depth))
+    #     point_3d.append((rear_size, rear_size, rear_depth))
+    #     point_3d.append((rear_size, -rear_size, rear_depth))
+    #     point_3d.append((-rear_size, -rear_size, rear_depth))
+    #
+    #     front_size = 100
+    #     front_depth = 100
+    #     point_3d.append((-front_size, -front_size, front_depth))
+    #     point_3d.append((-front_size, front_size, front_depth))
+    #     point_3d.append((front_size, front_size, front_depth))
+    #     point_3d.append((front_size, -front_size, front_depth))
+    #     point_3d.append((-front_size, -front_size, front_depth))
+    #     point_3d = np.array(point_3d, dtype=np.float64).reshape(-1, 3)
+    #
+    #     # Map to 2d image points
+    #     (point_2d, _) = cv2.projectPoints(point_3d,
+    #                                       rotation_vector,
+    #                                       translation_vector,
+    #                                       self.camera_matrix,
+    #                                       self.dist_coeefs)
+    #     point_2d = np.int32(point_2d.reshape(-1, 2))
+    #
+    #     # Draw all the lines
+    #     cv2.polylines(image, [point_2d], True, color, line_width, cv2.LINE_AA)
+    #     cv2.line(image, tuple(point_2d[1]), tuple(
+    #         point_2d[6]), color, line_width, cv2.LINE_AA)
+    #     cv2.line(image, tuple(point_2d[2]), tuple(
+    #         point_2d[7]), color, line_width, cv2.LINE_AA)
+    #     cv2.line(image, tuple(point_2d[3]), tuple(
+    #         point_2d[8]), color, line_width, cv2.LINE_AA)
+    #
+    # def draw_axis(self, img, R, t):
+    #     points = np.float32(
+    #         [[30, 0, 0], [0, 30, 0], [0, 0, 30], [0, 0, 0]]).reshape(-1, 3)
+    #
+    #     axisPoints, _ = cv2.projectPoints(
+    #         points, R, t, self.camera_matrix, self.dist_coeefs)
+    #
+    #     img = cv2.line(img, tuple(axisPoints[3].ravel()), tuple(
+    #         axisPoints[0].ravel()), (255, 0, 0), 3)
+    #     img = cv2.line(img, tuple(axisPoints[3].ravel()), tuple(
+    #         axisPoints[1].ravel()), (0, 255, 0), 3)
+    #     img = cv2.line(img, tuple(axisPoints[3].ravel()), tuple(
+    #         axisPoints[2].ravel()), (0, 0, 255), 3)
+
+    def draw_axes(self, img, R, t):
+        """
+        OX is drawn in red, OY in green and OZ in blue.
+        """
+        return cv2.drawFrameAxes(img, self.camera_matrix, self.dist_coeefs, R, t, 30)
+
+    @staticmethod
+    def get_pose_marks(marks):
+        """Get marks ready for pose estimation from 68 marks"""
+        pose_marks = [marks[30], marks[8], marks[36], marks[45], marks[48], marks[54]]
+        return pose_marks
+
+    @staticmethod
+    def rot_params_rm(R):
+        from math import pi,atan2,asin, fabs
+        # x轴
+        pitch  = (180 * atan2(-R[2][1], R[2][2]) / pi)
+        f = (0 > pitch) - (0 < pitch)
+        pitch = f * (180 - fabs(pitch))
+        # y轴
+        yaw = -(180 * asin(R[2][0]) / pi)
+        # z轴
+        roll = (180 * atan2(-R[1][0], R[0][0]) / pi)
+        f = (0 > roll) - (0 < roll)
+        roll = f * (180 - fabs(roll))
+        if not fabs(roll) < 90.0:
+            roll = f * (180 - fabs(roll))
+        rot_params = [pitch, yaw, roll]
+        return rot_params
+
+    @staticmethod
+    def rot_params_rv(rvec_):
+        from math import pi, atan2, asin, fabs
+        R = cv2.Rodrigues(rvec_)[0]
+        # x轴
+        pitch  = (180 * atan2(-R[2][1], R[2][2]) / pi)
+        f = (0 > pitch) - (0 < pitch)
+        pitch = f * (180 - fabs(pitch))
+        # y轴
+        yaw = -(180 * asin(R[2][0]) / pi)
+        # z轴
+        roll = (180 * atan2(-R[1][0], R[0][0]) / pi)
+        f = (0 > roll) - (0 < roll)
+        roll = f * (180 - fabs(roll))
+        rot_params = [pitch, yaw, roll]
+        return rot_params
+
+    def imageEulerAngle(self, img_points):
+        # 这里的img_points对应的是facePoints的第三个返回值,注意是facePoints而非facesPoints
+        # 对于facesPoints而言,需要把第三个返回值逐一取出再输入
+        # 把列表转为矩阵,且编码形式为float64
+        img_points = np.array(img_points, dtype=np.float64)
+        rvec, tvec = self.solve_pose_by_68_points(img_points)
+        # 旋转向量转旋转矩阵
+        R = cv2.Rodrigues(rvec)[0]
+        # theta = np.linalg.norm(rvec)
+        # r = rvec / theta
+        # R_ = np.array([[0, -r[2][0], r[1][0]],
+        #                [r[2][0], 0, -r[0][0]],
+        #                [-r[1][0], r[0][0], 0]])
+        # R = np.cos(theta) * np.eye(3) + (1 - np.cos(theta)) * r * r.T + np.sin(theta) * R_
+        # 旋转矩阵转欧拉角
+        eulerAngle = self.rot_params_rm(R)
+        # 返回一个元组和欧拉角列表
+        return (rvec, tvec, R), eulerAngle
+
+
+# if __name__ == "__main__":
+#     # 示例
+#     from hyService.utils import Debug
+#     dg = Debug()
+#     image_input = cv2.imread("./test.jpg")  # 读取一张图片, 必须是三通道或者灰度图
+#     fd68 = FaceDetection68()  # 初始化人脸关键点检测类
+#     dets_, landmark_, point_list_ = fd68.facePoints(image_input)  # 输入图片. 检测单张人脸
+#     # dets_, landmark_, point_list_ = fd68.facesPoints(input_image)  # 输入图片. 检测多张人脸
+#     img = fd68.draw_points(image_input, landmark_)
+#     dg.cv_show(img)
+#     pe = PoseEstimator68(image_input)
+#     _, ea = pe.imageEulerAngle(point_list_)  # 输入关键点列表, 如果要使用facesPoints,则输入的是point_list_[i]
+#     print(ea)  # 结果

hivisionai/hycv/__init__.py

@@ -0,0 +1 @@
+from .utils import cover_mask, get_box, get_box_pro, filtering, cut, zoom_image_without_change_size

hivisionai/hycv/error.py

@@ -0,0 +1,16 @@
+"""
+定义hycv的一些错误类型，其实和hyService大致相同
+"""
+class ProcessError(Exception):
+    def __init__(self, err):
+        super().__init__(err)
+        self.err = err
+    def __str__(self):
+        return self.err
+
+class WrongImageType(TypeError):
+    def __init__(self, err):
+        super().__init__(err)
+        self.err = err
+    def __str__(self):
+        return self.err

hivisionai/hycv/face_tools.py

@@ -0,0 +1,427 @@
+import cv2
+import os
+import onnxruntime
+from .mtcnn_onnx.detector import detect_faces
+from .tensor2numpy import *
+from PIL import Image
+import requests
+from os.path import exists
+
+
+def download_img(img_url, base_dir):
+    print("Downloading Onnx Model in:", img_url)
+    r = requests.get(img_url, stream=True)
+    filename = img_url.split("/")[-1]
+    # print(r.status_code) # 返回状态码
+    if r.status_code == 200:
+        open(f'{base_dir}/{filename}', 'wb').write(r.content) # 将内容写入图片
+        print(f"Download Finshed -- {filename}")
+    del r
+
+class BBox(object):
+    # bbox is a list of [left, right, top, bottom]
+    def __init__(self, bbox):
+        self.left = bbox[0]
+        self.right = bbox[1]
+        self.top = bbox[2]
+        self.bottom = bbox[3]
+        self.x = bbox[0]
+        self.y = bbox[2]
+        self.w = bbox[1] - bbox[0]
+        self.h = bbox[3] - bbox[2]
+
+    # scale to [0,1]
+    def projectLandmark(self, landmark):
+        landmark_= np.asarray(np.zeros(landmark.shape))
+        for i, point in enumerate(landmark):
+            landmark_[i] = ((point[0]-self.x)/self.w, (point[1]-self.y)/self.h)
+        return landmark_
+
+    # landmark of (5L, 2L) from [0,1] to real range
+    def reprojectLandmark(self, landmark):
+        landmark_= np.asarray(np.zeros(landmark.shape))
+        for i, point in enumerate(landmark):
+            x = point[0] * self.w + self.x
+            y = point[1] * self.h + self.y
+            landmark_[i] = (x, y)
+        return landmark_
+
+
+def face_detect_mtcnn(input_image, color_key=None, filter=None):
+    """
+    Inputs:
+    - input_image: OpenCV Numpy.array
+    - color_key: 当color_key等于"RGB"时,将不进行转换操作
+    - filter：当filter等于True时，将抛弃掉置信度小于0.98或人脸框面积小于3600的人脸
+    return:
+    - faces: 带有人脸信息的变量
+    - landmarks: face alignment
+    """
+    if color_key != "RGB":
+        input_image = cv2.cvtColor(input_image, cv2.COLOR_BGR2RGB)
+
+    input_image = Image.fromarray(input_image)
+    faces, landmarks = detect_faces(input_image)
+
+    if filter:
+        face_clean = []
+        for face in faces:
+            confidence = face[-1]
+            x1 = face[0]
+            y1 = face[1]
+            x2 = face[2]
+            y2 = face[3]
+            w = x2 - x1 + 1
+            h = y2 - y1 + 1
+            measure = w * h
+            if confidence >= 0.98 and measure > 3600:
+                # 如果检测到的人脸置信度小于0.98或人脸框面积小于3600,则抛弃该人脸
+                face_clean.append(face)
+        faces = face_clean
+
+    return faces, landmarks
+
+
+def mtcnn_bbox(face, width, height):
+    x1 = face[0]
+    y1 = face[1]
+    x2 = face[2]
+    y2 = face[3]
+    w = x2 - x1 + 1
+    h = y2 - y1 + 1
+
+    size = int(max([w, h]) * 1.1)
+    cx = x1 + w // 2
+    cy = y1 + h // 2
+    x1 = cx - size // 2
+    x2 = x1 + size
+    y1 = cy - size // 2
+    y2 = y1 + size
+
+    dx = max(0, -x1)
+    dy = max(0, -y1)
+    x1 = max(0, x1)
+    y1 = max(0, y1)
+
+    edx = max(0, x2 - width)
+    edy = max(0, y2 - height)
+    x2 = min(width, x2)
+    y2 = min(height, y2)
+
+    return x1, x2, y1, y2, dx, dy, edx, edy
+
+
+def mtcnn_cropped_face(face_box, image, width, height):
+    x1, x2, y1, y2, dx, dy, edx, edy = mtcnn_bbox(face_box, width, height)
+    new_bbox = list(map(int, [x1, x2, y1, y2]))
+    new_bbox = BBox(new_bbox)
+    cropped = image[new_bbox.top:new_bbox.bottom, new_bbox.left:new_bbox.right]
+    if (dx > 0 or dy > 0 or edx > 0 or edy > 0):
+        cropped = cv2.copyMakeBorder(cropped, int(dy), int(edy), int(dx), int(edx), cv2.BORDER_CONSTANT, 0)
+    return cropped, new_bbox
+
+
+def face_landmark_56(input_image, faces_box=None):
+    basedir = os.path.dirname(os.path.realpath(__file__)).split("mtcnn.py")[0]
+    mean = np.asarray([0.485, 0.456, 0.406])
+    std = np.asarray([0.229, 0.224, 0.225])
+    base_url = "https://linimages.oss-cn-beijing.aliyuncs.com/"
+
+    if not exists(f"{basedir}/mtcnn_onnx/weights/landmark_detection_56_se_external.onnx"):
+        # download onnx model
+        download_img(img_url=base_url + "landmark_detection_56_se_external.onnx",
+                     base_dir=f"{basedir}/mtcnn_onnx/weights")
+
+    ort_session = onnxruntime.InferenceSession(f"{basedir}/mtcnn_onnx/weights/landmark_detection_56_se_external.onnx")
+    out_size = 56
+
+    height, width, _ = input_image.shape
+    if faces_box is None:
+        faces_box, _ = face_detect_mtcnn(input_image)
+
+    if len(faces_box) == 0:
+        print('NO face is detected!')
+        return None
+    else:
+        landmarks = []
+        for face_box in faces_box:
+            cropped, new_bbox = mtcnn_cropped_face(face_box, input_image, width, height)
+            cropped_face = cv2.resize(cropped, (out_size, out_size))
+
+            test_face = NNormalize(cropped_face, mean=mean, std=std)
+            test_face = NTo_Tensor(test_face)
+            test_face = NUnsqueeze(test_face)
+
+            ort_inputs = {ort_session.get_inputs()[0].name: test_face}
+            ort_outs = ort_session.run(None, ort_inputs)
+
+            landmark = ort_outs[0]
+
+            landmark = landmark.reshape(-1, 2)
+            landmark = new_bbox.reprojectLandmark(landmark)
+            landmarks.append(landmark)
+
+        return landmarks
+
+
+
+REFERENCE_FACIAL_POINTS = [
+    [30.29459953, 51.69630051],
+    [65.53179932, 51.50139999],
+    [48.02519989, 71.73660278],
+    [33.54930115, 92.3655014],
+    [62.72990036, 92.20410156]
+]
+
+DEFAULT_CROP_SIZE = (96, 112)
+
+
+def _umeyama(src, dst, estimate_scale=True, scale=1.0):
+    """Estimate N-D similarity transformation with or without scaling.
+    Parameters
+    ----------
+    src : (M, N) array
+        Source coordinates.
+    dst : (M, N) array
+        Destination coordinates.
+    estimate_scale : bool
+        Whether to estimate scaling factor.
+    Returns
+    -------
+    T : (N + 1, N + 1)
+        The homogeneous similarity transformation matrix. The matrix contains
+        NaN values only if the problem is not well-conditioned.
+    References
+    ----------
+    .. [1] "Least-squares estimation of transformation parameters between two
+            point patterns", Shinji Umeyama, PAMI 1991, :DOI:`10.1109/34.88573`
+    """
+
+    num = src.shape[0]
+    dim = src.shape[1]
+
+    # Compute mean of src and dst.
+    src_mean = src.mean(axis=0)
+    dst_mean = dst.mean(axis=0)
+
+    # Subtract mean from src and dst.
+    src_demean = src - src_mean
+    dst_demean = dst - dst_mean
+
+    # Eq. (38).
+    A = dst_demean.T @ src_demean / num
+
+    # Eq. (39).
+    d = np.ones((dim,), dtype=np.double)
+    if np.linalg.det(A) < 0:
+        d[dim - 1] = -1
+
+    T = np.eye(dim + 1, dtype=np.double)
+
+    U, S, V = np.linalg.svd(A)
+
+    # Eq. (40) and (43).
+    rank = np.linalg.matrix_rank(A)
+    if rank == 0:
+        return np.nan * T
+    elif rank == dim - 1:
+        if np.linalg.det(U) * np.linalg.det(V) > 0:
+            T[:dim, :dim] = U @ V
+        else:
+            s = d[dim - 1]
+            d[dim - 1] = -1
+            T[:dim, :dim] = U @ np.diag(d) @ V
+            d[dim - 1] = s
+    else:
+        T[:dim, :dim] = U @ np.diag(d) @ V
+
+    if estimate_scale:
+        # Eq. (41) and (42).
+        scale = 1.0 / src_demean.var(axis=0).sum() * (S @ d)
+    else:
+        scale = scale
+
+    T[:dim, dim] = dst_mean - scale * (T[:dim, :dim] @ src_mean.T)
+    T[:dim, :dim] *= scale
+
+    return T, scale
+
+
+class FaceWarpException(Exception):
+    def __str__(self):
+        return 'In File {}:{}'.format(
+            __file__, super.__str__(self))
+
+
+def get_reference_facial_points_5(output_size=None,
+                                inner_padding_factor=0.0,
+                                outer_padding=(0, 0),
+                                default_square=False):
+    tmp_5pts = np.array(REFERENCE_FACIAL_POINTS)
+    tmp_crop_size = np.array(DEFAULT_CROP_SIZE)
+
+    # 0) make the inner region a square
+    if default_square:
+        size_diff = max(tmp_crop_size) - tmp_crop_size
+        tmp_5pts += size_diff / 2
+        tmp_crop_size += size_diff
+
+    if (output_size and
+            output_size[0] == tmp_crop_size[0] and
+            output_size[1] == tmp_crop_size[1]):
+        print('output_size == DEFAULT_CROP_SIZE {}: return default reference points'.format(tmp_crop_size))
+        return tmp_5pts
+
+    if (inner_padding_factor == 0 and
+            outer_padding == (0, 0)):
+        if output_size is None:
+            print('No paddings to do: return default reference points')
+            return tmp_5pts
+        else:
+            raise FaceWarpException(
+                'No paddings to do, output_size must be None or {}'.format(tmp_crop_size))
+
+    # check output size
+    if not (0 <= inner_padding_factor <= 1.0):
+        raise FaceWarpException('Not (0 <= inner_padding_factor <= 1.0)')
+
+    if ((inner_padding_factor > 0 or outer_padding[0] > 0 or outer_padding[1] > 0)
+            and output_size is None):
+        output_size = tmp_crop_size * \
+                      (1 + inner_padding_factor * 2).astype(np.int32)
+        output_size += np.array(outer_padding)
+        print('              deduced from paddings, output_size = ', output_size)
+
+    if not (outer_padding[0] < output_size[0]
+            and outer_padding[1] < output_size[1]):
+        raise FaceWarpException('Not (outer_padding[0] < output_size[0]'
+                                'and outer_padding[1] < output_size[1])')
+
+    # 1) pad the inner region according inner_padding_factor
+    # print('---> STEP1: pad the inner region according inner_padding_factor')
+    if inner_padding_factor > 0:
+        size_diff = tmp_crop_size * inner_padding_factor * 2
+        tmp_5pts += size_diff / 2
+        tmp_crop_size += np.round(size_diff).astype(np.int32)
+
+    # print('              crop_size = ', tmp_crop_size)
+    # print('              reference_5pts = ', tmp_5pts)
+
+    # 2) resize the padded inner region
+    # print('---> STEP2: resize the padded inner region')
+    size_bf_outer_pad = np.array(output_size) - np.array(outer_padding) * 2
+    # print('              crop_size = ', tmp_crop_size)
+    # print('              size_bf_outer_pad = ', size_bf_outer_pad)
+
+    if size_bf_outer_pad[0] * tmp_crop_size[1] != size_bf_outer_pad[1] * tmp_crop_size[0]:
+        raise FaceWarpException('Must have (output_size - outer_padding)'
+                                '= some_scale * (crop_size * (1.0 + inner_padding_factor)')
+
+    scale_factor = size_bf_outer_pad[0].astype(np.float32) / tmp_crop_size[0]
+    # print('              resize scale_factor = ', scale_factor)
+    tmp_5pts = tmp_5pts * scale_factor
+    #    size_diff = tmp_crop_size * (scale_factor - min(scale_factor))
+    #    tmp_5pts = tmp_5pts + size_diff / 2
+    tmp_crop_size = size_bf_outer_pad
+    # print('              crop_size = ', tmp_crop_size)
+    # print('              reference_5pts = ', tmp_5pts)
+
+    # 3) add outer_padding to make output_size
+    reference_5point = tmp_5pts + np.array(outer_padding)
+    tmp_crop_size = output_size
+    # print('---> STEP3: add outer_padding to make output_size')
+    # print('              crop_size = ', tmp_crop_size)
+    # print('              reference_5pts = ', tmp_5pts)
+    #
+    # print('===> end get_reference_facial_points\n')
+
+    return reference_5point
+
+
+def get_affine_transform_matrix(src_pts, dst_pts):
+    tfm = np.float32([[1, 0, 0], [0, 1, 0]])
+    n_pts = src_pts.shape[0]
+    ones = np.ones((n_pts, 1), src_pts.dtype)
+    src_pts_ = np.hstack([src_pts, ones])
+    dst_pts_ = np.hstack([dst_pts, ones])
+
+    A, res, rank, s = np.linalg.lstsq(src_pts_, dst_pts_)
+
+    if rank == 3:
+        tfm = np.float32([
+            [A[0, 0], A[1, 0], A[2, 0]],
+            [A[0, 1], A[1, 1], A[2, 1]]
+        ])
+    elif rank == 2:
+        tfm = np.float32([
+            [A[0, 0], A[1, 0], 0],
+            [A[0, 1], A[1, 1], 0]
+        ])
+
+    return tfm
+
+
+def warp_and_crop_face(src_img,
+                       facial_pts,
+                       reference_pts=None,
+                       crop_size=(96, 112),
+                       align_type='smilarity'): #smilarity cv2_affine affine
+    if reference_pts is None:
+        if crop_size[0] == 96 and crop_size[1] == 112:
+            reference_pts = REFERENCE_FACIAL_POINTS
+        else:
+            default_square = False
+            inner_padding_factor = 0
+            outer_padding = (0, 0)
+            output_size = crop_size
+
+            reference_pts = get_reference_facial_points_5(output_size,
+                                                        inner_padding_factor,
+                                                        outer_padding,
+                                                        default_square)
+
+    ref_pts = np.float32(reference_pts)
+    ref_pts_shp = ref_pts.shape
+    if max(ref_pts_shp) < 3 or min(ref_pts_shp) != 2:
+        raise FaceWarpException(
+            'reference_pts.shape must be (K,2) or (2,K) and K>2')
+
+    if ref_pts_shp[0] == 2:
+        ref_pts = ref_pts.T
+
+    src_pts = np.float32(facial_pts)
+    src_pts_shp = src_pts.shape
+    if max(src_pts_shp) < 3 or min(src_pts_shp) != 2:
+        raise FaceWarpException(
+            'facial_pts.shape must be (K,2) or (2,K) and K>2')
+
+    if src_pts_shp[0] == 2:
+        src_pts = src_pts.T
+
+    if src_pts.shape != ref_pts.shape:
+        raise FaceWarpException(
+            'facial_pts and reference_pts must have the same shape')
+
+    if align_type == 'cv2_affine':
+        tfm = cv2.getAffineTransform(src_pts[0:3], ref_pts[0:3])
+        tfm_inv = cv2.getAffineTransform(ref_pts[0:3], src_pts[0:3])
+    elif align_type == 'affine':
+        tfm = get_affine_transform_matrix(src_pts, ref_pts)
+        tfm_inv = get_affine_transform_matrix(ref_pts, src_pts)
+    else:
+        params, scale = _umeyama(src_pts, ref_pts)
+        tfm = params[:2, :]
+
+        params, _ = _umeyama(ref_pts, src_pts, False, scale=1.0/scale)
+        tfm_inv = params[:2, :]
+
+    face_img = cv2.warpAffine(src_img, tfm, (crop_size[0], crop_size[1]), flags=3)
+
+    return face_img, tfm_inv
+
+
+if __name__ == "__main__":
+    image = cv2.imread("/home/parallels/Desktop/IDPhotos/input_image/03.jpg")
+    face_detect_mtcnn(image)
+
+

hivisionai/hycv/idphoto.py

@@ -0,0 +1,2 @@
+from .idphotoTool.idphoto_cut import IDphotos_create
+from .idphotoTool.idphoto_change_cloth import change_cloth