YOLOv7w6Pose
YOLOv7w6 Pose Estimation Model. You can find more details at https://github.com/WongKinYiu/yolov7#pose-estimation.
![YOLOv7w6Pose Example](../../images/yolov7_w6_pose_demo.jpg)
Overall
- Framework: PyTorch
- Model format: ONNX
- Model task: Pose Estimation
- Source: https://github.com/WongKinYiu/yolov7#pose-estimation.
Usage
import cv2
from furiosa.models.vision import YOLOv7w6Pose
from furiosa.runtime.sync import create_runner
yolo_pose = YOLOv7w6Pose()
with create_runner(yolo_pose.model_source()) as runner:
image = cv2.imread("tests/assets/yolov5-test.jpg")
inputs, contexts = yolo_pose.preprocess([image])
output = runner.run(inputs)
yolo_pose.postprocess(output, contexts=contexts)
Inputs
The input is a 3-channel image of 384, 640 (height, width).
- Data Type:
numpy.uint8
- Tensor Shape:
[1, 3, 384, 640]
- Memory Format: NHWC, where
- N - batch size
- C - number of channels
- H - image height
- W - image width
- Color Order: RGB
- Optimal Batch Size (minimum: 1): <= 4
Outputs
The outputs are 3 numpy.float32
tensors in various shapes as the following.
You can refer to postprocess()
function to learn how to decode boxes, classes, and confidence scores.
Tensor | Shape | Data Type | Data Type | Description |
---|---|---|---|---|
0 | (1, 18, 48, 80) | float32 | NCHW | |
1 | (1, 153, 48, 80) | float32 | NCHW | |
2 | (1, 18, 24, 40) | float32 | NCHW | |
3 | (1, 153, 24, 40) | float32 | NCHW | |
4 | (1, 18, 12, 20) | float32 | NCHW | |
5 | (1, 153, 12, 20) | float32 | NCHW | |
6 | (1, 18, 6, 10) | float32 | NCHW | |
7 | (1, 153, 6, 10) | float32 | NCHW |
Pre/Postprocessing
furiosa.models.vision.YOLOv7w6Pose
class provides preprocess
and postprocess
methods.
preprocess
method converts input images to input tensors, and postprocess
method converts
model output tensors to a list of PoseEstimationResult
.
You can find examples at YOLOv7w6Pose Usage.
furiosa.models.vision.YOLOv7w6Pose.preprocess
Preprocess input images to a batch of input tensors
Parameters:
Name | Type | Description | Default |
---|---|---|---|
images |
Sequence[Union[str, ndarray]]
|
Color images have (NHWC: Batch, Height, Width, Channel) dimensions. |
required |
with_scaling |
bool
|
Whether to apply model-specific techniques that involve scaling the model's input and converting its data type to float32. Refer to the code to gain a precise understanding of the techniques used. Defaults to False. |
False
|
Returns:
Type | Description |
---|---|
Tuple[ndarray, List[Dict[str, Any]]]
|
a pre-processed image, scales and padded sizes(width,height) per images. The first element is a stacked numpy array containing a batch of images. To learn more about the outputs of preprocess (i.e., model inputs), please refer to YOLOv7w6Pose Inputs. The second element is a list of dict objects about the original images. Each dict object has the following keys. 'scale' key of the returned dict has a rescaled ratio per width(=target/width) and height(=target/height), and the 'pad' key has padded width and height pixels. Specially, the last dictionary element of returning tuple will be passed to postprocessing as a parameter to calculate predicted coordinates on normalized coordinates back to an input image coordinator. |
furiosa.models.vision.YOLOv7w6Pose.postprocess
Postprocess output tensors to a list of PoseEstimationResult
. It transforms the model's output into a list of PoseEstimationResult
instances.
Each PoseEstimationResult
contains information about the overall pose, including a bounding box, confidence score, and keypoint details such as nose, eyes, shoulders, etc.
Please refer to the followings for more details.
Keypoint
The Keypoint
class represents a keypoint detected by the YOLOv7W6 Pose Estimation model. It contains the following attributes:
Attribute | Description |
---|---|
x |
The x-coordinate of the keypoint as a floating-point number. |
y |
The y-coordinate of the keypoint as a floating-point number. |
confidence |
Confidence score associated with the keypoint as a floating-point number. |
See the source code for more details.
PoseEstimationResult
The PoseEstimationResult
class represents the overall result of the YOLOv7W6 Pose Estimation model. It includes the following attributes:
Attribute | Description |
---|---|
bounding_box |
A list of four floating-point numbers representing the bounding box coordinates of the detected pose. |
confidence |
Confidence score associated with the overall pose estimation as a floating-point number. |
nose |
Instance of the Keypoint class representing the nose keypoint. |
left_eye |
Instance of the Keypoint class representing the left eye keypoint. |
right_eye |
Instance of the Keypoint class representing the right eye keypoint. |
left_ear |
Instance of the Keypoint class representing the left ear keypoint. |
right_ear |
Instance of the Keypoint class representing the right ear keypoint. |
left_shoulder |
Instance of the Keypoint class representing the left shoulder keypoint. |
right_shoulder |
Instance of the Keypoint class representing the right shoulder keypoint. |
left_elbow |
Instance of the Keypoint class representing the left elbow keypoint. |
right_elbow |
Instance of the Keypoint class representing the right elbow keypoint. |
left_wrist |
Instance of the Keypoint class representing the left wrist keypoint. |
right_wrist |
Instance of the Keypoint class representing the right wrist keypoint. |
left_hip |
Instance of the Keypoint class representing the left hip keypoint. |
right_hip |
Instance of the Keypoint class representing the right hip keypoint. |
left_knee |
Instance of the Keypoint class representing the left knee keypoint. |
right_knee |
Instance of the Keypoint class representing the right knee keypoint. |
left_ankle |
Instance of the Keypoint class representing the left ankle keypoint. |
right_ankle |
Instance of the Keypoint class representing the right ankle keypoint. |
See the source code for more details.
Source code in furiosa/models/vision/yolov7_w6_pose/postprocess.py
Furthermore, for convenience, the YOLOv7w6Pose model includes an example visualize function. The image at the top of this document was generated using this utility function. The code used is as follows:
Usage
import cv2
from furiosa.models.vision import YOLOv7w6Pose
from furiosa.runtime.sync import create_runner
yolo_pose = YOLOv7w6Pose()
with create_runner(yolo_pose.model_source()) as runner:
image = cv2.imread("tests/assets/pose_demo.jpg")
inputs, contexts = yolo_pose.preprocess([image])
output = runner.run(inputs)
results = yolo_pose.postprocess(output, contexts=contexts)
yolo_pose.visualize(image, results[0])
cv2.imwrite("./pose_result.jpg", image)
furiosa.models.vision.YOLOv7w6Pose.visualize
This visualize function is an example of how to visualize the output of the model. It draws a skeleton of the human body on the input image in in-place manner.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
image |
ndarray
|
an input image |
required |
results |
List[PoseEstimationResult]
|
a list of PoseEstimationResult objects |
required |
Source code in furiosa/models/vision/yolov7_w6_pose/__init__.py
148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 |
|