easycv.core.post_processing package

easycv.core.post_processing.affine_transform(pt, trans_mat)

Apply an affine transformation to the points.

Parameters

• pt (np.ndarray) – a 2 dimensional point to be transformed

• trans_mat (np.ndarray) – 2x3 matrix of an affine transform

Returns

Transformed points.

Return type

np.ndarray

easycv.core.post_processing.flip_back(output_flipped, flip_pairs, target_type='GaussianHeatmap')

Flip the flipped heatmaps back to the original form.

Note

batch_size: N num_keypoints: K heatmap height: H heatmap width: W

Parameters

• output_flipped (np.ndarray[N, K, H, W]) – The output heatmaps obtained from the flipped images.

• flip_pairs (list[tuple()) – Pairs of keypoints which are mirrored (for example, left ear – right ear).

• target_type (str) – GaussianHeatmap or CombinedTarget

Returns

heatmaps that flipped back to the original image

Return type

np.ndarray

easycv.core.post_processing.fliplr_joints(joints_3d, joints_3d_visible, img_width, flip_pairs)

Flip human joints horizontally.

Note

num_keypoints: K

Parameters

• joints_3d (np.ndarray([K, 3])) – Coordinates of keypoints.

• joints_3d_visible (np.ndarray([K, 1])) – Visibility of keypoints.

• img_width (int) – Image width.

• flip_pairs (list[tuple()]) – Pairs of keypoints which are mirrored (for example, left ear – right ear).

Returns

Flipped human joints.

• joints_3d_flipped (np.ndarray([K, 3])): Flipped joints.

• joints_3d_visible_flipped (np.ndarray([K, 1])): Joint visibility.

Return type

tuple

easycv.core.post_processing.fliplr_regression(regression, flip_pairs, center_mode='static', center_x=0.5, center_index=0)

Flip human joints horizontally.

Note

batch_size: N num_keypoint: K

Parameters

• regression (np.ndarray([..., K, C])) –

  Coordinates of keypoints, where K is the joint number and C is the dimension. Example shapes are: - [N, K, C]: a batch of keypoints where N is the batch size. - [N, T, K, C]: a batch of pose sequences, where T is the frame

  number.

• flip_pairs (list[tuple()]) – Pairs of keypoints which are mirrored (for example, left ear – right ear).

• center_mode (str) – The mode to set the center location on the x-axis to flip around. Options are: - static: use a static x value (see center_x also) - root: use a root joint (see center_index also)

• center_x (float) – Set the x-axis location of the flip center. Only used when center_mode=static.

• center_index (int) – Set the index of the root joint, whose x location will be used as the flip center. Only used when center_mode=root.

Returns

Flipped human joints.

• regression_flipped (np.ndarray([…, K, C])): Flipped joints.

Return type

tuple

easycv.core.post_processing.get_affine_transform(center, scale, rot, output_size, shift=(0.0, 0.0), inv=False)

Get the affine transform matrix, given the center/scale/rot/output_size.

Parameters

• center (np.ndarray[2, ]) – Center of the bounding box (x, y).

• scale (np.ndarray[2, ]) – Scale of the bounding box wrt [width, height].

• rot (float) – Rotation angle (degree).

• output_size (np.ndarray[2, ] | list(2,)) – Size of the destination heatmaps.

• shift (0-100%) – Shift translation ratio wrt the width/height. Default (0., 0.).

• inv (bool) – Option to inverse the affine transform direction. (inv=False: src->dst or inv=True: dst->src)

Returns

The transform matrix.

Return type

np.ndarray

easycv.core.post_processing.get_warp_matrix(theta, size_input, size_dst, size_target)

Calculate the transformation matrix under the constraint of unbiased. Paper ref: Huang et al. The Devil is in the Details: Delving into Unbiased Data Processing for Human Pose Estimation (CVPR 2020).

Parameters

• theta (float) – Rotation angle in degrees.

• size_input (np.ndarray) – Size of input image [w, h].

• size_dst (np.ndarray) – Size of output image [w, h].

• size_target (np.ndarray) – Size of ROI in input plane [w, h].

Returns

A matrix for transformation.

Return type

matrix (np.ndarray)

easycv.core.post_processing.rotate_point(pt, angle_rad)

Rotate a point by an angle.

Parameters

• pt (list[float]) – 2 dimensional point to be rotated

• angle_rad (float) – rotation angle by radian

Returns

Rotated point.

Return type

list[float]

easycv.core.post_processing.transform_preds(coords, center, scale, output_size, use_udp=False)

Get final keypoint predictions from heatmaps and apply scaling and translation to map them back to the image.

Note

num_keypoints: K

Parameters

• coords (np.ndarray[K, ndims]) –

  • If ndims=2, corrds are predicted keypoint location.

  • If ndims=4, corrds are composed of (x, y, scores, tags)

  • If ndims=5, corrds are composed of (x, y, scores, tags, flipped_tags)

• center (np.ndarray[2, ]) – Center of the bounding box (x, y).

• scale (np.ndarray[2, ]) – Scale of the bounding box wrt [width, height].

• output_size (np.ndarray[2, ] | list(2,)) – Size of the destination heatmaps.

• use_udp (bool) – Use unbiased data processing

Returns

Predicted coordinates in the images.

Return type

np.ndarray

easycv.core.post_processing.warp_affine_joints(joints, mat)

Apply affine transformation defined by the transform matrix on the joints.

Parameters

• joints (np.ndarray[..., 2]) – Origin coordinate of joints.

• mat (np.ndarray[3, 2]) – The affine matrix.

Returns

Result coordinate of joints.

Return type

matrix (np.ndarray[…, 2])

easycv.core.post_processing.oks_nms(kpts_db, thr, sigmas=None, vis_thr=None)

OKS NMS implementations.

Parameters

• kpts_db – keypoints.

• thr – Retain overlap < thr.

• sigmas – standard deviation of keypoint labelling.

• vis_thr – threshold of the keypoint visibility.

Returns

indexes to keep.

Return type

np.ndarray

easycv.core.post_processing.soft_oks_nms(kpts_db, thr, max_dets=20, sigmas=None, vis_thr=None)

Soft OKS NMS implementations.

Parameters

• kpts_db –

• thr – retain oks overlap < thr.

• max_dets – max number of detections to keep.

• sigmas – Keypoint labelling uncertainty.

Returns

indexes to keep.

Return type

np.ndarray

Submodules

easycv.core.post_processing.nms module

easycv.core.post_processing.nms.oks_iou(g, d, a_g, a_d, sigmas=None, vis_thr=None)

Calculate oks ious.

Parameters

• g – Ground truth keypoints.

• d – Detected keypoints.

• a_g – Area of the ground truth object.

• a_d – Area of the detected object.

• sigmas – standard deviation of keypoint labelling.

• vis_thr – threshold of the keypoint visibility.

Returns

The oks ious.

Return type

list

easycv.core.post_processing.nms.oks_nms(kpts_db, thr, sigmas=None, vis_thr=None)

OKS NMS implementations.

Parameters

• kpts_db – keypoints.

• thr – Retain overlap < thr.

• sigmas – standard deviation of keypoint labelling.

• vis_thr – threshold of the keypoint visibility.

Returns

indexes to keep.

Return type

np.ndarray

easycv.core.post_processing.nms.soft_oks_nms(kpts_db, thr, max_dets=20, sigmas=None, vis_thr=None)

Soft OKS NMS implementations.

Parameters

• kpts_db –

• thr – retain oks overlap < thr.

• max_dets – max number of detections to keep.

• sigmas – Keypoint labelling uncertainty.

Returns

indexes to keep.

Return type

np.ndarray

easycv.core.post_processing.pose_transforms module

easycv.core.post_processing.pose_transforms.fliplr_joints(joints_3d, joints_3d_visible, img_width, flip_pairs)

Flip human joints horizontally.

Note

num_keypoints: K

Parameters

• joints_3d (np.ndarray([K, 3])) – Coordinates of keypoints.

• joints_3d_visible (np.ndarray([K, 1])) – Visibility of keypoints.

• img_width (int) – Image width.

• flip_pairs (list[tuple()]) – Pairs of keypoints which are mirrored (for example, left ear – right ear).

Returns

Flipped human joints.

• joints_3d_flipped (np.ndarray([K, 3])): Flipped joints.

• joints_3d_visible_flipped (np.ndarray([K, 1])): Joint visibility.

Return type

tuple

easycv.core.post_processing.pose_transforms.fliplr_regression(regression, flip_pairs, center_mode='static', center_x=0.5, center_index=0)

Flip human joints horizontally.

Note

batch_size: N num_keypoint: K

Parameters

• regression (np.ndarray([..., K, C])) –

  Coordinates of keypoints, where K is the joint number and C is the dimension. Example shapes are: - [N, K, C]: a batch of keypoints where N is the batch size. - [N, T, K, C]: a batch of pose sequences, where T is the frame

  number.

• flip_pairs (list[tuple()]) – Pairs of keypoints which are mirrored (for example, left ear – right ear).

• center_mode (str) – The mode to set the center location on the x-axis to flip around. Options are: - static: use a static x value (see center_x also) - root: use a root joint (see center_index also)

• center_x (float) – Set the x-axis location of the flip center. Only used when center_mode=static.

• center_index (int) – Set the index of the root joint, whose x location will be used as the flip center. Only used when center_mode=root.

Returns

Flipped human joints.

• regression_flipped (np.ndarray([…, K, C])): Flipped joints.

Return type

tuple

easycv.core.post_processing.pose_transforms.flip_back(output_flipped, flip_pairs, target_type='GaussianHeatmap')

Flip the flipped heatmaps back to the original form.

Note

batch_size: N num_keypoints: K heatmap height: H heatmap width: W

Parameters

• output_flipped (np.ndarray[N, K, H, W]) – The output heatmaps obtained from the flipped images.

• flip_pairs (list[tuple()) – Pairs of keypoints which are mirrored (for example, left ear – right ear).

• target_type (str) – GaussianHeatmap or CombinedTarget

Returns

heatmaps that flipped back to the original image

Return type

np.ndarray

easycv.core.post_processing.pose_transforms.transform_preds(coords, center, scale, output_size, use_udp=False)

Get final keypoint predictions from heatmaps and apply scaling and translation to map them back to the image.

Note

num_keypoints: K

Parameters

• coords (np.ndarray[K, ndims]) –

  • If ndims=2, corrds are predicted keypoint location.

  • If ndims=4, corrds are composed of (x, y, scores, tags)

  • If ndims=5, corrds are composed of (x, y, scores, tags, flipped_tags)

• center (np.ndarray[2, ]) – Center of the bounding box (x, y).

• scale (np.ndarray[2, ]) – Scale of the bounding box wrt [width, height].

• output_size (np.ndarray[2, ] | list(2,)) – Size of the destination heatmaps.

• use_udp (bool) – Use unbiased data processing

Returns

Predicted coordinates in the images.

Return type

np.ndarray

easycv.core.post_processing.pose_transforms.get_affine_transform(center, scale, rot, output_size, shift=(0.0, 0.0), inv=False)

Get the affine transform matrix, given the center/scale/rot/output_size.

Parameters

• center (np.ndarray[2, ]) – Center of the bounding box (x, y).

• scale (np.ndarray[2, ]) – Scale of the bounding box wrt [width, height].

• rot (float) – Rotation angle (degree).

• output_size (np.ndarray[2, ] | list(2,)) – Size of the destination heatmaps.

• shift (0-100%) – Shift translation ratio wrt the width/height. Default (0., 0.).

• inv (bool) – Option to inverse the affine transform direction. (inv=False: src->dst or inv=True: dst->src)

Returns

The transform matrix.

Return type

np.ndarray

easycv.core.post_processing.pose_transforms.affine_transform(pt, trans_mat)

Apply an affine transformation to the points.

Parameters

• pt (np.ndarray) – a 2 dimensional point to be transformed

• trans_mat (np.ndarray) – 2x3 matrix of an affine transform

Returns

Transformed points.

Return type

np.ndarray

easycv.core.post_processing.pose_transforms.rotate_point(pt, angle_rad)

Rotate a point by an angle.

Parameters

• pt (list[float]) – 2 dimensional point to be rotated

• angle_rad (float) – rotation angle by radian

Returns

Rotated point.

Return type

list[float]

easycv.core.post_processing.pose_transforms.get_warp_matrix(theta, size_input, size_dst, size_target)

Calculate the transformation matrix under the constraint of unbiased. Paper ref: Huang et al. The Devil is in the Details: Delving into Unbiased Data Processing for Human Pose Estimation (CVPR 2020).

Parameters

• theta (float) – Rotation angle in degrees.

• size_input (np.ndarray) – Size of input image [w, h].

• size_dst (np.ndarray) – Size of output image [w, h].

• size_target (np.ndarray) – Size of ROI in input plane [w, h].

Returns

A matrix for transformation.

Return type

matrix (np.ndarray)

easycv.core.post_processing.pose_transforms.warp_affine_joints(joints, mat)

Apply affine transformation defined by the transform matrix on the joints.

Parameters

• joints (np.ndarray[..., 2]) – Origin coordinate of joints.

• mat (np.ndarray[3, 2]) – The affine matrix.

Returns

Result coordinate of joints.

Return type

matrix (np.ndarray[…, 2])