rofunc.utils.datalab.poselib.poselib.skeleton.skeleton3d#

1.  Module Contents#

1.1.  Classes#

SkeletonTree

A skeleton tree gives a complete description of a rigid skeleton. It describes a tree structure over a list of nodes with their names indicated by strings. Each edge in the tree has a local translation associated with it which describes the distance between the two nodes that it connects.

SkeletonState

A skeleton state contains all the information needed to describe a static state of a skeleton. It requires a skeleton tree, local/global rotation at each joint and the root translation.

SkeletonMotion

1.2.  API#

class rofunc.utils.datalab.poselib.poselib.skeleton.skeleton3d.SkeletonTree(node_names, parent_indices, local_translation, local_orientation=None)[source]#

Bases: rofunc.utils.datalab.poselib.poselib.core.Serializable

A skeleton tree gives a complete description of a rigid skeleton. It describes a tree structure over a list of nodes with their names indicated by strings. Each edge in the tree has a local translation associated with it which describes the distance between the two nodes that it connects.

Basic Usage:
>>> t = SkeletonTree.from_mjcf(SkeletonTree.__example_mjcf_path__)
>>> t
SkeletonTree(
    node_names=['torso', 'front_left_leg', 'aux_1', 'front_left_foot', 'front_right_leg', 'aux_2', 'front_right_foot', 'left_back_leg', 'aux_3', 'left_back_foot', 'right_back_leg', 'aux_4', 'right_back_foot'],
    parent_indices=tensor([-1,  0,  1,  2,  0,  4,  5,  0,  7,  8,  0, 10, 11]),
    local_translation=tensor([[ 0.0000,  0.0000,  0.7500],
            [ 0.0000,  0.0000,  0.0000],
            [ 0.2000,  0.2000,  0.0000],
            [ 0.2000,  0.2000,  0.0000],
            [ 0.0000,  0.0000,  0.0000],
            [-0.2000,  0.2000,  0.0000],
            [-0.2000,  0.2000,  0.0000],
            [ 0.0000,  0.0000,  0.0000],
            [-0.2000, -0.2000,  0.0000],
            [-0.2000, -0.2000,  0.0000],
            [ 0.0000,  0.0000,  0.0000],
            [ 0.2000, -0.2000,  0.0000],
            [ 0.2000, -0.2000,  0.0000]])
)
>>> t.node_names
['torso', 'front_left_leg', 'aux_1', 'front_left_foot', 'front_right_leg', 'aux_2', 'front_right_foot', 'left_back_leg', 'aux_3', 'left_back_foot', 'right_back_leg', 'aux_4', 'right_back_foot']
>>> t.parent_indices
tensor([-1,  0,  1,  2,  0,  4,  5,  0,  7,  8,  0, 10, 11])
>>> t.local_translation
tensor([[ 0.0000,  0.0000,  0.7500],
        [ 0.0000,  0.0000,  0.0000],
        [ 0.2000,  0.2000,  0.0000],
        [ 0.2000,  0.2000,  0.0000],
        [ 0.0000,  0.0000,  0.0000],
        [-0.2000,  0.2000,  0.0000],
        [-0.2000,  0.2000,  0.0000],
        [ 0.0000,  0.0000,  0.0000],
        [-0.2000, -0.2000,  0.0000],
        [-0.2000, -0.2000,  0.0000],
        [ 0.0000,  0.0000,  0.0000],
        [ 0.2000, -0.2000,  0.0000],
        [ 0.2000, -0.2000,  0.0000]])
>>> t.parent_of('front_left_leg')
'torso'
>>> t.index('front_right_foot')
6
>>> t[2]
'aux_1'

Initialization

Parameters:

  • node_names (List[str]) – a list of names for each tree node

  • parent_indices (Tensor) – an int32-typed tensor that represents the edge to its parent. -1 represents the root node

  • local_translation (Tensor) – a 3d vector that gives local translation information

  • local_orientation – a 4d vector that gives local orientation information

property node_names#
property parent_indices#
property local_translation#
property local_orientation#
property num_joints#

number of nodes in the skeleton tree

classmethod from_dict(dict_repr, *args, **kwargs)[source]#
to_dict()[source]#
classmethod from_mjcf(path: str) → rofunc.utils.datalab.poselib.poselib.skeleton.skeleton3d.SkeletonTree[source]#

Parses a mujoco xml scene description file and returns a Skeleton Tree. We use the model attribute at the root as the name of the tree.

Parameters:

path (string) –

Returns:

The skeleton tree constructed from the mjcf file

Return type:

SkeletonTree

parent_of(node_name)[source]#

get the name of the parent of the given node

Parameters:

node_name (string) – the name of the node

Return type:

string

index(node_name)[source]#

get the index of the node

Parameters:

node_name (string) – the name of the node

Return type:

int

drop_nodes_by_names(node_names: rofunc.utils.datalab.poselib.poselib.core.List[str], pairwise_translation=None) → rofunc.utils.datalab.poselib.poselib.skeleton.skeleton3d.SkeletonTree[source]#
keep_nodes_by_names(node_names: rofunc.utils.datalab.poselib.poselib.core.List[str], pairwise_translation=None) → rofunc.utils.datalab.poselib.poselib.skeleton.skeleton3d.SkeletonTree[source]#
class rofunc.utils.datalab.poselib.poselib.skeleton.skeleton3d.SkeletonState(tensor_backend, skeleton_tree, is_local)[source]#

Bases: rofunc.utils.datalab.poselib.poselib.core.Serializable

A skeleton state contains all the information needed to describe a static state of a skeleton. It requires a skeleton tree, local/global rotation at each joint and the root translation.

Example:
>>> t = SkeletonTree.from_mjcf(SkeletonTree.__example_mjcf_path__)
>>> zero_pose = SkeletonState.zero_pose(t)
>>> plot_skeleton_state(zero_pose)  # can be imported from `.visualization.common`
[plot of the ant at zero pose
>>> local_rotation = zero_pose.local_rotation.clone()
>>> local_rotation[2] = torch.tensor([0, 0, 1, 0])
>>> new_pose = SkeletonState.from_rotation_and_root_translation(
...             skeleton_tree=t,
...             r=local_rotation,
...             t=zero_pose.root_translation,
...             is_local=True
...         )
>>> new_pose.local_rotation
tensor([[0., 0., 0., 1.],
        [0., 0., 0., 1.],
        [0., 1., 0., 0.],
        [0., 0., 0., 1.],
        [0., 0., 0., 1.],
        [0., 0., 0., 1.],
        [0., 0., 0., 1.],
        [0., 0., 0., 1.],
        [0., 0., 0., 1.],
        [0., 0., 0., 1.],
        [0., 0., 0., 1.],
        [0., 0., 0., 1.],
        [0., 0., 0., 1.]])
>>> plot_skeleton_state(new_pose)  # you should be able to see one of ant's leg is bent
[plot of the ant with the new pose
>>> new_pose.global_rotation  # the local rotation is propagated to the global rotation at joint #3
tensor([[0., 0., 0., 1.],
        [0., 0., 0., 1.],
        [0., 1., 0., 0.],
        [0., 1., 0., 0.],
        [0., 0., 0., 1.],
        [0., 0., 0., 1.],
        [0., 0., 0., 1.],
        [0., 0., 0., 1.],
        [0., 0., 0., 1.],
        [0., 0., 0., 1.],
        [0., 0., 0., 1.],
        [0., 0., 0., 1.],
        [0., 0., 0., 1.]])
Global/Local Representation (cont. from the previous example)
>>> new_pose.is_local
True
>>> new_pose.tensor  # this will return the local rotation followed by the root translation
tensor([0., 0., 0., 1., 0., 0., 0., 1., 0., 1., 0., 0., 0., 0., 0., 1., 0., 0.,
        0., 1., 0., 0., 0., 1., 0., 0., 0., 1., 0., 0., 0., 1., 0., 0., 0., 1.,
        0., 0., 0., 1., 0., 0., 0., 1., 0., 0., 0., 1., 0., 0., 0., 1., 0., 0.,
        0.])
>>> new_pose.tensor.shape  # 4 * 13 (joint rotation) + 3 (root translatio
torch.Size([55])
>>> new_pose.global_repr().is_local
False
>>> new_pose.global_repr().tensor  # this will return the global rotation followed by the root translation instead
tensor([0., 0., 0., 1., 0., 0., 0., 1., 0., 1., 0., 0., 0., 1., 0., 0., 0., 0.,
        0., 1., 0., 0., 0., 1., 0., 0., 0., 1., 0., 0., 0., 1., 0., 0., 0., 1.,
        0., 0., 0., 1., 0., 0., 0., 1., 0., 0., 0., 1., 0., 0., 0., 1., 0., 0.,
        0.])
>>> new_pose.global_repr().tensor.shape  # 4 * 13 (joint rotation) + 3 (root translation
torch.Size([55])

Initialization

property rotation#
property object_poses#
set_object_poses(object_poses)[source]#
property is_local#

is the rotation represented in local frame?

Return type:

bool

property invariant_property#
property num_joints#

number of joints in the skeleton tree

Return type:

int

property skeleton_tree#

skeleton tree

Return type:

SkeletonTree

property root_translation#

root translation

Return type:

Tensor

property global_transformation#

global transformation of each joint (transform from joint frame to global frame)

property global_rotation#

global rotation of each joint (rotation matrix to rotate from joint’s F.O.R to global F.O.R)

property global_translation#

global translation of each joint

property global_translation_xy#

global translation in xy

property global_translation_xz#

global translation in xz

property local_rotation#

the rotation from child frame to parent frame given in the order of child nodes appeared in .skeleton_tree.node_names

property local_transformation#

local translation + local rotation. It describes the transformation from child frame to parent frame given in the order of child nodes appeared in .skeleton_tree.node_names

property local_translation#

local translation of the skeleton state. It is identical to the local translation in .skeleton_tree.local_translation except the root translation. The root translation is identical to .root_translation

property root_translation_xy#

root translation on xy

property global_root_rotation#

root rotation

property global_root_yaw_rotation#

root yaw rotation

property local_translation_to_root#

The 3D translation from joint frame to the root frame.

property local_rotation_to_root#

The 3D rotation from joint frame to the root frame. It is equivalent to The root_R_world * world_R_node

compute_forward_vector(left_shoulder_index, right_shoulder_index, left_hip_index, right_hip_index, gaussian_filter_width=20)[source]#

Computes forward vector based on cross product of the up vector with average of the right->left shoulder and hip vectors

classmethod from_dict(dict_repr: rofunc.utils.datalab.poselib.poselib.core.OrderedDict, *args, **kwargs) → rofunc.utils.datalab.poselib.poselib.skeleton.skeleton3d.SkeletonState[source]#
to_dict() → rofunc.utils.datalab.poselib.poselib.core.OrderedDict[source]#
classmethod from_rotation_and_root_translation(skeleton_tree, r, t, is_local=True)[source]#

Construct a skeleton state from rotation and root translation

Parameters:

  • skeleton_tree (SkeletonTree) – the skeleton tree

  • r (Tensor) – rotation (either global or local)

  • t (Tensor) – root translation

  • is_local (bool, optional, default=True) – to indicate that whether the rotation is local or global

classmethod zero_pose(skeleton_tree)[source]#

Construct a zero-pose skeleton state from the skeleton tree by assuming that all the local rotation is 0 and root translation is also 0.

Parameters:

skeleton_tree (SkeletonTree) – the skeleton tree as the rigid body

classmethod zero_pose_wo_qbhand(skeleton_tree)[source]#

Construct a zero-pose skeleton state from the skeleton tree by assuming that all the local rotation is 0 and root translation is also 0.

Parameters:

skeleton_tree (SkeletonTree) – the skeleton tree as the rigid body

local_repr()[source]#

Convert the skeleton state into local representation. This will only affects the values of .tensor. If the skeleton state already has is_local=True. This method will do nothing.

Return type:

SkeletonState

global_repr()[source]#

Convert the skeleton state into global representation. This will only affects the values of .tensor. If the skeleton state already has is_local=False. This method will do nothing.

Return type:

SkeletonState

drop_nodes_by_names(node_names: rofunc.utils.datalab.poselib.poselib.core.List[str], estimate_local_translation_from_states: bool = True) → rofunc.utils.datalab.poselib.poselib.skeleton.skeleton3d.SkeletonState[source]#

Drop a list of nodes from the skeleton and re-compute the local rotation to match the original joint position as much as possible.

Parameters:

  • node_names (List of strings) – a list node names that specifies the nodes need to be dropped

  • estimate_local_translation_from_states (boolean) – the boolean indicator that specifies whether or not to re-estimate the local translation from the states (avg.)

Return type:

SkeletonState

keep_nodes_by_names(node_names: rofunc.utils.datalab.poselib.poselib.core.List[str], estimate_local_translation_from_states: bool = True) → rofunc.utils.datalab.poselib.poselib.skeleton.skeleton3d.SkeletonState[source]#

Keep a list of nodes and drop all other nodes from the skeleton and re-compute the local rotation to match the original joint position as much as possible.

Parameters:

  • node_names (List of strings) – a list node names that specifies the nodes need to be dropped

  • estimate_local_translation_from_states (boolean) – the boolean indicator that specifies whether or not to re-estimate the local translation from the states (avg.)

Return type:

SkeletonState

retarget_to(joint_mapping: Dict[str, str], source_tpose_local_rotation, source_tpose_root_translation: numpy.ndarray, target_skeleton_tree: rofunc.utils.datalab.poselib.poselib.skeleton.skeleton3d.SkeletonTree, target_tpose_local_rotation, target_tpose_root_translation: numpy.ndarray, rotation_to_target_skeleton, scale_to_target_skeleton: float, z_up: bool = True) → rofunc.utils.datalab.poselib.poselib.skeleton.skeleton3d.SkeletonState[source]#

Retarget the skeleton state to a target skeleton tree. This is a naive retarget implementation with rough approximations. The function follows the procedures below.

Steps:

  1. Drop the joints from the source (self) that do not belong to the joint mapping with an implementation that is similar to “keep_nodes_by_names()” - take a look at the function doc for more details (same for source_tpose)

  2. Rotate the source state and the source tpose by “rotation_to_target_skeleton” to align the source with the target orientation

  3. Extract the root translation and normalize it to match the scale of the target skeleton

  4. Extract the global rotation from source state relative to source tpose and re-apply the relative rotation to the target tpose to construct the global rotation after retargetting

  5. Combine the computed global rotation and the root translation from 3 and 4 to complete the retargeting.

  6. Make feet on the ground (global translation z)

Parameters:

  • joint_mapping (Dict[str, str]) – a dictionary of that maps the joint node from the source skeleton to the target skeleton

  • source_tpose_local_rotation (Tensor) – the local rotation of the source skeleton

  • source_tpose_root_translation (np.ndarray) – the root translation of the source tpose

  • target_skeleton_tree (SkeletonTree) – the target skeleton tree

  • target_tpose_local_rotation (Tensor) – the local rotation of the target skeleton

  • target_tpose_root_translation (Tensor) – the root translation of the target tpose

  • rotation_to_target_skeleton (Tensor) – the rotation that needs to be applied to the source skeleton to align with the target skeleton. Essentially the rotation is t_R_s, where t is the frame of reference of the target skeleton and s is the frame of reference of the source skeleton

  • scale_to_target_skeleton (float) – the factor that needs to be multiplied from source skeleton to target skeleton (unit in distance). For example, to go from cm to m, the factor needs to be 0.01.

Return type:

SkeletonState

retarget_to_by_tpose(joint_mapping: Dict[str, str], source_tpose: rofunc.utils.datalab.poselib.poselib.skeleton.skeleton3d.SkeletonState, target_tpose: rofunc.utils.datalab.poselib.poselib.skeleton.skeleton3d.SkeletonState, rotation_to_target_skeleton, scale_to_target_skeleton: float) → rofunc.utils.datalab.poselib.poselib.skeleton.skeleton3d.SkeletonState[source]#

Retarget the skeleton state to a target skeleton tree. This is a naive retarget implementation with rough approximations. See the method retarget_to() for more information

Parameters:

  • joint_mapping (Dict[str, str]) – a dictionary of that maps the joint node from the source skeleton to the target skeleton

  • source_tpose (SkeletonState) – t-pose of the source skeleton

  • target_tpose (SkeletonState) – t-pose of the target skeleton

  • rotation_to_target_skeleton (Tensor) – the rotation that needs to be applied to the source skeleton to align with the target skeleton. Essentially the rotation is t_R_s, where t is the frame of reference of the target skeleton and s is the frame of reference of the source skeleton

  • scale_to_target_skeleton (float) – the factor that needs to be multiplied from source skeleton to target skeleton (unit in distance). For example, to go from cm to m, the factor needs to be 0.01.

Return type:

SkeletonState

class rofunc.utils.datalab.poselib.poselib.skeleton.skeleton3d.SkeletonMotion(tensor_backend, skeleton_tree, is_local, fps, object_poses=None, *args, **kwargs)[source]#

Bases: rofunc.utils.datalab.poselib.poselib.skeleton.skeleton3d.SkeletonState

clone()[source]#
property invariant_property#
property global_velocity#

global velocity

property global_angular_velocity#

global angular velocity

property fps#

number of frames per second

property time_delta#

time between two adjacent frames

property global_root_velocity#

global root velocity

property global_root_angular_velocity#

global root angular velocity

property object_poses#
classmethod from_state_vector_and_velocity(skeleton_tree, state_vector, global_velocity, global_angular_velocity, is_local, fps)[source]#

Construct a skeleton motion from a skeleton state vector, global velocity and angular velocity at each joint.

Parameters:

  • skeleton_tree (SkeletonTree) – the skeleton tree that the motion is based on

  • state_vector (Tensor) – the state vector from the skeleton state by .tensor

  • global_velocity (Tensor) – the global velocity at each joint

  • global_angular_velocity (Tensor) – the global angular velocity at each joint

  • is_local (boolean) – if the rotation ins the state vector is given in local frame

  • fps (int) – number of frames per second

Return type:

SkeletonMotion

classmethod from_skeleton_state(skeleton_state: rofunc.utils.datalab.poselib.poselib.skeleton.skeleton3d.SkeletonState, fps: int)[source]#

Construct a skeleton motion from a skeleton state. The velocities are estimated using second order gaussian filter along the last axis. The skeleton state must have at least .dim >= 1

Parameters:

  • skeleton_state (SkeletonState) – the skeleton state that the motion is based on

  • fps (int) – number of frames per second

Return type:

SkeletonMotion

classmethod from_dict(dict_repr: rofunc.utils.datalab.poselib.poselib.core.OrderedDict, *args, **kwargs) → rofunc.utils.datalab.poselib.poselib.skeleton.skeleton3d.SkeletonMotion[source]#
to_dict() → rofunc.utils.datalab.poselib.poselib.core.OrderedDict[source]#
classmethod from_fbx(fbx_file_path, skeleton_tree=None, is_local=True, fps=120, root_joint='', root_trans_index=0, *args, **kwargs) → rofunc.utils.datalab.poselib.poselib.skeleton.skeleton3d.SkeletonMotion[source]#

Construct a skeleton motion from a fbx file (TODO - generalize this). If the skeleton tree is not given, it will use the first frame of the mocap to construct the skeleton tree.

Parameters:

  • fbx_file_path (string) – the path of the fbx file

  • fbx_configs (dict) – the configuration in terms of {“tmp_path”: …, “fbx_py27_path”: …}

  • skeleton_tree (SkeletonTree, optional) – the optional skeleton tree that the rotation will be applied to

  • is_local (bool, optional, default=True) – the state vector uses local or global rotation as the representation

  • fps (int, optional, default=120) – FPS of the FBX animation

  • root_joint (string, optional, default="" or the first node in the FBX scene with animation data) – the name of the root joint for the skeleton

  • root_trans_index (int, optional, default=0 or the root joint in the parsed skeleton) – index of joint to extract root transform from

Return type:

SkeletonMotion

crop(start: int, end: int, fps: rofunc.utils.datalab.poselib.poselib.core.Optional[int] = None)[source]#

Crop the motion along its last axis. This is equivalent to performing a slicing on the object with […, start: end: skip_every] where skip_every = old_fps / fps. Note that the new fps provided must be a factor of the original fps.

Parameters:

  • start (int) – the beginning frame index

  • end (int) – the ending frame index

  • fps (int, optional) – number of frames per second in the output (if not given the original fps will be used)

Return type:

SkeletonMotion

retarget_to(joint_mapping: Dict[str, str], source_tpose_local_rotation, source_tpose_root_translation: numpy.ndarray, target_skeleton_tree: rofunc.utils.datalab.poselib.poselib.skeleton.skeleton3d.SkeletonTree, target_tpose_local_rotation, target_tpose_root_translation: numpy.ndarray, rotation_to_target_skeleton, scale_to_target_skeleton: float, z_up: bool = True) → rofunc.utils.datalab.poselib.poselib.skeleton.skeleton3d.SkeletonMotion[source]#

Same as the one in SkeletonState. This method discards all velocity information before retargeting and re-estimate the velocity after the retargeting. The same fps is used in the new retargetted motion.

Parameters:

  • joint_mapping (Dict[str, str]) – a dictionary of that maps the joint node from the source skeleton to the target skeleton

  • source_tpose_local_rotation (Tensor) – the local rotation of the source skeleton

  • source_tpose_root_translation (np.ndarray) – the root translation of the source tpose

  • target_skeleton_tree (SkeletonTree) – the target skeleton tree

  • target_tpose_local_rotation (Tensor) – the local rotation of the target skeleton

  • target_tpose_root_translation (Tensor) – the root translation of the target tpose

  • rotation_to_target_skeleton (Tensor) – the rotation that needs to be applied to the source skeleton to align with the target skeleton. Essentially the rotation is t_R_s, where t is the frame of reference of the target skeleton and s is the frame of reference of the source skeleton

  • scale_to_target_skeleton (float) – the factor that needs to be multiplied from source skeleton to target skeleton (unit in distance). For example, to go from cm to m, the factor needs to be 0.01.

Return type:

SkeletonMotion

retarget_to_by_tpose(joint_mapping: Dict[str, str], source_tpose: rofunc.utils.datalab.poselib.poselib.skeleton.skeleton3d.SkeletonState, target_tpose: rofunc.utils.datalab.poselib.poselib.skeleton.skeleton3d.SkeletonState, rotation_to_target_skeleton, scale_to_target_skeleton: float, z_up: bool = True) → rofunc.utils.datalab.poselib.poselib.skeleton.skeleton3d.SkeletonMotion[source]#

Same as the one in SkeletonState. This method discards all velocity information before retargeting and re-estimate the velocity after the retargeting. The same fps is used in the new retargetted motion.

Parameters:

  • joint_mapping (Dict[str, str]) – a dictionary of that maps the joint node from the source skeleton to the target skeleton

  • source_tpose (SkeletonState) – t-pose of the source skeleton

  • target_tpose (SkeletonState) – t-pose of the target skeleton

  • rotation_to_target_skeleton (Tensor) – the rotation that needs to be applied to the source skeleton to align with the target skeleton. Essentially the rotation is t_R_s, where t is the frame of reference of the target skeleton and s is the frame of reference of the source skeleton

  • scale_to_target_skeleton (float) – the factor that needs to be multiplied from source skeleton to target skeleton (unit in distance). For example, to go from cm to m, the factor needs to be 0.01.

Return type:

SkeletonMotion