rofunc.learning.RofuncRL.agents.mixline.ase_hrl_agent#

1.  Module Contents#

1.1.  Classes#

ASEHRLAgent

Adversarial Skill Embeddings (ASE) agent for hierarchical reinforcement learning (HRL) using pre-trained low-level controller.

1.2.  API#

class rofunc.learning.RofuncRL.agents.mixline.ase_hrl_agent.ASEHRLAgent(cfg: omegaconf.DictConfig, observation_space: Optional[Union[int, Tuple[int], gym.Space, gymnasium.Space]], action_space: Optional[Union[int, Tuple[int], gym.Space, gymnasium.Space]], memory: Optional[Union[rofunc.learning.RofuncRL.utils.memory.Memory, Tuple[rofunc.learning.RofuncRL.utils.memory.Memory]]] = None, device: Optional[Union[str, torch.device]] = None, experiment_dir: Optional[str] = None, rofunc_logger: Optional[rofunc.logger.BeautyLogger] = None, amp_observation_space: Optional[Union[int, Tuple[int], gym.Space, gymnasium.Space]] = None, motion_dataset: Optional[Union[rofunc.learning.RofuncRL.utils.memory.Memory, Tuple[rofunc.learning.RofuncRL.utils.memory.Memory]]] = None, replay_buffer: Optional[Union[rofunc.learning.RofuncRL.utils.memory.Memory, Tuple[rofunc.learning.RofuncRL.utils.memory.Memory]]] = None, collect_reference_motions: Optional[Callable[[int], torch.Tensor]] = None, task_related_state_size: Optional[int] = None)#

Bases: rofunc.learning.RofuncRL.agents.base_agent.BaseAgent

Adversarial Skill Embeddings (ASE) agent for hierarchical reinforcement learning (HRL) using pre-trained low-level controller.

“ASE: Large-Scale Reusable Adversarial Skill Embeddings for Physically Simulated Characters”. Peng et al. 2022. https://arxiv.org/abs/2205.01906

Rofunc documentation: https://rofunc.readthedocs.io/en/latest/lfd/RofuncRL/ASE.html

Initialization

Parameters:

  • cfg – Configuration

  • observation_space – Observation space

  • action_space – Action space

  • memory – Memory for storing transitions

  • device – Device on which the torch tensor is allocated

  • experiment_dir – Directory where experiment outputs are saved

  • rofunc_logger – Rofunc logger

  • amp_observation_space – cfg[“env”][“numASEObsSteps”] * NUM_ASE_OBS_PER_STEP

  • motion_dataset – Motion dataset

  • replay_buffer – Replay buffer

  • collect_reference_motions – Function for collecting reference motions

  • task_related_state_size – Size of task-related states

act(states: torch.Tensor, deterministic: bool = False)#
store_transition(states: torch.Tensor, actions: torch.Tensor, next_states: torch.Tensor, rewards: torch.Tensor, terminated: torch.Tensor, truncated: torch.Tensor, infos: torch.Tensor)#
update_net()#

Update the network