robometer/rewardeval_ui
4
1�
2 �lWi% � �v � d Z ddlmZmZmZ ddlZddlmZ ddlmZm Z ddl3m Z ddl m
Z
G d� d e � � ZdS )4a0 5Data generator for reward alignment evaluation.6 7This generator creates subsequence samples from trajectories for progress prediction evaluation.8For each trajectory, it creates multiple subsequences (0:2, 0:4, 0:6, etc.) and formats them9as PreferenceSample objects that can be evaluated by the model.10� )�Dict�List�AnyN)�tqdm)�ProgressSample�11Trajectory)�RFMBaseSampler)� rank_0_printc �� � � e Zd ZdZ ddedz dedef� fd�
Zd eee e12f fd13�Z d ed ee e14f d eee e15f fd
�Z de
d efd�Zd� Zd� Z� xZS )�RewardAlignmentSamplerz�16 Data generator that creates subsequence samples for reward alignment evaluation.17 18 For each trajectory, creates subsequences of frames (0:2, 0:4, 0:6, etc.)19 and formats them as PreferenceSample objects for evaluation.20 N� T�max_trajectories�21frame_step�use_frame_stepsc 22 �t �� t � � j di |�� || _ || _ || _ | � � � | _ t dt | j � � � d| j r't t | j23 � � | j � � nt | j24 � � � d�| j �� � d S )Nz25Generated z& reward alignment sample indices from �
trajectories��verbose� ) �super�__init__r r r � _generate_all_sample_indices�sample_indicesr26 �len�min�robot_trajectoriesr )�selfr r r �kwargs� __class__s ��B/scr/aliang80/reward_fm/rfm/data/samplers/eval/reward_alignment.pyr z RewardAlignmentSampler.__init__ s
�� � �����"�"�6�"�"�"� 0�� �$��� .�� � "� ?� ?� A� A����
\��T� 0�1�1�
\�
\� VZ� Vk� ZM�Y\�]`�ae�ax�]y�]y�{� |Q� ZR� ZR� ZR� qt� uy� uL� qM� qM�
\�
\�
\��L� 27� 28� 29� 30� 31� 32� �returnc �n � g }| j }| j �B| j t | j � � k r%| j � | j | j � � }t dt |� � � d�| j �� � |D ]8}| j | }|� | � ||� � � � �9|S )zJGenerate all possible subsequence sample indices (not the actual samples).Nz#Generating subsequence samples for r r )33r r r �
_local_random�sampler34 r �dataset�extend� _generate_indices_for_trajectory)r r �trajectories_to_process�traj_idx�trajs r r z3RewardAlignmentSampler._generate_all_sample_indices/ s� � ��� #'�"9� � � � ,��1F��T�Md�Ie�Ie�1e�1e�&*�&8�&?�&?��@W�Y]�Yn�&o�&o� #�� ]�#�6M�2N�2N� ]� ]� ]�gk�gs� 35� 36� 37� 38� 0� Y� Y�H��<��)�D� � !� !�$�"G�"G��RV�"W�"W� X� X� X� X� � r! r* r+ c 39 �@ � |d }g }| j rft | j |dz | j � � D ]F}t t |� � � � }|� ||||d |d dd�� � �Gn&|� ||d |d dd�� � |S ) z�Generate sample indices for a single trajectory.40 41 Args:42 traj_idx: Index of the trajectory in the dataset43 traj: Trajectory dictionary44 45 Returns:46 List of sample index dictionaries47 �48num_framesr
�frames�idT)r* �
frame_indicesr- �49video_pathr/ r F)r* r1 r/ r )r �ranger �list�append)r r* r+ r- �indices�end_idxr0 s r r( z7RewardAlignmentSampler._generate_indices_for_trajectoryB s� � � �,�'�50��� � � � �� �*�q�.�$�/�R�R�
�
�� $�U�7�^�^� 4� 4�
���� (�%2�",�"&�x�.� �t�*�'+�
� � � � � �
�
�N�N� $� "�8�n��4�j�#(� � �
�
�
� �r! �sample_idx_infoc �b � |d }|� dd� � }| j | }|rG|d }|d }d|d |d |r|d nd51|d �}| � |||� � � }n)d|d |d d
�}| � ||�� � }t |d�� � } | S )z9Generate a single subsequence sample from stored indices.r* r Tr0 r- �reward_alignmentr/ r1 �����r )�data_gen_strategyr/ r1 r r- )r+ r0 �metadata)r; r/ r1 )r+ r< �progress)�52trajectory� sample_type)�getr& �_get_traj_from_datar )53r r7 r* r r+ r0 r- r< r> r% s54 r � _generate_sample_from_indicesz4RewardAlignmentSampler._generate_sample_from_indicesf s � �"�:�.��)�-�-�.?��F�F���|�H�%�� � � +�O� <�M�(� �6�J� &8��4�j� -�l� ;�3@� G�m�B� /� /�a� (� � �H� �1�1�� +�!� 2� � �J�J� &8��4�j� -�l� ;�� �H� �1�1��!� 2� � �J�55 �:�:�N�N�N���
r! c �* � t | j � � S �N)r r )r s r �__len__z RewardAlignmentSampler.__len__� s � ��4�&�'�'�'r! c �B � | � | j | � � S rD )rB r )r �idxs r � __getitem__z"RewardAlignmentSampler.__getitem__� s � ��1�1�$�2E�c�2J�K�K�Kr! )Nr
T)�__name__�56__module__� __qualname__�__doc__�int�boolr r r �strr r r( �dictr rB rE rH �
__classcell__)r s @r r r s, �� � � � � �� � (,�� $� 57� 58� ��*�59� �60� � 61� 62� 63� 64� 65� 66�& �d�4��S��>�.B� � � � �&"�� "�D��c��N� "�W[�\`�ad�fi�ai�\j�Wk� "� "� "� "�H'�T� '�n� '� '� '� '�R(� (� (�L� L� L� L� L� L� Lr! r )rL �typingr r r �torchr �rfm.data.dataset_typesr r �rfm.data.samplers.baser �rfm.utils.distributedr67 r r r! r �<module>rW s� ��� � #� "� "� "� "� "� "� "� "� "� � � � � � � � � � � =� =� =� =� =� =� =� =� 1� 1� 1� 1� 1� 1� .� .� .� .� .� .�L� L� L� L� L�^� L� L� L� L� Lr!