Aigenthix/Graph_RAG
0
1�
2 "R6jY � �\ � S r SSKJrJrJr SSKr\R " \5 r " S S5 r g)zDocument chunking service� )�List�Dict�AnyNc � � \ rS rSrSrSS\S\4S jjrS\S\\\ 4 S\3\\\ 4 4S jr SS4\ S\\\ 4 S \S\5\\\ 4 4S jjr S
r
g)�ChunkerService� z+Service for splitting documents into chunks�6chunk_size�overlapc � � Xl X l g )N�r r7 )�selfr r8 s �a/Users/ssp/Downloads/SSP7/Claude folder/KnowledgeHut/POC1/backend/app/services/chunker_service.py�__init__�ChunkerService.__init__ s � �$�� � � �text�metadata�returnc �~ � / nUR 5 nSnSnU[ U5 : au [ XPR - [ U5 5 nSR XEU 5 nUR UU0 UEUUUS.ES.5 US-
nXpR -9 nU[ U5 : a Mu [ R S[ U5 S35 U$ )z$Split text into chunks with metadatar � )� chunk_index�10start_word�end_word��chunk_idr r � �Created z chunks from text) �split�len�minr �join�appendr11 �logger�info) r
r r �chunks�words� start_idx� chunk_idx�end_idx�12chunk_texts r r* �ChunkerService.chunk_text s� � � ���13�14� ��� �� ��#�e�*�$��)�o�o�5�s�5�z�B�G� ���%�'�":�;�J� �M�M� %�"� � � �#,�"+� '� � �
�
��N�I� �,�,�.�I�! �#�e�*�$�$ � � �h�s�6�{�m�+<�=�>��
r �df�rows_per_chunkc �< � / nSn[ S[ U5 U5 H[ nUR XfU- nUR 5 nUR UU0 UEUU[ Xc- [ U5 5 S.ES.5 US-
nM] [ R S[ U5 S35 U$ )zSplit DataFrame into chunksr )r � row_start�row_endr r r z chunks from DataFrame)�ranger �iloc� to_stringr" r r# r$ ) r
r, r r- r% r( �i�chunk_dfr* s r �chunk_dataframe� ChunkerService.chunk_dataframe1 s� � � ��� ��q�#�b�'�>�2�A��w�w�q�~�#5�6�H�!�+�+�-�J� �M�M� %�"� � � �#,�!"� "�1�#5�s�2�w� ?� � �
�
��N�I� 3�" � � �h�s�6�{�m�+A�B�C��
r r N)i �2 )�15 )�__name__�16__module__� __qualname__�__firstlineno__�__doc__�intr �strr r r r* r6 �__static_attributes__� r r r r s� � �5� �3� �s� � �� � �s�C�x�.� � 17�d�3��8�n� � �J !� � � � �s�C�x�.� � � �18 19�d�3��8�n� � � r r )20r> �typingr r r �logging� getLoggerr: r# r rB r r �<module>rF s- �� � "� "� � � � �8� $��D� Dr