Dwild/ChuanhuChatGPT
0
1a
2 �&d� � @ s� d dl mZ d dlZd dlZd dlmZ dd� Zdd� Zdd � Zd3d � Z dd
d�Z4ddd�Z dZ dZ
dZdZedkr�e d�Zeed � eed � dS )� ��SimpleNamespaceN)�Documentc s4 | j }|j|j � � fdd�}|� }dd||dd�S )uu Prepare table查找边界, 要求page为原始page5 6 From https://github.com/jsvine/pdfplumber/issues/2427 c s" g } � D ]}| t j�|�7 } q| S )z3See https://github.com/jsvine/pdfplumber/issues/127)�8pdfplumber�utilsZ
rect_to_edges)�edges�c��cs� �I/Users/treediagram/Documents/Developer/ChuanhuChatGPT/modules/pdf_func.py�curves_to_edges
s z-prepare_table_config.<locals>.curves_to_edges�explicit�9 )Zvertical_strategyZhorizontal_strategy�explicit_vertical_lines�explicit_horizontal_linesZintersection_y_tolerance)� root_page�curvesr )� crop_page�pager
r r r r �prepare_table_config s �r c sZ t | �}t|d �dks(t|d �dkr,| S dd� | jj|d�D �� � fdd�}| �|�S ) Nr r r c S s g | ]10}|j �qS r )�bbox)�.0�tabler r r �11<listcomp>" � z*get_text_outside_table.<locals>.<listcomp>)Ztable_settingsc s$ � fdd��t �fdd��D �� S )z2Check if the object is in any of the table's bbox.c sT � d � d d }� d � d d }| \}}}}||koR||k oR||koR||k S )zMSee https://github.com/jsvine/pdfplumber/blob/stable/pdfplumber/table.py#L404�top�bottom� �x0�x1r )�_bboxZv_midZh_midr r r r ��objr r � obj_in_bbox% s zFget_text_outside_table.<locals>.not_within_bboxes.<locals>.obj_in_bboxc 3 s | ]}� |�V qd S )Nr )r Z__bbox)r$ r r � <genexpr>+ r zDget_text_outside_table.<locals>.not_within_bboxes.<locals>.<genexpr>)�anyr" ��bboxes)r# r$ r �not_within_bboxes# s z1get_text_outside_table.<locals>.not_within_bboxes)r �lenr � find_tables�filter)r �tsr) r r' r �get_text_outside_table s 12r. c C s | j dddg d�d�S )NTr � )�fontname�size� object_type)Zkeep_blank_charsZ y_toleranceZ x_toleranceZ extra_attrs)�
extract_words)r r r r �<lambda>0 r r4 c C s� g }| j \}}}}t| �D ]@}tf i |��}|jdkrJ|�|j� |j}q|jdkr|j}qdd� t| �||||f��D �}||| �||||f�fS )N� ZAbstractc S s g | ] }|d �qS )�textr )r �ir r r r @ r z/get_title_with_cropped_page.<locals>.<listcomp>) r r3 r r1 �appendr6 r r � within_bbox) �13first_page�titler r r r �wordZ title_bottom� user_infor r r �get_title_with_cropped_page3 s 14 15 r> Tc C st g }| D ]f}|rd|j dd|jd |jfdd�}|j |jd d|j|jfdd�}|�|� |�|� q|�|� q|S )Nr r T)�relative)r9 �width�heightr8 )�pages�16two_column� new_pagesr �left�rightr r r �get_column_cropped_pagesD s 17 rG c C sD t �� j}|t �d�kr&t �� �d� t�| ���r}t|jd �\}}}t |g|jdd � |�}g }dd� } d }18t19|�D ]�\} } t | �} t | �D ]�}
t
f i |
��}
|
jdk�r"|20d u r�| | j|
j|
j�}21nF|22jr�|23j|24jk�r|25j|26jk�r| j|27_|�|28� | | j|
j|
j�}29|30j�|
j� q�d|31_|32j�|
j� q�qz| j|33_|�|34� |D ]D}t �d |j� d35|j|jf� d t|j�� �� t � d � |j�� �qPW d � n1 �s�0 Y d � |�}d � |�}d
|� d|� d�}t36|�D ]@\} }d � |j�|_|d| � d|j� d�d � |j� d 7 }�q�t �� �|� t |d|id�S )N�DEBUG�INFOr r/ c S s t g ||d| d g d�S )NT)�name�name_top� name_bottom�record_chapter_name�37page_start� page_stopr6 r )rN rK rL r r r r4 \ s �zparse_pdf.<locals>.<lambda>� Fz section: z pages:z word-count:� zArticle Title: z, Information:�38zThe z th Chapter z: r; )r6 �39extra_info) �logging� getLogger�level� getLevelName�setLevelr �openr> rB rG � enumerater. r3 r r1 � page_numberr r rM rL rK rO r8 rJ r6 �inforN r* �debug�joinr )�filenamerC rV �pdfr; r= r: rD ZchaptersZcreate_chapterZ cur_chapter�idxr r< r7 r6 Zchapterr r r � parse_pdfQ sH 40 "4142,64344,rb a* 451. Who are the authors?462. What is the process of the proposed method?473. What is the performance of the proposed method? Please note down its performance metrics.484. What are the baseline models and their performances? Please note down these baseline methods.495. What dataset did this paper use?50a 51You are a researcher helper bot. You can help the user with research paper reading and summarizing. 52 53Now I am going to send you a paper. You need to read it and summarize it for me part by part. 54 55When you are reading, You need to focus on these key points:{}56a� 57You are a researcher helper bot. You can help the user with research paper reading and summarizing. 58 59Now I am going to send you a paper. You need to read it and summarize it for me part by part. 60 61When you are reading, You need to focus on these key points:{},62 63And You need to generate a brief but informative title for this part.64Your return format:65- title: '...'66- summary: '...'67zXYou are a researcher helper bot. Now you need to read the summaries of a research paper.�__main__z./build/test.pdfr= r; )T)T)�typesr r rT � llama_indexr r r. r3 r> rG rb Z BASE_POINTSZREADING_PROMPTZREADING_PROMT_V2�SUMMARY_PROMPT�__name__�z�printr r r r �<module> s" 68
69C