cathub/LocalChatGPT
0
1a
2 8 d~; � @ s$ d dl mZ d dlmZmZ d dlZd dlZd dlZd dlZd dl Z d dl3m4Z5 d dl Z d dl m
Z
d dlZd dlZd dlT d dlT d dlT d dlmZ er�d dlmZ G dd � d e�Zd6Zd Zd Zd
d� Zd dd�Zd dd�Z de d ddddfdd�Z de d dfdd�Z e d dfdd �Z dS ) � )� annotations)�
TYPE_CHECKING�ListN)�tqdm)�ddg)�*)� TypedDictc @ s e Zd ZU ded< ded< dS )�
DataframeDataz List[str]�headersz List[List[str | int | bool]]�dataN)�__name__�7__module__� __qualname__�__annotations__� r r �G/Users/treediagram/Documents/Developer/huggingface/modules/chat_func.pyr s 8r z You are a helpful assistant.�history� templatesc C s9 dd| � �d�}t |�g|�}||||d|ddd�}|r>t} nt} tj�d�pXtj�d�}10tj�d �pptj�d11�} i } |12r�t�d |13� �� |14| d < | r�t�d
| � �� | | d< tj j15t kr�t�dtj j16� �� | r�t j
tj j17||d| | d�}
nt j
tj j18||d| d�}
|
S )Nzapplication/jsonzBearer )z Content-Type�
Authorization� r )�model�messages� temperature�top_p�n�stream�presence_penalty�frequency_penaltyZ19HTTP_PROXY�20http_proxy� HTTPS_PROXY� https_proxyu 使用 HTTP 代理: �httpu 使用 HTTPS 代理: �httpsu 使用自定义API URL: T)r21 �jsonr �timeout�proxies)r22 r# r r$ )�construct_system�timeout_streaming� timeout_all�os�environ�get�logging�info�shared�state�api_url�API_URL�requests�post)�openai_api_key�
system_promptr r r r �selected_modelr23 �payloadr$ r r r% �responser r r � get_response$ sX ��24 � �r9 � c # s� � ���fdd�} t �d� d} d}
d���t|�� ��td�� | rX��| df� n��|df� d}t� �dkr�tt|��}tt|��| }n tt|��}� �|� t �d|� �� | � V zt| |�||d|�}W nZ t j25j �y t t
t �| � V Y d S t j26j�y4 t t t �| � V Y d S 0 | � V d}|�� D �]x}|
dk�rf|
d 7 }
�qJ|
d 7 }
|�rJ|�� }t|�}zt�|d27d � �}W n@ tj�y� t �|� ||7 }d |� ��| � V Y �qJY n0 |d28k�rJd |d
d v �rJ|d
d d }tt� �dd��|dk�r0| � V �q�z | |d
d d d } W n8 t�y� t d tt� �� �| � V Y �q�Y n0 t| ��d< �d d | |29 f�d< � d d 7 < | � V �qJd S )Nc s ���� fS )Nr r ��all_token_counts�chatbotr � status_textr r �get_return_valuen s z(stream_predict.<locals>.get_return_valueu 实时回答模式r: r u 开始实时传输回答……u 输入token计数: Tr � u6 JSON解析错误。请重置对话。收到的内容: �delta�choices�
finish_reason)r �stop�contentul API回复中找不到内容。很可能是Token计数达到上限了。请重置对话。当前Token计数: �����)r, r- �append�construct_user�construct_assistant�len� count_tokenr& r9 r2 �30exceptions�ConnectTimeout�standard_error_msg�connection_timeout_prompt�error_retrieve_prompt� ReadTimeout�read_timeout_prompt�31iter_lines�decoder# �loads�JSONDecodeError�construct_token_message�sum�KeyError�str)r4 r5 r �inputsr= r< r r r6 �32fake_input�display_appendr? Z
partial_words�counterZuser_token_countZsystem_prompt_token_countr8 Zerror_json_str�chunkZ chunklengthrC r r; r �stream_predicta s�
33 � 34� 35� 363738 �39��40�� r` c C sp t �d� |�t|�� |�td�� | r:|�| df� n|�|df� |�tt|��� zt| ||||d|�} W n� tjj y� t41t t } ||| |f Y S tjj
y� t42t t } ||| |f Y S tjj� y� t43t t } ||| |f Y S 0 t�| j�} | d d d d }
t|
�|d< |d d |
|44 f|d< | d d45 }|t|� |d< t|�} ||| |fS ) Nu 一次性回答模式r: FrB r �messagerE rF �usage� total_tokens)r, r- rG rH rI rK r9 r2 rL rM rN rO rP �46ProxyError�proxy_error_prompt�SSLError�ssl_error_promptr# rU �textrX rW )r4 r5 r r[ r= r< r r r6 r\ r] r8 r> rE Ztotal_token_countr r r � predict_all� sF
47� 48� ri Fu 中文Tc
c s49 t �dtjj |� tjj � ||dfg |d|fV | dkrDd} | r�d}t �|� ||dfg |||fV t| | d�}d}||dfg |||fV t| ||||| �\}}}||||fV d S d}g }|50�r�t |d d51�}|}g }t52|�D ]�\}}t �d |d � d
|� �� t j �
|d �j}|�d|d � d|d � d|d � �� |�|d � d|� d|d � d�� q�dd�|� }tt��d|��dd�|���d| �}nd}t| �dk�r.tt }t �|� |�|df� t|�dk�r|�t|�� |�d� |�d� n t|�|d < ||dfg |||fV d S t|�� �dk�rntt }t �|� ||dfg |||fV d S |�r�t �d � t| |||||||| ||d � }|D ]6\}}}}tj j �r�tj � � d S ||||fV �q�n>t �d � t | |||||||| ||d � \}}}}||||fV t �d |� �� t|�d k�rp|d! d" |k�rpt �d#tjj |d! d" � tjj � |�r|t }nt!}t"|�|k�r|
�rd$|� d%|� �}t �|� ||||fV t#| |||||||d& | d'� }|D ]&\}}}}d(|� �}||||fV �q�d S ))Nu 输入为：r: u 开始生成回答……u! 跟随问题语言（不稳定）ug the same language as the question, such as English, 中文, 日本語, Español, Français, or Deutsch.u< 构建索引中……（这可能需要比较久的时间）)�file_srcu* 索引构建完成，获取回答中……� )� max_resultsu 搜索结果r u ：�href�[z]"�bodyz"53URL: z. [z](z)54z55 56z{query}z
{web_results}z{reply_language}�3 r �����u 使用流式传输)r\ r] u 不使用流式传输u# 传输完毕。当前token计数为rF rE u 回答为：u 精简token中�/� )r6 u3 Token 达到上限，已自动降低Token计数至 )$r, r- �colorama�Fore�BLUE�Style� RESET_ALL�construct_index�chat_air � enumerate�urllib3�util� parse_url�hostrG �join�
replace_today�WEBSEARCH_PTOMPT_TEMPLATE�replacerJ rN �
no_apikey_msgrH �strip� no_input_msgr` r. r/ � interrupted�recoverri �max_token_streaming�
max_token_allrX �reduce_token_size)r4 r5 r r[ r= r< r r r r6 Z
use_websearch�files�reply_language�should_check_token_count�msg�indexr> Z57old_inputsZlink_references�search_resultsZ web_results�idx�resultZ domain_name�iterZ max_tokenr r r �predict� s� 58 (&�59���6061 6263�
646566�
� ���67� 68r� c69
c s� t �d� t|�dkr.||t� d�|fV d S |�� |�� d }70|�� t| |||71||||||| d� } t �d� | D ]72} | V qvt �d� d S )Nu 重试中……r u 上下文是空的rE )r r6 r� u 重试完毕)r, r- rJ rN �popr� )
r4 r5 r r= � token_countr r r r6 r� r[ r� �xr r r �retry� s0 73 �
74r� c75
c s� t �d� t| ||t|||||d| d� }76t �d|� �� d} |77D ]�\}}} }
t|
|�}| rf|d d� }d} |dkr�|d| d � ng }|dkr�|
| d � ng }d |� d78�}|||d tt|�dkr�t|�nd� |fV q@t �|� t �d � d S )
Nu 开始减少token数量……F)r6 r� r� z chatbot: rF Tr rq u 保留了最近u 轮对话u ，u 减少token数量完毕)r, r- r� �summarize_prompt�find_nrW rJ rX )r4 r5 r r= r� r r Zmax_token_countr6 r� r� �flagr> �previous_token_countZnum_chatr� r r r r� � s> 79�
80 ��81r� )Nr: )Nr: )!�82__future__r �typingr r r, r# r) r2 r| r rt Zduckduckgo_searchr �asyncio�aiohttp�modules.presets�modules.llama_func�
modules.utils�modules.sharedr. r r �initial_prompt� HISTORY_DIR�
TEMPLATES_DIRr9 r` ri �MODELSr� r� r� r r r r �<module> sV G �83s �84=�85 �86/�