CoolFace
Modelpublic

hymenjj/llama-cpp-python-prebuilt

sourceHugging Faceupdated 7mo agoView on Hugging Face
0likes
errors.cpython-311.pyc87 linesDownload Raw Back to __pycache__
1200�i�����ddlmZddlZddlZddlZddlmZmZmZddl	m3Z4mZmZm
Z
mZmZddlmZddlmZmZmZddlmZddlmZdd	lmZmZmZGd5�de��ZGd�d
��ZGd�de��Z dS)�)�annotationsN)�compile�Match�Pattern)�Callable�	Coroutine�Optional�Tuple�Union�Dict)�	TypedDict)�Request�Response�
HTTPException)�JSONResponse)�APIRoute)�CreateCompletionRequest�CreateEmbeddingRequest�CreateChatCompletionRequestc�<�eZdZUdZded<ded<ded<ded<dS)	�
ErrorResponsezOpenAI style error response�str�message�typez
Optional[str]�param�codeN)�__name__�6__module__�__qualname__�__doc__�__annotations__���B/usr/local/lib/python3.11/site-packages/llama_cpp/server/errors.pyrrsB�������%�%��L�L�L�
�I�I�I����������r#rc�B�eZdZdZed	d���Zed	d���ZdS)7�ErrorResponseFormattersa1Collection of formatters for error responses.8 9    Args:10        request (Union[CreateCompletionRequest, CreateChatCompletionRequest]):11            Request body12        match (Match[str]): Match object from regex pattern13 14    Returns:15        Tuple[int, ErrorResponse]: Status code and error response16    �request�?Union['CreateCompletionRequest', 'CreateChatCompletionRequest']�return�Tuple[int, ErrorResponse]c	�"�t|�d����}t|�d����}|j}t|d��rd}nd}dt	|�||pd|z||��ddd	�17��fS)z+Formatter for context length exceeded error���messagesz�This model's maximum context length is {} tokens. However, you requested {} tokens ({} in the messages, {} in the completion). Please reduce the length of the messages or completion.z�This model's maximum context length is {} tokens, however you requested {} tokens ({} in your prompt; {} for the completion). Please reduce your prompt; or completion length.�r�invalid_request_error�context_length_exceeded�rrrr)�int�group�18max_tokens�hasattrr�format)r'�match�context_window�
prompt_tokens�completion_tokensrs      r$r1z/ErrorResponseFormatters.context_length_exceeded/s����U�[�[��^�^�,�,���E�K�K��N�N�+�+�
�#�.���7�J�'�'�	�J�
�G�C�
��M��N�N��"�'�a�=�8��!�	��)��*�192021222324252627	28r#c�|�t|�d����}d|�d�}dt|ddd���fS)	z#Formatter for model_not_found errorr-zThe model `z` does not existr/r0N�model_not_foundr2)rr4r)r'r8�29model_pathrs    r$r=z'ErrorResponseFormatters.model_not_foundUsU������Q���(�(�30�<�31�<�<�<���M��(��"�	32�33�34�35�	36r#N)r'r(r)r*)rrrr �staticmethodr1r=r"r#r$r&r&#s_������	�	��#37�#38�#39��\�#40�J�
41�
42�
43��\�
44�
45�
46r#r&c���eZdZUdZed��ejed��ejiZde	d<	ddd
�Z47d�fd�Z�xZS)�RouteErrorHandlerz>Custom APIRoute that handles application errors and exceptionsz9Requested tokens \((\d+)\) exceed context window of (\d+)zModel path does not exist: (.+)z�Dict['Pattern[str]', Callable[[Union['CreateCompletionRequest', 'CreateChatCompletionRequest'], 'Match[str]'], Tuple[int, ErrorResponse]]]�pattern_and_formattersN�error�	Exception�body�cOptional[Union['CreateChatCompletionRequest', 'CreateCompletionRequest', 'CreateEmbeddingRequest']]r)r*c��|�mt|ttf��rQ|j���D]7\}}|�t
|����}|�|||��cS�8tdt
|����tj	���tjtj	���dtt
|��ddd���fS)z2Wraps error message in OpenAI style error responseNzException: )�filei��internal_server_errorr2)
�48isinstancerrrB�items�searchr�print�sys�stderr�	traceback�	print_excr)�selfrCrE�pattern�callbackr8s      r$�error_message_wrapperz'RouteErrorHandler.error_message_wrapper}s�����49��'�+�
�!50�!51��&*�%@�%F�%F�%H�%H�
1�
1�!������s�5�z�z�2�2���$�#�8�D�%�0�0�0�0�0�%�	�(�C��J�J�(�(�s�z�:�:�:�:�����,�,�,�,��M���J�J�(���	52�53�54�55�	56r#�4Callable[[Request], Coroutine[None, None, Response]]c�Z����t������d��fd�}|S)zgDefines custom route handler that catches exceptions and formats57        in OpenAI style error responser'rr)rc���K�	tj��}�58|���d{V��}ttj��|z59dz��}|�|jd<|S#t$r}|�d}~wt60$r�}|����d{V��}	d|vr
tdi|��}nd|vr
tdi|��}ntdi|��}n#t61$rd}YnwxYw��62||���\}}	td|	i|���cYd}~Sd}~wwxYw)	Ni�zopenai-processing-msr.�prompt)rCrErC)�status_coder")�time�perf_counterr3�headersrrD�jsonrrrrUr)r'�	start_sec�response�elapsed_time_ms�unauthorized�exc�	json_bodyrErZ�
error_message�original_route_handlerrRs          ��r$�custom_route_handlerzARouteErrorHandler.get_route_handler.<locals>.custom_route_handler�s������'
� �-�/�/�	�!7�!7��!@�!@�@�@�@�@�@�@��"%�t�'8�':�':�Y�'F�$�&N�"O�"O��>M�;O�� �!7�8���� �
#�
#�
#�"�"������
�
�
�")�,�,�.�.�0�0�0�0�0�0�	� �!�Y�.�.�8�D�D�)�D�D�
��"�Y�.�.�6�C�C��C�C��� 6�B�B�	�B�B���� � � � ��D�D�D� �����.�.�S�t�.�D�D���!�#��m�,� +��������������7
���sN�AA�63D�&A(�(
D�5D�.B?�>D�?C�D�
C�/D�=D�D)r'rr)r)�super�get_route_handler)rRrgrf�	__class__s` @�r$riz#RouteErrorHandler.get_route_handler�sI�����"'���!:�!:�!<�!<��(	�(	�(	�(	�(	�(	�(	�T$�#r#)N)rCrDrErFr)r*)r)rV)
rrrr rr&r1r=rBr!rUri�
__classcell__)rjs@r$rArAfs���������H�H�	��H�	64�	65�"�:���.�	66�	67�"�2�
	������6
�#68�#69�#70�#71�#72�J2$�2$�2$�2$�2$�2$�2$�2$�2$�2$r#rA)!�73__future__rrNrPr[�rerrr�typingrrr	r74rr�typing_extensionsr
�fastapirrr�fastapi.responsesr�fastapi.routingr�llama_cpp.server.typesrrrrr&rAr"r#r$�<module>rts���"�"�"�"�"�"�75767778���������&�&�&�&�&�&�&�&�&�&�D�D�D�D�D�D�D�D�D�D�D�D�D�D�D�D�'�'�'�'�'�'�����������79+�*�*�*�*�*�$�$�$�$�$�$����������������I����@80�@81�@82�@83�@84�@85�@86�@87�Fn$�n$�n$�n$�n$��n$�n$�n$�n$�n$r#