gr8testgad-1/Movie-Review-Analysis
0
1�
2 �5eg � �~ � d dl Z d dlZ e j d� � Z ee j j j � � Zej Z3d� Z dS )� N�en_core_web_smc �b � t | � � }d� |D � � }d� |� � S )Nc �T � g | ]%}|j t v�|j t v� |j ��&S � )�text� stopwords� punctuations)�.0�tokens �MC:\Users\Israel Anaba\Desktop\Movie_Review_Analysis\src\text_preprocessing.py�4<listcomp>z process_text.<locals>.<listcomp> s3 � �
l�
l�
l�U�U�Z�y�-H�-H�U�Z�_k�Mk�Mk�e�j�Mk�Mk�Mk� � )�nlp�join)r �doc�tokenss r � process_textr 5 s2 � �6
�d�)�)�C�
l�
l�c�
l�
l�
l�F� �8�8�F� � �r ) �spacy�string�loadr �set�lang�en�7STOP_WORDSr � punctuationr r r r r �<module>r sb ��
� � � �
�
�
�
��e�j�!�"�"�� �C��8�
�(� )� )� ��!� � � � � � r