kalilurrahman/tnelectionpredictor2026
0
1�
2 ���iv � �T � S SK r S SKJr S SKrS SKrS SKJrJrJrJ r " S S5 r3g)� N)�
BeautifulSoup)�List�Dict�Any�Optionalc 4 �� � \ rS rSrS rSS\S\S\\R 4S jjr S\S\4S jr5S\ \ \\4 4S jr
SS \S6\S\ \ \\4 4S jjrSS \S
\S\ \ \\4 4S jjrS\S\ \\4 4S jrSrg)�ElectionScraper� c � � [ R " 5 U l U R R R SS05 SU l SU l g )Nz7User-AgentzsMozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36� )� r )�requests�Session�session�headers�update� max_retries�delay_between_requests)�selfs �]C:\Users\HP\OneDrive\Documents\GitHub\tn-election-predictor\tn-predictor-final\src\scraper.py�__init__�ElectionScraper.__init__ sJ � � �'�'�)�� � � � � � �#�#� � P�%8� � ���&,��#� �url�method�returnc 9 � � [ U R 5 Hf n U R R " X!4SS0UD6nUR 5 [10 R " [ R " U R 6 5 Us $ g ! [ R R ab n[ SUS- SU SU 35 X@R S-11 : a [12 R " SU- 5 S nAM� [ SU S 35 S nA g S nAff = f)13N�timeout� zAttempt r
z failed for z: � zMax retries reached for z . Skipping.)�ranger r �request�raise_for_status�time�sleep�random�uniformr r �14exceptions�RequestException�print)r r r �kwargs�attempt�response�es r �
_make_request� ElectionScraper._make_request s� � � �T� -� -�.�G�
� �<�<�/�/��R�R�R�6�R���)�)�+��15�16�6�>�>�4�+F�+F�G�H� �� /� �� �&�&�7�7�
����1��
�\�#��b���D�E�� -� -�� 1�1��J�J�q�G�|�,�,�� 4�S�E��E�F� ��
�s �A!B� D� AC<�'C<�<Dc �p � [ SU 35 U R U5 nU( a UR $ SU 3$ )NzScraping custom URL: z Failed to retrieve content from )r* r/ �text)r r r- s r �scrape_custom_url�!ElectionScraper.scrape_custom_url s< � �
�%�c�U�+�,��%�%�c�*�� ��=�=� �5�c�U�;� ;r c �� � [ S5 SSSS.SSSS.S S17S S.S S
S S./n[ S5 [ R " [ R " U R18 6 5 U$ )Nz8Attempting to scrape ECI candidate data (Placeholder)...zThiru. M.K. Stalin�DMK�Kolathur)�name�party� constituencyz Thiru. Edappadi K. Palaniswami�AIADMK�EdappadizThiru. K. Annamalai�BJPzCoimbatore SouthzThiru. Kamal HaasanzMakkal Needhi Maiamz.Returning dummy candidate data for ECI scrape.�r* r$ r% r& r' r )r �dummy_candidatess r �%scrape_election_commission_candidates�5ElectionScraper.scrape_election_commission_candidates( sk � �
�H�I�)�E�:� V�5��Zd� e�*�U�L^� _�*�5J�\n� o� 19�� �>�?� �20�21�6�>�>�4�#>�#>�?�@� � r �query� num_articlesc �2 � [ SU S35 / nSU S3nU R U5 nU( d
[ S5 / $ [ UR S5 nUR SSS 9nUR SS22S 9n[ S [ U5 S [ U5 S
35 [
[ [ U5 [ U5 U5 5 GH n Xy R S5 n23U24( a U25R SS9OSn U26( a SU27R ; a U28S OSn X� R SSS 9n
U
( a< U
R S5 Vs/ s H o�R SS9PM nnU( a US OSnOSnU S:w d M� U S:w d M� U R S5 ( a& U R S5 S R S5 S n UR X�UUS.5 GM [ S[ U5 S35 [ R " [ R" " U R$ 6 5 U$ s snf ) Nz#Scraping news articles for query: '�'z https://www.google.com/search?q=z&tbm=nwsz)Failed to fetch news search results page.z html.parser�div�n0jRee)�class_�MgUUOzFound z potential headlines and z potential sources.�aT)�stripzN/A�href�#�B6bxjdr z/url?q=r
z&sa=U)�titler �sourcerB zScraped z29 articles.)r* r/ r r2 �find_all�lenr! �min�find�get_text�attrs�30startswith�split�appendr$ r% r& r' r )r rB rC �
articles_data�31search_urlr- �soup� headlines�sources�i� headline_tag�headline�link�source_info_tag�tag�source_time_partsrP s r �scrape_news_articles�$ElectionScraper.scrape_news_articles4 s� � �
�3�E�7�!�<�=��
�7��w�h�G�32��%�%�j�1��� �=� >��I� �X�]�]�M�:���M�M�%��M�9� ��-�-��g�-�6��
��s�9�~�&�&?��G� �~�M`�a�b��s�3�y�>�3�w�<� �F�G�A�$�<�,�,�S�1�L�<H�|�,�,�4�,�8�e�H�+7�F�l�FX�FX�<X�<��'�^a�D� %�j�o�o�e�H�o� E�O� �IX�Ia�Ia�bg�Ih�$i�Ih�#�\�\��\�%=�Ih�!�$i�1B�*�1�-��� ���5� �T�S�[��?�?�9�-�-� �:�:�i�0��3�9�9�'�B�1�E�D� �$�$�x�PV�af�%g�h� H� ���]�+�,�J�7�8� �33�34�6�>�>�4�#>�#>�?�@� � �� %js �8H�keywords�countc �� � [ SU S35 SU S3SSS.SU S 3S35SS./US - - n[ S 5 [ R " [ R " U R36 6 5 U$ )
Nz,Attempting to scrape Twitter for keywords: 'z' (Placeholder)...z This is a dummy tweet about �.�37DummyUser1z382026-04-04)r2 �user�datezAnother random thought on z from a fake account.� FakeTweeterr z Returning dummy Twitter data.r> )r rh ri � dummy_tweetss r �scrape_twitter_feed�#ElectionScraper.scrape_twitter_feedV s� � �
�<�X�J�FX�Y�Z�3�H�:�Q�?� �_k� l�1�(��;P�Q�[h�r~� �39� �a�Z�� � �-�.� �40�41�6�>�>�4�#>�#>�?�@��r �constituency_namec � � [ SU S35 USSSS/SS/S .n[ S425 [ R " [ R " U R43 6 5 U$ ) Nz Scraping data for constituency: z (Placeholder)...�ChennaizCurrent MLA NamezWater Scarcity�Infrastructurer6 r; )r8 �district�
incumbent_mla� major_issues� key_partiesz"Returning dummy constituency data.r> )r rs �44dummy_datas r �get_constituency_page_data�*ElectionScraper.get_constituency_page_data` sd � �
�0�1B�0C�CT�U�V�%�!� /� -�/?� @� !�8�,� 45�46� �2�3� �47�48�6�>�>�4�#>�#>�?�@��r )r r r N)�GET)zTamil Nadu election�49 )�2 )�__name__�50__module__� __qualname__�__firstlineno__r �strr r �Responser/ r3 r r r@ �intrf rq r r| �__static_attributes__� r r r r s� � �-��� �c� ��QY�Qb�Qb�Hc� � <�S� <�S� <�51 �t�D��c��N�7K� 52 � �#� �UX� �bf�gk�lo�qt�lt�gu�bv� �D �C� �� �T�$�s�TW�x�.�EY� � �C� �D��c��N� r r ) r �bs4r r$ r& �typingr r r r r r� r r �<module>r� s# �� � � �
� ,� ,�d� dr