jacky5124/TLDR-TrendingTopics
0
1---2title: TLDR TrendingTopics3emoji: 🌖4colorFrom: purple5colorTo: yellow6sdk: gradio7sdk_version: 5.5.08app_file: app.py9pinned: false10license: llama3.111short_description: Summarizer of trending topics and news articles12---13 14# TLDR: Trending Topics15 16## What is this Space about?17 18TLDR: Trending Topics is an AI with grounding that summarizes relevant news of trending topics. 19If you like to read news to keep yourself updated about what is happening, but you find yourself 20too busy to read full news articles, then this Space is just for you! By simply selecting a 21country (currently either Canada or United States), you can instantly browse a wide range of 22news topics trending in that country within six hours, and for each trending topic, you can 23view an accurate and succinct summary based on the linked news articles given below the summary. 24Thanks to great understanding of natural language and flexible instruction following, Meta Llama 3.1 25Instruct models make such task possible by grounding with data from reliable APIs and prompt 26engineering to generate output in desired format. With this Space, you can get to know all the 27trending news in just seconds and save time for what matter to you the most!28 29## Architecture30 3132 33## Backend Repositories34 35- [GitHub repository of the cache service](https://github.com/jacky5124/TLDR-TrendingTopics-Cache)36- [GitHub repository of the scheduled jobs](https://github.com/jacky5124/TLDR-TrendingTopics-Functions)37 38## Workflow39 40The scheduled jobs execute the following to fetch trending topics and generate summaries:41 421. Call Bing Search API for trending topics in either Canada or United States.432. Call Brave Search API for news articles as snippets with the trending topics as queries.443. Use Meta Llama 3.1 8B Instruct to filter the snippets to keep only the relevant ones.454. Use Meta Llama 3.1 8B Instruct again to summarize the relevant snippets for each trending topic.465. Cache the summaries as well as original links to the news articles for all trending topics.47 48This Space only accesses the results cached by the scheduled jobs. The results are available until the scheduled jobs run again, which overwrite the cache with new results.49 50## Design Decision51 52The above workflow is run only periodically. Since there could be around 30 trending topics returned 53by Bing Search API, to cover every trending topic, Brave Search API must be called 30 times, each of 54which returns up to 5 news articles as snippets. However, Brave Search API sometimes could return 55news articles that are irrelevant to the given trending topic, so each returned news article must be 56checked for relevance, which means Meta Llama 3.1 8B Instruct is called for every pair of trending 57topic and its queried news article, and there are up to 150 such pairs. Finally, Meta Llama 3.1 8B 58Instruct is used to produce a summary for every trending topic, and this means another 30 calls. 59As a complete workflow is both computationally expensive and pricey, to accommodate any number of 60users, it is made to run by a pre-defined schedule to share the same results as much as possible. 61This strategy works because the only input of the process is just a country name, and users should 62see consistent results for the same country name at the same time. The only downside of this strategy 63is that users are not able to access real-time results, and the results could be up to 6 hours old.64 65The backend is hosted on Microsoft Azure. To save cost, everything except the cache is run on a 66serverless basis. Since the scheduled jobs are run only once per six hours, and the average time of 67a complete workflow is one minute, serverless approach maximizes resource utilization, and Functions 68on Azure provide the best solution to it. Similarly, Meta Llama 3.1 8B Instruct is also made 69available as serverless API rather than being provided by dedicated GPU instances, because the 70generated results are already accurate without fine-tuning the model, so that default weights on the 71API can be used as it is, and the model is only utilized once per six hours by the scheduled jobs. 72The cache service and Redis are containerized instances, which means they can be scaled up or down 73to handle any kind of user traffic with ease. Microsoft Azure provides all these features, and it 74is easy to monitor every part of the backend running on the cloud.75 76