End-to-End Text Summarization and Analysis System implemented for the IRWA module in my Y3S1.
Key Features of the System:
• Text Summarization : Summarizes news articles to 30% of their original content using cosine similarity, ensuring a concise yet meaningful representation of the text.
• Sentiment Analysis : Evaluated several models, including CatBoost, XGBoost, and RandomForest, for sentiment classification. XGBoost was selected as the top-performing model based on model evaluation metrics, delivering superior accuracy and reliability.
• Keyword/Topic Extraction : Utilized the Latent Dirichlet Allocation (LDA) model to identify key topics and extract relevant keywords, enhancing content understanding.
• Model Training and Evaluation : Comprehensive preprocessing, including data cleaning, feature scaling, and balancing, was performed. Models were rigorously evaluated using confusion matrices, classification reports, and accuracy metrics to ensure optimal performance.