Anomaly Detection _ Data Scraping From Websites

Introduction

Anomaly Detection

This course introduces anomaly detection using unsupervised machine learning techniques. Anomaly detection is crucial for identifying outliers in datasets, which may represent errors, unusual events, or significant insights. By applying methods such as isolation forests through the PyCaret library, participants will learn how to preprocess datasets, train models, and interpret results to improve the accuracy of machine learning workflows

Data Scraping from Websites

This course covers the fundamentals of web scraping using Python libraries such as Requests, BeautifulSoup, and Pandas. The focus is on extracting vehicle safety data from the IIHS (Insurance Institute for Highway Safety) website, storing test results in structured formats like DataFrames, and exporting reports to Excel. Through this, participants will learn to automate data collection while considering limitations of scraping methods.

Learning Objectives

Anomaly Detection

- Understand the concept and importance of anomaly detection.

- Learn how to prepare datasets by handling null values and splitting into training/testing sets.

- Implement anomaly detection models using PyCaret.

- Analyze results and evaluate model performance for unsupervised problems.

- Apply anomaly detection to improve clustering model performance.

Data Scraping from Websites

- Understand the basics of web scraping and its ethical considerations.

- Use Python libraries (Requests, BeautifulSoup, Pandas) for data extraction.

- Scrape structured vehicle safety test data from IIHS website.

- Filter data using Test IDs and user-defined parameters.

- Export collected data into Excel for reporting and analysis.