This tutorial explains how to use SweetViz to create feature profiles of a pandas dataframe and both view the profile in the notebook and save to an HTML file.


This tutorial uses:

Open up your jupyter notebook and insert the following into the first cell:

import statsmodels.api as sm
import pandas as pd
import numpy as np
import sweetviz as sv

Reading the data

The data is from rdatasets imported using the Python package statsmodels.

df = sm.datasets.get_rdataset('flights', 'nycflights13').data

Feature Engineering

Convert the times from floats or ints to hour and minutes

Convert some of the fields into more meaningful fields to better understand the time flights depart and arrive. Next the original fields are dropped as they are now redundant.

df['arr_hour'] = df.arr_time.apply(lambda x: int(np.floor(x/100)))
df['arr_minute'] = df.arr_time.apply(lambda x: int(x - np.floor(x/100)*100))
df['sched_arr_hour'] = df.sched_arr_time.apply(lambda x: int(np.floor(x/100)))
df['sched_arr_minute'] = df.sched_arr_time.apply(lambda x: int(x - np.floor(x/100)*100))
df['sched_dep_hour'] = df.sched_dep_time.apply(lambda x: int(np.floor(x/100)))
df['sched_dep_minute'] = df.sched_dep_time.apply(lambda x: int(x - np.floor(x/100)*100))
df.rename(columns={'hour': 'dep_hour',
                   'minute': 'dep_minute'}, inplace=True)
df.drop(columns=['time_hour', 'dep_time', 'sched_dep_time', 'arr_time', 'sched_arr_time', 'dep_delay'], inplace=True)

Profile Features

The parameter pairwise_analysis defaults to auto, so the pairwise analysis is only run if the number of features is below a threshold. As this data contains more features than that threshold, pairwise_analysis is set to on to cause it to be run.

report = sv.analyze(df, pairwise_analysis='on')

Show profile with notebook widgets


Save the profile to an HTML file

By default, show_html will open the HTML document in the browser. Setting open_browser to false prevents the page from being opened.

report.show_html("nyc_flights_report.html", open_browser=False)

No-code/low-code data prep and visualization

Request Demo
Try for Free