LATEST NEWS:

Grab your copy of the free SaaS Metric Playbook! : Vital metrics you’ll need from leaders in SaaS and VC. >>

Rasgo Feature Store for Data Science
Tutorials
Community Dropdown button
Rasgo Quick Start
Data Analysis
Accelerators
Feature Engineering
Docs
Docs Homepage
Rasgo Quickstart
Tutorials
Insight Accelerators
Data Analysis
Model Accelerators
Feature Engineering
Community
Slack
GitHub
Community
Community Dropdown button
Slack
GitHub
Blogs
close menu overlay button
Login
Try For Free
Try For Free!
menu icon button

Sign-Up For Your
Free 30-Day Trial!

Rasgo can be configured to your data and dbt/git environments in under 20 minutes. Book time with your personal onboarding concierge and we'll get you all setup!

Not ready for a free trial?
Not ready for a free trial?
Private Demo
Click here to schedule time for a private demo
Book Demo
Try Rasgo’s FREE SQL Generator
A low-code web app to construct a SQL Query
SQL Generator
Tutorials that help Data Scientists get their pandas on.

Model Selection

Model Selection

Tutorials

How To Calculate r^2 with scikit-learn

How To Calculate RSME using Scikit-learn

How To Calculate MAPE using Scikit-learn

How to Calculate MAE Using Scikit-learn

How To Calculating Log Loss Using Scikit-learn

How To Calculate AUC With Scikit-learn

How To Build Regression models with catboost

How To Build Classification Models With Catboost

How to Do Model Type Selection with PyCaret

How To Do Scikit-Learn Cross-Validation Splits

How To Do Scikit-Learn Group Cross-Validation Splits

How To Do Scikit-Learn Stratified Cross-Validation Splits

Feature Selection Using Mutual Information in Scikit-learn

How To Do A Train Test Split With Scikit-learn

How To Do Time Series Split With PyRasgo

Scikit-Learn Time Series Split

Additional Featured Engineering Tutorials

Data Cleaning

Feature Transformation

Feature Selection

Feature Profiling

Feature Importance

How To Calculate r^2 with scikit-learn

This tutorial explains how to calculate r^2 from scikit-learn on a regression model from catboost.

During this tutorial you will build and evaluate a model to predict arrival delay for flights in and out of NYC in 2013.

Packages

This tutorial uses:

  • pandas
  • statsmodels
  • statsmodels.api
  • numpy
  • scikit-learn
  • sklearn.metrics
  • sklearn.model_selection
  • catboost

Open a new Jupyter notebook and import the following:


import statsmodels.api as sm
import pandas as pd
import numpy as np
from sklearn.metrics import r2_score
from sklearn.model_selection import train_test_split

from catboost import CatBoostRegressor, Pool

Reading the data

The data is from rdatasets imported using the Python package statsmodels.


df = sm.datasets.get_rdataset('flights', 'nycflights13').data
df.info()


RangeIndex: 336776 entries, 0 to 336775
Data columns (total 19 columns):
 #   Column          Non-Null Count   Dtype  
---  ------          --------------   -----  
 0   year            336776 non-null  int64  
 1   month           336776 non-null  int64  
 2   day             336776 non-null  int64  
 3   dep_time        328521 non-null  float64
 4   sched_dep_time  336776 non-null  int64  
 5   dep_delay       328521 non-null  float64
 6   arr_time        328063 non-null  float64
 7   sched_arr_time  336776 non-null  int64  
 8   arr_delay       327346 non-null  float64
 9   carrier         336776 non-null  object 
 10  flight          336776 non-null  int64  
 11  tailnum         334264 non-null  object 
 12  origin          336776 non-null  object 
 13  dest            336776 non-null  object 
 14  air_time        327346 non-null  float64
 15  distance        336776 non-null  int64  
 16  hour            336776 non-null  int64  
 17  minute          336776 non-null  int64  
 18  time_hour       336776 non-null  object 
dtypes: float64(5), int64(9), object(5)
memory usage: 48.8+ MB

Feature Engineering

Handle null values


df.isnull().sum()

year                 0
month                0
day                  0
dep_time          8255
sched_dep_time       0
dep_delay         8255
arr_time          8713
sched_arr_time       0
arr_delay         9430
carrier              0
flight               0
tailnum           2512
origin               0
dest                 0
air_time          9430
distance             0
hour                 0
minute               0
time_hour            0
dtype: int64

As this model will predict arrival delay, the Null values are caused by flights did were cancelled or diverted. These can be excluded from this analysis.


df.dropna(inplace=True)

Convert the times from floats or ints to hour and minutes


df['arr_hour'] = df.arr_time.apply(lambda x: int(np.floor(x/100)))
df['arr_minute'] = df.arr_time.apply(lambda x: int(x - np.floor(x/100)*100))
df['sched_arr_hour'] = df.sched_arr_time.apply(lambda x: int(np.floor(x/100)))
df['sched_arr_minute'] = df.sched_arr_time.apply(lambda x: int(x - np.floor(x/100)*100))
df['sched_dep_hour'] = df.sched_dep_time.apply(lambda x: int(np.floor(x/100)))
df['sched_dep_minute'] = df.sched_dep_time.apply(lambda x: int(x - np.floor(x/100)*100))
df.rename(columns={'hour': 'dep_hour',
                   'minute': 'dep_minute'}, inplace=True)
                   

Prepare data for modeling

Set up train-test split


target = 'arr_delay'
y = df[target]
X = df.drop(columns=[target, 'flight', 'tailnum', 'time_hour', 'year', 'dep_time', 'sched_dep_time', 'arr_time', 'sched_arr_time', 'dep_delay'])
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=.2, random_state=1066)

Fit the model

Build the regression model


categorical_features = X_train.select_dtypes(exclude=[np.number])

train_pool = Pool(X_train, y_train, categorical_features)
test_pool = Pool(X_test, y_test, categorical_features)

model = CatBoostRegressor(iterations=500, max_depth=5, learning_rate=0.05, random_seed=1066, logging_level='Silent')
model.fit(X_train, y_train, eval_set=test_pool, cat_features=categorical_features, use_best_model=True, early_stopping_rounds=10)


Using r2_score from scikit-learn, calculate the r^2.


r2 = r2_score(y_test, model.predict(X_test))
r2

0.9418282555971598
Try RasgoQL

Open source data transformations, without having to write SQL. Choose from a wide selection of predefined transforms that can be exported to DBT or native SQL.

Explore on Github
No items found.
How To Calculate r^2 with scikit-learn
How To Calculate RSME using Scikit-learn
How To Calculate MAPE using Scikit-learn
How to Calculate MAE Using Scikit-learn
How To Calculating Log Loss Using Scikit-learn
How To Calculate AUC With Scikit-learn
How To Build Regression models with catboost
How To Build Classification Models With Catboost
How to Do Model Type Selection with PyCaret
How To Do Scikit-Learn Cross-Validation Splits
How To Do Scikit-Learn Group Cross-Validation Splits
How To Do Scikit-Learn Stratified Cross-Validation Splits
Feature Selection Using Mutual Information in Scikit-learn
How To Do A Train Test Split With Scikit-learn
How To Do Time Series Split With PyRasgo
Scikit-Learn Time Series Split

© RASGO Intelligence, Inc. All rights reserved.

TUtorials
Rasgo Quick StartInsight AcceleratorsData AnalysisModel AcceleratorsFeature Engineering
COMMUNitY
GitHubSlackBlog
COMPANY
About Careers Privacy PolicyTerms of ServiceContact