How to find optimal parameters for CatBoost using GridSearchCV for Classification?
2

How to find optimal parameters for CatBoost using GridSearchCV for Classification?

This recipe helps you find optimal parameters for CatBoost using GridSearchCV for Classification
In [1]:
def Snippet_200():
    print()
    print(format('How to find optimal parameters for CatBoost using GridSearchCV for Classification','*^82'))

    import warnings
    warnings.filterwarnings("ignore")

    # load libraries
    from sklearn import datasets
    from sklearn.model_selection import train_test_split
    from sklearn.model_selection import RandomizedSearchCV
    from scipy.stats import uniform as sp_randFloat
    from scipy.stats import randint as sp_randInt
    from catboost import CatBoostClassifier

    # load the iris datasets
    dataset = datasets.load_wine()
    X = dataset.data; y = dataset.target
    X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25)

    model = CatBoostClassifier()
    parameters = {'depth'         : sp_randInt(4, 10),
                  'learning_rate' : sp_randFloat(),
                  'iterations'    : sp_randInt(10, 100)
                 }

    randm = RandomizedSearchCV(estimator=model, param_distributions = parameters,
                               cv = 2, n_iter = 10, n_jobs=-1)
    randm.fit(X_train, y_train)

    # Results from Random Search
    print("\n========================================================")
    print(" Results from Random Search " )
    print("========================================================")
    print("\n The best estimator across ALL searched params:\n",
          randm.best_estimator_)
    print("\n The best score across ALL searched params:\n",
          randm.best_score_)
    print("\n The best parameters across ALL searched params:\n",
          randm.best_params_)
    print("\n ========================================================")

Snippet_200()

How to find optimal parameters for CatBoost using GridSearchCV for Classification*

/Users/nilimesh/anaconda3/lib/python3.6/site-packages/sklearn/model_selection/_search.py:841: DeprecationWarning: The default of the `iid` parameter will change from True to False in version 0.22 and will be removed in 0.24. This will change numeric results when test-set sizes are unequal.
  DeprecationWarning)

0:	learn: 0.1708323	total: 62.3ms	remaining: 2.87s
1:	learn: 0.0741225	total: 66.2ms	remaining: 1.49s
2:	learn: 0.0413385	total: 69.6ms	remaining: 1.02s
3:	learn: 0.0349551	total: 77.9ms	remaining: 837ms
4:	learn: 0.0172439	total: 83.3ms	remaining: 700ms
5:	learn: 0.0143331	total: 93.8ms	remaining: 641ms
6:	learn: 0.0136467	total: 97.5ms	remaining: 557ms
7:	learn: 0.0104568	total: 101ms	remaining: 493ms
8:	learn: 0.0084645	total: 107ms	remaining: 452ms
9:	learn: 0.0069334	total: 113ms	remaining: 418ms
10:	learn: 0.0055324	total: 116ms	remaining: 381ms
11:	learn: 0.0049073	total: 121ms	remaining: 352ms
12:	learn: 0.0043357	total: 129ms	remaining: 337ms
13:	learn: 0.0037293	total: 132ms	remaining: 311ms
14:	learn: 0.0034793	total: 136ms	remaining: 290ms
15:	learn: 0.0031629	total: 144ms	remaining: 279ms
16:	learn: 0.0030079	total: 147ms	remaining: 260ms
17:	learn: 0.0027360	total: 151ms	remaining: 243ms
18:	learn: 0.0026357	total: 157ms	remaining: 231ms
19:	learn: 0.0024981	total: 161ms	remaining: 217ms
20:	learn: 0.0023519	total: 164ms	remaining: 203ms
21:	learn: 0.0022145	total: 168ms	remaining: 191ms
22:	learn: 0.0020650	total: 190ms	remaining: 198ms
23:	learn: 0.0020040	total: 203ms	remaining: 194ms
24:	learn: 0.0019376	total: 214ms	remaining: 188ms
25:	learn: 0.0018634	total: 218ms	remaining: 176ms
26:	learn: 0.0017796	total: 228ms	remaining: 169ms
27:	learn: 0.0017148	total: 232ms	remaining: 157ms
28:	learn: 0.0016426	total: 236ms	remaining: 146ms
29:	learn: 0.0016027	total: 239ms	remaining: 136ms
30:	learn: 0.0015376	total: 242ms	remaining: 125ms
31:	learn: 0.0015050	total: 246ms	remaining: 115ms
32:	learn: 0.0014783	total: 249ms	remaining: 106ms
33:	learn: 0.0014207	total: 253ms	remaining: 96.8ms
34:	learn: 0.0013793	total: 257ms	remaining: 88ms
35:	learn: 0.0013399	total: 268ms	remaining: 81.8ms
36:	learn: 0.0013060	total: 280ms	remaining: 75.6ms
37:	learn: 0.0012657	total: 290ms	remaining: 68.7ms
38:	learn: 0.0012342	total: 300ms	remaining: 61.5ms
39:	learn: 0.0012019	total: 305ms	remaining: 53.4ms
40:	learn: 0.0011690	total: 309ms	remaining: 45.2ms
41:	learn: 0.0011382	total: 312ms	remaining: 37.1ms
42:	learn: 0.0011222	total: 315ms	remaining: 29.3ms
43:	learn: 0.0011005	total: 324ms	remaining: 22.1ms
44:	learn: 0.0010792	total: 334ms	remaining: 14.8ms
45:	learn: 0.0010511	total: 338ms	remaining: 7.34ms
46:	learn: 0.0010211	total: 341ms	remaining: 0us

========================================================
 Results from Random Search
========================================================

 The best estimator across ALL searched params:
 <catboost.core.CatBoostClassifier object at 0x1a19f2b2e8>

 The best score across ALL searched params:
 0.6766917293233082

 The best parameters across ALL searched params:
 {'depth': 5, 'iterations': 47, 'learning_rate': 0.6312277198025861}

 ========================================================