evaluate#

datarobotx.evaluate(model, evaluation_data=None, data_partition=None, wait_for_autopilot=False)#

Show evaluation metrics and plots for a model.

DataRobot automatically calculates several model evaluation metrics and plots to help assess model performance.

This helper retrieves these metrics and plots and renders them in-notebook; if an external dataset is provided, it will automatically be uploaded and calculations requested.

For Binary Models with external evaluation data: evaluate will score the dataset against all DataRobot metrics for binary classification and display the lift chart, roc curve, and feature impact chart.

For Regression Models with external evaluation data: evaluate will score the dataset against all DataRobot metrics for regression and display the lift chart, residuals chart, and feature impact chart.

For Multiclass Models with external evaluation data: evaluate will score the dataset against all DataRobot metrics for multiclass and display the lift chart and feature impact chart.

Parameters:

  • model (Union[Model, ModelOperator]) – The drx model that should be used in the evaluation.

  • evaluation_data (Union[pd.DataFrame, str], optional) – The evaluation data to be used. If Pandas Dataframe, the data will be uploaded to DR and scored. If a string, should correspond to a previously uploaded external test set or an ai catalog id.

  • data_partition (str, optional) – The data parition to use for evaluation. If None, the latest available will be used. If evaluation_data is supplied, that will be used instead of data_partition. data_partition can be one of: [‘validation’, ‘crossValidation’, ‘allBacktests’, ‘holdout’, ‘backtest_2’, ‘backtest_3’, ‘backtest_4’, ‘backtest_5’, ‘backtest_6’, ‘backtest_7’, ‘backtest_8’, ‘backtest_9’, ‘backtest_10’, ‘backtest_11’, ‘backtest_12’, ‘backtest_13’, ‘backtest_14’, ‘backtest_15’, ‘backtest_16’, ‘backtest_17’, ‘backtest_18’, ‘backtest_19’, ‘backtest_20’]

  • wait_for_autopilot (bool = False) – If True, wait for autopilot to complete before evaluating the model.

Return type:

None

Notes

Only binary, regression and multiclass models are presently supported.