Deploy

Deploy any model to ML Ops

Custom models

Any model built outside of DataRobot is considered a ‘Custom Model’. If a model is built outside of DataRobot, deploy() will:

1. Dynamically generate a Dockerfile to build a custom environment for serving the provided model including:

   • Python interpreter version

   • Dependencies and associated versions This is accomplished by introspecting the environment in which deploy() is called.

   Faster builds

   Users can also supply an environment_id for an existing custom or DataRobot environment if building a custom environment is not desired behavior.

2. Serialize the model and any provided custom hooks, generating an appropriate custom.py file.

   See also

   The DataRobot User Models (DRUM) project has additional documentation on available custom hooks that can be specified.

3. Transmit configuration and serialized file to DataRobot and create a new:

   • Custom model environment

   • Custom model environment version

   • Custom model

   • Custom model version

4. Deploy the custom model into ML Ops.

   Custom models presently supported by drx.deploy()

   • Scikit-learn pipelines and estimators

   • Arbitrary model artifacts and hooks (including LLM-powered apps)

   • Extending deploy() to additional estimators should be straightforward upon request

Deploying DataRobot models

For drx-trained models, drx.deploy() is equivalent to calling deploy() on the model directly.

Usage

Example 1: Simple sklearn pipeline

Deploy

import datarobotx as drx

deployment = drx.deploy(pipe,
                        target='readmitted',
                        classes=['True', 'False'],
                        environment_id='5e8c889607389fe0f466c72d' # Python 3.9 Scikit-Learn dropin environment
                        )

Supporting example pipeline fitting code

from sklearn.ensemble import RandomForestClassifier
from sklearn.preprocessing import OrdinalEncoder
from sklearn.impute import SimpleImputer
from sklearn.compose import ColumnTransformer
from sklearn.pipeline import Pipeline
from sklearn.model_selection import train_test_split
import pandas as pd
import numpy as np

df = pd.read_csv('https://s3.amazonaws.com/datarobot_public_datasets/10k_diabetes.csv')
train, test_df = train_test_split(df, test_size=0.2)
y = train["readmitted"]
X = train.drop("readmitted", axis=1)
test_df = test_df.drop("readmitted", axis=1)
cat_cols = [
    "discharge_disposition_id",
    "medical_specialty",
    "admission_source_id",
    "diag_3",
    "admission_type_id",
    "age",
    "weight",
    "payer_code",
    "diag_2",
    "insulin",
    "diag_1",
    "change",
    "race",
    "diabetesMed",
    "pioglitazone",
    "glipizide",
    "A1Cresult",
    "glyburide",
    "repaglinide",
    "glyburide.metformin",
    "chlorpropamide",
    "acarbose",
    "glimepiride",
    "nateglinide",
    "max_glu_serum",
    "rosiglitazone",
    "gender",
    "metformin",
]
num_cols = [
    "number_diagnoses",
    "num_medications",
    "time_in_hospital",
    "number_inpatient",
    "num_procedures",
    "number_outpatient",
    "num_lab_procedures",
    "number_emergency",
]
rf_classifier = RandomForestClassifier(
    n_estimators=10, random_state=np.random.RandomState(42)
)
imputer = SimpleImputer(strategy="most_frequent")
categorical_transformer = Pipeline(
    steps=[
        ("imputer", imputer),
        (
            "encoder",
            OrdinalEncoder(handle_unknown="use_encoded_value", unknown_value=-1),
        ),
    ]
)
preprocessor = ColumnTransformer(
    transformers=[
        ("num", imputer, num_cols),
        ("cat", categorical_transformer, cat_cols),
    ]
)
pipe = Pipeline(steps=[("preprocessor", preprocessor), ("classifier", rf_classifier)])
pipe.fit(X, y)

Predict

test_df = pd.read_csv('https://s3.amazonaws.com/datarobot_public_datasets/10k_diabetes_20.csv')
predictions = deployment_2.predict(test_df)

Example 2: Huggingface question answering

Deploy with Unstructured target type

import datarobotx as drx
from transformers import pipeline

qa_pipeline = pipeline(
    "question-answering",
    model="mrm8488/bert-tiny-finetuned-squadv2",
    tokenizer="mrm8488/bert-tiny-finetuned-squadv2",
)
path = "./bert-tiny-finetuned-squadv2"
qa_pipeline.save_pretrained(path)

def load_model(input_dir):
    from transformers import AutoModelForQuestionAnswering, AutoTokenizer, pipeline

    model = AutoModelForQuestionAnswering.from_pretrained(input_dir + "/" + path)
    tokenizer = AutoTokenizer.from_pretrained(input_dir + "/" + path)
    return pipeline(
        "question-answering",
        model=model,
        tokenizer=tokenizer,
    )

def score_unstructured(model, data, query, **kwargs):
  import json

  try:
      data_dict = json.loads(data)
      outputs = model(data_dict)
      rv = {"answer": outputs["answer"]}
  except Exception as e:
      rv = {"completion": f"{e.__class__.__name__}: {str(e)}"}
  return json.dumps(rv)

hf_deployment = drx.deploy(
    path,
    extra_requirements=["transformers", "torch"],
    hooks={"load_model": load_model, "score_unstructured": score_unstructured},
)

Predict

inputs = {
    "context": "Healthcare tasks (e.g., patient care via disease treatment) and "
    + "biomedical research (e.g., scientific discovery of new therapies) require "
    + "expert knowledge that is limited and expensive. Foundation models present "
    + "clear opportunities in these domains due to the abundance of data across "
    + "many modalities (e.g., images, text, molecules) to train foundation models, "
    + "as well as the value of improved sample efficiency in adaptation due to the "
    + "cost of expert time and knowledge. Further, foundation models may allow for "
    + "improved interface design (§2.5: interaction) for both healthcare providers "
    + "and patients to interact with AI systems, and their generative capabilities "
    + "suggest potential for open-ended research problems like drug discovery. "
    + "Simultaneously, they come with clear risks (e.g., exacerbating historical "
    + "biases in medical datasets and trials). To responsibly unlock this potential "
    + "requires engaging deeply with the sociotechnical matters of data sources and "
    + "privacy as well as model interpretability and explainability, alongside "
    + "effective regulation of the use of foundation models for both healthcare and "
    + "biomedicine.",
    "question": "Where can we use foundation models?",
}
prediction = hf_deployment.predict_unstructured(inputs)

Deploy with TextGeneration target type

For GenAI applications, deployments can be created with Unstructured or TextGeneration target type. TextGeneration target type supports additional MLOps capabilities such as Data Export, Drift Tracking, and Batch Predictions but can presently only return a single string for each requested prediction (e.g. additional metadata cannot be returned). TextGeneration requires the Enable Monitoring Support for Generative Models feature flag.

When deploying the above example as a TextGeneration deployment, the score_unstructured hook becomes score:

def score(data, model, **kwargs):
    import pandas as pd

    try:
        inp = {"context": data.iloc[0]["context"], "question": data.iloc[0]["question"]}
        outputs = model(inp)
        rv = outputs["answer"]
    except Exception as e:
        rv = f"{e.__class__.__name__}: {str(e)}"
    return pd.DataFrame({"answer": [rv]})

And the drx.deploy() call becomes:

hf_deployment = drx.deploy(
    path,
    target_type="TextGeneration",
    target="answer",
    environment_id="64c964448dd3f0c07f47d040",  # GenAI drop-in env
    hooks={"load_model": load_model, "score": score},
)

And finally the prediction request becomes:

inp = {
    "context": [
        "Healthcare tasks (e.g., patient care via disease treatment) and "
        + "biomedical research (e.g., scientific discovery of new therapies) require "
        + "expert knowledge that is limited and expensive. Foundation models present "
        + "clear opportunities in these domains due to the abundance of data across "
        + "many modalities (e.g., images, text, molecules) to train foundation models, "
        + "as well as the value of improved sample efficiency in adaptation due to the "
        + "cost of expert time and knowledge. Further, foundation models may allow for "
        + "improved interface design (§2.5: interaction) for both healthcare providers "
        + "and patients to interact with AI systems, and their generative capabilities "
        + "suggest potential for open-ended research problems like drug discovery. "
        + "Simultaneously, they come with clear risks (e.g., exacerbating historical "
        + "biases in medical datasets and trials). To responsibly unlock this potential "
        + "requires engaging deeply with the sociotechnical matters of data sources and "
        + "privacy as well as model interpretability and explainability, alongside "
        + "effective regulation of the use of foundation models for both healthcare and "
        + "biomedicine."
    ],
    "question": ["Where can we use foundation models?"],
}
prediction = hf_deployment.predict(pd.DataFrame(inp))

Example 3: Thin, monitored OpenAI wrapper with secret-handling

Network access

This example requires the Enable Public Network Access for all Custom Models feature flag.

Prior to running this example, create a new credential in the DataRobot Credentials Management tool named OPENAI_API_KEY with credential type Basic; store any placeholder value in the Username field and place your OpenAI API key in the Password field.

import datarobotx as drx

def load_model(input_dir):
    import os
    import datarobot_drum as drum
    from langchain.llms import AzureOpenAI
    try:
        key = drum.RuntimeParameters.get("OPENAI_API_KEY")["password"]
    except ValueError:
        key = os.environ.get('OPENAI_API_KEY', '')
    llm = AzureOpenAI(temperature=0, 
                      openai_api_type="azure",
                      openai_api_base="https://your-azure-domain.openai.azure.com/",
                      openai_api_version="2023-05-15",
                      openai_api_key=key,
                      openai_organization="your-azure-domain",
                      deployment_name="text-davinci-003", 
                      model_name="text-davinci-003",
                      max_retries=0,
                      request_timeout=20)
    return llm

def score_unstructured(model, data, query, **kwargs):
    import json
    try:
        data_dict = json.loads(data)
        rv = {'completion': model(data_dict['prompt'])}
    except Exception as e:
        rv = {'error': f"{e.__class__.__name__}: {str(e)}"}
    return json.dumps(rv)

d = drx.deploy(model=None, 
               hooks={'load_model': load_model, 
                      'score_unstructured': score_unstructured}, 
               runtime_parameters=['OPENAI_API_KEY'],
               environment_id='64c964448dd3f0c07f47d040')  # DR py3.9 GenAI drop-in environment 

completion = d.predict_unstructured({'prompt':'What color is the sky?'})

In general, each value in the list passed as the runtime_parameters argument to deploy() will be treated as a credential to be resolved at deployment-time. A suitable model-metadata.yaml will be auto-generated to configure the parameters being requested, and an incremental model version created to bind the credentials to the runtime parameters prior to deployment.

In order to use runtime parameters, enable the feature flag “Enable the Injection of Runtime Parameters for Custom Models”.

Passing arbitrary artifacts

A path to a file or directory can be specified as the model positional argument to deploy(). In this case, each file in this path will be treated as an artifact to be included in the custom model version. This functionality can be used to include additional arbitrary artifacts/files required to power your unstructured app/model. Be careful to include appropriate hooks with your requirements as well as specifying any extra_requirements needed for your hooks to run.

Important considerations

Because deployment can be a long-running operation, it is recommended to test any custom hooks locally in your notebook environment before deploying.

If passing custom hooks to deploy(), be sure any import statements required by the hooks have executed prior to calling deploy() or are dynamic and occur within the hook definition itself. Failing to do so can result in unsuccessful deserialization of the hook inside the MLOps deployment environment at initialization time.

Generally, it is recommended to take advantage of the environment_id keyword argument to avoid the need to rebuild a custom environment while debugging. This can save significant deployment time. Keep in mind that your local dependencies and python version must align to deployment environment to guarantee successful functioning.

Make sure your local python version matches your environment python version!*

The process used by drx to create custom hooks from functions requires that your local python version
matches the python version of the environment you are deploying to. If they are not the same, the deployment will
likely break. You can check your local python version by running `python --version` in a terminal

The extra_requirements keyword argument can be used to explicitly enumerate additional requirements to include in your custom environment.

Fully custom environment example (not generally recommended)

deployment = drx.deploy(
    path, # Path to deployment model artifacts
    extra_requirements=["transformers", "torch"], # extra requirements to include in the environment
    hooks={"load_model": load_model, "score_unstructured": score_unstructured},
)

API Reference

deploy(model, *args[, target_type, target, ...])	Deploy a model to MLOps.
Deployment([deployment_id])	DataRobot ML Ops deployment.