MLflow：如何从现有运行中读取指标或参数？

Question

我尝试以这种方式读取指标： data, info = mlflow.get_run(run_id) print(data[1].metrics) # example of output: {'loss': 0.01}但它只得到最后一个值。是否可以手动读取特定指标的所有步骤？

Answer 1

我遇到了同样的问题，并且能够使用mlflow.tracking.MlflowClient（）.get_metric_history来获取指标的所有值。这将返回您使用 记录的每个值。mlflow.log_metric(key, value)

快速示例（未经测试）

import mlflow

trackingDir = 'file:///....'

registryDir = 'file:///...'

runID = 'my run id'

metricKey = 'loss'

client = mlflow.tracking.MlflowClient(

            tracking_uri=trackingDir,

            registry_uri=registryDir,

        )

metrics = client.get_metric_history(runID, metricKey)

从文档

get_metric_history（run_id，键）[源] 返回与给定衡量指标记录的所有值相对应的衡量指标对象列表。

参数 run_id – 运行的唯一标识符

键 – 运行中的指标名称

返回 mlflow.entities 的列表。指标实体（如果已记录），否则为空列表

from mlflow.tracking import MlflowClient

def print_metric_info(history):

    for m in history:

        print("name: {}".format(m.key))

        print("value: {}".format(m.value))

        print("step: {}".format(m.step))

        print("timestamp: {}".format(m.timestamp))

        print("--")

# Create a run under the default experiment (whose id is "0"). Since this is low-level

# CRUD operation, the method will create a run. To end the run, you'll have

# to explicitly end it. 

client = MlflowClient() 

experiment_id = "0" 

run = client.create_run(experiment_id) 

print("run_id:{}".format(run.info.run_id))

print("--")

# Log couple of metrics, update their initial value, and fetch each

# logged metrics' history. 

for k, v in [("m1", 1.5), ("m2", 2.5)]:

    client.log_metric(run.info.run_id, k, v, step=0)

    client.log_metric(run.info.run_id, k, v + 1, step=1)

    print_metric_info(client.get_metric_history(run.info.run_id, k))

client.set_terminated(run.info.run_id) 

Answer 2

是的，您可以使用 来获取实验和运行信息。它将 MLflow 数据结构作为字典返回，您可以对其进行迭代以提取 listcomp 中所需的内容。下面是一个示例：MlffowClient APIs

def print_experiment_details(experiment_id, run_id):

    """

    Method to print experiment run info and a specific run details

    :param experiment_id: MLflow experiment ID

    :param run_id: MLflow run ID within an experiment

    :return: none

    """

    print("Finished MLflow Run with run_id {} and experiment_id {}".format(run_id, experiment_id))

    # Use MlflowClient API to list experiments and run info

    client = MlflowClient()

    print("=" * 80)

    # Get a list of all experiments

    print("List of all Experiments")

    print("=" * 80)

    [print(pprint.pprint(dict(exp), indent=4))

     for exp in client.list_experiments()]

    print("=" * 80)

    print(f"List Run info for run_id={run_id}")

    print(pprint.pprint(dict(mlflow.get_run(run_id))))

此输出：

Running local model registry=sqlite:///mlruns.db

Finished MLflow Run with run_id 3f3b827dd6814649a2f84ebae09b26c6 and experiment_id 0

================================================================================

List of all Experiments

================================================================================

{   'artifact_location': './mlruns/0',

    'experiment_id': '0',

    'lifecycle_stage': 'active',

    'name': 'ODSC_TUTORIALS',

    'tags': {   'mlflow.note.content': 'This is experiment for getting started '

                                       'with MLflow ...'}}

None

================================================================================

List Run info for run_id=3f3b827dd6814649a2f84ebae09b26c6

{'data': <RunData: metrics={'metric_1': 0.9236238251076615,

 'metric_2': 1.6732389715754346,

 'metric_3': 2.249979396736294}, params={'n_estimators': '3', 'random_state': '42'}, tags={'mlflow.log-model.history': '[{"run_id": "3f3b827dd6814649a2f84ebae09b26c6", '

                             '"artifact_path": "sklearn-model", '

                             '"utc_time_created": "2020-03-18 '

                             '22:25:33.083332", "flavors": {"python_function": '

                             '{"loader_module": "mlflow.sklearn", '

                             '"python_version": "3.7.5", "data": "model.pkl", '

                             '"env": "conda.yaml"}, "sklearn": '

                             '{"pickled_model": "model.pkl", '

                             '"sklearn_version": "0.22.2.post1", '

                             '"serialization_format": "cloudpickle"}}}]',

 'mlflow.note.content': 'This Run is for getting started with MLflow ...',

 'mlflow.runName': 'LOCAL_REGISTRY',

 'mlflow.source.git.commit': '0a3c6a3739deab77631318eca7fb9690b6dbad66',

 'mlflow.source.name': '/Users/julesdamji/gits/tutorials/mlflow/labs/00_get_started.py',

 'mlflow.source.type': 'LOCAL',

 'mlflow.user': 'julesdamji'}>,

 'info': <RunInfo: artifact_uri='./mlruns/0/3f3b827dd6814649a2f84ebae09b26c6/artifacts', end_time=1584570333841, experiment_id='0', lifecycle_stage='active', run_id='3f3b827dd6814649a2f84ebae09b26c6', run_uuid='3f3b827dd6814649a2f84ebae09b26c6', start_time=1584570332914, status='FINISHED', user_id='julesdamji'>}

您可以在此处获取完整代码

希望有所帮助。

Answer 3

使用MLflow客户端（），您可以使用以下方法轻松获取所有或选定的参数和指标：MlflowClientget_run(id).data

# create an instance of the MLflowClient,

# connected to the tracking_server_url

mlflow_client = mlflow.tracking.MlflowClient(

    tracking_uri=tracking_server_url)

# list all experiment at this Tracking server

# mlflow_client.list_experiments()

# extract params/metrics data for run `test_run_id` in a single dict 

run_data_dict = mlflow_client.get_run(test_run_id).data.to_dictionary()

# list all params and metrics for this run (test_run_id)

# pprint(run_data_dict)

print(run_data_dict['params']['algo'])

print(run_data_dict['metrics']['RMSE'])

Answer 4

我残酷地解决了这个问题：我阅读了特定[metric_name]的原始文件，其中包含特定的[run_id]。

path = f'./mlruns/0/[run_id]/metrics/[metric_name]'

with open(path) as f:

    content = f.readlines()

metrics_for_step = [float(x.split(' ')[1]) for x in content]