Tracking Variables

When running your code in AskAnna, you might want to track relevant variables related to your run. With AskAnna it is possible to track variables and to get tracked variables from runs. You can use the Python SDK or the AskAnna API. The different ways to track and get variables are described here.

Track variables

Run environment, project and payload variables

The variables set by the worker (a.k.a. run environment), project or payload are automatically tracked in AskAnna. This will give you insight about which environment variables where set in the run environment.

For every variable we will set a label source that contain a value what the source of the variable is. You can overwrite a project with a payload variable. In AskAnna we will track both variables, but we will only use the latest as an environment variable in the run environment.

Secrets

We don't want to track confidential information like secrets. For example, if you mark a project variable as masked, AskAnna will not track the value of the masked variable. What will get tracked is the name of the variable and we will add a label is_masked.

To prevent that you accidentally try to track secrets we also exclude values from variables with a name in the following list. Similar to project variables, we will mask the value and add the label is_masked.

• Key
• Token
• Secret
• Password

Python

If you are using Python, the AskAnna Python SDK makes it easy for you to track variables. When you want to track a variable you only have to add two lines and make sure you installed the Python SDK.

The two lines:

from askanna import track_variable

track_variable(name, value)

When you run a job in AskAnna, every variable you track will be stored in the run. On the run page you find all variables that are tracked for that run. For the value, AskAnna support the following data types:

• integer
• float/numeric
• string
• date
• time
• datetime (ISO8601)
• boolean
• tag
• dictionary
• list

If the value is of type list, we also try to detect the type of data in the list. If we can detect a unique type of data, the data type list is extened with the type of data in the list:

• list_integer
• list_float (also when the values are a mix of types integer and float)
• list_string
• list_date
• list_time
• list_datetime
• list_boolean
• list_mixed (when multiple data types are detected)

It is also possible to add labels. You can add a label, a list of labels, or a dictionary with labels with a value. If you add a label without a value, then it will be processed as a label with value type tag. Some examples:

track_variable(..., label="label a")

track_variable(..., label=["label a", "label b"])

track_variable(..., label={
                           "type": "model name",
                           "accuracy_type": "R-squared",
                         })

It's also possible to track multiple variables at the same time. Use track_variables and add a dictionary that you want to track. Optionally, you can also add labels to track_variables.

from askanna import track_variables

track_variables({
                  "model name": "Random Forest",
                  "accuracy type": "R-squared",
                  "calc precision": True,
                }, label="label a")

Get tracked variables

Run page

On the run page, you van view the variables tracked for a run. In the next example we show a run with only worker variables that we track by default.

In the table you find find the name of the variable, and the value tracked. For variable we always set a label source that can contain values:

1. Run
2. Payload
3. Project
4. Worker

The run variables are tracked during the run time of the job. The other variable sources are set while building the run environment. The variables with label source payload, project and worker were also available as environment variables.

Next to the table view, you can select to view the variables JSON. Also you can download, or copy, the JSON with the variables for the run.

Python

You can use the tracked variables directly in Python. If you use run.get_variable you get the variables of a specific run. The output is a VariableList with the variables of the specified run:

from askanna import run

run_variables = run.get_variable(run_suuid="{RUN_SUUID}")

If you want to filter the variables, for example to only keep variables with the name model, you can use a filter:

variables_accuracy = run_variables.filter(name="model")

If you use the runs module, you can also get all variables from a job next to the other info this module provides. If you want to include variables, you should set include_variables to True. Note that by default we will return a maximum of 100 runs, you can increase the number_of_results value to get more runs.

from askanna import run

job_runs = run.list(
    job_suuid="{JOB_SUUID}",
    number_of_results=100,
    include_metrics=True
)

If you want to get run information including variables for a specific set of runs, you can list the runs you want to retrieve:

runs = run.list(
    run_suuid_list=["{RUN_SUUID_1}", "{RUN_SUUID_2}"],
    include_variables=True
)

API

You can use the AskAnna API to retrieve the variable information. It's possible to get all variables from a job, or to get the variables from a single run:

GET /v1/run/{RUN_SUUID}/variable/

If we go back to the example from the run page. If you only want get get the variables with name model, with the API you could filter them using the query param variable_name:

GET /v1/run/{RUN_SUUID}/variable/?variable_name=model