Tracking Variables
When running your code in AskAnna, you might want to track relevant variables related to your run. With AskAnna it is possible to track variables and to get tracked variables from runs. You can use the Python SDK or the AskAnna API. The different ways to track and get variables are described here.
Track variables
Run environment, project and payload variables
The variables set by the worker
(a.k.a. run environment), project
or payload
are automatically tracked in AskAnna. This will give you insight about which environment variables where set in the run environment.
For every variable we will set a label source
that contain a value what the source of the variable is. You can overwrite a project with a payload variable. In AskAnna we will track both variables, but we will only use the latest as an environment variable in the run environment.
Secrets
We don't want to track confidential information like secrets. For example, if you mark a project variable as masked, AskAnna will not track the value of the masked variable. What will get tracked is the name of the variable and we will add a label is_masked
.
To prevent that you accidentally try to track secrets we also exclude values from variables with a name in the following list. Similar to project variables, we will mask the value and add the label is_masked
.
- Key
- Token
- Secret
- Password
Python
If you are using Python, the AskAnna Python SDK makes it easy for you to track variables. When you want to track a variable you only have to add two lines and make sure you installed the Python SDK.
The two lines:
from askanna import track_variable
track_variable(name, value)
When you run a job in AskAnna, every variable you track will be stored in the run. On the run page you find all variables that are tracked for that run. For the value, AskAnna support the following data types:
- integer
- float/numeric
- string
- date
- time
- datetime (ISO8601)
- boolean
- tag
- dictionary
- list
If the value is of type list, we also try to detect the type of data in the list. If we can detect a unique type of data, the data type list
is extened with the type of data in the list:
- list_integer
- list_float (also when the values are a mix of types integer and float)
- list_string
- list_date
- list_time
- list_datetime
- list_boolean
- list_mixed (when multiple data types are detected)
Local run
If you run the code locally, there is no run SUUID set. We store a temporary JSON file with variables locally. We will print the location of the JSON file in case the run SUUID is not available.
It is also possible to add labels. You can add a label, a list of labels, or a dictionary with labels with a value. If you add a label without a value, then it will be processed as a label with value type tag. Some examples:
track_variable(..., label="label a")
track_variable(..., label=["label a", "label b"])
track_variable(..., label={
"type": "model name",
"accuracy_type": "R-squared",
})
It's also possible to track multiple variables at the same time. Use track_variables
and add a dictionary that you want to track. Optionally, you can also add labels to track_variables.
from askanna import track_variables
track_variables({
"model name": "Random Forest",
"accuracy type": "R-squared",
"calc precision": True,
}, label="label a")
Get tracked variables
Run page
On the run page, you van view the variables tracked for a run. In the next example we show a run with only worker variables that we track by default.
In the table you find find the name of the variable, and the value tracked. For variable we always set a label source
that can contain values:
- Run
- Payload
- Project
- Worker
The run
variables are tracked during the run time of the job. The other variable sources are set while building the run environment. The variables with label source payload
, project
and worker
were also available as environment variables.
Next to the table view, you can select to view the variables JSON. Also you can download, or copy, the JSON with the variables for the run.
Python
You can use the tracked variables directly in Python with the runs module. You can get all tracked variables from runs next to the other info this module provides. If you want to include variables, you should set include_variables
to True. Note that by default we will return a maximum of 100 runs, you can increase the limit
value to get more runs.
from askanna import run
job_runs = run.list(job="{{ JOB SUUID or JOB NAME}}",
limit=100,
include_variables=True)
If you want to get run information including variables for a specific set of runs, you can list the runs you want to retrieve:
runs = run.list(run_suuid_list=["{{ RUN SUUID 1 }}", "{{ RUN SUUID 2 }}"],
include_variables=True)
API
You can use the AskAnna API to retrieve the variable information. It's possible to get all variables from a job, or to get the variables from a single run:
GET /v1/run/{{ RUN SUUID }}/variable/
GET /v1/job/{{ JOB SUUID }}/variable_tracked/
The following parameters are available:
- limit / offset
- variable_name (string to query only variables with name "x")
- variable_value (string to query a specific value for variables)
- label_name (string to query variables that only has label "x")
- label_value (strig to query variables that only has label value "x")
If we go back to the example from the run page. If you only want get get the variable with name RUN_SUUID
, with the API you could filter them by using:
GET /v1/run/{{ RUN SUUID }}/variable/?variable_name=RUN_SUUID