Jobs
In an AskAnna project, you can create jobs that you specify in the askanna.yml. Jobs described as tasks you want to perform using AskAnna.
There are a couple of use cases for jobs in AskAnna:
- Run a (large) task in AskAnna, so you don’t have to run it locally on your machine
- If you want to run a series of commands, for example to retrain & update your model, the job is the place to be
- Using our API and Jobs, you can put a data science model in production and run it via other applications
Job overview
Jobs relates to a project. On the project page you will find a section listing jobs.
When you open the jobs section, you will find a table that lists the available jobs for that project. In this table you will find:
Name | Description |
---|---|
If you click on this arrow, a list of runs for this job opens | |
Name | Name of the job |
Runs | How many times the jobs has run |
Status | The status of the last job run |
You can find more details about the table with job runs on the runs page.
Job page
After you have created a job, you can open the job page related to that job. On this page you will find the Overview with:
- Description of the job
- Definition of the job
- Information how you can run the job
Also, you see a section Runs where you can find the runs for that job. On the run documentation you can find more information regarding this topic.
Below an example of a job page:
Job description
On the job page you can add/edit a description of the job. Everybody with access to the job can see this information.
Markdown support
The text editor support Markdown instructions. These instructions will automatically transferred into formatted text.
Create a job
You create a job via a manifest that defines the job’s behavior. You can add an askanna.yml file to your project code. In the askanna.yml file you can specify a job. After specifying the job, you can push/upload your code to AskAnna and AskAnna will create the job.
Job definition
An example of a job definition:
name of the job:
job:
- pip install -r requirements.txt
- python train_your_model.py --input_file {{ PAYLOAD_PATH }}
output:
result: models/model_summary.json
artifact:
- models/
name of the job (required)
The job definition starts with the name of the job. You cannot use the following names as a job name:
- push-target
- environment
- cluster
- variables
job (required)
Here you list the commands that you want to execute in the run. In the above example, the first command executed is installing the requirements. If that command finished successful, we will also run a second command that triggers a Python script.
output
In the output
you can define which files you want to store. There are two options:
- result
- artifact
In result
you can refer to a single file that contains the result of the run. For example, a JSON file with the summary of a train job or predictions.
In artifact
you can list directories or files that you want to save as artifact of the run.
Use variables
In the job run definition, you can also use variables. In the job example above the {{ PAYLOAD_PATH }}
is a variable. A variable can be used as a parameter in the job run manifest and is also available as environment variables in the run environment. Variables are created on different levels:
Note
Currently, the behavior of using variables as parameters in the job run definition is not consequent. In most cases you can refer to a variable via {{ VARIABLE_NAME }}
and sometimes you need to use ${VARIABLE_NAME}
. In case the first doesn’t work, please try the second variant. In the future we will uniform the solution.
AskAnna variables
AskAnna variables are always available in every job. The list of AskAnna variables:
Name | Description |
---|---|
AA_TOKEN | Token used to trigger the job |
JOBRUN_SUUID | The unique identifier for the job run |
JOBRUN_JOBNAME | The name of the job |
PROJECT_SUUID | The unique identifier for the project |
PACKAGE_SUUID | The unique identifier for the code package used |
PAYLOAD_SUUID | The unique identifier for the payload file |
PAYLOAD_PATH | The path of the payload file |
PAYLOAD_PATH
When you run a job, it’s possible to send a JSON data body. This JSON file is made available in the job run environment. The location of the payload file is available via the variable PAYLOAD_PATH
. This variable is also set as an environment variable for the run environment.
Project variables
Project variables are variables defined on project level. On the project page you can find the section variables. Here you can manage variables. The project variables are available via the name you defined and for both the job definition, as also as environment variables in the run environment.
For more information check the variables documentation.
Payload variables
There are two use cases for adding a JSON body to a request. The first is adding data that can be used in the run. For example, a JSON body with the most recent sales data. Another use case is that the JSON body contains variables that can be used in the job run.
AskAnna scans the first 10,000 characters of the JSON body. Every key will become a variable name, and every value the value of that variable. In case the Payload variable
and Project variable
has the same name, AskAnna will use the value of the Payload variable
.
Example of a JSON body:
{
"FROM_DATE": "2020-09-07",
"UNTIL_DATE": "2020-09-11",
"TRAINED_MODEL_ID": "k68o-Mi2L-mYte-7E54"
}
In the job run definition you can use these JSON keys as variables:
name of the job:
job:
- askanna artifact get --id {{ TRAINED_MODEL_ID }} --output "models/"
- pip install -r requirements.txt
- python your_model.py --from_date {{ FROM_DATE }} --until_date {{ UNTIL_DATE }}
In the above example, we use Payload variables
to define which trained model and period we want to use to run the model. With this setup you can easily switch between models and periods.
Run a job
In AskAnna you can trigger jobs using the web interface, CLI, Python SDK or API. In the section Running the job
you can find examples as well.
On the job page you can find information about how you can run a job in AskAnna. When new options become available, we will extend the option presented here.
Web interface
The fastest and easiest way of triggering a job is via the web interface. On the tab PLATFORM
you can add a payload and run the job. The payload is optional, so you can leave it empty. When you run the job, you will also see the status of the run and a button to open the run.
AskAnna CLI
First, install the AskAnna CLI to be able to start running a job via the command line:
askanna run
If you are running the command from a project directory with an askanna.yml
and the push-target
set to a valid project, then the CLI will ask you which job you want to run. If this information is not available, the CLI will check if you are a member of multiple workspaces or projects. If that's the case, a wizard guide you to select a job to run.
To make live easier, you can also specify the name of the job:
askanna run "JOB NAME"
Or the SUUID of the job:
askanna run --id JOB_SUUID
If you want to push your code, and then run the job you can combine these tasks in one command:
askanna run "JOB NAME" --push
Similar to the API, you can add JSON data as a string:
askanna run "JOB NAME" --data '{"example": "payload"}'
Or a JSON file:
askanna run "JOB NAME" --data-file 'input/payload.json'
For more options check the help:
askanna run -h
API
You can trigger jobs using our API. You can find all the information you need on the job page in the section Running the job
. For the API you will find a Shell
example. In the example you see:
- The URL including the job SUUID
- Your authorization token
- An example data payload
When you copy this code, you only need to change the data (second row) and you are ready to run it via your terminal. In case you are going to work often with APIs, we advise using a software platform like Postman.
The JSON data is optional. You can also run jobs without this content. If you add a JSON data body to the API request, it will become available as the payload (input) for the job run. This way you can add data, variables, or both to your job run.
Python
AskAnna SDK
First, install the Python AskAnna SDK. Then you can run a job in your Python script:
from askanna import run
run_info = run.start(job_suuid="JOB SUUID")
If you run the code from a project directory with an askanna.yml
and the push-target
set to a avlid project, you can also use the name of the job:
from askanna import run
run_info = run.start(job_name="JOB NAME")
It's also possible to include a dictionary with JSON data as input for the run:
from askanna import run
json_data = {"example": "payload"}
run_info = run.start(job_name="JOB NAME", data=json_data)
Request
In the tab PYTHON you can find an example of how you can use the AskAnna API to run a job in Python. When you trigger a job via Python, you can use the response for follow-up actions. For example to monitor when the jobs are finished so you can retrieve a result.
Run environment
The default run environment is Python 3.7 with 1 CPU. Besides Python 3.7 we also installed:
- Git
- Unzip
- Curl
- g++
- AskAnna CLI & SDK
If you want to have another run environment, you can always contact us ([email protected]).