Jobs

In an AskAnna project, you can create jobs that you specify in the askanna.yml. Jobs described as tasks you want to perform using AskAnna.

There are a couple of use cases for jobs in AskAnna:

1. Run a (large) task in AskAnna, so you don’t have to run it locally on your machine
2. If you want to run a series of commands, for example to retrain & update your model, the job is the place to be
3. Using our API and Jobs, you can put a data science model in production and run it via other applications

Job overview

Jobs relates to a project. On the project page you will find a section listing jobs.

When you open the jobs section, you will find a table that lists the available jobs for that project. In this table you will find:

You can find more details about the table with job runs on the runs page.

Job page

After you have created a job, you can open the job page related to that job. On this page you will find the Overview with:

• Description of the job
• Definition of the job
• Information how you can run the job

Also, you see a section Runs where you can find the runs for that job. On the run documentation you can find more information regarding this topic.

Below an example of a job page:

Job description

On the job page you can add/edit a description of the job. Everybody with access to the job can see this information.

Create a job

You create a job via a manifest that defines the job’s behavior. You can add an askanna.yml file to your project code. In the askanna.yml file you can specify a job. After specifying the job, you can push/upload your code to AskAnna and AskAnna will create the job.

Job definition

An example of a job definition:

name of the job:
  job:
    - pip install -r requirements.txt
    - python train_your_model.py --input_file {{ PAYLOAD_PATH }}
  output:
    result: models/model_summary.json
    artifact:
      - models/

name of the job (required)
The job definition starts with the name of the job. You cannot use the following names as a job name:

• push-target
• environment
• cluster
• variables

job (required)
Here you list the commands that you want to execute in the run. In the above example, the first command executed is installing the requirements. If that command finished successful, we will also run a second command that triggers a Python script.

output
In the output you can define which files you want to store. There are two options:

1. result
2. artifact

In result you can refer to a single file that contains the result of the run. For example, a JSON file with the summary of a train job or predictions.

In artifact you can list directories or files that you want to save as artifact of the run.

Use variables

In the job run definition, you can also use variables. In the job example above the {{ PAYLOAD_PATH }} is a variable. A variable can be used as a parameter in the job run manifest and is also available as environment variables in the run environment. Variables are created on different levels:

• AskAnna variables
• Project variables
• Payload variables

AskAnna variables

AskAnna variables are always available in every job. The list of AskAnna variables:

Project variables

Project variables are variables defined on project level. On the project page you can find the section variables. Here you can manage variables. The project variables are available via the name you defined and for both the job definition, as also as environment variables in the run environment.

For more information check the variables documentation.

Payload variables

There are two use cases for adding a JSON body to a request. The first is adding data that can be used in the run. For example, a JSON body with the most recent sales data. Another use case is that the JSON body contains variables that can be used in the job run.

AskAnna scans the first 10,000 characters of the JSON body. Every key will become a variable name, and every value the value of that variable. In case the Payload variable and Project variable has the same name, AskAnna will use the value of the Payload variable.

Example of a JSON body:

{
  "FROM_DATE": "2020-09-07",
  "UNTIL_DATE": "2020-09-11",
  "TRAINED_MODEL_ID": "k68o-Mi2L-mYte-7E54"
}

In the job run definition you can use these JSON keys as variables:

name of the job:
  job:
    - askanna artifact get --id {{ TRAINED_MODEL_ID }} --output "models/"
    - pip install -r requirements.txt
    - python your_model.py --from_date {{ FROM_DATE }} --until_date {{ UNTIL_DATE }}

In the above example, we use Payload variables to define which trained model and period we want to use to run the model. With this setup you can easily switch between models and periods.

Run a job

In AskAnna you can trigger jobs using the web interface, CLI, Python SDK or API. In the section Running the job you can find examples as well.

On the job page you can find information about how you can run a job in AskAnna. When new options become available, we will extend the option presented here.

Web interface

The fastest and easiest way of triggering a job is via the web interface. On the tab PLATFORM you can add a payload and run the job. The payload is optional, so you can leave it empty. When you run the job, you will also see the status of the run and a button to open the run.

AskAnna CLI

First, install the AskAnna CLI to be able to start running a job via the command line:

askanna run

If you are running the command from a project directory with an askanna.yml and the push-target set to a valid project, then the CLI will ask you which job you want to run. If this information is not available, the CLI will check if you are a member of multiple workspaces or projects. If that's the case, a wizard guide you to select a job to run.

To make live easier, you can also specify the name of the job:

askanna run "JOB NAME"

Or the SUUID of the job:

askanna run --id JOB_SUUID

If you want to push your code, and then run the job you can combine these tasks in one command:

askanna run "JOB NAME" --push

Similar to the API, you can add JSON data as a string:

askanna run "JOB NAME" --data '{"example": "payload"}'

Or a JSON file:

askanna run "JOB NAME" --data-file 'input/payload.json'

For more options check the help:

askanna run -h

API

You can trigger jobs using our API. You can find all the information you need on the job page in the section Running the job. For the API you will find a Shell example. In the example you see:

• The URL including the job SUUID
• Your authorization token
• An example data payload

When you copy this code, you only need to change the data (second row) and you are ready to run it via your terminal. In case you are going to work often with APIs, we advise using a software platform like Postman.

The JSON data is optional. You can also run jobs without this content. If you add a JSON data body to the API request, it will become available as the payload (input) for the job run. This way you can add data, variables, or both to your job run.

Python

AskAnna SDK

First, install the Python AskAnna SDK. Then you can run a job in your Python script:

from askanna import run

run_info = run.start(job_suuid="JOB SUUID")

If you run the code from a project directory with an askanna.yml and the push-target set to a avlid project, you can also use the name of the job:

from askanna import run

run_info = run.start(job_name="JOB NAME")

It's also possible to include a dictionary with JSON data as input for the run:

from askanna import run

json_data = {"example": "payload"}
run_info = run.start(job_name="JOB NAME", data=json_data)

Request

In the tab PYTHON you can find an example of how you can use the AskAnna API to run a job in Python. When you trigger a job via Python, you can use the response for follow-up actions. For example to monitor when the jobs are finished so you can retrieve a result.

Run environment

The default run environment is Python 3.7 with 1 CPU. Besides Python 3.7 we also installed:

• Git
• Unzip
• Curl
• g++
• AskAnna CLI & SDK

If you want to have another run environment, you can always contact us ([email protected]).