CLI Reference¶

Bodywork CLI commands for managing ML pipelines on Kubernetes. Note that bw can be used as an alias for bodywork in any of the commands listed below.

Get Version¶

$ bodywork --version

Prints the Bodywork version.

Validate Configuration File¶

The bodywork.yaml file can be checked for errors using,

$ bodywork validate

Returns a list of configuration file errors.

Options:

--check-files: Check that all paths specified as executable_module_path exist and can be reached by Bodywork, from the root directory where bodywork.yaml is located. This command assumes that bodywork.yaml is in the current working directory - if this is not the case, use the --file option to specify the path of bodywork.yaml.

Configure Cluster¶

$ bodywork configure-cluster

Prepare your Kubernetes cluster for use with Bodywork. Creates a dedicated Bodywork namespace, together with service accounts and roles for managing pipeline orchestration.

Run this command before you try to run any others

This only needs to be done once per-cluster. It will also be triggered automatically if you try and create deployment without it, but if you try to do anything else you will be met with errors.

Deploying Pipelines¶

Orchestrate stages on Kubernetes that run ML workloads and start ML services.

Create Deployment¶

$ bodywork create deployment GIT_REPO_URL

Deploy a pipeline to Kubernetes. Press CTRL + C to gracefully terminate deployments before they have finished.

Arguments:

GIT_REPO_URL: Location of the Git repository that contains the pipeline's codebase.

Options:

--branch: Branch of Git repository to deploy, defaults to your repository's default branch (e.g., master).
--retries: The number of times to retry the deployment, should any stage fail.
--ssh: If the Git repo is private, then provide a path to a valid SSH key file to enable access - e.g., ~/.ssh/id_rsa. Automatically creates a secret for this purpose.

Get Deployments¶

$ bodywork get deployments PIPELINE_NAME SERVICE_NAME

Get information on pipelines and the service created by them. If no options are specified, then a list of all pipelines with active services will be returned.

Arguments:

PIPELINE_NAME: (optional) List summary information on all active service stages associated with a pipeline. A pipeline's name is defined in bodywork.yaml.
SERVICE_NAME: (optional) List detailed information for a single service deployed by a pipeline.

Update Deployment¶

$ bodywork update deployment GIT_REPO_URL

Redeploy a pipeline to Kubernetes. Press CTRL + C to gracefully terminate deployments before they have finished.

Arguments:

GIT_REPO_URL: Location of the Git repository that contains the pipeline's codebase.

Options:

--branch: Branch of Git repository to deploy, defaults to your repository's default branch (e.g., master).
--retries: The number of times to retry the deployment, should any stage fail.

Delete Deployment¶

$ bodywork delete deployment PIPELINE_NAME

Delete all active services associated with a pipeline

Arguments:

PIPELINE_NAME: The name of the pipelines that created the services. A pipeline's name is defined in bodywork.yaml.

Manage Secrets¶

Pass secret data to the containers running pipeline stages - e.g., for authenticating with your cloud provider's API to access object storage. See Managing Credentials and Other Secrets and Injecting Secrets into Stage Containers.

Create Secrets¶

$ bodywork create secret SECRET_NAME \
    --group SECRETS_GROUP \
    --data SECRET_KEY_VALUE_PAIRS

Arguments:

SECRET_NAME: The name to give this secret - e.g., api-credentials.
SECRETS_GROUP: The group this secret belongs to - e.g., dev-environment
SECRET_KEY_VALUE_PAIRS: Secret data items as keys and values - e.g., USERNAME=api-username PASSWORD=api-password.

Secret groups will be created automatically if they do not already exist.

Get Secrets¶

$ bodywork get secret SECRET_NAME \
    --group SECRETS_GROUP \

If none of the arguments or options are specified, then bodywork get secrets will return a list of all secrets.

Arguments:

SECRET_NAME: (optional) The name of the secret. Provide this to print the secret to stdout.
SECRETS_GROUP: The secret group - e.g, dev-environment - to look in. If SECRET_NAME is not specified, then this will return a list of all secrets in the group.

Update Secrets¶

$ bodywork update secret SECRET_NAME \
    --group SECRETS_GROUP \
    --data SECRET_KEY_VALUE_PAIRS

Arguments:

SECRET_NAME: The name of the secret to update - e.g., api-credentials.
SECRETS_GROUP: The group this secret belongs to - e.g., dev-environment
SECRET_KEY_VALUE_PAIRS: Updated secret data items - e.g., USERNAME=new-api-username PASSWORD=-new-api-password REGION=api-region.

Delete Secrets¶

$ bodywork delete secret SECRET_NAME \
    --group SECRET_GROUP

Arguments:

SECRET_NAME: (optional) The name of the secret to delete.
SECRET_GROUP: The secret group - e.g, dev-environment - to look in. If SECRET_NAME is not specified, then this delete the entire group and all of the secrets within it.

Manage Cronjobs¶

Schedule pipeline runs using cronjobs.

Create Cronjob¶

$ bodywork create cronjob GIT_REPO_URL \
    --name NAME \
    --schedule CRON_SCHEDULE

Arguments:

GIT_REPO_URL: Location of the Git repository that contains the pipeline's codebase.
NAME: The cronjob's name - e.g., 'daily retraining'.
CRON_SCHEDULE: Valid cron expression - e.g. 0 * * * * will run the pipeline every hour.

Options:

--branch: Branch of Git repository to deploy, defaults to your repository's default branch (e.g., master).
--retries: The number of times to retry executing the pipeline, should any stage fail (default is 3).
--history-limit: The number of historical pipeline runs to retain logs for.
--ssh-key-path: If the Git repo is private, then use the SSH key at this location to enable access. Automatically creates a secret for this purpose.

Get Cronjobs¶

$ bodywork get cronjob NAME

Arguments:

NAME: (optional) The cronjob's name - e.g., 'daily retraining'. If omitted, then a list of all active cronjobs will be returned.

Options:

--history: If specified together with NAME, then return a list of historical cronjob runs, for which logs have been retained. Use this to get the name assigned to a specific run, to use with the --logs option (see below).
--logs: To return the logs of a historical pipeline run, specify this flag and use the specific name assigned to the historical run as the NAME argument.

Update Cronjob¶

$ bodywork update cronjob GIT_REPO_URL \
    --name NAME \
    --schedule CRON_SCHEDULE

Arguments:

GIT_REPO_URL: Location of the Git repository that contains the pipeline's codebase.
NAME: The name of the cronjob to update - e.g., 'daily retraining'.
CRON_SCHEDULE: Valid cron expression - e.g. 0 * * * * will run the pipeline every hour.

Options:

--branch: Branch of Git repository to deploy, defaults to your repository's default branch (e.g., master).
--retries: The number of times to retry executing the pipeline, should any stage fail.
--history-limit: The number of historical pipeline runs to retain logs for.

Delete Cronjob¶

$ bodywork delete cronjob NAME

Will also delete all historical logs associated with this cronjob.

Arguments:

NAME: The name of the cronjob to update - e.g., 'daily retraining'.

Advanced Debugging¶

$ bodywork debug SECONDS

Runs the Python time.sleep function for SECONDS. This is intended for use with the Bodywork image and kubectl - for deploying a container on which to open shell access for advanced debugging. For example, issuing the following command,

$ kubectl create deployment DEBUG_DEPLOYMENT_NAME \
    -n YOUR_NAMESPACE \
    --image bodyworkml/bodywork-core:latest \
    -- bodywork debug SECONDS

Will deploy the Bodywork container and run the bodywork debug SECONDS command within it. While the container is sleeping, a shell on the container in this deployment can be started. To achieve this, first of all find the pod's name, using,

$ kubectl get pods -n YOUR_NAMESPACE | grep DEBUG_DEPLOYMENT_NAME

And then open a shell to the container within this pod using,

$ kubectl exec DEBUG_DEPLOYMENT_POD_NAME -n YOUR_NAMESPACE -it -- /bin/bash

Once you're finished debugging, the deployment can be shut-down using,

$ kubectl delete deployment DEBUG_DEPLOYMENT_NAME -n YOUR_NAMESPACE