Octopus Deploy Runbook cloning with the REST API

Published in

Version 1

9 min readAug 18, 2023

Ever wondered how you can copy an Octopus Deploy Runbook from one Project to another? It can’t be done, well at least by using the User Interface. What you can do is use code, so by using the power of the Octopus REST API and some PowerShell we can make it possible. In this post, we will cover the end goal of cloning an Octopus Deploy Runbook Process, plus along the way look at other endpoints that could be useful for other tasks.

Normally, I would go down the route of using the native CLI for the product, which Octopus does have two versions, Octo and Octopus. These are useful tools to use, however, the inconsistency between them is not helpful. The Octo version is their latest version and what they want people to use but does not have all the features of the Octopus version. Further to that there are some features just not in either, which is why I resorted to using the REST API instead as this is more consistent, contains all interactions with Octopus and is well documented.

Get the Basics

Let us start off by getting all the details we are going to need to run our API requests.

The simplest one to start with is the Octopus Deploy Servers URL. If you are using a load balancer, then it will be this URL you need. Using a load balancer is always the advised setup, but if you are not using one you can get your Octopus Deploy node URL. We will also add the port number for the API endpoints to the end, which is 8082.

An example would be something like https://octopus.mydomain.com:8082

Then we need an API Key to authenticate, which will be used in the request header for authentication. You can generate an API Key by following the instructions on the Octopus Deploy guide How to Create an API Key. Remember this key is generated by a user so that user will need to have the correct permissions and access to carry out any of the tasks we will be executing. It is possible to set the key to not have an expiry, which means you do not need to rotate it. However, I would advise you to rotate this every 3 months to keep it fresh and prevent any security concerns.

This should result in some PowerShell like

# Define working variables 
$octopusURL = "https://octopus.mydomain.com:8082" 
$octopusAPIKey = "API-ABCDEF0GHIJK1LMNOP2QRSTUV3" 
$header = @{ "X-Octopus-ApiKey" = $octopusAPIKey }

The helper endpoints

These first few endpoints are resource endpoints that are used to get the information required for the cloning of the Runbook process and can be used in many other situations.

Get Space

In Octopus you can define different Spaces, where projects and other resources can be organized into.

Like all the other elements of Octopus they are referenced in the API requests as IDs, which can be retrieved from URLs in the browser when working on the UI, but to make life easier we can get it from an API endpoint using the user-friendly name. You can set within the Octopus- Deploy UI what the default Space is, which if you only have one Space, itwill be the Default Space. This means for the name you can use the value of `Default`.

You will also notice that there is no endpoint to get a particular resource by its name, so we are going to use the `all` endpoints, which will retrieve every resource for that type and then we will apply a filter.

Below we use Invoke-RestMethod on the /api/spaces/all endpoint and then filter by the declared Space name to get our particular Space object.

# Get Space 
$spaceName = "Default" 
Write-Host "Get Space $spaceName" 
$space = (Invoke-RestMethod -Method Get -Uri "$octopusURL/api/spaces/all" -Headers $header) | Where-Object { $_.Name -eq $spaceName } 
Write-Host "Space ID retrieved $($space.Id)"

Get project

Next is to get a Project, which will be used later when getting the target and source Project for the Runbooks. Again, we want to use the user-friendly name to get the resource rather than the ID, we will use this endpoint to get all projects to filter on.

You can see in this endpoint URL we require the Space ID that we would have gotten from the Space request previously. The endpoint is /api/<Space ID>/projects/all.

I have put this into a function to make it more reusable.

<#
.DESCRIPTION 
Get the Octopus Project details 

.PARAMETER name 
Octopus Project user readable name 
 
.EXAMPLE 
Get-Project -name "My Project Name" 
#> 
function Get-Project { 
  param ( 
    [string]$name 
  ) 

  Write-host "Get project by Name $name" 
  $project = (Invoke-RestMethod -Method Get -Uri "$octopusURL/api/$($space.Id)/projects/all" -Headers $header) | Where-Object { $_.Name -eq $name } 
  Write-Host "Project ID retrieved $($project.Id)" 

  return $project 
}

Get Runbook

The last helper function gets the Octopus Deploy Runbook, and also the Runbook’s Process. The Runbook will contain the descriptive details of the Runbook and its settings. The Runbook Process will contain all the Steps that will be executed when run. To clone the Runbook we need both.

For the Runbook, we are doing the same by getting all of the Runbooks from Octopus using the user-friendly name and the Space ID, but the endpoint does not contain the Project ID to filter the response, for example /api/<Space ID>/runbooks/all. As we are getting all the Runbooks from this request there could be many that have the same name (as the names do not need to be unique), so in the PowerShell we must filter by not just the Runbook name but also the Project ID. The Projects name is not stored on the Runbook object, which is the reason why we need to get the Projects details before.

Once we have got the Runbook details we can get the Runbook Process, which is easier as this endpoint relies on the Runbook Process ID. This ID is always in the format of runbookProcess-<Runbook ID>. Therefore, we do not need to do any filtering on the endpoint as it will directly get the resource, we are looking for at /api/<Space ID>/runbookProcesses/<Runbook Process ID>.

I have turned this into a single function as they will always be requested together to get the full Runbook details.

<# 
.DESCRIPTION 
Get the Octopus Runbook details and its process 

.PARAMETER projectId 
Octopus Project ID

.PARAMETER name 
Octopus Runbook user readable name

.EXAMPLE 
Get-Runbook -projectId "Projects-123" -name "My Runbook Name" 
#> 
function Get-Runbook { 
  param ( 
    [string]$projectId, 
    [string]$name 
  )

  Write-host "Get runbook by Name $name" 
  $runbook = (Invoke-RestMethod -Method Get -Uri "$octopusURL/api/$($space.Id)/runbooks/all" -Headers $header) | Where-Object { $_.Name -eq $name -and $_.ProjectId -eq $projectId } 
  Write-Host "Runbook ID retrieved $($runbook.Id)"

  Write-host "Get runbook process" 
  $runbookProcess = Invoke-RestMethod -Method Get -Uri "$OctopusUrl/api/$($space.Id)/runbookProcesses/RunbookProcess-$($runbook.Id)" -Headers $header 
  Write-Host "Runbook Process ID retrieved $($runbookProcess.Id)"

  return @{ 
    "runbook" = $runbook 
    "process" = $runbookProcess 
  } 
}

Clone the Runbook

Now we can start with the steps to clone the Runbook from one Project to another.

As below, we set the parameters with the user-friendly source Runbook and Project names, which we do the same for the target Project and this is where you can put the new Runbooks name that does not need to be the same but can. Then using the functions created before we can get the target and source information.

$sourceRunbookName = "Source Runbook Name" 
$sourceProjectName = "Source Project Name" 

$targetProjectName = "Target Runbook Name" 
$targetRunbookName = "Target Project Name" 

Write-Host "Get Source Information" 
$sourceProject = Get-Project -name $sourceProjectName  
$sourceRunbook = Get-Runbook -projectId $sourceProject.id -name $sourceRunbookName 

Write-Host "Get Target Information" 
$targetProject = Get-Project -projectName $targetProjectName

For the first part of the cloning process, we need to create the new Runbook in the target Project, for this, we can use as much or as little of the source Runbook settings. You may want to replicate things like the Retention Policy or Environment Scope, but you also do not need to if the target Project can have a different setting.

The create the new Runbook the endpoint we use is a POST endpoint unlike the previous, so we have a JSON body to send, which we create in the format of an Object. Below is the minimal information you need for the request, but not limited to.

Once filled out we can send the request which will return the newly created Runbook object.

Write-Host "Create Target Runbook $targetRunbookName in $($targetProject.id)" 
$jsonPayload = @{ 
  Name               = $targetRunbookName 
  ProjectId          = $targetProject.id 
  EnvironmentScope   = "All" 
  RunRetentionPolicy = @{ 
    QuantityToKeep    = 100 
    ShouldKeepForever = $false 
  }    
} 

Write-Host "Create Runbook $($jsonPayload.Name) in $($targetProject.id)" 
Invoke-RestMethod -Method Post -Uri "$octopusURL/api/$($space.Id)/runbooks" -Body ($jsonPayload | ConvertTo-Json -Depth 10) -Headers $header

You will notice that I do not collect the object from the request to use later and the reason for that is we still need to get the Runbook Process. Although this has also been created with the request above, it is not returned in the API response so to make things easier, we can use the previous function we declared earlier.

Write-host "Get target Runbook and Process" 
$targetRunbook = Get-Runbook -projectId $targetProject.id -name $($jsonPayload.Name)

Now that we have both the source Project, Runbook, Runbook Process and the target Project, Runbook, and Runbook Process, it is very simple to do the clone. The endpoint to update a Runbook Process (PUT) accepts the same Runbook Process object that is returned from the GET request. Therefore, as we have the Runbook Processes as PowerShell objects, we can make the target steps equal to the source steps. Finally, we can request the endpoint to update the target Runbook Process with the updated object.

Write-host "Update target Runbook Process" 
$targetRunbook.process.steps = $sourceRunbook.process.steps 
$jsonPayload = $targetRunbook.process | ConvertTo-Json -Depth 10 

Invoke-RestMethod -Method Put -Uri "$octopusURL/api/$($space.Id)/runbookProcesses/RunbookProcess-$targetRunbookId" -Headers $header -Body $jsonPayload

Bring it all together

Below is the full script we just went through step by step

$ErrorActionPreference = "Stop";

# Define working variables
$octopusURL = "https://octopus.mydomain.com:8082"
$octopusAPIKey = "API-ABCDEF0GHIJK1LMNOP2QRSTUV3"
$header = @{ "X-Octopus-ApiKey" = $octopusAPIKey }

$sourceRunbookName = "Source Runbook Name"
$sourceProjectName = "Source Project Name"

$targetProjectName = "Target Runbook Name"
$targetRunbookName = "Target Project Name"

# Get Space
$spaceName = "Default"
Write-Host "Get Space $spaceName"
$space = (Invoke-RestMethod -Method Get -Uri "$octopusURL/api/spaces/all" -Headers $header) | Where-Object { $_.Name -eq $spaceName }
Write-Host "Space ID retrieved $($space.Id)"

<#
.DESCRIPTION
Get the Octopus Project details

.PARAMETER name
Octopus Project user readable name

.EXAMPLE
Get-Project -name "My Project Name"
#>
function Get-Project {
  param (
    [string]$name
  )

  Write-host "Get project by Name $name"
  $project = (Invoke-RestMethod -Method Get -Uri "$octopusURL/api/$($space.Id)/projects/all" -Headers $header) | Where-Object { $_.Name -eq $name }
  Write-Host "Project ID retrieved $($project.Id)"

  return $project
}

<#
.DESCRIPTION
Get the Octopus Runbook details and its process

.PARAMETER projectId
Octopus Project ID

.PARAMETER name
Octopus Runbook user readable name

.EXAMPLE
Get-Runbook -projectId "Projects-123" -name "My Runbook Name"
#>
function Get-Runbook {
  param (
    [string]$projectId,
    [string]$name
  )

  Write-host "Get runbook by Name $name"
  $runbook = (Invoke-RestMethod -Method Get -Uri "$octopusURL/api/$($space.Id)/runbooks/all" -Headers $header) | Where-Object { $_.Name -eq $name -and $_.ProjectId -eq $projectId }
  Write-Host "Runbook ID retrieved $($runbook.Id)"

  Write-host "Get runbook process"
  $runbookProcess = Invoke-RestMethod -Method Get -Uri "$OctopusUrl/api/$($space.Id)/runbookProcesses/RunbookProcess-$($runbook.Id)" -Headers $header
  Write-Host "Runbook Process ID retrieved $($runbookProcess.Id)"

  return @{
    "runbook" = $runbook
    "process" = $runbookProcess
  }
}

Write-Host "Get Source Information"
$sourceProject = Get-Project -name $sourceProjectName 
$sourceRunbook = Get-Runbook -projectId $sourceProject.id -name $sourceRunbookName

Write-Host "Get Target Information"
$targetProject = Get-Project -projectName $targetProjectName

Write-Host "Create Target Runbook $targetRunbookName in $($targetProject.id)"
$jsonPayload = @{
  Name               = $targetRunbookName
  ProjectId          = $targetProject.id
  EnvironmentScope   = "All"
  RunRetentionPolicy = @{
    QuantityToKeep    = 100
    ShouldKeepForever = $false
  }   
}

Write-Host "Create Runbook $($jsonPayload.Name) in $($targetProject.id)"
Invoke-RestMethod -Method Post -Uri "$octopusURL/api/$($space.Id)/runbooks" -Body ($jsonPayload | ConvertTo-Json -Depth 10) -Headers $header

Write-host "Get target Runbook and Process"
$targetRunbook = Get-Runbook -projectId $targetProject.id -name $($jsonPayload.Name)

Write-host "Update target Runbook Process"
$targetRunbook.process.steps = $sourceRunbook.process.steps
$jsonPayload = $targetRunbook.process | ConvertTo-Json -Depth 10

Invoke-RestMethod -Method Put -Uri "$octopusURL/api/$($space.Id)/runbookProcesses/RunbookProcess-$targetRunbookId" -Headers $header -Body $jsonPayload

About the Author:
Christopher Pateman is a Senior Azure DevOps Engineer here at Version 1