Using vCloud Director API for ISO/OVA Upload
Introduction
The vCloud API offers a number of operations to explore and manipulate your vCloud Director environment. One area that may be of interest to users is its media upload features. The vCloud API allows you to upload OVF packages and ISO-format media images, allowing you to transfer files from your client system to a target catalog in your vCloud Director organization. The vCloud API is available as a REST API that can interact with HTTP requests. A useful resource to get started with the vCloud API is the vCloud API Programming Guide.
In this two-part article, I will illustrate how to explore and use the vCloud API with Python and the Python-Requests package. In the first part of this article, I will demonstrate how to set up your environment and obtain the contents of your organization’s catalogs. In part two of this article, I will illustrate the process of uploading both OVA packages and ISO-format media. The repository for the Python source for this project can be found on Github.
vCloud API Primer
Before going into development detail, let’s start with some background information on the vCloud API, which offers RESTful application development when creating tools which interact with vCloud Director. The following expand on some of the terms used in this article.
- Objects – The vCloud API defines objects – including organizations, catalogs, virtual data centers, and media – and represents them in XML with additional embedded attributes which can be retrieved from API responses.
- Links – Link elements provide references to objects and supported actions, providing a mechanism to access and operate on objects such as catalogs and VMs.
- REST Requests/Responses – Our application client will make HTTP requests using a vCloud API endpoint, then examine the returned response for status and retrieve information from the response body.
Project Environment
This article details developing a Python application on a GNU/Linux platform. The latest versions of Ubuntu, CentOS, Fedora and Redhat Enterprise (RHEL) come with Python 2.7 already installed. Next, virtualenv is a tool to create isolated Python environments, essentially keeping dependencies required by different projects in separate places. In our case, we want to create a new Python environment in which to run our client application. virtutalenv will create a folder in our project directory, containing all the necessary executables and packages that our Python project will need without affecting our system’s existing site-packages.
To begin, let’s install virtualenv via pip:
$ pip install virtualenv
Next, change directory into your project folder, and create a virtual environment for the project:
$ cd api_client $ virtualenv venv
This creates a new folder in the current directory, named venv, containing the Python executable files and a copy of the pip library which you can use to install additional packages. The name of the virtual environment, in this case, is venv but can be anything you choose.
To begin using the virtual environment, we need to activate it with:
$ source venv/bin/activate
The name of the current virtual environment appears on the left of the prompt to indicate that it’s active. Any package that you install using pip will be placed in the venv folder, isolated from your system Python installation.
The first package we’ll need is Python Requests, which facilitates HTTP requests to the vCloud Director API. The second package we need is lxml, which will be used to parse XML data from the API responses.
$ pip install requests $ pip install lxml
In order to keep your environment consistent, it’s a good idea to “freeze” the current state of the environment packages with:
$ pip freeze > requirements.txt
This creates a requirements.txt file, which contains a list of all the packages in the current environment and their respective versions. This is helpful in case you need to re-create the environment, allowing you to install the same packages using the same versions with:
$ pip install -r requirements.txt
This helps to ensure consistency across different deployments and across developers.
Let’s Get Started
In this article, we will perform the following tasks:
- Establish a connection to the vCloud API server using Python and the Python-Requests package
- Make a GET request on the /orgs endpoint of the vCloud API
- This will return a link corresponding to the organization object
- Make a GET request on the organization link to obtain catalog information
In order to use the vCloud API, we need the vCloud server hostname (or IP) and user credentials in order to authenticate and obtain a token which enables us to make additional REST calls on the API.
- vCloud hostname or IP
- This is the vCloud server to which REST calls will be made
- If you use the fully-qualified name of your vCloud server, your GNU/Linux localhost needs resolve the IP of the server
- Using the IP of the vCloud sever works as well
- User Credentials
- A user name and password in your vCloud organization with sufficient privileges to log in, view organization info, and access its catalog
- I used the user role: Organization Administrator
- User, credentials and access roles can be set up in vCloud Director within your Organization: Administration > Members > Users
- Given the vCloud organization user credentials, we can establish a Python Requests session with the following:
import requests session = requests.Session() session.auth = HTTPBasicAuth(username, password) session.headers = {'accept': 'application/*+xml;version=5.6'} session.verify = False
with the HTTP Accept header set to API version 5.6. Next, our client application must complete a login request to the vCloud API in order to receive an authorization token. This token needs to be included in all subsequent API requests. The login name format used is username@vcd-organization. The login API endpoint is https://vcd-hostname/api/sessions, where vcd-hostname is the fully-qualified name of your vCloud server or IP, and requires a POST call:
auth = session.post('{}/sessions'.format(vcd-hostname) if not 'x-vcloud-authorization' in auth.headers: raise Exception('Not authorized to access vCloud Director') session.headers.update({'X-Vcloud-Authorization': auth.headers['x-vcloud-authorization']})
Upon a successful response, the returned token can be stored in the session header for subsequent calls.
Now that we have a valid API token, we can make additional calls to the vCloud API. First, let’s make a GET call to the endpoint https://<vcd-hostname>/api/org:
import xml.etree.ElementTree as ET data = [] response = session.get('https://{}/org'.format(vcd-hostname)) if response.status_code == 200: root = ET.fromstring(response.text) for child in root: if child.attrib: data.append(child.attrib) return data
This returns a list of dictionary items with keys: href, type and name. The response body from the API GET call contains XML of the form:
<?xml version="1.0" encoding="UTF-8"?> <OrgList xmlns="https://www.vmware.com/vcloud/v1.5" href="" type="application/vnd.vmware.vcloud.orgList+xml" xmlns:xsi="https://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="https://www.vmware.com/vcloud/v1.5 https://10.10.1.18/api/v1.5/schema/master.xsd"> <Org href="https://vcd.martin.syr.lab/api/org/3fe52e47-8439-4f68-9028-22713be601a7" name="Marquis" type="application/vnd.vmware.vcloud.org+xml"/> </OrgList>
We are interested in the Org href, which references the organization corresponding to the user credentials that were used earlier. Using this href link in a subsequent GET call provides additional information that we are interested in, such as the organization’s catalog.
If we make a GET https://vcd-hostname/api/org/<uuid> API call with this Org href, we will find links to the Org catalogs:
data = [] response = session.get(org['href']) if response.status_code == 200: root = ET.fromstring(response.text) for child in root: if child.attrib and child.attrib['type'] == 'application/vnd.vmware.vcloud.catalog+xml': data.append(child.attrib) return data
The response body from the GET call contains XML of the form:
https://www.vmware.com/vcloud/v1.5" name="Marquis" id="urn:vcloud:org:3fe52e47-8439-4f68-9028-22713be601a7" href="https://vcd.martin.syr.lab/api/org/3fe52e47-8439-4f68-9028-22713be601a7" type="application/vnd.vmware.vcloud.org+xml" xmlns:xsi="https://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="https://www.vmware.com/vcloud/v1.5 https://10.10.1.18/api/v1.5/schema/master.xsd"> https://vcd.martin.syr.lab/api/vdc/8d4c17ef-f90f-4544-9e3f-a7105c01fa46" name="Marquis VDC" type="application/vnd.vmware.vcloud.vdc+xml"/> https://vcd.martin.syr.lab/api/tasksList/3fe52e47-8439-4f68-9028-22713be601a7" type="application/vnd.vmware.vcloud.tasksList+xml"/> https://vcd.martin.syr.lab/api/catalog/4b29817a-11ed-442b-9c5f-1ca722f131af" name="Vagrant" type="application/vnd.vmware.vcloud.catalog+xml"/> https://vcd.martin.syr.lab/api/catalog/4b29817a-11ed-442b-9c5f-1ca722f131af/controlAccess/" type="application/vnd.vmware.vcloud.controlAccess+xml"/> https://vcd.martin.syr.lab/api/catalog/4b29817a-11ed-442b-9c5f-1ca722f131af/action/controlAccess" type="application/vnd.vmware.vcloud.controlAccess+xml"/> https://vcd.martin.syr.lab/api/catalog/5c6fa902-bdcb-45dc-b1d8-04fd5424eb54" name="pub" type="application/vnd.vmware.vcloud.catalog+xml"/> https://vcd.martin.syr.lab/api/catalog/5c6fa902-bdcb-45dc-b1d8-04fd5424eb54/controlAccess/" type="application/vnd.vmware.vcloud.controlAccess+xml"/> https://vcd.martin.syr.lab/api/catalog/5c6fa902-bdcb-45dc-b1d8-04fd5424eb54/action/controlAccess" type="application/vnd.vmware.vcloud.controlAccess+xml"/> https://vcd.martin.syr.lab/api/admin/org/3fe52e47-8439-4f68-9028-22713be601a7/catalogs" type="application/vnd.vmware.admin.catalog+xml"/> https://vcd.martin.syr.lab/api/network/0a02fbef-e79d-4073-a6fa-16aa48d940c3" name="MarquisNet" type="application/vnd.vmware.vcloud.orgNetwork+xml"/> https://vcd.martin.syr.lab/api/network/312e30d8-1c59-48e4-b3f4-7d8f44f60208" name="NetflowNet" type="application/vnd.vmware.vcloud.orgNetwork+xml"/> https://vcd.martin.syr.lab/api/org/3fe52e47-8439-4f68-9028-22713be601a7/metadata" type="application/vnd.vmware.vcloud.metadata+xml"/> Marquis
From the response, we are interested in links with type application/vnd.vmware.vcloud.catalog+xml. In this case, there are two catalogs for this organization, named pub and Vagrant.
As the last step, let’s perform GET https://vcd-hostname/api/catalog/<uuid> API calls using the Link hrefs corresponding to the above catalog types to obtain the list of files residing in each of the organization’s catalogs. Looping over the list of catalogs returned from the previous code, we have
data = [] response = session.get(catalog['href']) if response.status_code == 200: root = ET.fromstring(response.text) for element in root: for all_tags in element.findall('.//'): if all_tags.attrib['type'] == 'application/vnd.vmware.vcloud.catalogItem+xml': data.append(all_tags.attrib['name']) return data
to yield the href links and names from each catalog. The response body from the GET API call contains XML of the form:
https://www.vmware.com/vcloud/v1.5" name="Vagrant" id="urn:vcloud:catalog:4b29817a-11ed-442b-9c5f-1ca722f131af" href="https://vcd.martin.syr.lab/api/catalog/4b29817a-11ed-442b-9c5f-1ca722f131af" type="application/vnd.vmware.vcloud.catalog+xml" xmlns:xsi="https://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="https://www.vmware.com/vcloud/v1.5 https://10.10.1.18/api/v1.5/schema/master.xsd"> https://vcd.martin.syr.lab/api/org/3fe52e47-8439-4f68-9028-22713be601a7" type="application/vnd.vmware.vcloud.org+xml"> https://vcd.martin.syr.lab/api/catalog/4b29817a-11ed-442b-9c5f-1ca722f131af/metadata" type="application/vnd.vmware.vcloud.metadata+xml"> https://vcd.martin.syr.lab/api/catalog/4b29817a-11ed-442b-9c5f-1ca722f131af/catalogItems" type="application/vnd.vmware.vcloud.catalogItem+xml"> https://vcd.martin.syr.lab/api/catalog/4b29817a-11ed-442b-9c5f-1ca722f131af/controlAccess/" type="application/vnd.vmware.vcloud.controlAccess+xml"> https://vcd.martin.syr.lab/api/catalog/4b29817a-11ed-442b-9c5f-1ca722f131af/action/controlAccess" type="application/vnd.vmware.vcloud.controlAccess+xml"> https://vcd.martin.syr.lab/api/catalogItem/f084ff8e-4132-4e2f-9b56-8fa63a743272" id="f084ff8e-4132-4e2f-9b56-8fa63a743272" name="precise-5.7.1.iso" type="application/vnd.vmware.vcloud.catalogItem+xml"> https://vcd.martin.syr.lab/api/catalogItem/8d233328-7fd5-4dc5-bfa5-179123b39178" id="8d233328-7fd5-4dc5-bfa5-179123b39178" name="API vApp Template" type="application/vnd.vmware.vcloud.catalogItem+xml">
which, in the case shown, corresponds to the Vagrant catalog. The catalog items reside under the <CatalogItem> XML tag, have type application/vnd.vmware.vcloud.catalogItem+xml, and include the name of the catalog item. This same name will appear in the vCloud Director web client.
Part 2 of this article goes through the process of uploading OVF packages and ISO-format media images into our organization catalog.