Category Archives: vRealize Automation (vCAC)

The right tool for the job

I work with vRealize Automation and vRealize Orchestrator on a daily basis. And I really enjoy doing so, especially the custom code development part. vRO gives a lot of flexibility and it’s not often that I’m unable to build what my customers need. Whetever the request I usually find a way to emply vRA and vRO in such a way that if fulfills the customers need. But more and more often do I wonder if we’re using the right tool for the job.

Today I presented a break-out session during the annual NLVMUG UserCon. In the presentation we emphasized the importance of using the right tool for the job. After all, you don’t drive a nail in the wall with a power drill. you can do so if you really want to but you’ll probably spent more time than needed putting up your new painting and likely destroy you power drill in the process. It’s similar in enterprise IT: You can use a customizable tool like vRA/vRO for nearly anything. But that doesn’t mean you should.

But if you can make it work anyway then why not? First of all: If you’re using a product to do something that it wasn’t originally intended to do you’ll spent a lot of time and money to make it do what you actually want.  But getting the product to do that is only the beginning. Now you need to maintain the product customizations. chances are something will break at the next product upgrade. So you postpone the upgrade, then postpone again and in the end the upgrade never happens because the risk is just too high.

Let me give an example: Let’s say you’re trying to deploy in-house developed code through different life cycle stages. You could argue that everything needs to run on a virtual machine so you start out by automating virtual machine deployment. You’ll probably use vRA or something similar to do that for you. After this first step you realize that the code does not run on an bare OS, you may need IIS or .NET or Java or a bunch of shared libraries. So you decide to automate the deployment of middleware software as well. But that still isn’t enough to run the code. You also need a database, a load balancer, an SSL certificate and last but not least: you need a way to deploy the code to your machines and configure the way it’s running.  Oh and of course all this needs to be triggered by the code repository and be completely self service. By the time you have implemented all this you’ll have written tons of custom installation scripts and integration workflows.

Automating code deployment can be tricky to say the least. And in my opinion all this difficulty stems from the fact that we’re starting with the VM as the unit of deployment. The actual unit of deployment is the code/application your developers are writing. By using the wrong data as input for the tool selection you ended up with the wrong tool.

Luckily there are tools designed for application deployment. One of them is called Cloud Foundry. If you use the Pivotal distribution you can set it up in a day or so. And then your developers can just run cf push and their code is running. In the cloud. Sounds a lot better than writing countless installation scripts and custom integrations doesn’t it? Also, the Cloud Foundry platform gives you loads of options you wouldn’t have out of the box with tools like vRA: auto-scaling, easy manual scaling, application health monitoring, service bindings, application statistics, centralized logging, custom logging endpoints and lots more.

There is one major “drawback” however: your applications need to be cloud native or 12factor apps. But you’ll have to transform your apps into cloud native apps at some point in the future anyways so why not start now?

 

Automated directory synchronization of the vRA Identity Manager

Disclaimer: The API documentation has not yet been released, therefor I would like to notice that this is currently an unsupported method of triggering a directory sync.

During a recent project the customer requested the functionality to create a new business group with just one click. This should be a function to onboard new teams into the vRA environment, including the creation of Reservations and Active Directory groups.

In vRA 6 this would not have been a problem at all, but starting at vRA 7 the Identity Manager was introduced. The Identity Manager, in short the connection from vRA to Active Directory (AD), synchronizes AD content on a specific schedule. This means that while specifying the different AD groups in the new Business Group, these will not be visible immediately but after a synchronization.

As the customer stated, it should be an automated process, a click on the button. Waiting for the synchronization to take place is not an option.. We are automating this, right?! Therefor my colleague Marco van Baggum (#vMBaggum blog) came up with the idea to automate the synchronization of the identity manager. In a shady corner Marco found the necessary API calls and off we go!

The first step is to create the a new HTTP-REST endpoint in vRO. Run the workflow “Add a REST host” located at Library / HTTP-REST / Configuration and use the following settings:

Name vRA
URL https://<vRA FQDN>/ e.g. https://itqlab-vra.itqlab.local/
Authentication NONE

* The other settings are dependent on how vRA is set-up and how vRO connects to it.

A new endpoint in the inventory should pop up at the HTTP-REST plugin. Now right click this endpoint and run the workflow to add the additional REST operations to it.

Name Get Directories
Method GET
URL template /SAAS/t/{tenant}/jersey/manager/api/connectormanagement/directoryconfigs

 

Name Get Directory Sync Executions
Method GET
URL template /SAAS/jersey/manager/api/connectormanagement/directoryconfigs/{directoryId}/syncexecutions

 

Name Invoke Directory Sync
Method POST
Content-type application/json
URL Template /SAAS/jersey/manager/api/connectormanagement/directoryconfigs/{directoryId}/syncprofile/sync

 

Name Login
Method POST
Content-type application/json
URL Template /SAAS/t/{tenant}/API/1.0/REST/auth/system/login

 

The images below show the configured operations in vRO

This slideshow requires JavaScript.

Now the endpoint and operations are created, import the workflow package attached to this post. (nl.itq.psi.vidm Workflows)

When the workflow package is imported, open the Configuration Elements tab and edit the Endpoints configuration element located under the ITQ folder. Select the correct HTTP-REST endpoint and REST-Operations, insert the correct username, password and tenant to connect to vRA. As a side-note, the used API calls can only be used with a vRA local account. Domain accounts will throw an “Invalid Credentials” error. Make sure that the user has rights to execute a Directory Sync in vRA.

Now go back to the workflow overview and expand ITQ / PSI / VIDM / Helpers. You should have the same overview as in the image below.

vRO Workflow structure

Now execute the “Synchronize active directory” workflow and the synchronization will start!

vRO Workflow execution
vRO Workflow execution

Please note that these workflows are not production ready yet and bugs may exist!

Download nl.itq.psi.vidm Workflows!

vRO Code – Finding VirtualMachines by Custom property

For the current project I’m involved in, I was asked to deliver a list of vRA deployed machines that have a Production status.

At first I have been writing a short piece of code that obtained all vRA managed machines and for each machine gathered the customer properties. Creating this workflow actually took less time than the execution itself as the environment has about 4200 managed objects. Next to the fact that this is time consuming to wait for, it will also generate a lot of load on the vRO service and the vRA IaaS API.

The developer in me felt like improving this and move the functionality to the vRA IaaS API, the API nevertheless has the custom properties linked to the virtual machine entity object. Eventually, after some research on ODATA queries and how to query for properties within linked entities, I was able to write the following ODATA filter:

Putting the filter and the vCAC IaaS plugin logic together will form the following script that can be used in either a workflow or an action:

To elaborate  a little bit on the code snippet above:

  • First the property and it’s value are being specified
  • The second step is to setup the filter with the property and value
  • The third step is to actually perform the call to vRA IaaS to return an array of vCAC:Entity based on the filter.
  • The last step in the code is to System.log() the names of the VirtualMachines that match the query.

When necessary to have vCAC:VirtualMachine objects instead of vCAC:entity objects change the last part of the code to:

 

Conclusion

Gathering virtualmachines based on specific properties can be a hassle using ODATA queries as in some cases it is not completely clear on how to structure the query. Eventually when the query is ready and working it shows to be much faster than creating a script to “hammer” the API for data.  The two screenshots below show the actually difference between the initial code and the improved code. The first screenshot is the original code, it errors out after 30 minutes of API calls. The second screenshot is a capture of the improved code, it runs for only 1 second to return the list of VirtualMachines matching the filter.

log get virtual machines by property and value error
First attempt ended up in an error returned by the vRA IaaS API after 30 minutes of performing API calls.

 

log get virtual machines by property and value
Second attempt with improved code. The runtime of the script is now only a matter of seconds.

vRO Code – Finding vRA IaaS entities using OData query

odata logo

As I’ve explained in an earlier blog the vRA API is split into two parts: CAFE and Iaas. This post is about the latter which still contains a lot of the vRA entities. When working with those entities you regularly need to find them first. One of the methods for finding vRA Iaas entities is using an odata query.

Continue reading

vRealize Automation: Missing Actions in Entitlements

After installing a new vRealize Automation 6.2.2 distributed environment a customer tried to add actions to an entitlement but default actions like Reconfigure were not visible. Also this Actions were not visible when we looked at \Administration\Catalog Management\Actions. The source of these actions are iaas-service so because it’s a new installation we checked if all the services were registered on the appliance via port 5480.

AutomateIt_Appliance

So the registration of the IaaS-Service was successful in this case, so it must be a registration of component, after investigating the install log we didn’t find any errors that point in a particular direction.

But, there is a easy fix:

1, Go to the IaaS Server Model Manager Data machine.

2, Browse to the café folder in Model Manager Data from the command line:
C:\Program Files (x86)\VMware\vCAC\Server\Model Manager Data\Cafe\

3, Run the command “Vcac-Config.exe registercatalogtypes –v”

When you now look at \Administration\Catalog Management\Actions you will see the actions appear, you don’t even have to restart servers or services 😉

AutomateIt_Actions

 

Advanced Options vRealize Automation – Hide Properties & Snapshots tab

When you look at the item details for a Virtual Machine in the vRealize Automation Portal there are several tabs which show information. Most of this information is usable for the customer, like the CPU, memory and storage configuration.

Automate-it-properties1

Besides this tabs the tabs Properties and Snapshots are visible, during a deployment of a Virtual Machine you often use advanced properties which are of no use to the customer. This is also the case for the Snapshots tab, you might have disabled snapshotting on your blueprint so why show the Snapshots tab?

We didn’t expect this being a problem, but in several customer environments we got a lot of questions of customers about the information on the properties tab and why they cannot make a snapshot on the Virtual Machine. So we looked if it was possible to make the tabs invisible.

And it is possible by editing a configuration file on the IaaS web server called EditVirtualMachine.aspx, in our case on the default location:

c:\Program Files (x86)\VMware\vCAC\Server\Website\Leases\

You can edit this file with an editor like Notepad++.

Properties tab:
On line 265 in the EditVirtualMachine file you need the add ClientVisible=”False” so the line looks like this:

This can also be done for the Snapshots tab by editing line 278 in the EditVirtualMachine file and add ClientVisible=”False”

When you now refresh the Item Details the tabs are gone, so you don’t have to restart any services.

Automate-it-properties2

The downside, you as an admin are also no longer able to see the properties for this VM in the portal. I added a simple workflow so the properties are viewable with the use of Orchestrator.

Get Properties of vRA_VirtualMachine.workflow

HOWTO: Speed up vRA template deployments

[ Since flowgrab is no longer amongst us you can get it from dropbox now ]

I recently did some scalability and load testing on a vRA deployment. One of the problems I encountered was that the template deployments,which seemed reasonably fast suddenly became a bottleneck. So obviously I created a workflow to fix that. So here is howto: speed up vRA template deployments.

TL;DR: if you’re too lazy to read an just want the workflow click here.

Let me explain the scenario first: Usually when you deploy a virtualmachine from a vSphere template it is reasonably quick. After all it just copies a file from disk to disk. If you happen to have an all flash array (my customer does) it is probably pretty fast.  Especially when your array is VAAI compatible. Because that would offload the whole copy action to the array.A typical VAAI accelerated template deployment takes around 4 seconds on an EMC X-IO.

But what happens if your template is in one cluster and needs to be deployed to another cluster which has a separate storage array. Then VAAI can’t  do anything for you and the template will be copied over the network. Which could be rather slow compared to a VAAI accelerated deployment. More in the region of 2 minutes instead of 4 seconds.

So before we get to the solution let me answer a few questions:

1: Why do you have the template on another cluster? Well… this customer has more than 10 different clusters with their own storage. Why that is is a completely different sotry. Anyhow, I wasn’t going to copy the template to each of them manually. Keeping them up to date would be a nightmare.  Second reason is that that strategy would require another blueprint for each cluster. Which is another nightmare to maintain.

2: Why don’t you use a template LUN shared between all clusters? This was actually not physically possible and also otherwise unwanted. It also wouldn’t fix the VAAI issue since it would require copying from one storage bo to another. It would be faster then copying over the network, that’s for sure.

3: 2 minutes isn’t that long. Why bother? Actually, consider the fact that vRA by default only does 2 deployments in parallel. That means that if you kick-off 50 deployments it takes at least 48 minutes before it even start deploying the last request. That is an unacceptable  long delay and even causes time-outs in vRA. I tried bumping up the number of parallel deployments but that slowed the deployments so much they never finished within vRA time-outs.

So…. I didn’t want a separate blueprint for each target cluster and I didn’t want to copy the template manually. The solution that remains is having the template copied by a workflow and then overwrite the template that the blueprint is going to use. Can we do that? You we can!  :). Turns out there is a custom propery called __clonefrom which contains the template name. If you overwrite this property during the buildingMachine state it will just use that machine to clone from.

to automate this process I created a workflow that:

  1. gets the template name from the __clonefrom property
  2. gets the cluster name of the cluster where vRA is going to deploy the machine
  3. add the cluster name to the tempate name and checks if such template exists
  4. If it doesn’t exist it will clone the template that is configured in the blueprint to the target cluster and adds the cluster name to the template name so we can find it next time we deploy something to the same cluster.
  5. overwrite the __cloneid property and then let vRA do it’s jobs

selectTemplate workflow

That’s it. This will make sure you have VAAI enabled deployments on each cluster. In my case it decreased the template deployment time from around 2 minutes to 4 seconds. This is so fast tha vsphere deployment is done before vRA can kick-off the next one.

you can download the workflow from dropbox. Use at you own risk! It’s tested wit vRA 6.2.1 and vSphere 5.5. Should work on any vRA 6.x version or even 5.x but I didn’t test that. Not sure about vRA 7. I’ll let you know when I had a change to test that.

vRA 7 Blueprint designer preview

One of the big changes you’ll see in vRA 7 is the new blueprint designer. If you can’t wait until the GA release, here is a vRA 7 bleuprint designer preview.

In summary these are the differences from the old pre-7 blueprints:

  • Graphical canvas
  • Unified blueprint. No difference between single machine, multi-machine or application blueprints
  • NSX integration. You can now drag and drop NSX object into your blueprint canvas
  • Ability to use blueprints in blueprints Yes, nested blueprints!
  • Ability to set dependencies between elements in the blueprint
  • Ability to use vRO workflow in a blueprint. This is now called XaaS, formerly ASD
  • Last but not least: The designer is on a separate tab in the vRA GUI not hidden in the infrastructure tab.

So basically all the good elements of application director (or appD or application services) are now integrated into the vRA blueprint designer. On top of that we can now use workflows directly in a blueprint. Which is exciting to me really.

This is what the designer looks like when you just created a new blueprint:

vra7-canvas-empty

And this is what it looks like when you would create a typical two tier application:

vra7-canvas-populated

Please note that these screenshots are taken from a BETA version so the GA release might look a little bit different.

As you can tell from the screenshots all features are now nicely integrated. You no longer need to hack NS integration into the product or use vRO to actually integrate app services and vRA.

I don’t want to complain because this is a huge step forward but…. The thing I’m still missing though is a higher abstraction layer on top of the blueprints. The bueprint author still has to decide one which reservation policy the VM will land for example. If you have different environments you’ll end up with blueprint sprawl. A better approach would be a form that asks functional questions like: is it for dev or prod? and then populates the different properties in the blueprint. I used to build that kind of functionality in workflows and then publish those in the catalog instead of the actual blueprints. Hopefully VMware will build something into the product to facilitate this.

VMworld news: vRealize Automation 7

I’m currently at VMworld Barcelona where this morning VMware announced the new version of vRA. It’s going to be called vRA7. As far as I know it’s still not GA (general availability)  but the Beta program is in full swing.

This morning I attended a a vExpert deepdive session hosted by @virtualJad. Here is an overview of some of the new features:

  • Simplified architecture You no longer need 24 machines for an enterprise deployment.
  • Simplified installation: From download to up and running in 20 minutes. All wizard driven so anybody can run a vRA PoC without assistance from PSO or other consultants.
  • One converged blueprint. No difference between IaaS and ASD (which is now called XaaS by the way) and application blueprint
  • Blueprint designer is now a beautiful canvas which allows for visio style drag and drop design.
  • Nested blueprints: You can use other blueprints in your blueprint (nice!)
  • Eventbroker: instead of having a few workflow stub we can now create policies that define when to kickoff a workflow. There are 60 different lifecycle events to which you can attach workflows. And each event has multiple stages (pre, event, after).

This event broker makes the product so much more extensible than what it currently is. The possibilities are almost endless. The other nice thing about this is that it is policy driven and defined by the vRA admin. So extensibility is now no longer part of the workflows. This means you can give the workflow designer to an application architect while still making sure that important IPAM or CMDB workflow is kicked off with each deployment. The application architect can consume XaaS workflows to extend his own blueprint.

In summary: really cool stuff, you’ll be reading lots more about it here in the coming months. I know, I know, I haven’t blogged in a while but I promise you’ll see some good vRA 7 stuff here on this blog!