Windows Server 2012, PowerShell 3.0 and DevOps, Part 2

This concludes my two part series.  In my first post, I provided some background information about PowerShell and DevOps.  In this post, I’ll provide you a bunch of specifics.  PowerShell 3.0, like Windows Server 2012, has a ton of new features and enhancements so I’ll only scratch the surface. 

While PowerShell has always been focused on the goals of DevOps, PowerShell 3.0 and Windows Server 2012 take this to a new level.  With Windows 2012, we shifted our focus from being a great OS for a server to being a cloud OS for lots of servers and the devices that connect them whether they are physical or virtual, on-premise or off-premise.  In order to achieve this, we needed major investments in:

1. Automating everything
2. Robust and agile automation
3. Making it easier for operators to automate
4. Make it easier for developers to build tools

Automating Everything
Windows Server 2008/R2 shipped with ~230 cmdlets.  Windows Server 2012 beats that by a factor of over 10 shipping ~ 2,430 cmdlets.  You can now automate almost every aspect of the server.  There are cmdlets for networking, storage, clustering, RDS, DHCP, DNS, File Servers, Print, SMI-S etc. – the list goes on.  If you’ve read blogs about Windows Server 2012, you’ve seen how many things can be done using PowerShell.  If you haven’t kept up to date, check out Jose Barreto’s File Server blog posts, Yigal Edery’s Private Cloud blog posts, Ben Armstrong’s Virtual PC Guy’s Blog posts, the Clustering and High-Availability blog posts or Natalia Mackevicius’ Partner and Customer blog posts and you’ll see what I mean.  Windows Server 2012 is, by far, the most automatable version of Windows ever.

There are already a large number of hardware and software partners that are shipping PowerShell cmdlets and those that haven’t released them yet are  working to quickly deliver them in the next versions of their products.  This was very clear at the recent MMS conference in Las Vegas and I think you’ll see even more support at TechEd.   You should definitely make sure that any product you buy delivers a full set of PowerShell cmdlets.  If it doesn’t, you should think twice and do some due diligence to make sure you are getting a product that is current and is still being invested in.  If they didn’t do PowerShell, what other things they missing?  The good news is that a lot of the products will support PowerShell by the time Windows Server 2012 ships and that the products that have delivered cmdlets found it easy to do and mention the very positive customer feedback they get.  EVERY product that ships PowerShell cmdlets, increases their investment in PowerShell in their next release.

Robust and agile automation

Workflow
We integrated the Windows Workflow Foundation engine into PowerShell to make it simple and easy to automate things that take a long time, that operate against a very large scale, or that require the coordination of multiple steps across multiple machines.  Traditionally Windows Workflow has been a developer-only tool requiring visual studio and a lot of code to create a solution.  We’ve made it an in-the-box solution that operations can easily create a solution using their existing PowerShell scripting skill.  Workflow provides direct support for parallel execution, operation retries, and the ability to suspend and resume operations.  For example, a workflow can detect a problem that requires manual intervention, notify the operator of this condition and then suspend operations until the operator corrects the situation and resumes the workflow.

Operators can use any of the available Workflow designers to create workflows.  However we took it a step further and simplified authoring by extending the PowerShell language with the workflow keyword.  Any operator or developer can now easily author a workflow using the tools that ship in all Windows SKUs.  The behavior of a workflow are different than a function and it has a few more rules but if you know how to write a PowerShell function, you are 80% of the way to being able to write a workflow.  Authoring workflows using PowerShell is much easier than working with XAML and many of us easier to understand than Workflow designer tools.  You also get the benefit of being able to paste them into email and have someone be able to read/review it without having to install special tools.  Below is an example workflow which operates on multiple machines in parallel collecting inventory information in parallel on each of the machines.

The command below will get this inventory information from a list of servers contained in servers.txt and output the results to a file.  If any of the servers is unavailable, the workflow will attempt to contact the server every 60 seconds for an hour.

Workflow is exactly what DevOps practitioners need to reliably and repeatably perform operations.  One of the key techniques of DevOps is A/B testing where two versions of software are deployed and run for a period of time.  They are measured against some goodness metric (e.g. increased sales) and then the winning version is deployed to all machines.  The workflow capabilities allow PowerShell to perform operations against a large number of machines over a large period of time making it easy to automate A/B testing.

Scheduled jobs
We also seamlessly integrated Task Scheduler and PowerShell jobs to make it simple and easy to automate operations that either occur on a regular schedule or in response to an event occurring.  Below is a workflow which is meant to run forever.  It collects configuration information (disk info) and then suspends itself.  The workflow is started and given a well-known name “CONFIG”.  We’ll resume this workflow using Task Scheduler.  In the example, we register a ScheduledJob to run every Friday at 6pm and after every system startup.  When one of the triggers occurs, the scheduled job runs and resumes the workflow using its well-known name.  The workflow then collects the configuration information, putting it into a new file, and suspends itself again.

Robust Networking
In previous releases, PowerShell shipped with remoting disabled by default and required operators to go to each machine and issue the Enable-PSRemoting cmdlet in order to remotely manage it.  As a Cloud OS, remote management of servers via PowerShell is now the mainstream scenario, so we’ve reduced the steps required and enabled PowerShell remoting by default in all server configurations.  We did extensive security analysis and testing to ensure that this was safe.

In Wojtek Kozaczynski’s blog post on Standards-Based management, he described how we made WS-MAN our primary management protocol and kept COM and DCOM for backwards compatibility.  WS-MAN is a Web-Services protocol using HTTP and HTTPS.  While these are effectively REST protocols, PowerShell establishes a session layer on top of these to reuse a remote process for performance and to take advantage of session state.  These sessions were robust in the face of modest network interruptions but would occasionally break when operators managed servers from their laptops over Wi-Fi networks while roaming between buildings.  We’ve enhanced the session layer of WSMAN.  By default, it will survive network interruptions up to 3 minutes.   Disconnected Sessions support was added to PowerShell sessions which give users the option to disconnect from an active remote session and later reconnect to the same session, without losing state or being forced to terminate task execution. You can even connect to the session from a different computer (just like a remote desktop session).

Easier for operators to automate
We wanted to significantly lower the skill level required to successfully automate a complex solution.  Ultimately we want to create a world where operators think about what they want, type it and get it.  Every customer’s needs and scenarios are different so they need to script their own solutions.  Our goal is to make it simple and easy to author scripts gluing together high level task oriented abstractions.  The number one factor in making it simple is cmdlet coverage.  That is why having ~2,430 cmdlets makes Windows Server 2012 so much easier to automate.  A number of these cmdlets are extremely effective in dealing with the messy, real-world life of datacenters.  We have cmdlets to work with REST APIs, JSON objects and even to get, parse and post web pages from management applications if required.

PowerShell 3.0 simplifies the language and utility cmdlets to reduce the steps and syntax necessary to perform an operation.  Below is an example showing the old way of doing something and the new simplified syntax.

PowerShell3.0 improves the authoring tools operators use to create scripts and author workflows.  PowerShell-ISE now supports rich IntelliSense, snippets, 3rd party extensibility and a Show-Command window which makes it easy to find exactly the right command and parameters you need to accomplish a task.

Easier for developers to build tools
Developers have always loved scripting with PowerShell because of its power, its use of C language conventions and its ability to program against .Net objects.  PowerShell 3.0 cleans up a number of seams in dealing with .NET and objects and expands to allow developers to use PowerShell in a much wider range of scenarios.

Tool building enhancements
PowerShell 3.0 now has an Abstract Syntax Tree (AST).  This allows new classes of intelligent tools to create, analyze, and manipulate PowerShell scripts.  One of the Microsoft cloud services depends upon a very large number of PowerShell scripts to run all aspects of the service.  Their development team used the AST to develop a script analysis tool to enforce a set of scripting best practices for their operators.  The public AST is the reason why IntelliSense is freakishly powerful.  It uses the AST to reason about the actual behavior of the program.

We modified a number of key areas of PowerShell to make them easier for developers to use and extend to write their own tools.  This includes access to our serializer, API improvements, and an extensibility model for PowerShell_ISE.

Scripting enhancements
PowerShell 3.0 now uses the .NET Dynamic Language Runtime (DLR) technology.  PowerShell monitors how a script is executing and will compile the script or portions of the script on the fly to optimize performance.  Performance varies but some scripts run 6 times faster in 3.0.

Intellisense (and tab completion on the command line) now work with .NET namespaces and types.

It is able to reason about the program and use variable type-inferencing to improve the quality of the IntelliSense.

We extended our hashtable construct with two variations which make it much easier for developers to get the behavior they want:

Platform building enhancements
We have streamlined the process to support delegated administration scenarios.  PowerShell 3.0 allows you to register a remoting endpoint, configure what commands it makes available and specify what credentials those command should run as.  This allows you to let regular uses run a well-defined set of cmdlets using Admin privileges.  We’ve simplified the process of defining which cmdlets are available to using a declarative session configuration file.

PowerShell 3.0 is also available as an optional component of WINPE.

Windows Server 2012 and PowerShell 3.0 are excellent DevOps tools
DevOps is a new term and there is some disagreement about what it entails but at the heart it is all about making change safe through automation and bridging the gap between operators and developers.  There is a lot to do in this area but Windows Server 2012 and PowerShell 3.0 make excellent progress towards accomplishing those goals.  PowerShell won’t be the only tool in your DevOps toolbox but it should be in every DevOps toolbox.  Download the beta today and find out for yourself.