TechEd: Comparing Microsoft and VMware Private Clouds (MDC-B352)

This was Part 2 of a two part series on comparing VMware and Microsoft virtualization/Cloud offerings. Part 1 was focused on the hypervisor and how Hyper-V and ESXi compare. I had a schedule conflict with part 1, so I didn’t attend it. This is part 2, focusing on the private cloud offerings. I thought Microsoft did a decent job in the 75 minutes provided. VMware has a leg up in areas, while other areas Microsoft has a leg up or a longer track record (such as Operations and Configuration manager).

A lot of differences in both products were not discussed, and would take a lot more time than 75 minutes. But it’s clear with Windows Server 2012 R2 and System Center 2012 R2 that they are making rapid and big strides in the private cloud and virtualization arena. Now that VMware and Microsoft appear to be on a yearly release cadence, I see the “Cloud OS” battle really heating up. MS has a lot of ground to make up, and they clearly knew it.

Private Cloud Technologies

Speaker acknowledges this is not a perfect comparison, as some products from each vendors package up features differently. For example, vCloud Director does a lot more than just self-service, but MS VMM has vCloud directly-like functionality not found in vCenter. So you can’t exactly line up products and say they are the same. But combine the entire stack from each vendor to really see how they shape up instead of doing per-product comparisons.

  • Hypervisor: Microsoft – Hyper-V; VMware – vSphere Hypervisor
  • VM Management – Microsoft – VMM; VMware – vCenter Server
  • Self-Service – Microsoft – App Controller; VMware – vCloud Director.
  • Monitoring – Microsoft – Operations Manager; VMware – vCenter Operations Management Suite
  • Protection – Microsoft – Data Protection Manager; VMware – vSphere Data Protection
  • Service Management – Microsoft – Service Manager ; VMware – vCloud Automation Center
  • Automation – Microsoft – Orchestrator; VMware – vCenter Orchestrator

Private Cloud Software Licensing

For both suites both vendors license the products by the socket basis. You can buy some VMware products a la carte, and some lesser known products aren’t included in the vCloud Suite. So depending on what features you need, you may need a different set up products.

  • Microsoft – System Center 2012 SP1 (per socket) & Hyper-V
  • VMware – vCloud Suite & vCenter

Key Focus Area for this Session

  • Granular App & Service Deployment
  • Deeper insight and remediation
  • Protection for key apps andworkloads
  • Hybrid Infrastructure
  • Costs

Granular App & Service Deployment

  • On VMware you use templates to deploy standardized templates. Templates are simple, but static.
  • In VMM you also have a dedicated Library to VM templates (like VMware) and service templates
  • In VMM you can have lots of templates all pointing to the same VHDX image (templates can have different features/etc.). Or small, medium, large, etc. templates all pointing to the same OS image.
  • In VMM you can add roles/features to the guest VM template and capture them in the template
  • You can have separate guest profile, and can marry up them with a hardware profile and a VDHX image without using any extra disk space
  • In VMM you can add applications, such as SQL, and easily create a template
  • VMM can directly configure App-V server packages and inject them into the VM template
  • VMM 2012 has a concept of service templates. Service template allows you to build and model multi-tier services. Ability to configure scale out rules, for example. Drag and drop VM templates onto a canvas and you can customize the VM properties.
  • Anything you can do in VMM you can do in PowerShell
  • VMM is more about delivering services to the business unit, not just deploying individual VMs
  • “Create Cloud” button in VMM. Defines resources, networks, load balancers, VIP templates, Port classifications (NIC), Storage, library, define capacity quotas (vCPUs, memory, storage, VMs, etc.). Ability to select hypervisor (Hyper-V, VMware, XenServer).

Service Manager

  • IT self-service management portal, built on SharePoint (also a full helpdesk ticketing system)
  • ITaaS offering
  • Plugs into VMM, Orchestrator
  • BI is built into service manager for deep reporting
  • Download “Cloud Service Process Pack” which pre-configures VMM, Service Manager and Orchestrator for a self-service VM portal

Orchestrator

  • Custom automation with minimal scripting needed
  • MS Orchestrator has a lot of plug-ins for third party products and hardware (integration packs)

Operations Manager

  • Extensible with MS and third-party management packs. Veeam MP can do deep monitoring of VMware environments.
  • Veeam MP is not free, so if you want to monitor VMware with SCOM you will have to license the excellent MP
  • OpsMgr can also monitor network infrastructure (switch CPU usage, memory, port-level stats, etc.)
  • Maintains the relationship between VMs and physical hardware such as switch ports, etc.
  • Server-side, client-side and synthetic transactions for application monitoring
  • Global Service Monitor (GSS) – MS Azure based global services that will test your private cloud app

Visual Studio Integration

  • VMM Library is accessible from Visual Studio
  • Team Foundation Server can use the “Test & Lab Manager” which will spin out VMs for automated dev testing via VMM

System Center Advisor

  • Provides configuration guidance around specific workloads (SQL, etc.) for troubleshooting. Free from MS.

Data Protection Manager

  • Supports Windows server, SQL server, SharePoint, Exchange, Dynamics
  • Up to every 15 minute differential backups
  • DPM can backup to Azure and tape
  • Changed block tracking for VM backups
  • Cluster aware – integrates with CSV
  • Item-level restore
  • DPM has no inline dedupe, but VMware data protection does

Heterogeneous Environments

  • VMM can connect to and provide basic management of vCenter
  • Can use VMM service templates on VMware hosts
  • Many integration and management packs for third party software and hardware (HP, NetApp, Cisco, etc.)

Hybrid Infrastructure

  • Private cloud (VMM can manage XenServer, vSphere, Hyper-V)
  • System Center can link to Service Provider and Azure
  • Single Sign on with AD (Azure)
  • Integrated with DEV (Team Foundation)

Cost Scenario

Cost scenarios can be extremely tricky and misleading. Plus large enterprises will likely get big discounts from both VMware and Microsoft. So take the numbers below with a grain of salt. Not in the cost calculation is the cost of the guest operating systems, since it was assumed both used the same OSes so the cost was a wash. The costs were only for the hypervisor and cloud stack.

The speaker didn’t mention the Microsoft ECI license (enrollment for core infrastructure). This combines the operating system and system center stack licenses into a single SKU, licensed by the socket. The datacenter edition of ECI allows unlimited VM deployment and management using all cloud features. Even if you are a 100% VMware shop for the hypervisor,  you may still have the ECI license if you use system center components (such as SCCM or SCOM). So you may already be fully licensed from the MS perspective and incur no additional software costs for the MS cloud stack.

  • Example: 500 VM Private cloud; 15:1 VM to host ratio; 34 hosts, 2 sockets with 16 cores; Windows Server licensing additional; comprehensive management; 68 licenses of Windows server datacenter
  • 68 CPUs Hyper-V: $0; 68 CPUs of System Center $122K
  • 68 CPUs vCloud Enterprise Suite $781K, vCenter $5K

TechEd: Building Windows 8 Image Engineering (WCA-B351)

This session covered the process of building a Windows 8 image. There are a variety of ways to build your image, which could range from custom built scripts to using MS provided tools. The big takeaway from this session was to use MDT 2012 Update 1 (or later) to create your customized Windows images. The resulting WIM and ISOs can be used with any MS or third party deployment product. MDT can inject drivers, software, and run custom scripts. It can even inject Windows update patches, using a repeatable and automated method. This would enable you to product frequent Windows images, that follow your business process.

Imaging Process

  • 1) Identify requirements for the master image – Use the new PoC offering to capture requirements
  • 2) Create automated image engineering task sequences using MDT 2012 U1 deployment workbench
  • 3) Automate as much as possible using MDT functions and scripting
  • You can fully automate the WIM build process and even bake-in Windows update patches

Identify Requirements

  • 32-bit or 64-bit or both? Look at both hardware and software compatibility. Best bet is to do both.
  • Thick, thin or hybrid images? Thin image is just the base OS with only minor changes/additions. Thick image is packed with applications and changes. Thick images are good for call centers or training labs.
  • Deployment – How will the image be delivered to client machines? MDT can create images used for any deployment method be it MS or third-party tools

How about Office?

  • Recommend to bake Office into the image.
  • Able to automate the Office installation through transforms

Proof of Concept Jumpstart Kit (Free)

  • Proof of concept jumpstart offer on connect. Lots of documents and pre-created scripts. Download: Windows 7 kit Windows 8 kit
  • Hydration kit creates 5 pre-configured VMs for a DC, MDT, and other services with pre-created customized settings and eval OS images
  • Contains infopath form to walk you through the configuration requirements gathering process
  • Solution Kit for Win8 adds a lot of custom tasks not in the base MDT kit

Deployment Basics

  • Build a reference image answer file (XML file) – Windows SIM (system image manager)
  • Create Bootable Windows PE Media – Windows ADK
  • Build and Capture a reference device – WinPE/DISM/ImageX
  • Build a deployment answer file – Windows SIM
  • Migrate data and settings – USMT
  • Deploy reference image – WinPE/DISM/ImageX

MDT 2012 Update 1

  • Basically just a file share with all the components needed to build the image
  • MDT is a platform that simplifies and automates the build process

Image Engineering Process

  • Install the vanilla operating system (Windows 8) – Use a VM for this
  • Customize the OS and install core applications/utilities
  • Sysprep and capture the machine with imagex (creates .WIM)

Other resources: Deployment Guys blog

TechEd: Hyper-V 2012 R2 Networking Deep Dive (MDC-B380)

Hyper-V in Windows Server 2012 R2 brings a lot of new networking features to the table. This was a deep dive session on what’s new in R2, how MS’s network virtualization works, and how it benefits customers. While Server 2012 brought huge gains to the network stack, R2 rounds out the feature set and makes it ready for large enterprise deployments.

Introduction

  • Three primary goals: Cloud scale performance and diagnostics; Comprehensive SDN, core infrastructure enhancements
  • Requirements to transform networking:
  • 1)  Deliver networking as part of a pooled resource, automated infrastructure
  • 2) Ensure multi-tenant isolation, scale and performance is what you expect
  • 3) Expand datacenter capacity seamlessly as per business needs
  • 4) Reduce operation complexity
  • What is Software Defined Networking (SDN)? Enables software to dynamically manage the network
  • 1) Abstract virtual networks away from physical networks (allow flexibility)
  • 2) Spanning policies across physical and virtual networks
  • 3) Controlling datacenter traffic flow

Hyper-V Network Virtualization (HNV)

  • Multiple virtual networks on a physical network
  • Each virtual network has an illusion it is running as a physical network
  • Overlays physical network
  • Encapsulating using NVGRE protocol
  • Workload owner Benefits: Seamless migration to the cloud, move n-tier topology to the cloud, preserve policies VM settings, IP addresses
  • Enterprise benefits: Private cloud datacenter consolidation and efficiencies, extension of datacenter into hybrid cloud, incremental integration of acquired company network infrastructure
  • Hoster benefits: Bring your own IP, bring your own network topology, scalable multi-tenancy

Windows Server 2012 R2 Enhancements

  • HVN is part of the Hyper-V switch (prior to 2012 R2 it was a NDIS filter)
  • Dynamically learn customer addresses
  • Support Hyper-V clustering
  • Enhanced performance and diagnostics
  • Able to ping the default gateway (if allowed)

Hyper-V Networking Virtualization Concepts

  • VM Network: Network isolation boundary; routing between VM networks must be explicit; comprised of one or more subnets
  • Virtual Subnet (VSID): Broadcast boundary
  • Routing between VM networks is via gateways (now built-in to WS2012 R2, or use third party)
  • Able to re-use IP addresses in different VM networks (bring your own IP)
  • Two kinds of gateways:
  • 1) Default gateway (.1), routes between VMs on different virtual subnets. Built into the HNV filter running on each host
  • 2) HVN gateway: Required to communicate outside a virtual network. Comes in different forms (VPN for site-to-site; load balancing and NAT for internet access; forwarding gateway for in datacenter physical machine access).
  • Partners can also provide gateway (F5 Big-IP software gateway, Iron Networks, and others)
  • Encapsulation: Network virtualization using Generic Routing encapsulation (NVGRE). Provider packet/IP is what the physical networks see, customer packet is encapsulated inside the provider packet and is what the VM see. Provider IPs must be routable on the physical network.

HVN Architecture

  • HVN is automatically enabled for all adaptors
  • New hybrid forwarding in Hyper-V switch
  • New in R2 is the ability of switch extensions (e.g. Nexus 1000v) can see provider and customer packets, not just customer packets like in WS2012
  • Combination of SR-IOV and HVN is not currently supported (since packets bypass the virtual switch). SR-IOV is designed for only extremely high traffic and trusted VMs.

Learning IP Addresses in Virtual Networks

  • New to WS2012 R2 is the ability to learn IP addresses in the customer space, vice explicit addresses set in 2012
  • Broadcast/Multicast support is new in R2
  • Enables new scenarios (DHCP in the virtual network, host and guest clustering)
  • Efficient implementation (uses hardware for Provider Address multicast if configured)
  • if no HW multicast is configured it falls back to intelligent provider address unicast replication – Only one unicast packet not matter how many VMs are on the host
  • Supports many address resolution protocols: DAD, NUD, ARP for IPv4 and IPv6
  • Reliable ARP proxy

Enhanced Performance and Diags

  • HNV + NIC Teaming is now allowed (new in R2)
  • Inbound and outbound spread on virtualized traffic
  • NVGRE Encapsulated Task Offload – Most offloads break when using NVGRE (LSO, RSS, VMQ)
  • Emulex and Mellanox announced NVGRE task offload in hardware
  • Showed a graph where Emulex shows line speed throughput with offload, with big decrease in CPU utilization
  • Look for the Message Analyzer (new netmon) is in beta – Can decode NVGRE packets. Can filter on CA or PA packets
  • Ping -p allows you to ping provider IPs
  • In CA address space you can use test-vmnetworkadapter
  • HVN responds to ICP request to the default gateway – Allows pinging the IP address of the CA default gateway

TechEd: Pass the Hash: Preventing Lateral Movement (ATC-B210)

This session was presented by Mark Simos, Aaron Margosis from Microsoft. Pass the hash (PtH) is an extremely common method that companies become owned by the bad guys. This session covered what PtH is, how it works, and some mitigation techniques. Microsoft recently released an extensive whitepaper on mitigating PtH attacks, which is well worth a read. Microsoft is releasing an updated version this week, so check for the 2013 version if you already have looked at the December 2012 version. The download link is in the session notes.

Background

  • The problem is that the internet is a source of a lot of good, with easy and instant IP connectivity.
  • The information that is worth money is stored on a computer (criminal organization, intelligence agency are after your data)
  • A lot of different organizations hack for various reasons. A logging company in Brazil hacked a rainforest quota system so they could log more and make more money.
  • Hackers use your own systems against you. They take your admin credentials and the consequences can be deadly. 99% of the time they hack you for data/IP theft.
  • The bad guys have the power to completely erase your data and render PCs unbootable if they wanted. An example was shared of a middle east oil company where 75% of the company’s 30,000 PCs were completely wiped and data files replaced with an animated gif of a burning American flag.
  • Lateral movement: A attacker uses the credentials obtained from a compromised computer to gain access to another computer of the same value to the organization.
  • Privilege escalation: The attacker uses the credentials obtained from a compromised computer to gain access to another computer of a higher value to the organization.

Attacks

  • Typical pass the has attack:
  • 1) Starts in a phishing email or watering hole attack (find a place users already goes, and stake it out by compromising the website) and targets workstations in masse. System needs to be compromised at the system-level, not just in the user space.
  • 2) Running as local admin the bad guys takes credentials for lateral traversal
  • 3) Bad guy acquires domain admin credentials
  • 4) Bad guy has direct or indirect access to read/write/destroy data and systems in the environment
  • If all computers have the same local admin password, then one compromised PC has compromised all PCs
  • Removing admin rights from users significantly raises the bar on compromising a PC. Zero day exploit, unpatched application, etc.
  • Most pass the hash attacks are human speed attacks (not automated malware) using a remote human controller (remote shell)
  • Windows credential editor v1.4 beta by Amplia Security (download link)
  • wce -w can also display plain text password as well
  • wce has early code for using “pass the ticket” (Kerberos) attacks, but very uncommon
  • pwdump7.exe was also demoed
  • Easily can recompile the wce binary to hide from A/V software
  • In real world attacks a complete domain can be compromised in 6 minutes to 24 hours
  • Why can’t Microsoft just patch this? Local admins own the box, so they can look at any aspect of the OS and dump the passwords. MS should change the entry points, encryption, etc. and hackers will quickly release new tools. Microsoft hinted something was coming down the pipeline in the future to help, but was extremely vague.

Practical Mitigations

  • Download the full Microsoft whitepaper for PtH mitigation here. Check back for the June 2013 version, as a new version will be released shortly.
  • Mitigation 1: Restrict and protect high privileged domain accounts. Excellent effectiveness. Medium effort required.
  • Don’t allow domain admins to logon to workstations
  • Don’t create service accounts that use domain admin creds
  • Mitigation 2: Restrict and protect local accounts with admin privileges. Excellent effectiveness. Low effort required.
  • Explicitly deny network and remote desktop logon rights for all administrative local accounts
  • Create random passwords for local accounts with administrative privileges
  • Mitigation 3: Restrict inbound traffic using the Windows firewall. Excellent effectiveness, medium effort required.
  • “Local account token filter policy” should be enabled (don’t set this to 1) as it opens up to pass the hash attack
  • Whitepaper updated this week for Lync peer-to-peer considerations
  • If you implement these three mitigations, then you are head and shoulders above nearly all other organizations. Attackers will have a much harder time in compromising your systems. Certainly doable, but these raise the bar significantly.

Other Mitigations (that don’t help that much)

  • Disable NTLM (very costly in terms of implementing and testing. Likely break A LOT of software/hardware like printers)
  • Smart cards and multifactor authentication (random password hashes are still stored and  used in the background. They are static hashes that never expire, so almost worse than password hashes that require regular changing)
  • Jump servers (has good value for other reasons, not just pass the hash). Keystroke loggers and other malware reduce their effectiveness for PtH attacks.
  • Rebooting workstations and servers. Many service accounts and services may use domain credentials, so they are cached upon reboot.

Microsoft also showed off a high level diagram of a security architecture that drastically mitigates PtH attacks. Microsoft professional services. The slide below shows the high level enhanced security environment.

pass the hash

TechEd: Prospecting for Windows 8 Gold (WCA-B360)

This session was by Mark Minasi, who is one of the must hear TechEd speakers. Highly entertaining and highly informative. If you ever come to TechEd, you must attend one of his sessions. This session was focused on Windows 8, and going beyond the arguably ugly skin and going under the covers to the hidden gold. If you can get past the UX issues with Win8, there are a lot of great features under the covers, including big security improvements.

Windows 8 is worth trying out

  • Best reason: Domain join your tablet
  • Learn the shortcut keys to navigate Windows
  • Windows Key + D get back to the desktop from the start screen
  • Windows + E Opens Explorer
  • Windows + . (cycles through snap options)
  • Windows + z (shows options)
  • Alt-F4 closes Modern app windows
  • Windows + x (lots of goodies)
  • Windows + c (for charms)
  • Windows + I (settings)
  • Windows Page Up/Down swaps Modern screen on dual monitors
  • Windows + o locks orientation

Understanding the new Apps

  • Modern Apps, Windows Store Apps, Immersive Apps
  • Very sandboxed and extremely hard to write malware within the app
  • You can screw up your own profile settings but not system settings
  • Non-admin users can install apps
  • App deployment story is quite different
  • Four ways to get a store app:
  • 1) User installs it herself with the Windows Store application
  • 2) User installs it himself from a private “company app store” the admin created
  • 3) User finds a provisioned app that is on the computer (up to 24 apps)
  • 4) User runs a PowerShell command “add-appxpackage” to install the app (side loading)
  • Codeplex has a free Company app store tool
  • If an administrator installs a Modern app, it does NOT install it for all users. Only the user can install apps for themselves.
  • Provision apps in your image
  • Each 64K of an appx package has a hash, and if any byte changes, the app kills itself
  • To provision a Modern app you must have the appx package. You can’t get the appx package from the app store yourself. You must contact the developer/company to get the package.

Sideloading Apps

  • Group policy setting to enable side loading
  • Win8 home cannot side load
  • Only WindowsRT and Windows 8 Pro/Enterprise can sideload
  • Applications must be digitally signed (can use your own CA)
  • Enterprise comes with a license to sideload, must be purchased for professional
  • Powershell: import-certificate to load a certificate into the Windows store
  • Domain joined enterprise server has a built-in free license
  • Windows Professional requires license (MS sells them for $30 each in packs of 100)

New Cool Stuff

  • Native 4K support (faster, cheaper, bigger drives)
  • Windows 8 recognizes SSDs and turns off defragmenter and uses the TRIM command
  • Most of the new SAN like storage spaces from server 2012 are in Windows 8
  • You could mount ISOs and VHDs from Explorer
  • Chkdsk is way smarter and faster
  • chkdsk /f /sdcleanup driveletter: finds and removes dead SIDs on ACLs
  • chkdsk /scan runs at low priority several times a day and makes mental notes on stuff to fix later
  • chkdsk /spotfix will just fix the list of problems
  • Powershell: repair-volume (but does NOT warn you when it takes a volume offline)
  • Recovery tip: make a recovery stick
  • F8 doesn’t take you safe mode anymore
  • Create a recovery disk on a USB stick from the control panel (search on “recovery”)

Security Upgrades

  • UEFI support means “secure boot” which means rootkits and bootkits are nearly impossible
  • Hyper-V 2012/R2 can now create UEFI VMs
  • Early launch anti-malware protection (ELAM)
  • Defender protects against malware now
  • Look at “offline defender” for cold scanning a suspected infected machine

PowerShell Goodies

  • 2000+ PowerShell commandlets
  • disk cmdlets: get-volume, clear-disk, get-tpm, set-partition changes drive letters easily
  • networking: add-vpnconnection, set-dnsclientserveraddress, get-smbopenfile
  • -scheduledtask commandlets
  • printing: get-printerdriver, add-printerdriver (admin rights needed), add-printer, get-printer,

Other Goodies

  • Use the Windows 8 ADK tomake a bootable USB stick:
  • makewinpemedia /ufd c:\winpe4-64 h:
  • WinPE 4.0 supports PowerShell
  • “Refresh” returns your PC back to a known state
  • Roaming profiles can be limited to “primary” PCs using set-aduser to limit roaming settings
  • powercfg /batteryreport

TechEd: IaaS with the Azure Pack (MDC-B364)

This session covers how to develop on-prem IaaS (Infrastructure as a service) using the Azure pack for Windows Server 2012 R2 and VMM 2012 R2. The session was more developer oriented than I thought from the description, so I ended up leaving a bit early since I’m not a developer. However, in the beginning the speaker did several demos of what the Azure pack does, which I found very useful. He then dove into the back-end details on how it all worked and what you have to do to build your own on-prem Azure VM gallery.

Hinted at in this session, and other sessions, is a possible roadmap feature where Microsoft would provide pre-configured gallery templates for certain Microsoft products like System Center and SQL. You would then be able to tweak the config, and easily built up a service catalog, and deploy MS services on Hyper-V in a highly controlled, standardized, and automated way. The R2 Windows Server and System Center release have a lot of the building blocks to enable those features in the future. Given the accelerated release cadence of MS’s cloud platform, customers will get new features much faster than they historically have.

Introduction

  • MS is hyper-focused on consistent cloud experience across the clouds (on-prem, Azure, service provider) at all layers (UX, APIs, PowerShell)
  • IaaS (Infrastructure as a service) – Elastic tiers
  • Customer requests: Enable templates to be deployed to any cloud, Provide a gallery of applications, Provide console access to remote VMs, anaging standalone VMs is not enough
  • Vision (not 100% delivered in R2): A consistent service model amongst Windows Server, System Center and Windows Azure for composing, deploying and scaling virtualized applications and workloads.
  • Four pillars: Portal User experience, deployment artifacts, management APIs, on-prem, hosted clouds and Azure
  • Consistent IaaS Platform: Delivered on portal user experience (Azure Pack), deployment artifacts, management APIs, Clouds

Demo #1

  • Showed a gallery for the VM role (new to Azure). Lists various services (SQL srever, IIS web server, SharePoint, etc.) that the admin has configured and curated. Gallery shows different versions of the same template, and can be tied to a subscription. When deploying a VM you can define the number of instances, for scale-out.
  • VM container, and Application container concepts (application payload is delivered into an OS)
  • The Gallery wizard prompts for a number of service properties (website name, admin names, VM sizes, etc.).
  • Shows a usage portal, which lists cores, RAM, storage, and VM usage. Also lists instances, IP address, disks, subscription, VM operations (power, stop, reset, etc.). Scale slider for increasing VM count.
  • Shows the ability to create a virtual network  (e.g. creating a site-to-site VPN) in the Azure pack.
  • Shows the ability to open a console to a Linux VM, or a VM without a network or OS

Iaas Architecture

  • Stack is: Hyper-V, VMM, Orchestrator, Operations manager, and two portals (tenant and service admin)
  • Steps to setup:
  • Load application extensions to VMM
  • Create a gallery item (VMM role template)
  • Create a service admin
  • Expose to tenant

Remote Console

  • Requires a new RDP client to support the new console version
  • Trust is established between all components (Azure Pack, Hyper-V, RDS gateway)
  • RDPTLSv2 is the new protocol

How to Build your Gallery

  • Definitions: VIEWDEF, RESDEF, RESEXT (consistent naming across Azure and on-prem/service provider)
  • REDEF: Virtual machine role resource definition (VM size, OS settings, OS image reference)
  • RESEXT: Your Application (roles, features, OS image requirements, etc.)
  • VIEWDEF: User GUI experience definition (parameters, grouping, ordering, validation, etc.)
  • RESCONFIG: RESDEF parameter values, single deployment, versioned (e.g. hard coded port number, etc.)
  • Uses JSON not XML files (make it more REST and portal friendly format)
  • Good support for command line installers/scripting (integrate PowerShell desired state, Puppet, etc.)
  • First class support for SQL deployments, IIS, etc. to make it very easy to configure
  • Built-in full localization support with a default language (which you can change)

TechEd: Windows Server 2012 R2 IPAM for Clouds (MDC-B376)

Starting off Day 3 of TechEd 2013 is a session on Windows Server 2012 R2 networking for cloud services. He covers what’s new in Windows Server 2012 R2 IPAM (and touching on DNS and DHCP). Windows Server 2012 shipped with major new features, and R2 builds on those features and better integrates them. IPAM in WS2012 was pretty bare bones, but far better than not managing your address space or using Excel spreadsheets that are never up to date.

IPAM in R2 gets a lot of major new features, and deep integration with VMM 2012 R2 to manage virtualized multi-tenant datacenters.  Some features like GUI-based scheduled DNS/DHCP record import are still missing, but are fully exposed through powershell for easy scripting. If you aren’t using an IPAM tool today, take a good look at Windows Server 2012 R2. The preview version will be out later this month, so you don’t have to wait long to try it out.

Windows Server 2012 ReCap

  • Existing IPAM options: Spreadsheets, in-house tools, commercial appliances
  • In-box IPAM: Compliments DNS and DHCP services. Ability to organize, assign and monitor IPv4 and IPv6 addresses.
  • Automatic discovery of DC, DHCP and DNS servers dynamic IP addresses
  • Track and audit changes and provide real-time view of service status
  • Multi-server management to manage all DNS and DHCP servers
  • DHCP and DNS have major new features: DHCP failover (active/active config), DHCP policies (group difference devices and assign different address to them (e.g. printers, phones for proxy settings, etc.). DNSSEC cache poisoning protection.

R2 Address Space Demo:

  • Shows IPAM DHCP scope utilization and health status
  • Shows you can now group IP address blocks by geographic regions. You can then filter views by region and drill down into countries or regions and see all scopes and IP address assignments.

Server 2012 R2 Enhancements

  • WS 2012 R2 Network environment: Host or Enterprise, multi-tenant and multiple datacenters with virtual networks
  • Ability to setup DHCP failover across datacenters
  • Supports virtual networks (administered by Fabric administrators)

IPAM 2012 R2 Enhancements

  • IPAM now manages and monitors both physical and virtual addresses
  • Integrated with VMM 2012 R2 and makes all address info available to VMM
  • All-new role based access control in IPAM. Granular control over what admin tasks people can perform.
  • Plan, design and administer IP address schemes of virtualized datacenters
  • Support network isolation WNV, VLAN
  • Enhanced service monitoring
  • Single and multi-entity configuration of reservations, scopes, failovers, policies, filters, etc.
  • External database support (SQL)
  • CIM based PowerShell – 100% parity with GUI

Virtualized Networks

  • Provider address space: Physical network address space
  • Logical networks in VMM are customer address space
  • Customer can bring in their own address space, which may overlap with other address spaces
  • Must deploy network virtualized networks (e.g. NVGRE) to keep address spaces isolated

IPAM-VMM Integration

  • IPAM has a view of both physical and virtualized address space
  • Network admin tasks (fabric layer): Configure address space, subnets, pools, VLANs. Then creates subnets, pools and logical networks, and then the config is pushed to VMM. Changes in VMM are pushed back to IPAM. Conflict detection, notification and updates, changes and meta-data are all synchronized. All configuration is done in IPAM by the network admin.

IPAM-VMM Demo

  • New “Virtualized Address Space” node in IPAM
  • “Managed by Service” column that shows VMM or IPAM service that controls the subnet config
  • “Service instance” column shows which VMM instance is assigned that subnet. Subnet now appears in VMM console.
  • Shows VMM synchronization with IPAM when subnets are pushed to VMM
  • When creating a VM network in VMM, he shows that the config is pushed to IPAM as a customer network

Role Based Access Control

  • Granular admin control within IPAM, DNS, DHCP. Five step process:
  • 1) Define a user role (operations an admin can perform)
  • 2) Define business hierarchy model based on the desired administration levels and controls
  • 3) Define access policy based on configured use role and access scope and associate users or groups
  • 4) Set/associate access scope to objects in IPAM
  • 5) New access control for leaf nodes or inherited from parent

DHCP/DNS Integration

  • Monitoring: Server availability, DHCP scope utilization, DNS zone health, DHCP failover health
  • Management: DHCP server, scopes, properties, options, filters, policies, classes, DNS records, etc.

DHCP Management and RBAC Demo

  • Shows the ability in IPAM to configure DHCP scope failover on remote DHCP servers
  • Shows the new “Access control” node in IPAM. 12+ pre-configured roles. Shows the ability to create a new custom role. Dozens of operations available to delegate and add to a custom role.
  • Shows the ability to create network hierarchies (e.g. in a city you can create a building).
  • Shows the ability to create an “Access Policy”, then bind the access policy to a DHCP scope for delegation
  • Shows the creation of a new R2 “FQDN” DHCP policy in the IPAM tool. Able to specify that all clients that do NOT contain *.contoso.com in their hostname get registered in DNS with guest.com instead.

External System Integration

  • IPAM PowerShell interface facilitates integration with other external systems like SCCM and MAP toolkit
  • Integration with AD Directory Services enables synchronization of site and services and subnets information

TechEd: Building Clouds on Server 2012 R2 (MDC-B312)

This session was a firehose of information on the design considerations when building your private cloud based on Server 2012 R2. There are ton of new features in WS2012 and R2, so this was a high level roadmap on how to figure out what you want to implement. Bottom line is that with WS2012 R2 and System Center 2012 R2, you have a full Cloud stack available. The 2012 releases built the foundation, but had some missing pieces. The R2 release rounds out those holes, and unifies the release schedule and simplifies the experience.

Introduction

  • Windows Server 2012 is Cloud optimized
  • Clouds are dynamic, multi-tenant, high scale, low cost, manageable and extensible
  • Major new cloud enabling features in Server 2012, released last year
  • 2012 built  a strong platform, but was not a full cloud solution

WS2012 R2 Improvements

  • Live migration is much faster
  • Live migration from 2012 servers
  • Shared VHDX clustering
  • Automated block-level storage tiering
  • write-back cache
  • Per-share auto-redirection to scale-out file servers
  • Dedupe of VDI workloads
  • iSCSI target VHDX support
  • Multi-tenant site-to-site VPN gateway
  • Hyper-V NAT and forwarding gateway
  • vRSS
  • NIC teaming dynamic-mode
  • Desired state configuration
  • Datacenter abstraction layer
  • All aligned with System Center 2012 R2

Blueprint for a Cloud

  • Build your managment stack
  • Start provisioning compute nodes and storage
  • Then you scale out as needed
  • This is a cloud “stamp”
  • Publish a self-service portal or APIs
  • Add network gateways
  • Add users

Infrastructure

  • Think about: workloads, networking, storage, resiliency

Designing for the workload

  • Cloud-aware stateless apps or stateful apps?
  • IaaS cloud can support both but with different design considerations
  • What are the workloads performance requirements
  • 2 socket servers offer the best ROI
  • Some workloads will benefit from hosts with SR-IOV
  • Are workloads trusted? Think about level of isolation between workloads and QoS policies
  • Keep it simple and manageable
  • Can’t optimize a unified infrastructure for all possible workloads
  • Standardize VMs, self-service based, managed to an SLA

Network Design

  • Traffic isolation considerations (tenant generated traffic) and hoster/datacenter traffic (cluster traffic, storage, live migration mgtmt, etc.)
  • Use physical isolation as needed, port ACLs, QoS & VM QoS
  • Between tenants and datacenter: separate networks
  • Between tenant VMs of different tenants: Hyper-V network virtualization & VM QoS
  • Hardware offloads for NICs: HW QoS (DCB), RDMA, RSC, RSS, VMQ, IPsecTo, SR-IOV
  • For storage, if using SMB 3.0, then the NIC would benefit from RDMA feature
  • R2: can also use RDMA for Live Migration
  • Look at RSS and RSC for the NIC which support management (Live Migration, management)
  • Look at IPsecTO and VQM for VM guest NICs
  • SR-IOV bypasses the extensible switch
  • R2: vRSS (spreads NIC traffic load across multiple VM cores

Storage Design

  • Hyper-V servers with internal SAS disks is a perfectly acceptable if you don’t need super high HA
  • 2012: Can pool shared JBOD SAS array for some good HA
  • Scaling options: Block based FC or iSCSI or file based (lower cost w/ high performance)
  • Block based enables storage offload with ODX, and high IOPS

Resiliency Approaches

  • Infrastructure – VMs not designed to handle failures, HA at server level, failover clustering as another layer of protection. High end servers, redundant power and apps.
  • App-Level Resiliency – Cloud-aware apps can sustain failures without infrastructure dependency

WS2012 Representatitve Configurations

  • Three different approaches are fully documented and validated by Microsoft:
  • aka.ms/CloudBlog
  • aka.ms/CloudConfigs
  • aka.ms/CloudPowerShell

How do you deploy and configure?

  • In 2012 it was a mixture of GUI and a lot of PowerShell
  • With R2 and aligning with system center 2012 R2, it is much much easier
  • “Physical computer profile” is new in SC2012R2 – Deploy Hyper-V to bare metal
  • Demo showed provisioning a new scale out file server and creating a file share, all from a GUI

Scaling Considerations

  • Compute (Hyper-V) cluster size
  • Larger clusters improve overall efficiency
  • Consider clustering across failure domains (e.g. cross-rack)
  • Storage: Need JBODs with appropriate number of SAS interfaces

Management Stack Improvements In R2

  • Provides a unified Powershell method to manage physical devices, such as switches
  • MS created a logo program that vendors can certify against
  • MS open sourced the OMI standard for anyone to use
  • Desired State Configuration (DSC) MDC-B302 session

Windows Azure Pack

  • Same self-service portal as Azure
  • Common management experience
  • Workload portability
  • As future services are delivered in Azure, they will transfered into the private cloud

TechEd: What’s new in SC VMM 2012 R2 (MDC-B357)

This session was mostly a demo of VMM 2012, where the speaker also covered some enhancements in the upcoming R2 release. Many of the Hyper-V 2012 R2 features have already been covered in other sessions, so there wasn’t a lot of new content. But he did a good walk through of several scenarios using VMM. I had forgotten that VMM can also provision storage from a physical array, automate SAN switch zoning, and present storage to a Hyper-V host, all within the VMM GUI. So you no longer have to pull out SAN tools, then your SAN switch GUI, then your virtualization management tool.

One thing to note is that the Azure Pack and the System Center App controller product are different products. A question was asked whether they would be merged down the road, and the speaker could not comment about futures. But one would hope they unify the provisioning portals and experience, and I expect they will down the road.

Introduction

  • Cloud OS: Three datacenters: On-prem, Windows Azure, Service Provider
  • Many customers will have assets across all three clouds
  • Customers need a consistent set of building blocks
  • The hyper-v that ships to customers is the same version that powers all of Azure
  • This session will focus on the on-prem and service provider clouds

What is the Cloud?

  • Term is way over used and misunderstood
  • Pool compute, storage and networking
  • Allocatable on demand
  • Automate everything – In VMM everything is Powered by PowerShell (500+ commandlets)
  • Metered
  • self-service

VMM – Enabling the Cloud

  • Storage – Can use any kind of storage you wish – SAN (iSCSI, FC), or SMB 3.0
  • Networking – In R2 VMM can manage physical switch configuration. NVGRE, PVLANs, etc.
  • Compute – Intel and AMD processor support
  • Virtualization Support – Hyper-V, VMware, Citrix XenServer
  • Do not name your cloud after a department, it is a pool of compute power. Cloud is an SLA construct.
  • User roles can be departments (Finance, HR, etc.). Construct an AD group, assign people, and assign access to appropriate cloud resources.
  • Model your application you are deploying so you can enable self-service

Announcing the Cisco Nexus 1000v for Hyper-V is now available for production usage.

VMM Investments in 2012 R2

  • Services, VMs, Clouds, Networking, Storage, Infastructure
  • Think of a “stamp” as a consistent configuration of Storage, compute, edge components and management
  • Later this year: All System Center components will be available a service template for fast and standard deployment
  • Physical Computer profile is a new feature for a scale-out file server Hyper-V host
  • 2012: VMM can appropriately zone SAN switches and provision storage from an array, such as 3PAR
  • Enables ODX to copy VMs from the library to production
  • Guest clustering using a shared VHDX file. No iSCSI or FC required.
  • Service template supports first node having a different configuration from other nodes, so you can automate cluster builds
  • VMM integrates with IPAM, so you can push/pull network configs with each other
  • VMM will warn on physical switch VLAN misconfigurations with switches that support OMI management
  • VMM can remediate network config problems on physical switches, if the network team allows it
  • Directly deploy and configure gateway (site-to-site VPN, NAT, or virtual to physical gateway) settings
  • Site-to-site VPN optionally supports iBGP
  • Site-to-site VPN supports third party devices such as Juniper or Cisco concentrators, or another Windows server
  • Delegation: Per-cloud delegated permissions
  • New and very rich SCOM management pack for VMM

TechEd: SQL Virtualization and Management Best Practices (MDC-B328)

This session focused on how to manage a virtualized SQL workload with system center. By using VMM, Operations Manager, and other SC components, you can automate, monitor, and easily deploy new SQL instances in a cookie cutter manner. This session applies to you whether you use VMware or Hyper-V hypervisors. Much of the session was live demos, so I don’t have extensive notes from the session. But it was a good eye opening experience on how well the System Center stack plays together, and can orchestrate your datacenter.

Benefits of Virtualizing SQL with System Center

  • Performance and scalability
  • Flexible storage and availablity
  • Depoyment and management
  • Portability of development workloads
  • On demand platform provisioning
  • Lower costs

Pitfalls of Virtualizing SQL

  • SQL server VM sprawl – Huge problem! Create an approval process.
  • Licensing challenges
  • Additional layer of monitoring

System Center 2012 Benefits

  • Deploy SQL server using SCVMM on Hyper-V or VMware
  • Provide self-service capabilities using SC Service Manager
  • Manage SQL server automations using SC Orchestrator
  • Manage SQL server operations using SC Operations Manager
  • Self-service backup and restore of SQL services using SC Data Protection Manager

SCVMM and SQL Server

  • Ability to deploy SQL server VMs on Hyper-V and VMware
  • Ability to create a SQL server profile to standarize VM templates and configuration
  • Ensures a SQL server is deployed every time exactly as you want, every single time
  • You can provide a SQL .ini configuration file for a standardized deployment config
  • You can mix and match SQL profiles with different operating systems

SQL Server Task Automation

  • Use SC Orchestrator (SCORCH)
  • Standardizes automated task management
  • Create a process in SCORCH which can be performed in a workflow manner
  • Enables end to end automation
  • Eg. Create a SQL backup, or database snapshot. With Service Manager you could provide user self service for SQL activities.
  • Complex tasks may take a day or two to configure in ORCH, but many tasks can be done in 1 hour or less of work

SQL Server Monitoring

  • Monitor SQL synthetic transactions and perspectives
  • Monitor SQL queries using application performance monitoring
  • Manage SQL server using distributed application
  • Ability to trend SQL database response times