In my pre-sales role, when explaining the technical aspect of 1E Nomad, I’m often asked about the impact Nomad has on the end user’s personal computer. Remember, the entire premise is to leverage the existing computers in the estate to facilitate downloading and distribution of Microsoft System Center Configuration Manager (SCCM) content of all kinds. Being concerned about the possible adverse impact this may impart on the users of those machines is a natural concern. Simply saying “trust me, they won’t even know it’s there!” is not an adequate response. This article is intended to convey some real world actual metrics to illustrate the impacts imparted in two different scenarios: a data center server hosting the source content; and, the remote client machine used to download and redistribute that content.
Let’s Review a Bit First
Before we get into the deep end of the technical discussion that follows, let’s take a moment to review what 1E Nomad actually is in the first place, and show a simple illustration of the general concept.
Whenever an enterprise is in the business of managing the deployed estate of personal computers, more often than not they are using SCCM. This very mature solution is the overwhelming market leader in this space. It is a robust, scalable and feature rich tool that manages virtually every aspect of Enterprise level systems management, from hardware and software inventory, desired state compliance, remote control, and arguably the most important of all, software distribution. This software distribution content covers the gamut of routine security patch updates, application software programs, and operating system images of all kinds. As you can imagine, this can mean moving a LOT of content (data) around the enterprise. This doesn’t always go over well with the Network Engineering team responsible for maintaining all those wide area network (WAN) links. One cannot accept a machine on the far end of a WAN link running a 1gb Visio install when the source content is at the headquarters data center, and the user’s machine is half a world away across a number of router hops.
To overcome this problem, standard SCCM design incorporates a system role known as a Distribution Point (DP). This server role is deployed at the remote office and provides a server-server pipeline between the data center and the remote offices. The Administrator uses this role to copy source content from the data center and stage it locally on the DP. This is accomplished in such a fashion that the WAN between the two is not adversely impacted as the content is replicated to the remote DP(s). The user’s machine then installs Visio from that local copy. Sound simple? It is, except for a few key things. Servers and their operating system (OS) are expensive, regardless of whether they are physical or virtual. They need to be maintained and patched like any other server. They hold an instance of IIS, providing yet another potential attack surface for evil doers. They pose additional headaches for the SCCM Administrator managing all the content that needs to be staged, often across hundreds of DPs. Then you have the challenge of managing the hard core road warrior who rarely or never is on a network connection in a traditional office, but lives and dies via remote access methods. Getting even a small and simple set of monthly security patch updates to that person is a totally different challenge. Then there are the OS deployment and imaging challenges, but we won’t even go there for this simple discussion! All of this administrative effort imposes hard and soft dollar costs to the enterprise. Network links can still be overwhelmed if things go wrong, and they do. And on and on… there was a good reason why my telephone was speed-dial #1 in the Network Operations Center when I was an active Administrator!
1E Nomad was designed to make all of these problems go away. It does so by eliminating virtually all of those remote DPs, and instead passes on the chore to existing end user computers in those remote locations at a minimum. Ideally, Nomad is used everywhere! I’ll address that reasoning in a subsequent article. Today, I’ll focus on the basics of how the Nomad agent does its task of downloading and sharing content to the local office, across any kind, speed, and size of WAN links, regardless of the number of router hops, and doing so with no impact to the business traffic that is also traversing the same links. The means by which Nomad is able to accomplish this is using what is known as a Reverse-QoSTM technique. This allows a single machine on the remote subnet to be dynamically “elected” by its Nomad peers to assume the role of downloading the designated content that the remote machines require, and then sharing that content with its peers. Nomad is initially invoked by its partner, the SCCM client agent using what is known by SCCM as the “alternate content provider”, 1E Nomad. When SCCM needs content, Nomad is called by SCCM, and the process begins thru to completion. Once all of the content is received by those that require it, the installation proceeds normally. Nomad is done. Of course, the content is retained by all of those Nomad clients in their secure local cache to respond to any future needs for it. Consequently, all systems management content is only ever transferred once and once only to the remote network.
All of the previous description can be a bit confusing. The following brief animation attempts to illustrate this very basic concept in actual operation, from a data center DP hosting the source content (e.g. that Visio installation media mentioned earlier) across an imaginary WAN of whatever topology to a small branch office of five machines: two laptops, and three desktops. The premise: the SCCM and Nomad client agents are installed; there is an SCCM Visio deployment targeting these machines, and the SCCM client then invokes the alternate content provider Nomad, and the process begins:
I should also add that in the above animation, when the original master comes back on-line (and the Nomad service starts), it sees that it is partially through the download. It then makes a simple content request to its Nomad peers asking if anyone happens to have that Visio package it was in the middle of downloading. At that point, any of the other Nomad peers holding that content will simply respond affirmatively and share the remaining/missing content to complete the process. Likewise, the road warrior, who is essentially a “remote office” of a single machine that happens to be in a hotel room or home office, goes through the exact same process, electing itself as the “master” as there are no peers involved. It proceeds to start downloading the content as long as the connection is present. Once lost, it will simply pick up where it left off upon the next connection to the corporate network. This process continues until all of the content is downloaded.
Now that we have all of that out of the way, lets get back to the original question this article is intended to address: “What’s the hit on the systems running Nomad?”. To answer that, I’ll address first the data center DP serving out content to remote Nomad “master” machines across the enterprise that are actively calling for it. I then illustrate a “worst case” scenario on the downstream client side: the elected master machine that is not only downloading the source content for itself, but is also actively sharing it with its peers on the subnet. The impact on a receiving client is even less so not shown here. Now let’s review each scenario.
Impact on Performance of a Distribution Point
While Nomad typically eliminates the need for 90% or more of existing SCCM Distribution Points (DPs), there will always be a few that remain. Nomad architecture requires the Nomad agent be installed on those DPs servicing Nomad clients. This allows tight security validation of staged and delivered content (e.g. LSZ hash checking, eliminating any man-in-the-middle attack surface), as well as providing for a Remote Differential Compression (RDC) capability at the binary (not byte) level as and when source content is updated. This allows Nomad to securely deliver content initially, as well as securely delivering only updated data to the clients rather than redistributing an entire package.
The following illustrates the typical performance metrics of a large distribution point server while servicing many Nomad client distribution activities. While this is clearly a high end server with 16 cores, it is clear that the CPU impacts imposed by the Nomad clients in the download process are negligible.
Likewise, the overall disk I/O, measured over a substantial time interval, tells a similar story: overall disk performance is well within all acceptable parameters throughout the monitoring interval. What is also interesting about this particular timeline is that 3/12/2013 also happened to be Patch Tuesday, meaning that routine software update activities are typically unusually high as a result, yet there is no appreciable impact on this system as seen in the following graph.
Impact On A Client Machine in the Master Role
When an SCCM deployment is targeted to a remote group of Nomad equipped machines, as we illustrated earlier one of those machines is dynamically elected as the master machine from among all Nomad clients at the site to assume the responsibility of contacting the remote DP and downloading the required content for them all. This single machine then shares that content with its peers in real time throughout the download process. Consequently, that machine is the “busiest” of all the machines on that subnet in the remote office. This presents the “worst case” utilization scenario as it is not only downloading content across the WAN using the Reverse-QOS TM technology, it is also sharing that content with all of the other targeted machines on its subnet.
In this section we monitor CPU utilization, network utilization, and memory on an elected master Nomad device, while it is serving a 500MB file to another Nomad peer on the same subnet. This same machine is also actively running Outlook and streaming internet video at the same time.
In the graphs below, the turquoise box throughout the graphs coincides with the 500MB file being served to a peer from this master.
1. The CPU utilization was minimally impacted during the transfer
2. Network utilization reached 2% during this test
3. Memory performance also had minimal impact, reaching 15.5% during testing
4. Disk activity is documented in the last graph
CPU Usage by Processor
5. Overall system CPU usage is shown below. The Nomad transfer activity is shown predominantly on CPU 0 in red.
CPU Usage by Process – All Processes
6. All system activity is shown below, but only the Outlook and Internet Explorer processes are significant.
7. Outlook reached approximately 25%, while Internet Explorer was around 10%.
CPU Usage by Process – Nomad/CCMexec Processes Only
8. Only Nomad and CCMexec processes shown for clarity.
9. The highest CPU Usage for the Nomad associated processes on the master devices was CCMexec.exe at 0.25%. This occurred while the 500MB upload test was taking place. Overall CPU consumption was very low.
Client Disk Utilization by Process
10. All system disk activity with noticeable spikes is shown below in red for the system.exe process while transferring the 500MB file to peer.
11. This disk usage did not have a noticeable impact on the device while the video was streaming.
The laws of physics are clearly in play here, so it would be irresponsible to state that there is zero impact to a client computer; however, it should be clear from the above that the impact of Nomad, whether on a DP servicing a large number of simultaneous remote Nomad master machines, or on a single Nomad client in the act of servicing a download as a master agent is essentially negligible. When I’m asked this question by a prospective customer, I tell them that an end user working on a machine where Nomad is working has virtually no knowledge that this activity is taking place at all. The most an overly observant individual might notice is a busy disk I/O light flashing for no “apparent” reason. Nomad simply “works” from end to end with no effort on the part of the Administrator other than ticking the box enabling Nomad in the SCCM administrative console. The user’s system just “gets” what it needs, when it needs it, while the user continues on about his or her business.
Simplicity itself! I trust this high level of detail and transparency helps as you research and evaluate solutions to aid the challenges facing all SCCM Administrators managing content distribution and related WAN utilization concerns.
This article is tightly focused on only one aspect of the Nomad process. There are far more that will be of interest to the serious researcher. The following links provide additional reading on many aspects of 1E Nomad itself and related topics that also may be of interest.