Leveraging Nomad FanOut to accelerate Windows OSD

Scenario
In this scenario, we were testing Nomad’s FanOut functionality in a ConfigMgr 2012 R2 production environment where a new remote branch office was opening, which has slow WAN links to the central data centre. The customer’s objective was to completely build a large volume of devices within a limited time frame using standard ConfigMgr OSD tools integrated with Nomad.
The method used to achieve the objective outlining leveraging Nomad FanOut can be equally applied to an existing remote branch office or dedicated build centre where the requirement is to perform OS migrations out of hours (e.g. over an evening), on a large volume of devices.
What is the purpose of FanOut?
FanOut significantly accelerates the distribution of SCCM content between Nomad peers. From rapidly deploying software updates to delivering enterprise wide OS migrations in record time. This blog explains how the Nomad FanOut feature can help supercharge your OS migrations.
Native Nomad distribution model
By default, Nomad cache share (NomadSHR) is limited to six concurrent connections on an elected Nomad Master. If a seventh device requires content and attempts to connect to the Nomad Master it will receive a “maximum connections limit” response message which can be seen in the NomadBranch.log. The device will then back off and will wait a short period before attempting to connect to the Nomad master again. When a free connection becomes available, it will establish the connection and start downloading content.
NB: The six concurrent connection restriction only occurs when a Nomad Master already has all or majority of content. If a Nomad Master is pulling content from a DP, it can serve more than six Nomad peers by using a round robin approach whereby Nomad peers connect, catch up to as much content as the Nomad Master currently has downloaded, and then disconnects allowing other Nomad peers to connect. This process is repeated until Nomad peers have downloaded all the content. The reason for this behaviour is due to the fact that typically the LAN is much faster than the WAN.
In the event that the Nomad Master already has all or a majority of the content, this is where FanOut becomes beneficial as demonstrated below.
Nomad with FanOut enabled
When the Nomad Peer receives the “maximum connections limit” response message from the Nomad Master, rather than waiting for an available connection to the Nomad Master, the Nomad Peer requests connection to a FanOut peer within the local subnet. This is known as a FanOut Request.
Devices will respond to the FanOut Requests if they meet certain criteria. Further information can be found at https://help.1e.com/display/NMD55/Nomad+FanOut
The Nomad Peer will attempt to connect to the first FanOut Peer that responds to its FanOut Request.
FanOut can scale out up to six FanOut peers at any one time if they are concurrently active and downloading content from the Nomad Master (due to the 6 connection limit to the Nomad share). In this way, rather than limiting the distribution to six Nomad peers concurrently, 43 devices (1 Nomad Master, 6 FanOut peers and 6 Nomad Peers connected to each FanOut Peer) can simultaneously be downloading the content from the Nomad Master and FanOut peers, speeding up the content deployment exponentially on large subnets.
However, if there are other devices within the local subnet that are idle and have already downloaded all of the package, then FanOut can scale out to greater than six FanOut peers at any one time. In such a scenario, FanOut can scale out to potentially more than 43 devices downloading content concurrently, supercharging the content deployment even further on larger subnets.
Nomad in WinPE
By default, Nomad installed in WinPE is not capable of becoming a FanOut peer (i.e. a provider of package content) because file sharing is not supported under WinPE (WinPE cannot act as a SMB share server, only as a client). However, Nomad installed in WinPE with FanOut enabled is capable of requesting content by broadcasting a FanOut request (if there are no available connections to the Nomad Master) and connecting to FanOut Peers that respond on the local subnet.
Therefore, Nomad can share content when it is in the full Windows OS to a computer running in WinPE, but a computer in WinPE can only share content using ConnectionLess mode. Further information on ConnectionLess mode can be found at https://help.1e.com/display/NMD55/Peer-to-peer+enhancements
Enabling FanOut
FanOut is disabled by default. To enable Nomad FanOut functionality, it must be configured via registry settings on all full Windows OS platforms hosting Nomad and in the “Install Nomad and configure Nomad in WinPE” custom task sequence action.
To enable Nomad FanOut you must add/modify the SpecialNetShare Nomad Registry value on devices hosting Nomad, and in the ““Install and configure Nomad in WinPE” OSD task sequence action properties, to include bit 6 (0x40). E.g. SpecialNetShare=0x40
Further information on configuring SpecialNetShare can be found at https://help.1e.com/display/NMD55/SpecialNetShare
Testing FanOut Enabled OS Deployments
As this is a bare metal scenario, when using Nomad the best practice and pre-requisite prior to the start of the deployment is to “pre-cache” the OS deployment content to designated peer devices on the local subnet to ensure devices being built will obtain the content from the local peer when executing the OS deployment task sequence. A pre-caching task sequence will achieve this goal.
NB: Ideally in a refresh scenario OS deployment content would be pre-cached to the local hard drives of the devices being deployed. Therefore at the start of the deployment there is no need to pull content from a local peer as local content has been pre-cached locally. A dynamic pre-caching task sequence can be used to achieve this goal.
In the following scenario, Nomad FanOut enabled OS deployments were performed in a remote site with slow WAN links to the central data centre. OSD content had already been “Pre-Cached” to eight devices. Nomad FanOut is enabled and 50 computers needed to be fully built simultaneously over the course of an evening. The build process was automated further by using 1E PXE Everywhere which was installed on all the “Precached” devices to deliver the boot image.
The first six devices that were being built connected to the elected Nomad Master when they executed the “Prestage content using Nomad” custom step in the OS deployment task sequence. Any subsequent devices that were being built and requested the same content from the Nomad Master received a maximum connection limit message from the Nomad Master, at which point the Nomad peer broadcasted a FanOut Request.
The first of the remaining seven Pre-Cached devices which responded to the FanOut Request became a FanOut peer as it had a free connection to its Nomad cache, and was able to serve content to up to six Nomad peers. In this way, the other Pre-Cached devices responded to FanOut Requests as they were triggered. As a result there was zero waiting time for free connections, and all devices were able to concurrently download content when needed from a peer without interruption. Therefore, there was no added time to the overall build time due to delays downloading content. Using Nomad FanOut in this deployment allowed scheduled completion to deliver 50 fully built computers well within the deployment plan timescales.
NB: As usually is the case when attempting to build a large number of devices; they are started consecutively in a phased approach rather than simultaneously. Therefore it is a possible that by the time the latest device is started, previous devices have entered the full Windows OS stage of the build. At this stage, the Nomad cache has been restored and content activated making it available to share and respond to FanOut requests if the Nomad Master has all its connections used up.
NB: The “Prestage content using Nomad” task sequence step usually contains a number of package content required to build the device. After the Nomad peer has completed downloading a particular package it will take some time to hash the content for validation. At this point it has disconnected from the Nomad Master or FanOut Peer, freeing up a connection and allowing another Nomad peer to connect.
Conclusion
Assuming that eight devices are running in the full OS with all the content Pre-Cached. Nomad FanOut enabled OS deployments can scale out up to 54 devices (1 Nomad Master, 8 FanOut peers and 6 Nomad Peers connected to each FanOut Peer) concurrently building and accessing OS deployment content at the same time without any wait time. With more Pre-Cached devices, Nomad FanOut can potentially scale out even further, supercharging OS deployments exponentially in mass build scenarios to meet your deployment plan timescales and quotas.
Terminology
Nomad Master – The device that wins the Nomad election on the subnet and is the central resource for the download
FanOut Peer – A device connected to the Nomad Master share that responds to FanOut requests and allows other machines to connect to it
Nomad Peer – A device that requires the download