Table of Contents
An interesting case ringed through my door today and I want to share it with you!
Scenario
A customer reported that part of their company got acquired by another, they wanted to split a production domain environment into two. They were reluctant to deploy a new set of domain controllers and use any migration solution such as Active Directory Migration Tool (ADMT) or Quest and their were looking for my advise in this scenario.
To oversimplify, he had 6 domain controllers (DC), and the end goal was to end up with two separate environments sharing the same domain name and Active Directory objects.
One section with DC1, DC2 and DC3, let’s call it section A. And the others with DC4, DC5 and DC6, let’s call this section B.
Original customer plan
- Set the first domain controller section in subnet A
- Set the second domain controller section in subnet B
- Block communication between subnet A and subnet B
- Seize FSMO roles from each group
- From first group of DCs, metadata cleanup second group of DCs
- From second group of DCs, metadata cleanup first group of DCs
Scenario analysis
This is not a common request; If you don’t think twice, you could say that it is something completely out of support, even searching for documentation seems like a waste of time. So, I looked at the problem in a different way, a way that I could find supporting documentation and tested scenarios.
Put yourself in the shoes of the DC1, suddenly three of your partners lose contact with you beyond the firewall gates. Now, you must find a new path to retrieve and send your changes (Replication), you also have clients that you did not see before coming to you asking for your services, you just had to create a few more users and you don’t see your RID master.
The situation I just described will happen to these DCs, but these DCs could also experience these due other reasons, for example:
- When you have to force demote a DC due to software or hardware failure
- When an environment gets targeted by a Ransomware and some of the domain controllers get encrypted and the administrator “pulls down the levers” to stop the infection ASAP
With that in mind, we could formulate a plan for those scenarios and indirectly prepare the customer to achieve his end goal. In essence it would be like having an announced failure or announced attack with a defined date and time to get prepared.
Action plan
- Healthy environment: Make sure the environment is healthy, fix any Active Directory Replication, DFS-Replication, GPO inconsistencies, patch your domain controllers and overall, make sure your environment is completely healthy before moving on. DCDiag is a good start to notice any potential problems that need to be fixed.
- Take a backup: While the only way to go back is with a Forest restore after doing a metadata and it is very costly, make sure you have that option in case something unexpected occurs, your job will be at risk if you don’t take a backup prior to this operation.
- Documentation: Document what organizational units, computers, servers and users will belong to section A and what computers should belong to section B, this ideally can be done per OU, depending on the design, it may be necessary to re-structure your OUs, it will make the cleanup work that comes later way easier.
- Pre-Configure DHCP for your clients: Apply the desired DHCP settings to the clients prior to the network separation, this is a critical step as it allows the clients to be in the proper subnet and point to the DNS server that will still be available after the network separation.
- Pre-Configure the DNS on your DCs: Make sure the forwarders on each section point to an accessible DNS server after the network separation. This will especially avoid unnecessary replication issues.
- Consider non-AD Aware applications: These type of services were not designed to work with DCLocator API and they need to be pointed directly to a DC Name or IP address to get communication with a DC, ideally those applications will be documented, but if that is not the case, then an extensive job of auditing should be done to find those applications, this involves creating a site that has no subnets mapped to it, changing priority on SRV DNS records, and often times this involves testing what occurs if one of the servers goes down (intentionally cutting any inbound communication from the DC), this reveals what applications will naturally search for the next closest DCs, while what other apps require manual intervention to recover.
- Test before proceeding: The hardest troubleshoot you could experience is not from the Domain Controllers after the change, but from the clients and member servers. After the above points are completed, a good idea is to purposely block the traffic between the two sections, keep an eye on what type of problems that are reported, the only types of tickets you want to get are the ones related to FSMO roles (specifically PDC and RID master), any other problem that has nothing to do with a FSMO role should be analyzed to understand the dependency of that client/user/member server and the section that will be separated in the future.
- Block the network communication: Once you are ready block the network communication, making sure the two sections cannot communicate to each other.
- Seize FSMO roles: In the section that is left without a FSMO, seize the role as soon as possible, prioritize the PDC-e followed by the RID master.
- Cleanup: From first group of DCs, metadata cleanup second group of DCs and from second group of DCs, metadata cleanup first group of DCs.
- Stale DNS records: Cleanup any DNS record that is no longer available on its respective network.
Fortunately, this was a single domain forest with no external or forest trusts, which simplified a few tasks. While we can be very careful in this procedure, due to the nature of this operation it is expected to have unexpected problems, so be ready to troubleshoot any problems that may arise.
This method, while complex, can provide a structured approach to domain separation, ensuring minimal disruption.
See you in the next one!
Leave a Reply