Active Directory Domain Services
Note: The Active Directory Domain Services platform will continue to be operated by AIT staff - service and application support and the governance of the application will transition to Identity Services.
Lots of people have been asking for AD to be the central authentication service because there are tons of issues with the Kerb trust, no way to support Win 8 right.
Phil does about 60% AD and 40% CIFS stuff. Justin and the OS X group need help with AD because they don't support the external trust in 10.7 and 10.8. They really want a central AD. They are pouring large amounts of time into solving authentication issues on lab macs.
Karen and Phil have always thought that a central AD needs to be a new, separate AD. ITS needs to move into that AD as part of the project.
"Kerberos is a mess and nightmare and it has been killing us." No password syncing, no two-way trust. LDAP info not updated in their AD.
Eric from Kerberos is a big fan of going to AD for central authN.
The CLC group has a really robust AD that they use for classroom management, and they are known as "The Windows Shop" on campus. Phil is trying to bring the Microsoft techniques into the AIT AD.
The Windows 8 authN problem don't allow domain\username, you have to use the upn format, no default login domain. This causes issues with scripts. There's been talk of escalating that problem through PSS. This does work in the new version of Kerberos 1.11.1. Kerberos build probably won't be done by the end of the summer.
***Definitely get more background/doco from Karen Mitchell.
Basic Service Information
|Item #||Topic||Service Information|
|1||Name of service||Active Directory Domain Services|
|2||Description of service|
Microsoft AD - Computers, User Accounts, GPOs, distributed management of AD, Phil's group is the gatekeeper of the service, delegated OUs. Delegated GPO creation. Only AIT staff can manage Kerberized user accounts. Delegated admins can create non-kerberized user accounts. Service generally lacking in support - whole chain of people who have managed the service, usually just one admin. Their cleanup or best practice policies haven't been enforced. Naming policies for non-kerberized account. The names should all be prefixed with the department name - for computer names, user accounts, groups, etc.
They have a disjoint namespace to support the external Kerberos realm.
List of typical existing uses of the service (may not be exhaustive)
|To meet all AD needs across the entire University - need to support an easy centralized management system for users, groups and systems. Single access forest, four child domains. People can currently request child domains, but now that the pipes to the other campuses are bigger, the need for child domains is less necessary. The number of domains has been diminishing from 12 down to 4.|
|4||Current departmental home of service||AIT|
|5||Current owner or sponsor of service||ITS|
|6||Link to current public-facing documentation of service||http://ait.its.psu.edu/services/identity-access-management/ad/ - also the technical whitepaper explaining the environment to a vendor <-- best technical document for explaining disjoint namespace and external realm.|
|7||Current customer contact information for service (email address, phone number of service desk, web form, etc.)||email@example.com|
|8||Complete and detailed list of existing SLAs and/or OLAs related to this service||None - no tech documentation team had the time to do this - AD is way low on the list because it's been around for years.|
|9||Existing uptime expectations for the service|
24x7 - replicated to four DCs, two virtual hosts in CMP, two hardware hosts in Library data center.
No DynDNS - that's where the disjoint namespace comes in - TNS does not allow dynDNS on campus. They have to request a pointer record to the service records on the DCs when they bring up a new DC.
|10||List of existing maintenance windows for this service||None because of replication.|
|11||Existing escalation and paging/alerting procedure for the service, including all contact information for the existing escalation path (including names, cell phone numbers and/or email addresses and types of contact to be initiated for various events)||Could go through the service desk, win-ad email, ticketing. Buck stops at Phil.|
|12||All customer-facing documentation for the service||See above|
|13||All service desk-facing documentation for the service||None - best practices on the front-facing web site. Don't work with users, work with other sys admins.|
|14||All systems administrator-facing documentation for the service||See above|
|15||All application administrator-facing documentation for the service||See above - supplement with MSDN and TechNet|
|16||All disaster recovery documentation for the service||Old DR plan - implemented forest functional level of 2008 R2. Phil was charged with re-writing the DR plan because they can now use the recycle bin. Also have a complete test domain test-access, including own test kerberos realm.|
|17||Current pipeline for requests for new or changed service features or functionality - list of all current requests, including requester, estimate of priority and dependencies (external and internal)||They do authorize things like schema changes - customers have to test it in the test domain. Mostly Microsoft-driven, Microsoft schema changes. They had a request for the Mac extensions, but that never made it into production.|
|18||Schedule for licensing/agreement renewals||University Microsoft licensing agreement. Buy TechNet subscriptions for test servers. Keep active TechNet subscriptions. DCs run 2008 R2 Enterprise - Uses the KMS server for client and server licenses, which are all included.|
|19||Any immediate licensing/agreement renewals that must be coordinated with ITS Financial||NA|
|20||Service performance reporting, lists of current metrics or measures, baselines for service management measures||Nagios for monitoring - DCs are IPv6 - monitor both v4 and v6 - replication, sysvol for connectivity, LDAP connectivity, CPU, disk, global catalog, memory, dynDNS, replication.|
Existing Customer and Stakeholder Information
|Item #||Topic||Service Information|
|1||List of major customers of this service, including name of services, type of service provided, brief description of service, contact information for people and departments using the service|
Open to sys admins at the university, must be in accordance with
Keep the admin_ accounts in an OU, look at who's in charge of each OU. They put the telephone number, emergency contact in the AD attributes for those users. Because of staff turnover, it's very hard to manage that.
Have a policy to expire admin accounts that haven't been logged in to for three months. Those are just disabled accounts. Phil removes the admin accounts from groups if they are disabled.
|2||List of minor customers of this service, less detail is acceptable, may not be complete||See above|
|Item #||Topic||Service Information|
|1||Hardware that supports the service, including complete list of servers, their hostnames, any aliases or virtual host names (for web servers, or LDAP, Kerberos or other services fronted by an NLB, aliased with a cname record, etc.), manufacturer/model, purchase date, warranty end date, physical location, current OS name and version, along side systems administrators of those servers|
4 DC, 2 virtual, 2 hardware (blades). No NLB
*Ask Karen Mitchell about the blades - for HW info - they had just renewed the warranties three years ago. Server 2008 R2 Enterprise
|2||List of current application administrators for the service||All non-disabled admin_ accounts in AD. In AIT, _admin account is the kerberos account for admins.|
Name of password safe group for each service in the shared IdS password safe
Please add in all relevant system, application and other types of usernames, passwords, private keys, etc, necessary to perform tasks, into a password safe group named for the service in the shared safe
There's a job that Jeff D'Angelo is responsible for - it's a cron job that creates the AES256 enctype enabled, sets a randomized password on the shadow account, sets up the name mapping with Kerberos.
Purging of user accounts used to fall under Bob Walters and Jeff D'Angelo. Phil thinks it's automated, but there are a large number of user accounts.
The passwords for those processes are under the control of Jeff D'Angelo and/or Bob W - get from them. The OS-level passwords will remain in AIT.
|4||List of all SSL or TLS certificates used by the service, the name of the certificate authority which signed the certificates, and the start and end dates of the validity of each certificate||No certificate services enabled within the AD - never thought of when the service was first brought up. No enterprise PKI in use.|
|5||Existing monitoring information for the service||See above|
|6||Existing monitoring information for the servers||See above|
|7||List of all cron jobs or scheduled tasks needed to maintain the service, along with descriptions of the task and processes||See above re: user creation/deprov and also group removal from admin_ accounts (Phil maintains that last one)|
|8||List of all manually run jobs or tasks needed to maintain the service, along with descriptions of the task and processes||Phil manually runs tasks to find incorrect names. Deprov might be a manual process (Either Jeff D'Angelo or Bob)|
|9||List of all software used to support the service, current application owner or subject matter expert for the software, commercial contacts for vended software (including phone, address and email), current pricing, current contract information including any licensing terms, restrictions, start and end dates, recurring maintenance requirements, payment addresses, payment dates, etc.||Microsoft Server 2008 R2 Active Directory Domain Services - one of the DCs is Server Core.|
|10||Any documentation or support web site URLs or mailing list addresses used to support the service, server, and software|
MSDN, TechNet, firstname.lastname@example.org mailing list.
Outgoing mailing list: email@example.com to admins, other people can sign up to that list, too.
Network of people list: firstname.lastname@example.org <-- big conglomeration of all tech/admin type people in the university
Big changes, go to both lists.
|11||The names and contact information of all hardware maintainers or hardware escalation path contacts, if hardware will not be maintained by IdS||Phil Swanzy (Technical Lead) - used to be Karen Mitchell, backup is Josh McCarl (Linux admin)|
|12||The names and contact information of all OS maintainers or OS escalation path contacts, if OS will not be maintained by IdS||Phil Swanzy|
|13||The names and contact information of all software/application maintainers or escalation path contacts, if software/application will not be maintained by IdS||See above - Phil Swanzy|
|14||A list of all modifications made to any base product (software installed in an OS from a non-OS repository, including name, description, version, dependencies; software patches either vendor-supplied or in-house, including source code or link to download the patch; changes to base configuration including but not limited to port number changes, SSL or TLS requirements, schema additions, etc.) Include the reason for each installation or modification and whether or not it is still required||Had to turn off Teredo tunneling for 6-to-4 because replication failed when it was on. They made turning off Teredo a requirement.|
|15||List of dependencies on any specific version of any software, OS or hardware used by the service, specifically, any reason for not upgrading to the newest version of any software or OS||2008 R2 Forest functional level AES256 enctype to the external Kerberos realm. Some customers still doing single des or arcfour, trying to get them to migrate off. Server 2012 test DC was trying to do arcfour and is requiring it, so can't log in to Server 2012 domain.|
|16||List of dependencies on any protocol or algorithm currently in use which must be maintained, and the reason for the requirement in each case||See above for Kerberos enctypes. With LDAP in AD, they don't consider it authoritative, they tell people to contact the AIT LDAP group for LDAP. They don't update their attributes, don't support LDAP for AD clients. People can do GSSAPI binds to LDAP. You can use it to authenticate. ET was doing some basic lookups. You can bind to the AD on port 636 using SSL. They were using IPSec before. Now they use built-in firewall rules. They don't block any traffic. ITS blocks CIFS and NFS at the border. No other standard Windows ports. Windows authN ports are open to the world. Karen Mitchell might have set up SSL - might not be using an InCommon cert *Check.|
|17||Location of all logs (including text files and syslog servers) related to the service||Logs go to syslog - Talk to John Kalbach - they are keeping logs probably forever, when they are rotated to TSM at some point. *They get requests from SOS for info on who logged on to what machine pretty frequently - we may need to adjust our log policy to suit.|
|18||List of all relevant logging and data retention policies for the service||See above - likely kept forever by John Kalbach|
|19||List of dependencies this service has on other systems or services, including name, description of systems or services and contact information for service owners of the dependencies|
Kerberos, TNS DNS, vSphere for the VMs
-No Exchange or other stuff is using AD, DNS is completely empty, load on the systems is very small
|20||Any unusual hardware requirements for the service (fiber SAN connection, solid state disks, threading or other CPU requirements, large memory requirements, large disk requirements, large number of spindles (fast disk), no VMs, etc.)||4GB of RAM, 60GB HDDs, 1 proc|
|21||Relevant filesystem paths for configuration files, storage, application installation, init scripts, maintenance scripts||Windows standard stuff, sysvol|
|22||List of all service principals required to run the service and their ownership information. Include any usernames and passwords in the password safe entry for the service.|
Realm trust wizard for a one-way trust with Kerberos.
They have how-tos on getting a direct trust.
|23||List of any interfaces to the service|
|24||Required administrative credentials necessary to maintain, modify or install the service via any interface (*nix command line, Windows shell, web app, etc.)||NA|
|25||Names, descriptions, and procedures related to any software roles or groups needed to maintain or interact with the service|
|26||Any relevant technical documentation not explicitly listed above, including things like LDAP bind paths, schemas, OIDs; Systems that provision a service automatically or manually; list of custom interfaces to the service; etc.|
|27||List the technical details, procedures and order of execution involved with backups and restoration of the service during a DR scenario||Backups to TSM - one DC does a backup to TSM.|
|28||List any specific order in which servers or services need to be shut down and started up||Never had the service go completely out, this should be in the DR document - Phil will be looking at MS' best practices when he re-writes the DR document.|
|29||URLs, usernames and passwords, certificates, or UMG names associated with any source control for source code related to any PSU-developed code related to the service||NA|
|30||Information on any development, test, acceptance or other environments used for testing changes to the service||A test environment, testaccess.psu.edu. Access to that is on a per-request basis. Minimum amount of user accounts from years ago were created and sync'ed into the separate test Kerberos . The layout has diverged from prod significantly as far as OU layout, people are getting rid of Exchange, many access accounts not in there. Two test Kerberos realms: testk5 and the dev realm krb5qa.aitstest.psu.edu. That Kerb can be up or down depending on if they're doing a new build of Kerb to production. That realm follows prod version. Current prod build of Kerberos is over 5 years old. QA kerb is new because they're been trying to update prod kerberos for 3 years. New Kerberos build solves a Windows 8 authentication issue that currently exists in the Access domain.|
|31||Standard operating procedures and/or checklists related to testing, integration, communication and release of changes to the service||No current standard procedures. No time to do because only one admin.|