Error opening remote console after Configuration Manger in place OS upgrade

Quick post about issues connecting to site with remote console after upgrading the server OS from Windows 2008 R2 to Windows Server 2012 R2

After the upgrade all remote console were unable to connect to the site

console-error

After reviewing the SMSAdminUI.log  on one of the remote devices, there were access denied errors.

\r\nSystem.Management.ManagementException\r\nAccess denied \r\n   at System.Management.ManagementException.ThrowWithExtendedInfo(ManagementStatus errorCode)
:System.Management.ManagementException\r\nAccess denied \r\n   at System.Management.ManagementException.ThrowWithExtendedInfo(ManagementStatus errorCode)

I granted one user local admin rights to the SCCM server, which was hosting the SMS Provider and the console access was restored.

I reviewed the DCOM permissions which were fine, however, the WMI permissions for the SMS Admins group were missing!

I check the WMI permissions using wmimgmt.msc and added the required permissions

wmi.PNG

and

wmi1.PNG

I removed the local admin permissions and tested again. Console access was working again

More detail

In order to access a local or remote SMS Administrator console, users must be members of the SMS Admins local group. The SMS Admins group is explicitly granted Enable Account and Remote Enable on the Root\SMS namespace. The SMS Admins group provides its members with access to the SMS Provider, through WMI. Add Users to the SMS Admins group when they need to access the SMS Administrator console, but do not have to be Local Administrators

Console access troubleshooting

https://support.microsoft.com/en-us/kb/317872

 

Configuration Manager Assessment

Things to consider when doing an Configuration Manager assessment:

Network environment:

What does your network topology look like?

What are the  WAN link speeds, network latency and bandwidth?

What network infrastructure and security devices are in place?

What ports and protocols are allowed through these devices?

What are the usage patterns for network resources, network utilization?

At which locations will you provide services?

What client systems are at each location?

How many devices are to be managed in total?

What users are at each location?

How do external/remote users connect to the corporate network?

Is support for 802.1X authentication required?

Is there a PKI implementation?

Is a POC environment available?

How are change requests handled?

Active Directory environment:

Do you have multiple AD forests?

What are the trust relationship if so?

Will you support computers in workgroups?

SCCM Operators

Who will use the system?

What tasks will they be required to do?

Server and Data Center infrastructure:

Is server infrastructure centralized in a few large data centers or is it distributed?

Are some data centers better connected than are others?

What are the hardware standards?

Is virtualization preferred?

Installed client base and hardware refresh cycle:

What is the hardware and operating system (OS) mix for the installed PC base?

How are new systems imaged?

Is PXE booting  to install images required?

What mobile devices are in use?

Is there a need to support embedded systems?

How often are systems replaced?

Are users allowed to bring their own systems?

Is there a planned OS upgrade?

Who are you deploying Windows to or for?

What OS are you deploying?

What applications do you want to deploy with the OS?

Will this be different for different user or system roles?

To which hardware models are you deploying the OS?

Will you support mobile device client such as smartphones, or Internet-only clients?

Will App-V be enabled for deploying virtualised applications?

How are clients being updated with Windows patches?

Are third party update patches required?

Existing SQL Server deployment:

Will you be using existing SQL servers?

Do these systems meet ConfigMgr requirements?

Are SQL servers clustered?

Are SQL reporting services deployed?

Storage and backup infrastructure:

What storage technologies are in use?

How is data replicated between storage systems?

Details of the design such as optimum server placement, hardware configuration,

and client installation methods depend on the IT infrastructure and services you

have in place.

How to verify if site maintenance tasks are running successfully

If you would like to know if the site maintenance tasks are running successfully you can either check the site status for errors or use the below SQL query

select *,
floor(DATEDIFF(ss,laststarttime,lastcompletiontime)/3600) as Hours,
floor(DATEDIFF(ss,laststarttime,lastcompletiontime)/60)- floor(DATEDIFF(ss,laststarttime,lastcompletiontime)/3600)*60 as Minutes,
floor(DATEDIFF(ss,laststarttime,lastcompletiontime))- floor(DATEDIFF(ss,laststarttime,lastcompletiontime)/60)*60 as TotalSeconds
from SQLTaskStatus

Completion status – 0 = Success

SMT

Clients not communicating with MP after in place OS upgrade – Configuration Manger 1602

After in place OS upgrade Windows 2008 R2 to Windows 2012 R2 the Configuration Manager MP and clients communication was not working.

A site reset did not resolve this issue

Useful errors 

During the OS upgrade the SMS folder in the cert store gets removed until the ConfigMgr services are restarted and then its gets recreated. After time some of the certs com back but not them all.

This is how the cert store looked before the OS upgrade. The Site server, Site system identification and SMS Provider certs were recreated, but the other were not.

certs

MP logs

MP_Policy.log

CHandlePolicyAssignmentRequest::CreatePolicyRequestStagingFile: cannot create or find policy request file        MP_PolicyManager        7/12/2016 10:08:22 PM        8772 (0x2244)

CHandlePolicyAssignmentRequest::Execute(): CreatePolicyRequestStagingFile() failed with error: 0x80070020.

MP_PolicyManager        7/12/2016 10:08:22 PM        8772 (0x2244)

CPolicyManagerHandler::HandleMessage(): SetComplete(DISCARD) called.        MP_PolicyManager        7/12/2016 10:08:22 PM        8772 (0x2244)

 Client logs

 Locationservices.log

Failed to verify message. Sending MP [MP server] not in cached MPLIST.        LocationServices        12/07/2016 13:10:38        24440 (0x5F78)

MPLIST requests are throttled for 00:26:05        LocationServices        12/07/2016 13:10:38        24440 (0x5F78)

Failed to send web service info Location Request Message        LocationServices        12/07/2016 13:10:38        18512 (0x4850)

Failed to verify message. Could not retrieve certificate from MPCERT.        LocationServices        12/07/2016 13:10:38        24440 (0x5F78)

MPCERT requests are throttled for 00:04:50        LocationServices        12/07/2016 13:10:38        24440 (0x5F78)

Failed to verify message. Sending MP [MP server] not in cached MPLIST.        LocationServices        12/07/2016 13:10:38        24440 (0x5F78)

MPLIST requests are throttled for 00:26:05        LocationServices        12/07/2016 13:10:38        24440 (0x5F78)

Failed to send web service info Location Request Message        LocationServices        12/07/2016 13:10:38        18512 (0x4850)

LocationServices::CCMVerifyServiceSignature: Unable to refresh Web Service MP server certificate        LocationServices        12/07/2016 13:10:38        18512 (0x4850)

LocationServices::VerifyDataSignature: Overall signature verification failed – 0x87D00309; checking if status message should be sent.        LocationServices        12/07/2016 13:10:38        18512 (0x4850)

Failed to verify message. Could not retrieve certificate from MPCERT.        LocationServices        12/07/2016 13:11:19        24232 (0x5EA8)

Attempting to refresh certificate information from AD LocationServices 12/07/2016 13:41:19 17684 (0x4514)
Failed to refresh certificate information from AD LocationServices 12/07/2016 13:41:19 17684 (0x4514)
Failed to verify Certificate with error 0x80070057. LocationServices 12/07/2016 13:41:19 17684 (0x4514)

Cause

After the OS upgrade some of the Site server certs were missing !

SMS Signing Certificate

SMS SMP Encryption Certificate

SMS Encryption Certificate

Resolution

Restore the certs from backup, reboot the server and wait for 1-2 hours !🙂

You should see the client side locationservices log

Attempting to refresh certificate information from AD LocationServices 12/07/2016 14:01:19 22964 (0x59B4)
Refreshed Certificate Information from AD LocationServices 12/07/2016 14:01:19 22964 (0x59B4)
Retrieved thumbprints from AD LocationServices 12/07/2016 14:01:19 22964 (0x59B4)
Successfully stored new Site Server Signing Certificate… LocationServices 12/07/2016 14:01:20 20400 (0x4FB0)
Name : Site Server LocationServices 12/07/2016 14:01:20 20400 (0x4FB0)
Thumbprint : D9B1FFFA03C94CB5BAE4A94E16D8A4E523826918 LocationServices 12/07/2016 14:01:20 20400 (0x4FB0)
Valid From: 2016-07-05, 15:21 LocationServices 12/07/2016 14:01:20 20400 (0x4FB0)
Valid To : 2116-06-12, 15:21 LocationServices 12/07/2016 14:01:20 20400 (0x4FB0)
Refreshed Site Signing Certificate over AD LocationServices 12/07/2016 14:01:20 20400 (0x4FB0)