Monitoring OpsMgr workgroup clients - Part 1: Installing and configuring the Root CA

May 18, 2015, 10:39 am

≫ Next: Monitoring OpsMgr workgroup clients - Part 2: Installing certificates and final configuration

≪ Previous: Operations Manager news from Ignite 2015

~ Irfan Rabbani | Senior Support Escalation Engineer

This article is the first in a series of posts on how to monitor System Center 2012 R2 Operations Manager clients that are not members of your Active Directory domain. The series will be broken out into three parts:

Part 1: Installing the Microsoft Certificate Authority Server for Operations Manager certificate based authentication

Part 2: Installing certificates and final configuration

Part 3: Installing and configuring a gateway

To begin part 1, we first need to look at how clients outside the domain will authenticate to our Operations Manager infrastructure. System Center Operations Manager 2007 & System Center 2012 Operations Manager use mutual authentication to communication with the agents. First the agent will try to communicate with Kerberos and when this is not possible, certificates will be used for the secure communication. If you happen to have agents that lie outside of your domain, such as in a DMZ, you’ll need to use certificates for agent to server communication.

If you already have an Enterprise Root CA then you may not need to install a new one, however be sure to check out the second part below where we configure the certificate template and make it available to clients.

Step 1: Install the Active Directory Certification Authority Role

1. Open Server Manager, then click Add Roles and Features. Click Next, then click Active Directory Certificate Services. Click Next two times to get to the screen below. Here you will select the server where you want to install the role. Once selected, click Next.

2. On the Select Role Services page, click Active Directory Certification Authority. Click Next three times.

3. Choose the following Features:

Certificate Authority
Certificate Enrollment Web Service
Certificate Authority Web Enrollment

Click Next three times.

4. On the last page, check settings and choose Install.

After the installation finishes, restart the computer.

Step 2: Configure the Root CA

Once the Active Directory Certification Authority role is installed, we need to configure it.

1. Open Server Manager and click AD CS

2. On the Configuration Required for AD Certificate Services page, choose More.

3. Choose Configure Active Directory Certificate Service on the destination server:

4. Check the credentials and click Next

5. Select Certificate Authority and Certification Authority Web Enrollment and click Next (we will cover the web enrollment later).

6. Choose Enterprise Root CA, click Next.

7. On the CA Type section choose Root CA and click Next.

8. Choose Create a new private key and click Next.

9. Specify the Certificate Server Cryptographic options (we left them with the default values) and click Next.

10. Fill in the Common name for this CA. I suggest you use a logical name (we used Enterprise-CA). Click Next.

11. Choose the Validity Period and choose Next two times (we left it default).

12. Check the Confirmation page and choose Configure.

13. For “Do you want to configure additional Role Services” choose Yes.

14. Choose Next and now choose the Certificate Enrollment Web Service.

15. Click Next three times and on the Specify the Service account section, choose the service user that is a member of the IIS_IUSRS group (this group is in the Active Directory) and choose Next.

16. Select the Enterprise CA and click Next.

17. Check the Confirmation page and choose Configure.

At this point we are done installing the Enterprise Root CA. Now we are going to make the certificate site secure because it’s necessary for web enrolment.

18. Click on Server Certificates, Create self-signed certificate.

19. Give it a friendly name (in our case we used CA1-Dc1) and click OK.

Step 3: Create the Operations Manager Certificate Template

This section explains how to make an OpsMgr 2012 R2 certificate template in Windows Server 2012.

NOTE In my Lab I installed the Root CA on the Domain Controller.

1. Open Server Manager, click Tools and click Certificate Authority.

2. Select the Enterprise CA, right-click Certificate Templates, then right-click Manage.

3. Right-click on IPSec (Offline request) and select Duplicate Template.

4. Leave the default to Windows Server 2003 and Windows XP/ Server 2003. This way we are always backwards compatible.

5. Go to the General tab and type a logical Template Display name and Template Name (we used OpsMgr Certificate and OpsMgrCertificate) and we changed the validity period to 5 years.

6. Go to the Request Handling tab and check the option to allow the private key to be exported.

7. Go to Cryptography and choose the minimum key size - we selected 2048. This is sufficient and takes less CPU time to process. Also check the Microsoft Enhanced Cryptographic Provider v1.0 button.

8. Go to the tab titled Extensions. Select the option Applications Policies and click Edit. Remove IP security IKE intermediate and add the following policies:

- Client Authentication
- Server Authentication

Click OK.

9. Go to the tab titled Security. Authenticated Users need to have Read& Enroll access. Click Apply and OK. The template is now created.

10. Click Apply and OK, the template is now created.

Step 4: Make the Template Available

Now that we have the template created, it’s time to make it available.

1. Open Server Manager, click Tools, click Certificate Authority, right-click Certificate Templates, New, Certificate Template.

2. Choose OpsMgr Certificate and click OK:

After these steps the OpsMgr Certificate template is displayed in the certificate templates.

Additional Information

Authentication and Data Encryption for Windows Computers: https://technet.microsoft.com/en-us/library/hh212810.aspx

How to Obtain a Certificate Using Windows Server 2008 Enterprise CA in Ops Manager 2007: http://technet.microsoft.com/en-us/library/dd362553.aspx

How to Obtain a Certificate Using Windows Server 2008 Stand-Alone CA in Ops Manager 2007: http://technet.microsoft.com/en-us/library/dd362655.aspx

How to Obtain a Certificate Using Windows Server 2003 Enterprise CA in Ops Manager 2007: http://technet.microsoft.com/en-us/library/bb735413.aspx

How to Obtain a Certificate Using Windows Server 2003 Stand-Alone CA in Ops Manager 2007: http://technet.microsoft.com/en-us/library/bb735417.aspx

That should take care of getting our Enterprise Root CA installed and configured. In our next installment we’ll talk about installing our certificates and completing final configuration in Operations Manager.

Irfan Rabbani| Senior Support Escalation Engineer | Microsoft GBS Management and Security Division

Get the latest System Center news onFacebookandTwitter:

System Center All Up: http://blogs.technet.com/b/systemcenter/

Configuration Manager Support Team blog: http://blogs.technet.com/configurationmgr/
Data Protection Manager Team blog: http://blogs.technet.com/dpm/
Orchestrator Support Team blog: http://blogs.technet.com/b/orchestrator/
Operations Manager Team blog: http://blogs.technet.com/momteam/
Service Manager Team blog: http://blogs.technet.com/b/servicemanager
Virtual Machine Manager Team blog: http://blogs.technet.com/scvmm

Microsoft Intune: http://blogs.technet.com/b/microsoftintune/
WSUS Support Team blog: http://blogs.technet.com/sus/
The RMS blog: http://blogs.technet.com/b/rms/
App-V Team blog: http://blogs.technet.com/appv/
MED-V Team blog: http://blogs.technet.com/medv/
Server App-V Team blog: http://blogs.technet.com/b/serverappv
The Surface Team blog: http://blogs.technet.com/b/surface/
The Application Proxy blog: http://blogs.technet.com/b/applicationproxyblog/

The Forefront Endpoint Protection blog : http://blogs.technet.com/b/clientsecurity/
The Forefront Identity Manager blog : http://blogs.msdn.com/b/ms-identity-support/
The Forefront TMG blog: http://blogs.technet.com/b/isablog/
The Forefront UAG blog: http://blogs.technet.com/b/edgeaccessblog/

OpsMgr 2012 R2

↧

Monitoring OpsMgr workgroup clients - Part 2: Installing certificates and final configuration

May 19, 2015, 10:53 am

≫ Next: Monitoring OpsMgr workgroup clients - Part 3: Installing and configuring a gateway

≪ Previous: Monitoring OpsMgr workgroup clients - Part 1: Installing and configuring the Root CA

~ Irfan Rabbani | Senior Support Escalation Engineer

This article is the second in my series of posts on how to monitor System Center 2012 R2 Operations Manager clients that are not members of your Active Directory domain. The series is broken out into three parts:

Part 1: Installing the Microsoft Certificate Authority Server for Operations Manager certificate based authentication

Part 2: Installing certificates and final configuration

Part 3: Installing and configuring a gateway

Here in part 2 we’ll be installing the Operations Manager certificates, configuring OpsMgr and then setting up monitoring for a workgroup client. Note that this scenario describes how to install an untrusted agent on a Windows Server 2008 R2 X64 computer that resides in an untrusted domain when no gateway is used. Information on the use of gateway servers will be covered in part 3.

Step 1. Open TCP port 5723

You will need to take the necessary steps to open port 5723 from the client to the management server in your environment. What those steps may be will vary depending on your network configuration. For further help with this you may need to contact your local network specialist.

To verify connectivity, go to the client and open the command prompt, then in the command prompt run telnet x.x.x.x 5732 (where x.x.x.x is the IP address of the management server). If everything is working, continue to the next step. If not, be sure that the firewall is open and is passing traffic on port 5723.

Step 2. Install the Root CA certificate on the client

1. From the client, start Internet Explorer.
2. Go to the certificate server website (e.g. http://MyServer/Certsrv)

3. Click on Download CA certificate, certificate chain, or CRL.
4. Click Download CA Certificate chain and save it.
5. Once the certificate is downloaded, open MMC and connect to Local Computer, then load the certificates snap-in (Local Computer).
6. Go to Trusted Root Certification Authorities and right-click, then select all tasks, import and import the Root CA.

Step 3. Install the computer certificate on the client

1. From the client, start Internet Explorer.
2. Go to the certificate server website:

3. Click on Request a certificate, choose Advanced certificate request then click Create and submit a request to this CA.
4. Choose the OpsMgr Certificate template, in the name tab choose the FQDN of the machine and fill in the same name for the friendly name.

5. Click Finish and install the certificate. Once installed, the certificate is stored by default in the personal certificate store, however the OpsMgr certificate need to be in the computer store so our next step is to move it.
6. To move the certificate, click Start, click Run, and run MMC.
7. On the File menu, click Add/Remove Snap-in then click Add and click Certificates, and then click Add, select Personal Store and Computer account, then click Finish.
8. Export the certificate from the personal store and import it to the local computer store (no drag and drop here).
9. Remove the certificate from the local user store.
10. Make sure that both the client and the server are reachable via hostname (just ping). If it’s not working add the computers in DNS or in the Host file (C:\Windows\System32\Drivers\ETC\Host).

Step 4. Install the Root CA Certificate on the Management Server

1. From the client, start Internet Explorer.
2. Go to the certificate server website:

3. Click on Download CA certificate, certificate chain, or CRL.
4. Click Download CA Certificate chain and save it.
5. Once the certificate is downloaded, open MMC and connect to Local Computer, then load the certificates snap-in (Local Computer).
6. Go to Trusted Root Certification Authorities, right-click, select all tasks, import and import the Root CA.

Step 5. Install the Computer Certificate on the Management Server

1. From the client, start Internet Explorer.
2. Go to the certificate server website:

3. Click on Request a certificate, choose Advanced certificate request, then click Create and submit a request to this CA.
4. Choose the OpsMgr Certificate template, then in the name tab choose the FQDN name of the machine and fill in the same name for the friendly name.
5. Click Finish and install the certificate. The certificate will now be stored in the personal certificate store, however the OpsMgr certificate actually needs to be in the computer store so we now must move it.
6. To move the certificate, click Start, click Run, and run MMC.
7. On the File menu, click Add/Remove Snap-in then click Add and click Certificates, and then click Add, select Personal Store and Computer account, then click Finish.
8. Export the certificate from the personal store and import it to the local computer store (no drag and drop here).
9. Remove the certificate from the local user store.
10. Make sure that both the client and the server are reachable via hostname (just ping). If it’s not working add the computers in DNS or in the Host file (C:\Windows\System32\Drivers\ETC\Host).

Step 6. Configure Operations Manager for Manually Installed Clients

1. First we need to make sure new agents are not rejected. To do this, go to the Operations Manager console –> Administration –> Settings –> Security and select Review new manual agent installations in pending management view.

2. Now we need to import the certificate on the management server. Go to the support tools folder and run MOMCertimport.exe.

3. Select the imported certificate and click OK. Make sure this completes successfully.

Step 7. Manually Install Agents on Clients and Run MOMCertImport

1. From the client, go to the OpsMgr agent installation folder. By default this is \\DISK\Program Files\System Center 2012\Operations Manager\Server\AgentManagement\ in the AMD64 or i386 folder. If not available you can simply copy this to the client. Run Momagent.MSI to install the agent.

2. Fill in the proper settings for the monitoring group (I used the settings below).

3. I prefer using the Local System account. Choose Next and Install.
4. If necessary, update the agent with the required updates.

5. Copy the Support Tools directory from the OpsMgr installation media to the client.

6. On the client machine, open a Command Prompt (Run As Administrator)
7. Go to the copied support tools directory and run MOMCertimport.exe:

8. Select the imported certificate and click OK.
9. Make sure the import was successful:

Step 8. Approve Agents in the Operations Manager Console

NOTE Be aware that it can take a while before the client shows up in the console.

1. Open theOperations Manager console and go to Administration –> Pending Management.
2.Right-clickthe client and select Approve.

That’s it! Your client is now monitored.

Troubleshooting

A gateway server or client without the proper certificate that tries to communicate with a management server using Kerberos from a non-trusted realm will log the following:

Log Name: Operations Manager
Source: OpsMgr Connector
Event ID: 21007
Task Category: None
Level: Error
Keywords: Classic
User: N/A
Computer: SERVER
Description:
The OpsMgr Connector cannot create a mutually authenticated connection to SERVER.DOMAIN.COM because it is not in a trusted domain.

=====

A gateway server or client agent using a certificate to communicate with a management server will log the following if the root certificate is not installed:

Log Name:      Operations Manager
Source:        OpsMgr Connector
Event ID:      21036
Task Category: None
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      OM12GW
Description:
The certificate specified in the registry at HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft Operations Manager\3.0\Machine Settings cannot be used for authentication. The error is The credentials supplied to the package were not recognized(0x8009030D).

Log Name:      Operations Manager
Source:        OpsMgr Connector
Event ID:      21016
Task Category: None
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      OM12GW
Description:
OpsMgr was unable to set up a communications channel to SERVER.DOMAIN.COM and there are no failover hosts. Communication will resume when SERVER.DOMAIN.COM is available and communication from this computer is allowed.

=====

If the computer certificate does not have a SubjectName that matches the full computer name then you will see the following:

=====

If the Extensions/Enhanced Key usage is not met, you will see the following event indicating that the certificate does not have Server Authentication/Client Authentication enabled.

Log Name:      Operations Manager
Source:        OpsMgr Connector
Event ID:      20050
Task Category: None
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      OM12GW.msft.com
Description:
The specified certificate could not be loaded because the Enhanced Key Usage specified does not meet OpsMgr requirements.   The certificate must have the following usage types:
Server Authentication (1.3.6.1.5.5.7.3.1)
Client Authentication (1.3.6.1.5.5.7.3.2)

=====

If the Root CA is different on the gateway server or client versus the management server, meaning the gateway or client got its certificate from a different root chain PKI than the management server, you will see the following.

On the gateway or client:

Log Name:      Operations Manager
Source:        OpsMgr Connector
Date:          4/9/2015 5:38:54 PM
Event ID:      20071
Task Category: None
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      OM12GW.msft.com
Description:
The OpsMgr Connector connected to 2012OpsMgrR2.MSFT.com, but the connection was closed immediately without authentication taking place. The most likely cause of this error is a failure to authenticate either this agent or the server . Check the event log on the server and on the agent for events which indicate a failure to authenticate.

Log Name:      Operations Manager
Source:        OpsMgr Connector
Date:          4/9/2015 5:38:52 PM
Event ID:      20070
Task Category: None
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      OM12GW.msft.com
Description:
The OpsMgr Connector connected to 2012OpsMgrR2.MSFT.com, but the connection was closed immediately after authentication occurred. The most likely cause of this error is that the agent is not authorized to communicate with the server, or the server has not received configuration. Check the event log on the server for the presence of 20000 events, indicating that agents which are not approved are attempting to connect.

Log Name:      Operations Manager
Source:        OpsMgr Connector
Date:          4/9/2015 5:38:55 PM
Event ID:      21016
Task Category: None
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      OM12GW.msft.com
Description:
OpsMgr was unable to set up a communications channel to 2012OpsMgrR2.MSFT.com and there are no failover hosts. Communication will resume when 2012OpsMgrR2.MSFT.com is available and communication from this computer is allowed.

On the management server:

Log Name:      Operations Manager
Source:        OpsMgr Connector
Date:          4/9/2015 3:36:07 PM
Event ID:      21002
Task Category: None
Level:         Warning
Keywords:      Classic
User:          N/A
Computer:      2012OpsMgrR2.MSFT.com
Description:
The OpsMgr Connector could not accept a connection from 192.168.2.54:51294 because mutual authentication failed.

Log Name:      Operations Manager
Source:        OpsMgr Connector
Date:          4/9/2015 3:36:07 PM
Event ID:      20067
Task Category: None
Level:         Warning
Keywords:      Classic
User:          N/A
Computer:      2012OpsMgrR2.MSFT.com
Description:
A device at IP 192.168.2.54:51294 attempted to connect but the certificate presented by the device was invalid. The connection from the device has been rejected. The failure code on the certificate was 0x800B010A (A certificate chain could not be built to a trusted root authority.).

Additional Information

How to Obtain a Certificate Using Windows Server 2008 Enterprise CA in Ops Manager 2007: http://technet.microsoft.com/en-us/library/dd362553.aspx

How to Obtain a Certificate Using Windows Server 2008 Stand-Alone CA in Ops Manager 2007: http://technet.microsoft.com/en-us/library/dd362655.aspx

How to Obtain a Certificate Using Windows Server 2003 Enterprise CA in Ops Manager 2007: http://technet.microsoft.com/en-us/library/bb735413.aspx

How to Obtain a Certificate Using Windows Server 2003 Stand-Alone CA in Ops Manager 2007: http://technet.microsoft.com/en-us/library/bb735417.aspx

Authentication and Data Encryption for Windows Computers: https://technet.microsoft.com/en-us/library/hh212810.aspx

That should take care of getting our certificates installed and Operations Manager configured for these untrusted clients. In my next installment I’ll talk about installing a gateway for use in larger environments.

Irfan Rabbani| Senior Support Escalation Engineer | Microsoft GBS Management and Security Division

Get the latest System Center news onFacebookandTwitter:

System Center All Up: http://blogs.technet.com/b/systemcenter/

OpsMgr 2012 R2

↧

Monitoring OpsMgr workgroup clients - Part 3: Installing and configuring a gateway

May 20, 2015, 8:58 am

≫ Next: KB: "Error Code 800706D3" occurs when you perform an Agent push installation to Windows Server 2012 computers

≪ Previous: Monitoring OpsMgr workgroup clients - Part 2: Installing certificates and final configuration

~ Irfan Rabbani | Senior Support Escalation Engineer

This article is the third in a series of posts on how to monitor System Center 2012 R2 Operations Manager clients that are not members of your Active Directory domain. The series is broken out into three parts:

Part 1: Installing the Microsoft Certificate Authority Server for Operations Manager certificate based authentication

Part 2: Installing certificates and final configuration

Part 3: Installing and configuring a gateway

In part 1 we implemented a certificate authority, then in part 2 we went through acquiring and configuring the necessary certificates and how to approve the untrusted clients. Now in part 3 we’ll be looking at the installation and configuration of the gateway server. Note that use a gateway server is optional. Everything necessary to monitor an untrusted client is included in parts 1 and 2, however a gateway can be beneficial in larger environments. Here’s a little detail to explain why.

Why use a gateway

Operations Manager requires that mutual authentication be performed between agents and management servers prior to the exchange of information between them. To secure the authentication process between the two, the process is encrypted. When the agent and the management server reside in the same Active Directory domain or in Active Directory domains that have established trust relationships, they make use of Kerberos V5 authentication mechanisms provided by Active Directory. When the agents and management servers do not lie within the same trust boundary, other mechanisms must be used to satisfy the secure mutual authentication requirement.

In Operations Manager, this is accomplished through the use of certificates issued for each computer. If there are many agent-monitored computers, this results in high administrative overhead for managing all those certificates. In addition, if there is a firewall between the agents and management servers, multiple authorized endpoints must be defined and maintained in the firewall rules to allow communication between them.

To reduce this administrative overhead, Operations Manager has a server role called the gateway server. Gateway servers are located within the trust boundary of the agents and can participate in the mandatory mutual authentication. Because they lie within the same trust boundary as the agents, the Kerberos V5 protocol for Active Directory is used between the agents and the gateway server. Each agent then communicates only with the gateway servers that it is aware of. The gateway servers communicate with the management servers.

To support the mandatory secure mutual authentication between the gateway servers and the management servers, certificates must be issued and installed, but only for the gateway and management servers. This reduces the number of certificates required, and in the case of an intervening firewall it also reduces the number of authorized endpoints to be defined in the firewall rules. The following illustration shows the authentication relationships in a management group using a gateway server.

Authentication with gateway server

Step 1: Register the Operations Manager Gateway

NOTE As a best practice, never install the gateway role and configure it with its machine certificate before the gateway server role registration process. Why? Because you can end up having the Gateway Health service accidently approved as an agent if manual agent approval is configured to Always Approve in the Operations Manager console, as shown here:

1. Copy Microsoft.EnterpriseManagement.GatewayApprovalTool.exe (pick the AMD64 folder if you are going to run it on a 64-bit machine) and Microsoft.EnterpriseManagement.GatewayApprovalTool.exe.CONFIG from the Support Tools directory on your installation media to the installation folder of your OpsMgr installation. In my case that’s C:\Program Files\System Center 2012\Operations Manager\Setup.

2. Approve the gateway server by opening a Command Prompt and running the following command:

Microsoft.EnterpriseManagement.gatewayApprovalTool.exe /ManagementServerName=<managementserverFQDN> /GatewayName=<GatewayFQDN> /Action=Create

In the example below, the name of gateway server OM12GW and the name of the management server to which the gateway reports is named Om12MS.DRH.INT.

If the approval is successful, you will see the approval of server <GatewayFQDN> completed successfully.

Troubleshooting Tips

The Microsoft.EnterpriseManagement.GatewayApprovalTool.exe command should only take about 15 seconds to run even on the slowest of servers, so if this takes significantly longer then you probably have an authentication issue somewhere. If you find yourself in this situation, here are some things to check:

1. Make sure that the user running Microsoft.EnterpriseManagement.GatewayApprovalTool.exe is doing it from one of the management servers, preferable the RMS in the case of Operations Manager 2007 R2, and from any management server in OpsMgr 2012 or OpsMgr 2012 R2.

2. Microsoft.EnterpriseManagement.GatewayApprovalTool.exe must be executed by a user account that has rights to the Operations Manager Database. Because of this I strongly recommend the use of the domain user account that is running your System Center Data Access Service, also known as the OMSDK service. If you are using a local system account to run the OMSDK service then use any account that has DB owner permissions on Operations Manager DB.

Step 2: Install the Operations Manager Gateway

Now we need to install the gateway by clicking the Gateway management server link in the setup splash screen. Below is a quick overview of the installation.

Give the management group a name. This can be found in the title bar of the console on the management server. Give the management server a name as well. The port number can be changed if desired. Only this one port needs to be open on the firewall which is the big advantage of using a gateway server!

Now we need to copy the MOMCertImport.exe tool to the gateway server and put it in the gateway installation path. We will use it later. In my case, this is C:\Program Files\System Center Operations Manager\Gateway.

Step 3: Run MOMCertImport.exe on the Gateway Server

To install our certificate, run the following command:

MOMCertImport.exe /SubjectName <gatewayServerName>

You should see a message similar to the one below.

You should also see an Event ID 20053 logged:

You’ll probably also see an Event ID 21007 similar to the one below:

OK, so at this point the gateway is registered in the Operations Manager DB and the role is installed, but it’s not communicating with my management server. Why?

This is because the gateway server is in workgroup or in domain that is different than Operations Manager management server domain and there is no trust, meaning we must configure the right certificate based authentication mechanism.

Wait a minute. What do you mean by ….Right…?

You can create different types of certificates with different intended purposes and import them in the gateway machine, but only the right certificate with the required authentication mechanism can work for gateway to agent or gateway to management server certificate based authentication.

From the gateway server (the one that is not in the domain) you don’t trust the Enterprise CA by default. That’s why you first have to get and install the Root CA certificate from the AD CS role.

Step 4: Import the Trusted Root Certificate

Here is where we begin configuring the required certificates. In order to configure certificates for this type of authentication we will need to do the following:

1. Import the trusted root certificate on the gateway server, the untrusted client computers and the management servers.
2. Create and install certificates (client, server) for the OpsMgr servers.
3. Create and install certificates (client, server) for the untrusted computers.
4. Export the certificate (client, server).
5. Allow manual agent installation.
6. Complete manual client agent installation.
7. Run MOMcertimport.exe on the computers where the certificates are imported.
9. Approve the agents once they successfully communicate with the management server via certificates.

IMPORTANT The trusted root certificate must be imported on all participating computers along with a computer specific certificate that will be used in certificate based authentication (e.g. agent and gateway, or gateway and management server, or agent and management server). Each computer will have one trusted root certificate and one machine specific certificate issued to the respective computer name.

By default, the trusted root certificate probably does not exist in our store. This can be verified by doing the following:

If the trusted root certificate is not here, the certificate from the Root CA needs to be added in this list.

To begin, open a web browser on the gateway server and go to the CA web service. My Certificate Authority server name is OM12DC1 so I go to the certsrv site on my Certificate Authority server which looks like this: http://OM12DC1/certsrv.

Add the certsrv website to Trusted Sites by going to Internet Options and under Security choose Trusted Sites click on Sites to add this site.

Since the certsrv website uses ActiveX, change the security settings of Trusted Sites so that ActiveX is allowed. In my case I go to http://OM12DC1/certsrv (or http://192.168.1.170/Certsrv since my CA server IP address is 192.168.1.170).

Here we need to request the CA chain:

If you don’t see these 2 popups, you need to enable ActiveX first.

Now import the Root Chain Certificate in the gateway server computer store.

The certificate is in the list now, meaning our gateway server will trust certificates issued by the Enterprise Root CA.

Now we need to request a computer certificate for our gateway server. This certificate will be used by the Health service on gateway server to communicate with other machines (e.g. management servers or other agents that use certificate based authentication).

Step 5: Request a Computer Certificate for the Gateway

Open a web browser on the gateway server and go to the CA web service. Select Request a certificate:

Select advanced certificate request:

Select Create and submit a request to this CA:

Select the template that we created earlier and fill in the Name and Friendly Name fields with the FQDN of the gateway server. Since mine is in a workgroup the NetBIOS name is sufficient.

NOTE If you are running a gateway server in a domain, be aware that you must chose the exact full name of the gateway server as it appears in System Properties.

Now the certificate is generated and we can install it.

But wait a minute…installed where???

We need to authenticate computers, and the certificate is imported in the user personal certificate store.

So now we need to open the Certificates MMC and export the certificate from user personal store “With private keys attached to certificate” and then import it into the computer personal certificate store.

Once complete, the certificate is now installed and you can verify that everything is installed correctly by opening the certificate and verifying that the certification path is OK.

Certificate Check List

You can use this checklist when importing certificates on a gateway server:

1. Make sure that the certificate is in the computer store, not the user store.

2. Check that the certification path is OK as per the screen shot below:

3. Always check that private keys are associated in the certificate.

4. Always make sure that the certificate Intended Purpose column shows that it’s for Server Authentication, Client Authentication. This is as we configured it in the template.

5. Always Run Momcertimport.exe (either 32-bit or 64-bit as appropriate) on the gateway, agent or management server where the certificate is installed.

This must be done on all servers where a computer specific certificate is imported in the MMC.

Also make sure that the exe you use is of the same version (32-bit vs 64-bit) and also make sure the file is from the same version of the OpsMgr server\agent you are running on the system:

1. Run a Command Prompt as an Administrator and navigate to the Operations Manager installation source folder.

2. Run MOMcertimport.exe and select the computer certificate you imported on this machine.

3. The certificate is now imported on this system.

4. Restart “OpsMgr Health Service” on the server.

If you get Event ID 21006, make sure the firewalls on the gateway and/or the management server are not blocking communication.

Lastly, don‘t forget to enable Agent Proxy for the gateway as it will act as a proxy for other systems connecting through the gateway server!

Step 6: Allow Manual Agent Installation

Before the first manual agent installation, the Operations Manager global setting must be changed from Reject new manual agent installations to Review new manual agent installation in pending management view in the Operations Manager console:

1. Open the Operations Manager console with administrator privileges.

2. Navigate to Administration –> Settings –> Server.

3. In the right-hand pane click Security.

4. On the General tab, select “Review new manual agent installation in pending management view” and click “OK” to finish.

Step 7: Manual OpsMgr 2012 Agent Installation

1. Logon to the server with administrator privileges.

2. From the Operations Manager installation media, double-click the SetupOM.exe file.

3. On the Start page, select Install Operations Manager 2012 Agent.

4. On the Welcome page click Next.

5. On the Destination Folder page leave the installation folder set to the default and click Next.

6. On the Management Group Configuration page leave the Specify Management Group information check box selected and click Next.

7. On the Management Group Configuration page, do the following:

- Type the Management Group Name
- Type the Management Server name. This can be the name of your gateway server if the agent uses one.
- Leave the default port of 5273.
- Click Next.

8. When the Agent Action Account page appears, leave it set to the default of Local System and then click Next.

9. On the Ready to Install page, review the settings and then click Install to display the Installing Systems Center Operations Manager Agent page.

10. When the Completing the Systems Center Operations Manager Agent Setup Wizard page appears, click Finish.

Check List:

1. Make sure the agent can Telnet to its management server or its gateway server using port 5723 (whichever is chosen during installation setup for this agent).

2. If you cannot Telnet to port 5723, check firewalls on the management/gateway server as well as the agent computer. Usually this occurs when the Health service on the management server or gateway is not running or communication is blocked.

3. If Telnet is successful then follow the same steps mentioned above and import the chain root certificate in the computer store of the agent and also request a computer name based certificate for this respective agent as well.

4. Make sure this agent certificate includes all features such as Private keys, Server authentication, Client Authentication. It must have Successful Chain Cert as per the certificate properties.

5. Remember that you must execute momcertimport to register the machine name specific certificate on this agent followed by Health service restart.

Additional Information

How to Deploy a Gateway Server: https://technet.microsoft.com/en-us/library/hh456445.aspx

How to Chain Gateways: https://technet.microsoft.com/en-us/library/hh456427.aspx

Authentication and Data Encryption for Windows Computers: https://technet.microsoft.com/en-us/library/hh212810.aspx

Irfan Rabbani| Senior Support Escalation Engineer | Microsoft GBS Management and Security Division

Get the latest System Center news onFacebookandTwitter:

System Center All Up: http://blogs.technet.com/b/systemcenter/

OpsMgr 2012 R2

↧

KB: "Error Code 800706D3" occurs when you perform an Agent push installation to Windows Server 2012 computers

May 20, 2015, 1:02 pm

≫ Next: Operations Manager Customer Feedback Survey ends this Friday (29th May)

≪ Previous: Monitoring OpsMgr workgroup clients - Part 3: Installing and configuring a gateway

When you push a Microsoft System Center 2012 R2 Operations Manager (OpsMgr 2012 R2) agent from a Windows Server 2012 based management server to a Windows Server 2012 based agent server, you receive the following error message:

The Operations Manager Server failed to open service control manager on computer server2012agent.contoso.local. Therefore, the Server cannot complete configuration of agent on the computer.
Operation: Agent Install
Install account: contoso\scom_admin
Error Code: 800706D3
Error Description: The authentication service is unknown.

Additionally, when you try to connect to the Services console of the agent, the connection is unsuccessful and you receive the following error message:

Windows was unable to open service control manager database on server2012agent.Error 1747: The authentication service is unknown.

For complete details as well as a resolution, please see the following:

KB3060495 - "Error Code 800706D3" occurs when you perform an Agent push installation to Windows Server 2012-based servers (https://support.microsoft.com/en-us/kb/3060495/)

J.C. Hornbeck| Solution Asset PM | Microsoft GBS Management and Security Division

Get the latest System Center news onFacebookandTwitter:

Main System Center blog: http://blogs.technet.com/b/systemcenter/

Configuration Manager Support Team blog: http://blogs.technet.com/configurationmgr/
Data Protection Manager Team blog: http://blogs.technet.com/dpm/
Orchestrator Team blog: http://blogs.technet.com/b/orchestrator/
Operations Manager Team blog: http://blogs.technet.com/momteam/
Service Manager Team blog: http://blogs.technet.com/b/servicemanager
Virtual Machine Manager Team blog: http://blogs.technet.com/scvmm

Microsoft Intune: http://blogs.technet.com/b/microsoftintune/
WSUS Support Team blog: http://blogs.technet.com/sus/
RMS blog: http://blogs.technet.com/b/rms/
App-V Team blog: http://blogs.technet.com/appv/
MED-V Team blog: http://blogs.technet.com/medv/
Server App-V Team blog: http://blogs.technet.com/b/serverappv

Forefront Endpoint Protection blog: http://blogs.technet.com/b/clientsecurity/
Forefront Identity Manager blog: http://blogs.msdn.com/b/ms-identity-support/
Forefront TMG blog: http://blogs.technet.com/b/isablog/
Forefront UAG blog: http://blogs.technet.com/b/edgeaccessblog/
Application Proxy blog: http://blogs.technet.com/b/applicationproxyblog/
The Surface Team blog: http://blogs.technet.com/b/surface/

OpsMgr 2012 R2

↧

Operations Manager Customer Feedback Survey ends this Friday (29th May)

May 27, 2015, 3:42 am

≫ Next: Support Tip: Pushing an Operations Manager agent fails with Error Code: 800706BA - "RPC server is unavailable"

≪ Previous: KB: "Error Code 800706D3" occurs when you perform an Agent push installation to Windows Server 2012 computers

The last date for the Operations manager customer feedback survey is 29th May. Please take some time to offer your feedback. Your feedback is very important to us.

If you have any problems responding to the survey, please feel free to send me an email.

https://aka.ms/scomcustomer

Suraj Suresh Guptha | Program Manager | Microsoft

Get the latest System Center news on Facebook and Twitter:

System Center All Up: http://blogs.technet.com/b/systemcenter/

↧

Support Tip: Pushing an Operations Manager agent fails with Error Code: 800706BA - "RPC server is unavailable"

June 1, 2015, 10:16 am

≫ Next: Support Tip: Synthetic transaction monitoring in Operations Manager fails with 404 error

≪ Previous: Operations Manager Customer Feedback Survey ends this Friday (29th May)

~ Arun Kumar | Support Escalation Engineer

When you are pushing an agent from an Operations Manager management server, you may encounter the following error:

The Operation Manager Server failed to perform specified operation on computer agent1.contoso.com.
Operation: Agent Install
Install account: contoso\Agent_action
Error Code: 800706BA
Error Description: The RPC server is unavailable

If you encounter this issue, it’s possible that either nonstandard ephemeral ports are being used, or that the ports are blocked at a firewall. For example, if nonstandard high range RPC ports have been configured, a network trace while pushing the agent will show a successful connection to RPC port 135 followed by a connection attempt using a nonstandard RPC port such as 15595 as shown below.

18748 MS Agent TCP TCP: Flags=CE....S., SrcPort=52457, DstPort=15595, PayloadLen=0, Seq=1704157139, Ack=0, Win=8192
18750 MS Agent TCP TCP:[SynReTransmit #18748] Flags=CE....S., SrcPort=52457, DstPort=15595, PayloadLen=0, Seq=1704157139, Ack=0,
18751 MS Agent TCP TCP:[SynReTransmit #18748] Flags=......S., SrcPort=52457, DstPort=15595, PayloadLen=0, Seq=1704157139, Ack=0, Win=8192

Since the port exemption for this non-standard range was not configured on the firewall, the packets are dropped and the connection is severed.

To give this a little perspective, in Operations Manager, monitoringHost.exe on the management server running the agent install task will connect to the agent's RPC service over TCP port 135, and in turn a secondary RPC connection is created on a standard RPC high range port. The same thing happens with WMI service as well, where monitoringHost.exe first connects to the port that the WMI service is listening on and then subsequently on a RPC high range port for some DCOM operation.

Now we know that in Windows Vista and above the RPC high range ports are 49152-65535 so that’s what we want to look for. To verify whether this is your issue, run the following command to see what RPC high port range is configured:

Netsh int ipv4 show dynamicportrange tcp

As per IANA standards, it should look something like this:

Protocol tcp Dynamic Port Range
---------------------------------
Start Port : 49152
Number of Ports : 16384

If you see a different start port then the problem may be that the firewall is not configured correctly to allow traffic on those ports. You can change the configuration on the firewall or you can run the command below to set the high range ports back to their default values:

Netsh int ipv4 set dynamicport tcp start=49152 num=16383

Note that you can also configure the RPC dynamic port range via the registry. See the following for more information:

154596 - How to configure RPC dynamic port allocation to work with firewalls (https://support.microsoft.com/en-us/kb/154596)

If everything appears to be configured correctly but you still experience the problem, it may be that one of the following conditions are true:

1. DCOM has been restricted to a certain port. To verify, open dcomcnfg.exe and traverse to dcomcnfg -> My Computer –> Properties –> Default Protocols and ensure that there is not custom setting there.

2. WMI is configured to use a custom endpoint. To check if you have a static endpoint configured for WMI, open dcomcnfg.exe and traverse to dcomcnfg -> My Computer –> DCOM Config -> Windows Management and Instrumentation –> Properties -> Endpoint and ensure that there is no custom setting here.

3. The agent computer is running the Exchange 2010 CAS role. The Exchange 2010 Client Access Service changes this port range to 6005 through 65535. The range was expanded to provide sufficient scaling for large deployments. Do not change these port values without fully understanding the consequences of doing such.

Arun Kumar | Support Escalation Engineer | Microsoft GBS Management and Security Division

Get the latest System Center news onFacebookandTwitter:

System Center All Up: http://blogs.technet.com/b/systemcenter/

System Center 2012 Operations Manager System Center 2012 R2 Operations Manager OpsMgr 2012 R2

↧

Support Tip: Synthetic transaction monitoring in Operations Manager fails with 404 error

June 2, 2015, 10:03 am

≫ Next: OpsMgr 2012 R2 now offers additional support for SQL Server 2008 R2 SP3

≪ Previous: Support Tip: Pushing an Operations Manager agent fails with Error Code: 800706BA - "RPC server is unavailable"

~ Arun Kumar | Support Escalation Engineer

You may encounter an issue in Operations Manager where you will be able to record web transactions just fine, but when running the test manually or when OpsMgr runs it per a schedule, it fails with a 404 error on some of the URLs.

This can potentially happen if URL redirection is being used and the URL contains a “#” character. For example, assume that the page on a website is using server side URL redirection and the URL redirection involves a "#" character. In this scenario, any application that uses WinHTTP to access this URL will fail and return an HTTP 404 error. If you investigate this using a fiddler trace you’ll see something similar to the following:

Agents 1st request

GET https://www.contoso.com HTTP/1.1
Connection: Keep-Alive
Accept: */*
Accept-Encoding: GZIP
Accept-Language: en-us
User-Agent: Microsoft Monitoring Agent 7.1.10184.0
Host: server.contoso.com

Server's response

HTTP/1.1 302 Found
Date:
Location: https:// www.contoso.com/en/location-chooser.html#origuri=/en.html
Content-Length: 267
Keep-Alive: timeout=10, max=100
Connection: Keep-Alive
Content-Type: text/html; charset=iso-8859-1

Note that the server is redirecting the client to the URL specified in the location tag in the above response (https://www.contoso.com/en/location-chooser.html#origuri=/en.html).

Agents 2nd request in response to server reply above

GET https:// www.contoso.com/en/locat HTTP/1.1 >>>>>> Note the URL here is truncated
Connection: Keep-Alive
Accept: */*
Accept-Encoding: GZIP
Accept-Language: en-us
User-Agent: Microsoft Monitoring Agent 7.1.10184.0
Host: server.contoso.com

Server Response

HTTP/1.1 404 Not Found >>>>> Since the URL is invalid we get a 404 Not Found
Date:
Last-Modified: Tue, 06 Jun 2015 19:05:05 GMT
Accept-Ranges: bytes
Content-Length: 5745
Keep-Alive: timeout=10, max=99
Connection: Keep-Alive
Content-Type: text/html; charset=utf-8

If you encounter this problem, the resolution is to install the Windows hotfix below:

3047732 - "HTTP 404" error when you access a website with URL redirection in Windows (https://support.microsoft.com/en-us/kb/3047732)

You’ll want to install the hotfix on the watcher node, then reboot the machine and re-run the test. It should now complete as expected.

Arun Kumar | Support Escalation Engineer | Microsoft GBS Management and Security Division

Get the latest System Center news onFacebookandTwitter:

System Center All Up: http://blogs.technet.com/b/systemcenter/

System Center 2012 Operations Manager System Center 2012 R2 Operations Manager OpsMgr 2012 R2

↧

OpsMgr 2012 R2 now offers additional support for SQL Server 2008 R2 SP3

June 5, 2015, 9:50 pm

≫ Next: Understanding your Linux Monitoring Challenges

≪ Previous: Support Tip: Synthetic transaction monitoring in Operations Manager fails with 404 error

We are pleased to announce that System Center 2012 R2 Operations Manager (OpsMgr 2012 R2) now supports SQL Server 2008 R2 SP3 as its database. If you choose to use SQL 2008 R2 SP3, below are a couple known issues and workarounds you’ll want to be aware of.

Issue: Fresh Install of SCOM 2012 R2 on SQL 2008 R2 SP3 will not work
Workaround: Deploy SCOM 2012 R2 on to SQL 2008 R2 SP2 and after installation upgrade the DB to SQL 2008 R2 SP3

Suraj Suresh Guptha | Program Manager | Microsoft

Get the latest System Center news on Facebook and Twitter:

Main System Center blog: http://blogs.technet.com/b/systemcenter/

↧

Understanding your Linux Monitoring Challenges

June 8, 2015, 11:31 am

≫ Next: Quick note on an issue you might encounter after installing Microsoft Security Update 3004375

≪ Previous: OpsMgr 2012 R2 now offers additional support for SQL Server 2008 R2 SP3

Hi Folks, we want to know more in-depth how your organization monitors Linux servers and challenges you face monitoring these Linux servers. We are NOT trying to sell you anything, we just want you understand your pains in this area.

If you are or could introduce us to your Linux monitoring administrators (folks responsible for tools like Nagios, Zabbix, Zenoss etc…) we would really appreciate it as this an area we want to understand.

We would like to have do a 30min call on this topic. Please fill out this 5 question form to let us know if you or your colleague is available. We would really appreciate your input on this topic.

https://www.surveymonkey.com/r/KK772MC

Thank you,

Satya Vel

↧

Quick note on an issue you might encounter after installing Microsoft Security Update 3004375

June 15, 2015, 9:10 am

≫ Next: KB: Domain controllers are unavailable in the Operations Manager console when no client agent is installed

≪ Previous: Understanding your Linux Monitoring Challenges

We’ve seen an issue relating to 3004375 that occurs due to a regression, and while it’s already been fixed (by installing 3023562), we wanted to take a minute and let you know about some of the details in case you happen to see it.

This issue manifests itself in a number of SDK clients (e.g. Exchange Correlation Engine, Console, Reporting Installation, Management Server Installation, etc.). In each of these clients the behavior may vary. For example, Exchange Correlation Engine might fail with error 714 in the Application Event Log, Reporting and Management Server installation might fail after hitting an exception from the SDK layer, the admin console might report a crash after encountering an exception from the SDK layer, etc.

With Operations Manager, the environment seems stable. Management Servers successfully read and write from the DB, send configuration to the agents, process Run As Accounts, run workflows, etc. However, the SDK layer, whenever approached by the SDK clients in a certain way, can generate an exception that causes failures. When this happens you will see an error message and stack trace similar to the following:

Error: An error has been detected by Operations Manager Root Management Server '<ms-server>'. Error message: Microsoft.EnterpriseManagement.Common.UnknownServiceException: The service threw an unknown exception. See inner exception for details. ---> System.ServiceModel.FaultException`1[System.ServiceModel.ExceptionDetail]: The type initializer for 'Microsoft.EnterpriseManagement.Mom.ServiceDataLayer.SdkDataAccess' threw an exception. (Fault Detail is equal to An ExceptionDetail, likely created by IncludeExceptionDetailInFaults=true, whose value is:
System.TypeInitializationException: The type initializer for 'Microsoft.EnterpriseManagement.Mom.ServiceDataLayer.SdkDataAccess' threw an exception. ----> System.Exception: Exception decrypting. The specified file could not be decrypted
   at Microsoft.EnterpriseManagement.Security.DPAPIWrapper.Decrypt(Byte[] cipherText)
   at Microsoft.EnterpriseManagement.Security.AsymmetricKeyManager.Initialize(Byte[] publicKey)
   at Microsoft.EnterpriseManagement.Security.SecureStorageManager.Initialize()
   at Microsoft.EnterpriseManagement.Mom.ServiceDataLayer.SdkDataAccess.get_secureStorageDatabaseManager()
   at Microsoft.EnterpriseManagement.Mom.ServiceDataLayer.SdkDataAccess..cctor()
   --- End of inner ExceptionDetail stack trace ---
   at Microsoft.EnterpriseManagement.Mom.ServiceDataLayer.SdkDataAccess..ctor()
   at CreateMicrosoft.EnterpriseManagement.Mom.ServiceDataLayer.SdkDataAccessBackCompatProxy()
   at Syst...).
   --- End of inner exception stack trace ---
   at Microsoft.EnterpriseManagement.DataAbstractionLayer.SdkDataAbstractionLayer.HandleIndigoExceptions(Exception ex)
   at Microsoft.EnterpriseManagement.DataAbstractionLayer.SdkDataAbstractionLayer.CreateChannel(TieredManagementGroupConnectionSettings managementGroupTier)
   at Microsoft.EnterpriseManagement.DataAbstractionLayer.SdkDataAbstractionLayer..ctor(DuplexChannelFactory`1 channelFactory, TieredManagementGroupConnectionSettings managementGroupTier, IClientDataAccess callback, CacheMode cacheMode)
   at Microsoft.EnterpriseManagement.DataAbstractionLayer.SdkDataAbstractionLayer.CreateEndpoint(ManagementGroupConnectionSettings connectionSettings, IClientDataAccess clientCallback)
   at Microsoft.EnterpriseManagement.DataAbstractionLayer.SdkDataAbstractionLayer.Connect(ManagementGroupConnectionSettings connectionSettings)
   at Microsoft.EnterpriseManagement.ManagementGroup..ctor(ManagementGroupConnectionSettings connectionSettings)
   at Microsoft.EnterpriseManagement.ManagementGroup.Connect(ManagementGroupConnectionSettings connectionSettings)
   at Microsoft.Exchange.Monitoring.CorrelationEngine.MomSdkProxy..ctor(String momServer, String domain, String user, SecureString password, String mpId, String extensionMPId)

In some cases such as console view load failures, you might just see the following information:

System.Exception: Exception decrypting. The specified file could not be decrypted
at Microsoft.EnterpriseManagement.Security.DPAPIWrapper.Decrypt(Byte[] cipherText)
at Microsoft.EnterpriseManagement.Security.AsymmetricKeyManager.Initialize(Byte[] publicKey)

For products other than System Center 2012 Operations Manager, such as Windows, you might find the following exception on debugging:

Error Code c00000a5 (ERROR_BAD_IMPERSONATION_LEVEL)

As mentioned earlier, this issue occurs due to a regression in cng.sys in 3004375. If you have this hotfix installed on the OpsMgr Management Server then it is likely that you are affected by this issue.

The corrected code has been shipped in the hotfix released in 3023562. After applying this fix the issue is resolved, just keep in mind that the SDK service might require a restart.

J.C. Hornbeck| Solution Asset PM | Microsoft GBS Management and Security Division

Get the latest System Center news onFacebookandTwitter:

System Center All Up: http://blogs.technet.com/b/systemcenter/

System Center 2012 Operations Manager System Center 2012 R2 Operations Manager OpsMgr 2012 R2

↧

KB: Domain controllers are unavailable in the Operations Manager console when no client agent is installed

June 17, 2015, 11:06 am

≫ Next: Private Cloud Health Feedback Request

≪ Previous: Quick note on an issue you might encounter after installing Microsoft Security Update 3004375

In the System Center Operations Manager admin console, some domain controllers may appear unavailable (appear dimmed) or in a Not Monitored state.

This behavior can occur if the Management Pack for Active Directory is installed and if the domain controllers in question do not have the Operations Manager client agent installed. Active Directory Topology Discovery tries to discover all domain controllers regardless of whether the agent is installed.

For complete details as well as a resolution, please see the following:

KB3068465 - Domain controllers are unavailable in the Operations Manager console when no client agent is installed (https://support.microsoft.com/en-us/kb/3068465)

J.C. Hornbeck| Solution Asset PM | Microsoft GBS Management and Security Division

Get the latest System Center news onFacebookandTwitter:

Main System Center blog: http://blogs.technet.com/b/systemcenter/

ConfigMgr 2012 R2

↧

Private Cloud Health Feedback Request

June 17, 2015, 2:42 pm

≫ Next: OMS Log Search API Documentation is here!

≪ Previous: KB: Domain controllers are unavailable in the Operations Manager console when no client agent is installed

For this post, we are glad to host Sarah Sinclair, Program Manager in Microsoft’s Private Cloud Solutions Group.

Hello Private Cloud Admins! My name is Sarah and I’m a Program Manager on the Private Cloud Solutions Group. We’re looking for feedback from customers like you about which private cloud system components you consider the most critical to your system health. To collect and aggregate this data, we’ve developed a quick wiki survey which pairs system components head-to-head and asks users to select the option whose failure would be most critical to their system. Users can also contribute alternative system components as desired. The survey can be found at the following link: http://privatecloudblogsurvey.azurewebsites.net/ Please note that there is no set end to the survey, and you can stop at any time. It will continue to generate pairings for you to consider. We ask that you consider taking five minutes to click through the survey options. Thank you for your time and contribution to our project!

↧

OMS Log Search API Documentation is here!

June 18, 2015, 11:58 am

≫ Next: OMS agent now supports W2K8 SP1 OS and 32bit systems

≪ Previous: Private Cloud Health Feedback Request

We're happy to announce that the OMS Log Search API is now available. This API allows direct access to OMS Log Search capabilities via the RESTful Azure Resource Manager API.

You can view the new Log Search API Documentation attached to this post.

We hope you will love this new functionality and use it to gain many new insights into your OMS data! Should you have any questions about the new Log Search API, please e-mail SCDATA@Microsoft.com. Happy searching!

↧

OMS agent now supports W2K8 SP1 OS and 32bit systems

June 23, 2015, 10:55 pm

≫ Next: System Center Open Source Software Management Packs Walkthrough Part 1: Apache HTTP Server

≪ Previous: OMS Log Search API Documentation is here!

We have now added support for Windows Server 2008 SP1 operating systems and 32bit systems. You just need to download the updated agent from the OMS portal ‘Settings’ tile then click the ‘Connected Sources’ tab. Download the agent MSI and run it on your servers.

We have a lot of customers using Operations Management Suite to collect events from their point of sale devices and we have got numerous requests to support 32bit systems as a lot of POS devices are 32bit. We are therefore excited to announce our 32bit systems support.

You can download the agent from the OMS portal as shown in the image below.

If you have any questions on how it works or run into any issues email us at...

↧

System Center Open Source Software Management Packs Walkthrough Part 1: Apache HTTP Server

June 24, 2015, 5:11 pm

≫ Next: Thinking of upgrading your OMS plan?

≪ Previous: OMS agent now supports W2K8 SP1 OS and 32bit systems

At Microsoft Ignite we announced the public availability of preview OSS Management Packs for System Center 2016 Technical Preview 2 - Operations Manager. The OSS Management Packs brings performance and health monitoring to open source favorites, Apache HTTP Server and MySQL Server. This blog post is the first of two OSS management walkthroughs that brings you one step closer to managing open source software with System Center Operations Manager. This walkthrough details a step by step guide for managing Apache HTTP Server.

The following is a quick glimpse into the feature set of the Apache HTTP Server management pack. (A full list of available metrics can be found in the Apache HTTP Server management pack guide’s appendix):

- Percentage of workers that are busy and idle

- Information on Apache HTTP Server CPU and memory consumption

- Virtual Host statistics (Requests and Errors per minute)

- SSL certificate expiration alerting

Requirements for Open Source Software management pack monitoring can be found in the respective management pack guide at the linked download page above.

Basic Setup

For this walkthrough my pre-existing environment consists of the following

System Center 2016 Technical Preview 2 Operations Manager
System Center 2016 Technical Preview 2 Management Packs for Open Source Software
Ubuntu Server 14.04 LTS with Apache HTTP Server 2.4 installed
1. Important: This Linux server is not already managed by System Center Technical Preview 2 Operations Manager

Managing the Ubuntu Server with System Center 2016 Technical Preview 2 Operations Manager

Managing a Linux server with System Center 2016 Technical Preview 2 Operations Manager maintains the same process used from previous version of System Center Operations Manager. For a more detailed look on how to discover Linux Servers with Operations Manager refer to the following blog post http://blogs.technet.com/b/kevinholman/archive/2012/03/18/deploying-unix-linux-agents-using-opsmgr-2012.aspx

Behind the scenes, the new Linux agents included in the Linux management packs for System Center 2016 Technical Preview 2 Operations Manager contain a new version of the included CIMOM Server, Open Management Infrastructure (OMI) 1.0.8 instead of 1.0.6.5. OMI 1.0.8 brings a variety of performance enhancements, most notably the support for multiple thread execution. Additionally the new Linux agents include separate OSS agents bundled inside for Apache HTTP Server and MySQL Server. These OSS agents are installed by default when a supported installation of Apache HTTP Server or MySQL Server are identified by the Linux agent.

If you would like to add OSS management to an already managed Linux server refer to instructions inside the management pack guide on extracting the OSS agents and installation.

Import Apache HTTP Server Library Management Pack and OSS Library Management Pack. The OSS management packs are both 7.6.1042.0
Run through the “Discovery Wizard” with the Linux server that has Apache HTTP Server pre-installed.

Also a great opportunity to try out a new feature in System Center 2016 TP2 Operations Manager -- using Run As credentials to discover a Linux Server

Enabling Apache HTTP Server monitoring

Apache HTTP Server monitoring consists of two parts, an Apache module and an Apache OMI CIM provider. The Apache module registers with Apache HTTP Server as an apache module, and leverages the Apache “APR” to gather near real-time data and high fidelity performance metrics. The Apache OMI CIM provider implements a CIM model for the Apache HTTP Server and communicates to the Apache module through a shared memory channel. A short architecture overview can be seen below.

In the previous steps the Apache OMI CIM provider was installed automatically with the Linux agent, but the Apache module requires a manual install*. In order to manually install the Apache module you can run a management pack task, or run a command locally on the Linux server with the Apache OSS agent installed. This blog uses the management pack task to deploy the Apache module.

*When an Apache module is installed the Apache HTTP Server is restarted. Plan the module deployment in accordance with your enterprise requirements.

Navigate to the Monitoring Pane > Apache HTTP Server > Apache HTTP Server Summary page. When Apache HTTP Server installation discovery runs, an instance of the Apache HTTP Server that is unmanaged appears. By default Apache HTTP Server installation discovery runs every four hours.
Select the Apache HTTP Server you wish to deploy the Apache module to and select the management pack task “Load Monitoring Module”
Ensure a “success” status is returned along with a proper restart of the Apache HTTP Server.
Optional: Run the “Update Discovered Properties” management pack task to expedite monitoring. Otherwise, wait until the next instance of Apache HTTP Server discovery runs, which by default is every four hours. After a few minutes the “IsManaged?” property changes to “true”, and the Health state appears for the Apache HTTP Server instance.

Congratulations!! You have successfully set up Apache HTTP Server monitoring with System Center 2016 TP2 Operations Manager. After a couple minutes Apache HTTP Server Virtual Hosts appear and Apache HTTP Server performance metrics beginning to populate.

We look forward to any feedback you may have and stay tuned for the next walkthrough for monitoring MySQL Server with System Center 2016 TP2 Operations Manager.

↧

Thinking of upgrading your OMS plan?

July 6, 2015, 12:28 pm

≫ Next: New Update Rollup for Microsoft Monitoring Agent now available

≪ Previous: System Center Open Source Software Management Packs Walkthrough Part 1: Apache HTTP Server

We now provide you an easy way to estimate what your monthly cost would be if you upgrade to our Standard or Premium pay-as-you-go plans.

In the OMS portal, click the ‘Usage’ tile and click the ‘Estimated cost’ arrow to get your monthly estimated cost given your current data consumption.

With the ‘Free’ plan you get 7 days of data retention and 500MB of daily upload limit.

The ‘Standard’ plan gives you 30 days or retention with no upload limit and the ‘Premium’ plan gives you 1 year of data retention with no upload limit.

For detailed steps on how to upgrade your plan can be found here: https://azure.microsoft.com/en-us/documentation/articles/operational-insights-setup-workspace/#upgrade-workspace-to-a-paid-plan

↧

New Update Rollup for Microsoft Monitoring Agent now available

July 7, 2015, 11:49 am

≫ Next: How to troubleshoot Operations Manager DB performance problems

≪ Previous: Thinking of upgrading your OMS plan?

The article below describes the issues that are fixed in a new update rollup for the Microsoft Monitoring Agent. Additionally, the article contains installation instructions for the update rollup.

Issues that are fixed in this update rollup

- Support for Windows Server 2008 and Windows Vista: Adds support for Windows Server 2008 and Windows Vista on-boarding to the Microsoft Operations Management Suite.

- Maintain original identification after Operations Management Suite is disabled and then re-enabled or after agent is reinstalled: Preserves the identification of the system during upgrades or when enabling and disabling the Operations Management Suite. Previously, disabling and then re-enabling a connection to the Operations Management Suite caused a new agent identification to be generated.

- Always use the Microsoft .NET Framework 4.0 for loading managed modules if it is available: Sets a preference for the .NET Framework 4.0 when managed code is loaded in management packs. The .NET Framework 2.0 is used if the .NET Framework 4.0 is not available. Previously, the .NET Framework 4.0 would be used on Windows Server 2012, Windows 8, or later versions of Windows operating systems while the .NET Framework 2.0 was loaded on earlier versions of Windows.

- Fixes the Apply button that is not working in the Microsoft Monitoring Agent control panel.

- Removes a dialog box that is displayed during a silent installation when a restart is required.

For complete details as well as a download link, please see the following:

KB3071396 - Update Rollup for Microsoft Monitoring Agent (https://support.microsoft.com/en-us/kb/3071396)

J.C. Hornbeck| Solution Asset PM | Microsoft GBS Management and Security Division

Get the latest System Center news onFacebookandTwitter:

Main System Center blog: http://blogs.technet.com/b/systemcenter/

OpsMgr 2012 R2 System Center 2012 R2 Operations Manager System Center 2012 Operations Manager

↧

How to troubleshoot Operations Manager DB performance problems

July 8, 2015, 9:27 am

≫ Next: System Center Open Source Software Management Packs Walkthrough Part 2: MySQL Server

≪ Previous: New Update Rollup for Microsoft Monitoring Agent now available

In System Center 2012 Operations Manager (OpsMgr 2012 or OpsMgr 2012 R2), one of the more common performance concerns surrounds Operations Manager Database and Data Warehouse insertion times. Here are some tips to help identify and troubleshoot problems concerning Database and Data Warehouse data insertion.

A typical sign of Operations Manager DB performance problems is the presence of Event ID 2115 events in the Operations Manager Event log. These events typically indicate that performance issues exist on the Management Server or the Microsoft SQL Server that is hosting the OperationsManager or OperationsManager Data Warehouse databases.

First a little background

Database and Data Warehouse write action workflows run on the Management Servers and these workflows first retain the data received from the Agents and Gateway Servers in an internal buffer. They then gather this data from the internal buffer and insert it into the Database and Data Warehouse. When the first data insertion has completed, the workflows will then create another batch.

The size of each batch of data depends on how much data is available in the buffer when the batch is created, however there is a maximum limit on the size of the data batch of up to 5000 data items. If the data item incoming rate increases, or the data item insertion throughput to the Operation Manager and Data Warehouse databases throughput is reduced, the buffer will then accumulate more data and the batch size will grow larger.

There are several write action workflows that run on a Management Server. These workflows handle data insertion to the Operations Manager and Data Warehouse databases for different data types. For example:

Microsoft.SystemCenter.DataWarehouse.CollectEntityHealthStateChange
Microsoft.SystemCenter.DataWarehouse.CollectPerformanceData
Microsoft.SystemCenter.DataWarehouse.CollectEventData
Microsoft.SystemCenter.CollectAlerts
Microsoft.SystemCenter.CollectEntityState
Microsoft.SystemCenter.CollectPublishedEntityState
Microsoft.SystemCenter.CollectDiscoveryData
Microsoft.SystemCenter.CollectSignatureData
Microsoft.SystemCenter.CollectEventData

When a Database or Data Warehouse write action workflow on a Management Server experiences slow data batch insertion (e.g. times in excess of 60 seconds), it will begin logging Event ID 2115 to the Operations Manager Event log. This event is logged every one minute until the data batch is inserted into the Database or Data Warehouse, or the data is dropped by the write action workflow module. As a result, Event ID 2115 will be logged due to the latency inserting data into the Database or Data Warehouse. Below is an example Event logged due to data dropped by the write action workflow module:

Event Type: Error
Event Source: HealthService
Event Category: None
Event ID: 4506
Computer: <RMS NAME>
Description:
Data was dropped due to too much outstanding data in rule "Microsoft.SystemCenter.OperationalDataReporting.SubmitOperationalDataFailed.Alert" running for instance <RMS NAME> with id:"{F56EB161-4ABE-5BC7-610F-4365524F294E}" in management group <MANAGEMENT GROUP NAME>.

Taking a deeper look

Event ID 2115 contains 2 significant pieces of information. First, the name of the workflow that is experiencing the problem, and second, the elapsed time since the workflow began inserting the last batch of data. For example:

Log Name: Operations Manager
Source: HealthService
Event ID: 2115
Level: Warning
Computer: <RMS NAME>
Description:
A Bind Data Source in Management Group <MANGEMENT GROUP NAME> has posted items to the workflow, but has not received a response in 300 seconds. This indicates a performance or functional problem with the workflow.
Workflow Id : Microsoft.SystemCenter.CollectPublishedEntityState
Instance : <RMS NAME>
Instance Id : {88676CDF-E284-7838-AC70-E898DA1720CB}

This particular Event ID 2115 message indicates that the workflow Microsoft.SystemCenter.CollectPublishedEntityState, which writes Entity State data to the Operations Manager database, is trying to insert a batch of Entity State data and it started 300 seconds ago. In this example, the insertion of the Entity State data has not yet finished. Normally inserting a batch of data should complete within 60 seconds.

If the Workflow ID contains Data Warehouse then the problem concerns the Operations Manager Data Warehouse. Otherwise, the problem concerns inserting data into the Operations Manager database!

Common causes

Insertion Problems

As the description of Event ID 2115 states, this may indicate a database performance problem or too much data incoming from the agents. Event ID 2115 simply indicates there is a backlog inserting data into the Database (either Operations Manager or Operations Manager Data Warehouse). These events can originate from a number of possible causes. For example, there could be a sudden, large amount of Discovery data, there could be a database connectivity issue, the database may be full, or possibly there is a disk or network related constraint.

In Operations Manager, discovery data insertion is a relatively labor intensive process. It can also come in bursts, where there is a significant amount of data being received by the Management Server. These bursts can cause temporary instances of Event IDs 2115, but if Event ID 2115 consistently appears for discovery data collection this can indicate either a Database or Data Warehouse insertion problem or Discovery rules in a Management Pack collecting too much discovery data.

Operations Manager configuration updates caused by Instance Space changes or Management Pack imports have a direct effect on CPU utilization on the Database Server and this can impact Database insertion times. Following a Management Pack import or a large instance space change, it is expected to see Event ID 2115 messages. For more information on this topic please see the following:

2603913 - How to detect and troubleshoot frequent configuration changes in Operations Manager (http://support.microsoft.com/kb/2603913)

In Operations Manager, expensive user interface queries can also impact resource utilization on the database which can lead to latency in Database insertion times. When a user is performing an expensive User Interface operation it is possible to see Event ID 2115 messages logged.

Databases full or offline

If the Operations Manager or Operations Manager Data Warehouse databases are out of space or offline, it is expected that the Management Server will continue to log Event ID 2115 messages to the Operations Manager Event log and the pending time will grow higher.

If the write action workflows cannot connect to the Operations Manager or Operations Manager Data Warehouse databases, or they are using invalid credentials to establish their connection, the data insertion will be blocked and Event ID 2115 messages will be logged accordingly until this situation is resolved.

Configuration and environmental issues

Event ID 2115 messages can also indicate a performance problem if the Operations Manager Database and Data Warehouse databases as well as the supporting environment are not properly configured. Some possible causes include the following:

The SQL Log or TempDB database is too small or out of space.
The Network link from the Operations Manager and Data Warehouse database to the Management Server is bandwidth constrained or the latency is large. In this scenario we recommend to Management Server to be on the same LAN as the Operations Manager and Data Warehouse server.
The data disk hosting the Database, logs or TempDB used by the Operations Manager and Data Warehouse databases is slow or experiencing a function problem. In this scenario we recommend leveraging RAID 10 and we also recommend enabling battery backed Write Cache on the Array Controller.
The Operations Manager Database or Data Warehouse server does not have sufficient memory or CPU resources.
The SQL Server instance hosting the Operations Manager Database or Data Warehouse is offline.

It is also recommend that the Management Server reside on the same LAN as the Operations Manager and Data Warehouse database server.

Event ID 2115 messages can also occur if the disk subsystem hosting the Database, logs or TempDB used by the Operations Manager and Data Warehouse databases is slow or experiencing a functional problem. We recommend leveraging RAID 10 and we also recommend enabling battery backed Write Cache on the Array Controller.

Troubleshooting

The first step in troubleshooting the performance related Event ID 2115 messages is to identify what data items are being returned within the event. For example, the Workflow ID indicates what type of data items (e.g. discovery, alerts, event, perf) as well as what database is involved. If the Workflow ID contains the term DataWarehouse, then the focus should be towards the Operations Manager Data Warehouse. In other instances the focus would be towards the Operations Manager Database.

Scenario 1

In the example below, the problem concerns the workflow Microsoft.SystemCenter.CollectSignatureData.

Event Type: Warning
Event Source: HealthService
Event Category: None
Event ID: 2115
Computer: <RMS NAME>
Description:
A Bind Data Source in Management Group <MANGEMENT GROUP NAME> has posted items to the workflow, but has not received a response in 300 seconds. This indicates a performance or functional problem with the workflow.
Workflow Id : Microsoft.SystemCenter.CollectSignatureData
Instance : <RMS NAME>
Instance Id : {F56EB161-4ABE-5BC7-610F-4365524F294E}

Resolution

We can identify the Performance Signature Data Collection Rules in this example by executing the following SQL query. This query should be executed in SQL Management Studio against the Operations Manager database.

-- Return all Performance Signature Collection Rules
Use OperationsManager
select
managementpack.mpname,
rules.rulename
from performancesignature with (nolock)
inner join rules with (nolock)
on rules.ruleid = performancesignature.learningruleid
inner join managementpack with(nolock)
on rules.managementpackid = managementpack.managementpackid
group by managementpack.mpname, rules.rulename
order by managementpack.mpname, rules.rulename

This query will return all Performance Signature Collection Rules and their respective Management Pack name. A column is returned for Management Pack name and Rule name.

The following Performance Monitor Counters on a Management Server will provide information concerning Database and Data Warehouse write action insertion batch size and time. If the batch size is growing larger, for example the default batch size is 5000 items, this indicates either the Management Server is slow inserting the data to the Database or Data Warehouse, or is receiving a burst of Data Items from the Agents or Gateway Servers.

· OpsMgr DB Write Action Modules(*)\Avg. Batch Size
· OpsMgr DB Write Action Modules(*)\Avg. Processing Time
· OpsMgr DW Writer Module(*)\Avg. Batch Processing Time, ms
· OpsMgr DW Writer Module(*)\Avg. Batch Size

From the Database and Data Warehouse write action account Average Processing Time counter, we can understand how long it takes on average to write a batch of data to the Database and Data Warehouse. Depending upon the amount of time it takes to write a batch of data to the Database, this may present an opportunity for tuning.

Scenario 2

If the SQL Server instance hosting the Operations Manager Database or Data Warehouse is offline, Event ID 2115 as well as Event ID 29200 will appear within the Operations Manager Event log. For example:

Log Name: Operations Manager
Source: HealthService
Date:
Event ID: 2115
Level: Warning
Description:
A Bind Data Source in Management Group MSFT has posted items to the workflow, but has not received a response in 60 seconds. This indicates a performance or functional problem with the workflow.
Workflow Id : Microsoft.SystemCenter.CollectEventData
Instance : name.contoso.local
Instance Id : {88676CDF-E284-7838-AC70-E898DA1720CB}

=====

Log Name: Operations Manager
Source: OpsMgr Config Service
Event ID: 29200
Level: Error
Description:
OpsMgr Config Service has lost connectivity to the OpsMgr database, therefore it can not get any updates from the database. This may be a temporary issue that may be recovered from automatically. If the problem persists, it usually indicates a problem with the database. Reason:
A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: Named Pipes Provider, error: 40 - Could not open a connection to SQL Server)

Resolution

To resolve the issue in this scenario, complete the following:

Connect to the server hosting the Operations Manager database.
Open the Services applet.
Verify that the SQL Server (MSSQLSERVER) service is started and running.
If the SQL Server (MSSQLSERVER) service is not started and running, start the service.

Once database connectivity is restored workflows should resume successfully storing data within the respective database. Event ID 31554 validates that the information has been written successfully:

Log Name: Operations Manager
Source: Health Service Modules
Event ID: 31554
Task Category: Data Warehouse
Level: Information
Description:
Workflow succeeded storing data in the Data Warehouse
One or more workflows were affected by this.
Workflow name: Microsoft.SystemCenter.DataWarehouse.CollectEventData
Instance name: name.contoso.local
Instance ID: {88676CDF-E284-7838-AC70-E898DA1720CB}

Scenario 3

Event ID 2115 caused by invalid RunAs credentials.

Resolution

Examine the Operations Manager Event log for the following events. These events typically indicate that the Data Warehouse SQL Server Authentication Account may have incorrect credentials.

Log Name: Operations Manager
Source: HealthService
Event ID: 7000
Task Category: Health Service
Level: Error
Description:
The Health Service could not log on the RunAs account <ACCOUNT NAME> for management group <MANGEMENT GROUP NAME>. The error is Logon failure: unknown user name or bad password.(1326L). This will prevent the health service from monitoring or performing actions using this RunAs account

=====

Log Name: Operations Manager
Source: HealthService
Event ID: 7015
Task Category: Health Service
Level: Error
Description:
The Health Service cannot verify the future validity of the RunAs account <ACCOUNT NAME> for management group <MANGEMENT GROUP NAME>. The error is Logon failure: unknown user name or bad password.(1326L).

To resolve the issue in this scenario, complete the following:

1. Open the Operations Manager console.
2. Select Administration.
3. Select Run As Configuration\Accounts.
4. Configure the appropriate credentials for the Data Warehouse SQL Server Authentication Account.

Scenario 4

Event ID 2115 caused by disk performance issues. This can occur when the data disk hosting the database, logs or TempDB used by the Operations Manager and Data Warehouse databases is slow or experiencing a problem. In this scenario we recommend leveraging RAID 10 and we also recommend enabling battery backed Write Cache on the Array Controller.

Resolution

First, capture these Physical Disk counters for all drives that contain SQL data or log files:

% Idle Time: How much disk idle time is being reported. Anything below 50 percent could indicate a disk bottleneck.
Avg. Disk Queue Length: This value should not exceed 2 times the number of spindles on a LUN. For example, if a LUN has 25 spindles, a value of 50 is acceptable. However, if a LUN has 10 spindles, a value of 25 is too high. You could use the following formulas based on the RAID level and number of disks in the RAID configuration:

RAID 0: All of the disks are doing work in a RAID 0 set
Average Disk Queue Length <= # (Disks in the array) *2
RAID 1: half the disks are “doing work”; therefore, only half of them can be counted toward Disks Queue
Average Disk Queue Length <= # (Disks in the array/2) *2
RAID 10: half the disks are “doing work”; therefore, only half of them can be counted toward Disks Queue
Average Disk Queue Length <= # (Disks in the array/2) *2
RAID 5: All of the disks are doing work in a RAID 5 set
Average Disk Queue Length <= # (Disks in the array/2) *2
Avg. Disk sec/Transfer: The number of seconds it takes to complete one disk I/O
Avg. Disk sec/Read: The average time, in seconds, of a read of data from the disk
Avg. Disk sec/Write: The average time, in seconds, of a write of data to the disk
Disk Bytes/sec: The number of bytes being transferred to or from the disk per second
Disk Transfers/sec: The number of input and output operations per second (IOPS)

NOTE The last three counters in this list should consistently have values of approximately .020 (20 ms) or lower and should never exceed.050 (50 ms). The following are the thresholds that are documented in the SQL Server performance troubleshooting guide:

Less than 10 ms: Very good
Between 10 - 20 ms: Good
Between 20 - 50 ms: Slow, needs attention
Greater than 50 ms: Serious I/O bottleneck
Disk Bytes/sec: The number of bytes being transferred to or from the disk per second
Disk Transfers/sec: The number of input and output operations per second (IOPS)

When % Idle Time is low (10 percent or less), this means that the disk is fully utilized. In this case, the last two counters in this list (Disk Bytes/sec and Disk Transfers/sec) provide a good indication of the maximum throughput of the drive in bytes and in IOPS, respectively. The throughput of a SAN drive is highly variable, depending on the number of spindles, the speed of the drives and the speed of the channel. The best bet is to check with the SAN vendor to find out how many bytes and IOPS the drive should support. If % Idle Time is low, and the values for these two counters do not meet the expected throughput of the drive, engage the SAN vendor for further troubleshooting assistance.

The following links provide deeper insight into troubleshooting SQL Server performance issues:

Scenario 5

Event ID 2115 is logged and a Management Server generates an "unable to write data to the Data Warehouse" alert in System Center Operations Manager. You also experience the following symptoms on the Management Server computer:

The Management Server generates one or more alerts that resemble the following:

Performance data collection process unable to write data to the Data Warehouse

Performance data collection process unable to write data to the Data Warehouse. Failed to store data in the Data Warehouse. Exception 'SqlException': Management Group with id '9069F7BD-55B8-C8E8-1CF9-4395F45527E2' is not allowed to access Data Warehouse under login 'DOMAIN\Action_Account' One or more workflows were affected by this. Workflow name: Microsoft.SystemCenter.DataWarehouse.CollectPerformanceData Instance name: dataWarehouseServer.domain.com Instance ID: {48936EE3-4E3E-BEE1-8C09-AFDAB8ECF236} Management group: Management Group Name.

The following event is logged in the Operations Manager event log on the Management Server:

Event Type: Warning
Event Source: HealthService
Event Category: None
Event ID: 2115
Date: date
Time: time
User: N/A
Computer: ManagementServerName
Description: A Bind Data Source in Management Group Management Group Name has posted items to the workflow, but has not received a response in 1712 seconds. This indicates a performance or functional problem with the workflow. Workflow Id : Microsoft.SystemCenter.DataWarehouse.CollectPerformanceData Instance : ManagementServerName.domain.com Instance Id : {C7FDDE2A-E0AA-4B80-70DE-1D50D9965221}

Resolution

This issue can occur if the Management Server does not have accounts that are specified for its data warehouse "Run As" profiles. This issue is more likely to affect a secondary Management Server. To resolve this problem complete the following:

On the computer that is running System Center Operations Manager, open the Operations Console.
In the navigation pane, click Administration.
Expand Security, and then click Run As Profiles.
In the Run As Profiles view, double-click Data Warehouse Account.
In the Run As Profile Properties - Data Warehouse Account properties dialog box, click the Run As Accounts tab, and then click New.
In the Run As Account list, click Data Warehouse Action Account.
In the Name list, click the Management Server that generated the alert.
Click OK two times.
Follow steps 4 through 8 to assign the appropriate Run As account to the following profiles:
- Data Warehouse Configuration Synchronization Reader Account
- Data Warehouse Report Deployment Account
- Data Warehouse SQL Server Authentication Account
For each profile, select the Run As account that matches the name of the Run As profile. For example, make the following assignments:
- Assign the Data Warehouse Configuration Synchronization Reader Account to the Data Warehouse Configuration Synchronization Reader Account profile.
- Assign the Data Warehouse Report Deployment Account to the Data Warehouse Report Deployment Account profile.
- Assign the Data Warehouse SQL Server Authentication Account to the Data Warehouse SQL Server Authentication Account profile.
On the Management Server that generated the alert, restart the OpsMgr Health Service.
In the Operations Manager event log on the Management Server, verify that event ID 31554 events are logged. Event ID 31554 indicates that the monitor state has changed to Healthy. This change resolves the alert.

Scenario 6

Event ID 2115 occurs on a server running HP MPIO FF DSM XP v3.01 to which there are no LUNs presented. When the user opens Performance Monitor and attempts to add a counter, Performance Monitor will hang and the handle count for this application increases rapidly.

Resolution

There are two workarounds for this issue:

Rename the HPPerfProv.dll file and reboot Windows. Performance Monitor will work without issue when the file is renamed and not loaded.
Have at least 1 LUN present on the system.

For more information on this issue please see the following HP support document:

Windows MPIO DSM for XP v3.01.00 - Will Have Windows Perfmon Hung In "No LUN Presented" Environment

J.C. Hornbeck| Solution Asset PM | Microsoft GBS Management and Security Division

Get the latest System Center news onFacebookandTwitter:

System Center All Up: http://blogs.technet.com/b/systemcenter/

System Center 2012 Operations Manager System Center 2012 R2 Operations Manager SCOM 2012 R2 SCOM2012R2

↧

System Center Open Source Software Management Packs Walkthrough Part 2: MySQL Server

July 8, 2015, 4:53 pm

≫ Next: You can now ingest Syslog data into OMS from Azure Linux VMs!

≪ Previous: How to troubleshoot Operations Manager DB performance problems

Two weeks ago we released a walkthrough of managingApache HTTP Server with System Center 2016 Technical Preview 2 Operations Manager. This week we continue with a walkthrough for managing MySQL Server with System Center 2016 TP2 Operations Manager. This walkthrough details a step by step guide for setting up and managing MySQL Server.

Here’s a quick look at the feature set for the MySQL Server management pack. A full list of available metrics can be found in the MySQL Server management pack guide in the Appendix section:

- Performance metrics for key cache, query cache, table cache, etc.

- Alerting on connections and query performance

- InnoDB buffer pool hit and usage statistics

- Information on disk space usage for the whole server and individual databases

Requirements for Open Source Software management pack monitoring can be found in the respective management pack guide at the linked download page above.

Basic Setup

For this walkthrough my pre-existing environment consists of the following

System Center 2016 Technical Preview 2 Operations Manager
System Center 2016 Technical Preview 2 Management Packs for Open Source Software
Ubuntu Server 14.04 LTS with MySQL Server 5.5 installed
1. Important: This Linux server is not already managed by System Center 2016 Technical Preview 2 Operations Manager

MySQL Server Monitoring Overview

MySQL Server monitoring consists of a MySQL OMI CIM Provider that uses the MySQL client libraries to retrieve statistics from the running MySQL Server. In order for the MySQL OMI CIM Provider to utilize the MySQL client libraries user credentials must be specified to authorize and facilitate the connection. A short architecture overview can be seen below.

Creating the MySQL Server user

To begin configuration of the MySQL Server monitoring user let’s login to the MySQL Server using the MySQL client. There are a variety of tools to access the MySQL Server. In this tutorial I use the mysql client included with the mysql-server distribution

On the Ubuntu 14.04 LTS Server run the following command to login to the MySQL client as root. Note: You may login as any user as long as the specified user has permissions to CREATE users and GRANT permissions.
1. mysql -u<INSERT USER> -p<INSERT PASSWORD>
if successful you are greeted with a terminal for the mysql client
In the MySQL command terminal enter the following to create a MySQL Server monitoring user with a specified password. In our case the user is "scomuser" and the password is "scompass"
1. CREATE USER ‘scomuser’@’localhost’ IDENTIFIED by ‘scompass’
Once the MySQL Server monitoring user is created we need to grant permissions so that the monitoring user can view the performance metrics MySQL Server collects.
1. The MySQL monitoring user requires SELECT privileges to the default 'mysql' tables for base performance monitoring. Run the following command to grant SELECT privileges for the 'mysql' database to the scomuser
- GRANT SELECT ON mysql.* TO ‘scomuser’@’localhost’;
The MySQL monitoring user also retrieves information about MySQL Databases the MySQL monitoring user has permissions to access. In order to grant privileges to the MySQL monitoring user use the following command as a template.
1. GRANT SELECT ON <INSERT DATABASE NAME>.* TO ‘scomuser’@’localhost’;

Managing the Ubuntu Server with System Center 2016 Technical Preview 2 Operations Manager

As mentioned in the Apache blog last week, the process for bringing a Linux server under management with this new version of Operations Manager remains largely unchanged. For a more detailed look on how to discover Linux servers with Operations Manager refer to the following blow post http://blogs.technet.com/b/kevinholman/archive/2012/03/18/deploying-unix-linux-agents-using-opsmgr-2012.aspx

Like the Apache provider the MySQL provider is included in the new Linux agents for System Center 2016 TP2 Operations Manager. During the Linux agent installation if a supported version of MySQL Server is installed the Linux agent installs the MySQL Server agent.

If you would like to add MySQL management to an already managed Linux server refer to the instructions inside the MySQL management pack guide on the download page.

Import the MySQL Server Library Management Pack and the OSS Library Management Pack. The version for both management packs is 7.6.1042.0
After the MySQL management packs are imported when the next discovery runs a MySQL Server instance appears in the console. MySQL Server discovery runs every four hours by default.

Enabling MySQL Monitoring

Now that we have the MySQL Server discovery completed in Operations Manager we have to configure the MySQL provider to use the MySQL Server monitoring user we created earlier. Thankfully, this can be done by distributing an authentication text file or running a management pack task. This guide uses the management pack tasks. Instructions for creating and distributing the authentication text file can be found in the management pack guide.

Click the “Not monitored” instance of the MySQL Server and open the “Task Pane” on the right hand side
Under MySQL Server (Linux) Tasks you can see two tasks “Set Default MySQL Credentials” and “Update Discovered Properties”
First run the “Set Default MySQL Credentials”
The task contains parameters including username, port, password, and bind address. Let’s click “Override” and change the username and password to the user we created in the earlier step as well as change the bind address to 127.0.0.1 for localhost.
1. The port option can be changed to a specific port if needed, otherwise the MySQL client communicates through the default 3306. If multiple instances of MySQL Server are running on the same Linux server refer to MySQL Management Pack Guide for authentication file configuration.
After clicking override and then run task we should receive a “True” for task output
Once the MySQL Monitoring user is configured we can run the “Update Discovered Properties” management pack task to initiate a discovery and immediately update properties.

Congratulations!! You have successfully set up MySQL Server monitoring with System Center 2016 TP2 Operations Manager. After a couple minutes MySQL Server Databases and MySQL Server performance metrics begin to populate.

We appreciate any comments you may have about MySQL Server or Apache HTTP Server monitoring, and look forward to your feedback.

↧

You can now ingest Syslog data into OMS from Azure Linux VMs!

July 9, 2015, 5:33 am

≫ Next: Private Cloud Health - Alert Criticality Feedback Request

≪ Previous: System Center Open Source Software Management Packs Walkthrough Part 2: MySQL Server

The Azure team has recently released the Azure Diagnostics extension for Linux VMs. If you use Azure Diagnostics, Operations Management Suite can ingest syslog data from storage accounts configured to store your diagnostics data. As of the publication of this blog post, OMS can ingest all logs routed through syslog or RsyslogD.

We’d like to share a high level view of how your data travels to OMS:

To enable this functionality, follow the steps below:

1. Enable Azure Diagnostics for your Linux machines

2. In the Classic Azure Portal find your Operational Insights workspace.

Note: If you can’t find your OMS workspace here, it means you haven’t linked your workspace to your Azure Subscription – follow the steps to link workspaces here

3. Select this workspace and click on the STORAGE tab.

4. Click ADD in the task bar at the bottom and select the storage account that contains your diagnostics data.

5. Select the DATA TYPE you’d like to pull from the storage account. At the moment, the only Linux log collection we support is Syslog (Linux)

6. Select the table to read from. For Syslog (Linux) your only choice is LinuxsyslogVer2v0.

7. Click the checkbox

That’s it! In 5 to 15 minutes you will start to see any new logs written to this table in your OMS workspace. Just search for Type=Syslog in the Log Search feature of OMS.

If you are not seeing your logs, or are having any other issues, please email us at scadata@microsoft.com– We are here to help. :)

↧