To make it clear right away, this is not going to be a “Building an SNMP Management Pack Tutorial” since there’s plentiful good ones out there already, and to be extra helpful I’m gonna include a few links right away:

• SNMP Setup and Simple Custom SNMP Discovery – Pretty much the basics
• SNMP Management Pack Example: NetApp Management Pack – Part 4 actually, but has the links to the other parts
• Creating SNMP Probe Based Monitors – No custom discovery, but a good and simple guide to SNMP Probes

It’s the second, the NetApp one, I’ve used as a guide to building the UPS management pack since it goes through the process of building your own filtered discovery using SystemOID to identify your hardware-classes and then building the monitors on top of those.

Let’s get to it

When building the discovery of my hardware classes I ran into problems. The discovery simply did not work. At first I got some strange errors about “invalid queries”, something that turned out to be related to me reading two guides–seriously though, pick one guide that is closest to what you want to achieve and stick to it–and mixing up the XPathQuery variables. Silly me.
I got those errors to go away and I was able to get a few objects to my base-class, but none of the hardware classes who was populated through the return value of an SNMP OID got discovered.
The only error I got this time was the following:

Log Name:      Operations Manager
Source:        Health Service Modules
Date:          2010-09-02 11:19:12
Event ID:      11001
Task Category: None
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      CENSORED
Description:
Error sending an SNMP GET message to IP Address XX.XX.XX.XX, Community String:=CENSORED, Status 0x6c.

One or more workflows were affected by this.

Workflow name: CENSORED.MP.CLASS.DISCOVERY
Instance name: CENSORED_DEVICENAME
Instance ID: {5C7EFB30-D885-8843-0DD7-EA86B4FD2311}
Management group: CENSORED

<IsWriteAction>false</IsWriteAction>
<IP>$Config/IP$</IP>
<CommunityString>$Config/CommunityString$</CommunityString>
<Version>1</Version>
<SnmpVarBinds>
	<SnmpVarBind>
		<OID>1.3.6.1.4.1.534.1.1.1.0</OID>
		<Syntax>0</Syntax>
		<Value VariantType="8"></Value>
	</SnmpVarBind>
	<SnmpVarBind>
		<OID>1.3.6.1.4.1.534.1.1.2.0</OID>
		<Syntax>0</Syntax>
		<Value VariantType="8"></Value>
	</SnmpVarBind>
	<SnmpVarBind>
		<OID>1.3.6.1.4.1.534.1.1.3.0</OID>
		<Syntax>0</Syntax>
		<Value VariantType="8"></Value>
	</SnmpVarBind>
</SnmpVarBinds>

This seems to be  most common on Windows 2008 and i guess it’s because of the UAC and the fact that opening the Control Panel isn’t running in administrative mode.

To work around this you need to run the msiexec command on the correct installation GUID from an administrative command prompt.

Besides running through the registry to find the GUID, one of the easier ways is this:

1. Open an administrative command prompt.
2. run wmic product
3. Locate your product by its name, the GUID (looks a bit like this {25097770-2B1F-49F6-AB9D-1C708B96262A}) directly after that is the one you want. Copy it.
4. run msiexec /i <PASTEYOURGUIDHERE>
5. Modify the agent as pleased

That’s pretty much it. Good luck.

Check the file permissions! And if that doesn’t help?

Check the share permissions! Yes, if it is a shared folder. And if that doesn’t help?

Check the file ownership! Great! But then what?

Well, the file could be in use, and then you would have to shut the locking process down and perhaps kick a user out. In a really bad scenario it could also be a symptom of a broken filesystem, a reserved filename (like “lpt1” or “PRN”) or even an invalid name (silly things like a space in the beginning or the end of a filename).
Another possible reason could actually be that the path to the file or folder is too long. You won’t actually get an error telling you that the filepath exceeds the 255 characters Windows can handle but a simple “Acces Denied”.

There are some, more or less tedious, work-arounds for the problem. Like renaming, starting from the root, all the directories to shorter ones or using the old DOS (8.3, like “dokume~1.doc”) names that windows can auto-generate for you. Personally, I have two favourite ways of handling this.

1. Map the parent-directory of the file/folder you are trying to access/delete as a network drive and access your files that way.
   This is particularly useful if the folder you are trying to access a DFS-share or perhaps a share on the central fileserver filepaths like “\servername01Central ProjectsCentral ServicesIT DepartmentDevelop Methods for Automatically Deploying New Central Servers2.2.1 Auto-Deploying SQL-Server 2005 ClusterDocumentsPreparationsWhitepapersSQL Server 2005 Failover Clustering White Paper.doc”
2. Create a new share to a folder further down the hierarchy. This works locally too if you are logged on to, say, SRV01, you create a new share on “D:FilesharesCentral ProjectsCentral ServicesIT DepartmentDevelop Methods for Automatically Deploying New Central Servers” called “Autodeploymethods” and access it from “\SRV01Autodeploymethods”. That way the filepath doesn’t exceed 255 characters.

Now. When designing fileservers, you really should think about how deep the filepaths may get. This is especially true on DFS-shares since you might have to deal with the full FQDN too, and not only the actual folder structure. Many big corporations I know uses “codes” for departments and assign a project ID (quite simply a number or maybe an abbreviation) to each project and uses theese for the fileshares too. Another scenario that could lead to similar problems are intranet sites where users can create and manage their own subsites and where filenames and folders are not stored in a database.

I have only seen this phenomena on Windows systems so far, and I’ve actually used a linux Live-CD on occasion when admin access is denied.

Read More:
http://support.microsoft.com/kb/320081

In case you wonder why on earth I post when there’s allready a “solution” out there; Pagerank, baby!
Not for me, but for the forum post making it show up earlier on google.

Some customers have notices the the system-logs on some Unix machines are completely swamped with “connection error”, “TCP Connect failed”, “TCP Session Lost” and similar and after a bit och research the problematic servers were narrowed down to those monitored by Operations Manager. Specifically, those who are targeted by a TCP Port Check.

It would seem like the TCP-connection never fully initializes on the target server. Kind of like knocking on your neighbours door and then hiding. Then when the door opens, no one is there.

Maybe there’s a setting somewhere to modify how “deep” a Port Check should go before closing. Perhaps fully initializing and then sending a proper “Close” instead of just cutting the connection. In a few extreme cases we have noticed that the target server even goes so far as to start a session, but never ending it since there’s no closure and finally having no sessions to spare for the real users. But on most servers it’s just an annoyance since the “real” errors is very hard to be found in all the connection related logs.

Anyway. Just a good thing to keep in mind when running TCP Port Checks from Operations Manager 2007. Keep an eye on the logs when implementing the port checks.

The MSMQ Management Pack seems to have a few problems with it’s discovery script that can lead to the following error showing up in the logs:

The process started at 13:34:40 failed to create System.Discovery.Data. Errors found in output:

C:Program FilesSystem Center Operations Manager 2007Health Service StateMonitoring Host Temporary Files 499788DiscoverQueues.vbs(107, 4) Microsoft VBScript runtime error: Subscript out of range: '[number: 0]'

Command executed: "C:WINDOWSsystem32cscript.exe" /nologo "DiscoverQueues.vbs" {615D37C9-477D-62E2-0833-6ECBF0E89A87} {A176AC83-CC31-01C3-5DE9-E2DFF64E7CC7} "MASKED.server.fqdn" "MSMQ" "true" "true" "False" "false"
Working Directory: C:Program FilesSystem Center Operations Manager 2007Health Service StateMonitoring Host Temporary Files 499788

One or more workflows were affected by this.

Workflow name: Microsoft.MSMQ.2003.DiscoverQueues

Instance name: MASKED.server.fqdn

Instance ID: {A176AC83-CC31-01C3-5DE9-E2DFF64E7CC7}

Management group: MASKED

This seems to be related to the discovery of public queues on some servers that has none. One quick fix, or rather work-around, is to override the discovery on these servers to set DiscoverPublic to False.

Error doing IIS Discovery

Error: 0x80070002
Details: The system cannot find the file specified.

One or more workflows were affected by this.

Workflow name: Microsoft.Windows.InternetInformationServices.2003.DiscoverBase
Instance name: Microsoft.Windows.InternetInformationServices.2003.ServerRole
Instance ID: {A81E4808-4D05-9BFE-4043-DC668527F2D0}
Management group: MASKED

Or…

Error doing IIS Discovery

Error: 0x80070006
Details: The handle is invalid.

One or more workflows were affected by this.

Workflow name: Microsoft.Windows.InternetInformationServices.2000.DiscoverWebSites26to50
Instance name: IIS Web Server
Instance ID: {D36DA76A-027F-8F3E-4160-115279A1E23A}
Management group: MASKED

I have been trying to figure out what file is missing and/or if the “invalid handle” is related. Possibly a file-handle? Could be but not neccesary since these two errors occur on different servers with increasing repeat-count (atleast once-a-day). The IIS MP does call the IIS*.VBS Scripts in %windir%System32 but as far as I can tell, on the systems I have tried it on, the scritps return valid data. This does by no means mean that there is no error and evidently I am missing something. But what? Does anyone have a clue to this?

References and other victims:

• “Error Doing IIS Discovery” on Google
• “SCOM: IIS Discovery Probe Module Failed Execution” on Codejnki’s Blog
• “Error doing IIS Discovery Error: 0×80070006″ on TechNet Forums
• “IIS Discovery Probe Module Execution Failure” on Digital Support Forum
• “IIS Discovery Probe Module Execution Failure” on EggHeadCafe
• “IISDiscoveryProbe Error – any ideas?” on Microsoft Discusson Groups

And no, neither of these provides even a hint to a working solution.

Symptoms

This “research” comes from getting an obscene amounts of Script or Executable Failed to run in the Operations Console. Each time it was the NetworkAdapterCheck.vbs script that could not create PropertyBagData. The error message copied from one of the alerts looks like this:

The process started at 14:29:26 failed to create System.PropertyBagData, no errors detected in the output. The process exited with 0

Command executed: "C:WINNTsystem32cscript.exe" /nologo "NetworkAdapterCheck.vbs" MASKEDCOMPUTERNAME 0 false true false
Working Directory: C:Program FilesSystem Center Operations Manager 2007Health Service StateMonitoring Host Temporary Files 2882781

One or more workflows were affected by this.

Workflow name: Microsoft.Windows.Server.2000.NetworkAdapter.NetworkAdapterConnectionHealth
Instance name: 0
Instance ID: {F4C478D3-38E5-8C29-3957-E3B7F486216E}
Management group: MASKED

This error repeats almost as often as the script is scheduled to run and appears on almost every Windows 2000 server.

Cause?

I am not really sure, but after a quite a bit of troubleshooting I am pretty sure it all boils down to a malformed WMI-query. What I basically did was to extract the script from the MP and dry-run it to see if I could find anything obvious, which I didn’t.
Since I didn’t have a good debugging too available, like in PrimalScript, I added the VBS equivalent of old-school printf debugging. I basically added

wscript.echo "Line XX:" & Err.Number

after each object/function call in the Main Sub.
The problem seems to be at line 118:

Set WMISet = WMIGetInstance("winmgmts:" + TargetComputer + "rootcimv2", query)

At line 119 I had a debug line that never executes and it would seem like the script is unable to process the query and exits without any error codes.

The query variable contains the following:

Win32_NetworkAdapter Where DeviceID='0' And ( NetConnectionStatus = '0' )

When I was debugging this I started the script like this:

cscript.exe /nologo "NetworkAdapterCheck.vbs" MASKED 0 false true false

I did find that if I change the parameters to this:

cscript.exe /nologo "NetworkAdapterCheck.vbs" MASKED 0 false false false

Then the script runs smoothly and produces an XML-output like this:

<Collection>
<DataItem type="System.PropertyBagData" time="2009-04-22T15:03:44.6495202+02:00" sourceHealthServiceId="C9F5A9F7-00A3-8A86-57B5-34FB634471A8">
<Property Name="State" VariantType="8">GOOD</Property>
</DataItem>
</Collection>

Comments

There’s quite the lot of similar errors posted on various forums on the internet but most of them ends up pointing towards the Antivirus Software as the perp and we have tried various exceptions on the server like excluding HealthExplorer.exe, MonitoringHost.exe and also the entire Health Service State folder with no improvement. But since I was able to run the scripts from other locations (not excluded) on the same server I find it hard to believe that the AV would be blocking it.

Event Type:    Warning
Event Source:    Health Service Modules
Event Category:    None
Event ID:    21405
Date:        2009-04-21
Time:        09:00:26
User:        N/A
Computer:    MASKED
Description:
The process started at 09:00:26 failed to create System.PropertyBagData, no errors detected in the output.  The process exited with 0</pre>
Command executed:    "C:WINNTsystem32cscript.exe" /nologo "NetworkAdapterCheck.vbs" MASKEDCOMPUTER.DOMAIN 7 false true false Working Directory:    C:Program FilesSystem Center Operations Manager 2007Health Service StateMonitoring Host Temporary Files 150958

One or more workflows were affected by this.

Workflow name: Microsoft.Windows.Server.2000.NetworkAdapter.NetworkAdapterConnectionHealth
Instance name: 7
Instance ID: {A6F89C78-1217-578E-B03D-5ED377A9A40B}
Management group: MASKED

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

Always the “NetworkAdapterCheck.vbs”. Always “The process exited with 0″ which usually means that life is A-OK.

This may be a result of your AV-software blocking the scripts.
The first thing to do would be to exclude HealthService.exe and MonitoringHost.exe. If this doesn’t work, try excluding the entire C:Program FilesSystem Center Operations Manager 2007Health Service State directory.

There is a bug-report on Microsoft Connect that you can add your vote to if you cannot get rid of the problem. But ofcourse, if you have a solution, do post it here or on the bug-report.