Thursday, November 11, 2010

Solarwinds Orion Network Performance Monitor Bug

I am *scary* good at finding bugs in software. Just ask the Cisco TAC. Or in today's case, ask Solarwinds support. This is a duplicate posting that I've also added to Solarwinds' user community site. If you use Orion NPM and send SNMP traps to another network management tool, READ AND HEED.

Thwack Post Title: NPM 10.0.0 SP1 Bug: Alert Action To Send SNMP Traps Actually BROADCASTS On Local Network

Many thanks to Mariusz from the Support team for helping me pin this down. I wanted to share with all since this might be happening under your nose!

We have Orion NPM 10.0.0 SP1 and have the "Alert me when a node goes down" alert configured with two trigger actions:

  1. Log Alert to NetPerfMon Event Log
  2. Send SNMP Trap to two hosts (Microsoft Operations Manager and Orion NCM).
A DBA told me earlier today that he noticed a server was receiving traps from our Orion poller. He noticed this in that server's Event Viewer Application Log.

With help from Mariusz and Wireshark, we found that the Orion NPM poller was actually broadcasting SNMP traps to! It seems that the workaround is to create a different trigger action for each SNMP Trap destination.  In other words, we changed our trigger actions to this:
  1. Log Alert to NetPerfMon Event Log
  2. Send SNMP Trap to Microsoft Operations Manage
  3. Send SNMP Trap to Orion NCM
As a matter of fact, for each additional valid IP destination we added to the trigger action, it appears that the Orion poller actually generated duplicate broadcasts for each SNMP trap.

If you use this feature of Orion, I recommend you check your settings and maybe run Wireshark on your poller to be sure you're not spewing broadcasts out to your entire server subnet.

Mariusz is filing this as a bug, and I'm not sure what all versions of Orion are impacted. Feel free to add your comments to this thread.