Forward Foglight Alarms to a Pager

A customer would like to forward alarms from Foglight with all the appropriate information, such as alarm time, source, severity, message, and the rule name, to a pager script which in turn pages the on-call DBA.

The content of this was written by my colleague Minh Nguyen. I validated the steps on Foglight 5.7.5 in October 2016.

Assumption: the Foglight Management Server (FMS) is running on Linux/Unix

1. Login to your Foglight console
2. Go to Dashboards => Administration => Rules & Notifications => Create Rule
3. A rule wizard will started. Create a rule with the following info:
   1. Simple rule
   2. No scoping query
   3. Event driven
   4. Event Name => AlarmSystemEvent
   5. Give it a description

4. Click Next. Under the Condition & Actions tab, enter the following codes block in the Condition area:

return @event.get("change/name").equals("Fire")

Click on the green icon to verify its syntax. 

5. Click Next. We will use the default schedule.

6. Click Next. We will define the Action Behaviors to fire action after 1 consecutive evaluation.

7. Click Next. We will leave Rule Variables blank.

8. Click Finish. The rule is created and saved.
9. Now we need to edit the rule to add severity variables and command action. Click Edit Rule.

10. Go to the Condition & Actions tab => Severity Level Variables to define parameters to pass to the pager script.

11. We will want to add the following variables:

How to add a severity level variable:

Step 1 – select the Type (in our case, all variables are Expression)

Step 2 – enter a name

Step 3 – enter the code blocks for the variable

Step 4 – add the variable

Repeats steps 1-4 above for the following parameters: (in RED are the code blocks). We recommended using

copy-and-paste to eliminate any typing errors:

1. Message:

@event.get("message")

2. Rule:

@event.get("ruleName")

3. Severity:

severity = @event.get("severityName");

if(severity.equals("Fatal"))

{

severity = "FATAL";

}

if(severity.equals("Critical"))

{

severity = "CRITICAL";

}

if(severity.equals("Warning"))

{

severity = "WARNING";

}

return severity;

4. Source:

obj = server["TopologyService"].getObject(@event.get("topologyObjectID"));

if (obj.get("monitoredHost")==null) {

return "Unknown";

}

return obj.get("monitoredHost/name");

5. Time:

@event.get("createdTime")

The variables should look like those below:

12. Click on the Action tab and add a Command Action.

13. Once it is added, click on the CommandAction link. It should take you to the Command Action properties page.

14. Click on Default link to change its value. It will pop up a separate Action Parameter Editor page.
15. Click on the User Defined tab. Enter the command string:

<full path to location of pager script> "@Time" @Severity "@Source" "@Message" "@Rule"

The string essentially runs the script and passes the rule variables created above as parameters to the script (see below).

For scripting purposes, the parameters passed are as follows:

Time=$1

Severity=$2

Source=$3

Message=$4

Rule=$5

16. Click Change. Click Go to Action List.

17. Click Save All to save all changes. Click Go to Rule List.

18. Go to Rule Management and enable the rule.

NOTE: that the customized rule is under the Non-Cartridge drop-down menu.