Creating an Alert Rule in Grafana

Overview

This guide provides step-by-step instructions for creating an alert rule in GRACE using Grafana. You will learn how to select a dashboard panel, define alert conditions, configure notifications, and verify your alert setup.

---

1. Navigate to the Desired Dashboard

1. Log in to Grafana.
2. Go to: Dashboard Builder → Dashboard.
3. Select the dashboard you want to monitor, for example: Kubernetes / Views / Pods.
4. Set the filters:
   • Namespace = project-$NAME_OF_YOUR_PROJECT (e.g., project-aak)
   • Pod name = e.g., d-tool-chat-app-aak-* (your application pod)

---

2. Make the Dashboard Editable

1. In the top-right corner of the dashboard, click Make editable.

---

3. Select the Panel for Your Alert

1. Choose the panel you want to monitor, for example: Memory Usage by container.
2. Note: A pod may contain multiple containers.

---

4. Create a New Alert Rule

1. In the top-right corner of the panel, click the three dots menu.
2. Select More → New Alert Rule.

---

5. Configure the Alert Rule

You will be taken to the alert rule configuration screen.

Step 5.1: Name the Alert Rule

• Example: tool memory usage

Step 5.2: Define the Alert Condition

1. Set the condition type: WHEN QUERY is above.
2. Enter the threshold value (in bytes), e.g., 90000000.
3. Click Preview alert rule condition to verify which containers trigger the alert.

Step 5.3: Add Folder and Labels

1. Create a folder for organizing alerts:
   • Click New Folder → Enter name (e.g., project-aak).

Step 5.4: Set Evaluation Behaviour

1. Click New evaluation group.
2. Enter a name for the group.
3. Set the Evaluation interval (how often rules are checked), e.g., 1m.
4. Click Create.

Step 5.5: Configure Notifications

1. In the Notification policies section, select the contact point you created earlier (e.g., test email contact).

Step 5.6: Configure Notification Message

1. Add a descriptive message to help identify why the alert was triggered.

---

6. Save the Alert Rule

1. Click Save rule and exit in the top-right corner.

---

7. Verify the Alert

• Ensure the alert triggers when the defined condition is met.
• Check that notifications are sent to the configured contact point.