Post-Incident Reviews

The time-sensitive nature of operational interruptions require response teams to place the restoration of business operations as their top priority. So, during this time, analyzing what went wrong or how they could improve are forced to become lower priorities.

Although they do not address those questions immediately, they are incredibly important, so teams must conduct a post-incident review – an analysis hosted by individuals involved in and affected by an incident that evaluates the cause and necessary improvements, to ensure effective incident response.

For those who are new to the IT world, just need a refresher or are hosting your first post-incident review, this guide is for you. It will provide you with a comprehensive understanding about post-incident reviews and how to manage them.

What is a Post-Incident Review?

After an incident has been resolved, it is imperative that teams set aside time for a post-incident review so that they have a better understanding about why an incident occurred and assess the actions taken for resolution.

Post-incident reviews’ main purpose are to evaluate data from the incident to identify why the incident occurred and ultimately, make internal improvements that will prevent the recurrence of similar incidents in the future.

During the post-incident review, the designated review team creates a detailed write-up outlining the time-line of an incident to share during the post-incident meeting. These meetings foster collective knowledge and enhanced collaboration, by involving everyone in the group discussion focusing on incident resolution and proposed next steps.

Overall, post-incident reviews encourage transparency among teams and empowers them to continuously improve their processes.

The Importance of Post-Incident Reviews and How They Facilitate Continuous Improvement

While post-incident reviews may seem fairly intuitive, it is essential for teams to understand the positive effects that follow post-incident reviews as well as how to conduct them.

One important thing to note about post-incident reviews is that they should be blame-free. It doesn’t matter who was there when the incident occurred, it just matters that it was resolved.

By practicing this, accountability and transparency is enhanced across the team. Team members will feel more at ease to offer solutions and opinions if the fear of being judged or reprimanded about the incident is eliminated.

This overall improves company culture by improving trust among team members and empowering open-lines of communication and collaboration.

Fostering open communication and collaboration also plays a large part in facilitating continuous learning and improvement. Many times, in a culture that does not encourage transparency, innovative and creative ideas are left unsaid. But, when encouraged to speak up and find innovative solutions, teams can more effectively improve their incident response processes during the post-incident review.

Purpose of Post-Incident Reviews

While already slightly touched upon, having a comprehensive understanding about the purpose of post-incident reviews goes a long way. The most basic purpose is to reflect on incidents and analyze what couldn’t be analyzed during the actual response process, but that is not the only reason to conduct a review.

Some of the other goals of post-incident reviews are:

Identifying Why an Incident Occurred

During a post-incident review, the review team analyzes an incident and the underlying factors that caused it. It is important to note that when determining the cause, there is not always one origin to the incident and there may be many contributing factors that must be analyzed to determine why an incident happened. Essentially, by identifying contributing factors, teams can pinpoint, and plan to eradicate potential vulnerabilities and inefficiencies within critical systems.

Creating a Transparent Culture

As mentioned previously, encouraging transparency in post-incident reviews is an excellent practice that can enhance company wide transparency that results in improved employee satisfaction and efficiency, and promotes accountability. This can also be useful as a first step for teams trying to drive a culture of accountability, by allowing management to get a glimpse of how the team will adapt to practicing transparency.

Preventing Future Incident Recurrences

Post-incident reviews allow IT teams to prevent incidents from recurring in the future, because they are able to pinpoint why an incident happened and repair the problems. After resolving an incident, they can also test the effectiveness of the fix, by measuring how often their monitoring tools are flagging similar events.

Improving the Incident Management Process and Workflows

While post-incident reviews ultimately analyze an incident and how it happened, they also evaluate the effectiveness of the incident response plan. When looking at the incident resolution process as a team, there are many points that can be brought up for improvement. Teams can pinpoint inefficiencies and significantly improve the response plan during a post-incident review.

How to Prepare for a Post-Incident Review

After an incident has been resolved, it is essential to immediately begin preparing for a post-incident review so that your team can make the changes needed to improve incident response and prevent future damage.

So, here are the steps that must be taken prior to the post-incident review:

Schedule a Post-Incident Review Meeting

When an incident occurs, the response team will categorize the event based on severity or urgency during incident response. This is a step in the incident response process that is important when scheduling a post-incident meeting as well. Generally, if the incident was high-priority, meaning that its impact was severe, it is important that the post-incident meeting is scheduled within the next few days. With that being said, it is a good practice to schedule any necessary post-incident review meetings within a week of the incident.

Invite Relevant Team Members

When planning a post-incident review, involving the right team members is essential for both effectiveness and efficiency of the meeting. If people are left out of the meeting there is the potential for information gaps due to their inability to contribute to the meeting and their reliance on relayed information. So, it is important to invite the incident responders, system administrators, customer representatives, and any other subject matter experts to ensure seamless communication and effective post-incident review.

Collect Incident Data and Documentation

In order to conduct an accurate post-incident review, you must gather all of the relevant data. This data can be found in many places including incident reports, monitoring system logs, incident alert management platforms, and both internal and external communication records. By compiling substantial data prior to the post-incident review, teams can facilitate better collaboration, because everyone will have access to the same information during the meeting.

Write the Post-Incident Report

Once the meeting is scheduled and all the relevant information has been gathered, the post-incident report must be created, so that information can be easily distributed at the meeting. Writing the post-incident report before the meeting also allows for an objective understanding of the incident that can be used for future documentation without any biases that may arise during the meeting.

The report should include these subjects:

Timeline of the Incident

There must be a detailed timeline of the incident from the time that it occurred to when the incident was resolved.

Incident Summary and Impact

A summary of what the incident consisted of and its impact is also a crucial addition to the post-incident report.

Action Items

During the meeting, action items are delegated to the right people and discussed. So, having them in the report allows for structure and documentation of these next steps.

External Message

If the incident caused disruptions, there must be a message formulated that will be delivered to the affected parties. This can be edited during the meeting, but a baseline message is recommended, so that it does not interrupt the meeting’s progress.

Post-Incident Review Meeting Agenda

To ensure that these meetings go smoothly and result in positive outcomes, you must take a structured approach. So, the following is a recommended agenda, for a post-incident meeting:

Summarize the Post-Incident Report

The post-incident report has substantial data about the incident that may not have been accessed by everyone yet. So, by sharing the contents of the report, teams can ensure that all parties are on the same page and equipped with the same information, to avoid miscommunication and allow for the best possible outcomes.

Identify Underlying Causes of the Incident

Once the post-incident report has been shared, teams can begin to identify the underlying causes of the incident. Some of the causes may have already been noted in the incident report, but as mentioned previously there is not always one cause to an incident. This ensures that all potential causes have been exhausted so that engineers can have a clear understanding of which systems may need extra attention.

Analyze Client Impacts

Incidents can have different effects on clients ranging from service disruptions to data breaches, so it is imperative that the effects are evaluated and noted to ensure that client issues are tended to.

Evaluate the Effectiveness of the Incident Response Plan

When reflecting on an incident, there are oftentimes areas where the response teams wish they could have performed better, in hindsight. So, by evaluating the effectiveness of the incident response plan after an incident, teams can discover potential opportunities that will improve the incident response process in the future. But again, this process should be blame-free – during this time teams should be analyzing the actual incident response process, not the performance of specific responders.

Assess the Communication and Collaboration During the Incident

Communication is key to great incident response, especially when it comes to efficient response times. After an incident occurs, incident alert management platforms are typically utilized to promptly deliver alerts to response teams, who may then need to escalate the incident. So, ensuring the seamless communication between response teams, management, subject matter experts, and their monitoring systems is crucial.

Discuss Next Steps to Preventing Future Incidents

Once the incident has been discussed and conclusions about its origins have been drawn, teams must delegate the action items. These should be recordable actions that various team members are tasked with that will aid in preventing future incidents. This could include fixing vulnerabilities, rewriting sections in the incident management plan, or investing in more suitable alerting, monitoring or communication platforms.

Conclude and Document

After all of the relevant information has been exhausted, teams can conclude the meeting and document their findings. Documenting the meeting is essential as it helps response teams when responding to future incidents and avoids the miscommunication or misinterpretation of meeting conclusions by providing written documentation.

Follow-Up Actions

Once the meeting has been completed, team members will begin executing their respective action items. It is crucial that these are accomplished and that management follows up on their completion. This helps to make sure that the meeting was meaningful and provoked improvements for the organization.

Post-Incident Review Best Practices

While the technical information is out of the way, post-incident reviews have a snowball effect on many other organizational aspects including company culture and incident management. So, these are the best practices that you should follow to enhance post-incident reviewing and in turn improve your business model:

Foster a Blame-Free Culture

With a blame-free post-incident review process, teams will be able to speak openly during incident reviews without feeling poorly about themselves and how they handled an incident. If there is no blame within the process, teams can facilitate a more objective post-incident review that promotes productivity and improvement.

Clearly Define Response and Review Teams

By clearly defining response and review teams, it is much easier for the organization as a whole to be on the same page about the incident management procedures and specific incidents. These teams will develop a strong understanding about the processes required to manage critical incidents ensuring they are promptly and effectively resolved according to company policy and that significant improvements can be made during post-incident reviews.

Prioritize Post-Incident Reviews and Following Actions

Incidents can be stressful, so wanting to push off post-incident reviews is normal, but this is not a good practice. By prioritizing these reviews, response teams will have a fresh memory that will allow them to give accurate and substantial information about an incident. Additionally, by conducting post-incident reviews soon after an incident occurs, teams can quickly make improvements to the affected systems preventing further damage.

Create a Well-Structured Documentation Database

Documentation databases are crucial for enhancing collective knowledge and improving procedures. Review teams can refer back to this database during post-incident reviews to gather information that may be helpful in identifying the cause of future incidents or changing response plan procedures. This is also helpful during incident response because response teams can quickly find information that could expedite their response, if the databases are well-structured and organized.

Integrate Post-Incident Review Findings and Insights into the Incident Response Plan

Oftentimes, when reviewing incidents there are lapses in response times or procedures that affect incident response. This reveals the importance of the continuous improvement of your incident response plan. So, teams should use post-incident review findings to enhance their incident response plan to ensure the most effective use of their time.

How OnPage Enhances Post-Incident Reviews

Post-incident reviews are significantly improved with technology. Manual procedures and logs can delay processes and decrease productivity. So, we want to offer OnPage as a solution to help enhance your post-incident reviews.

Some of the features of OnPage that can aid post-incident reviews are:

Post-Incident Audit Trails

When an alert is sent through OnPage, there is an audit trail that documents when an alert was delivered and read. This is an essential piece of information that can be logged when investigating an incident. Additionally, this can reveal the effectiveness of your incident response by evaluating how quickly your response team is actually reading alerts after they receive them and then analyzing their response times to see if they are taking immediate action.

Centralized Messaging Platform

OnPage also provides secure two-way messaging that facilitates seamless communication and collaboration during incidents. By having a centralized platform that the entire response team has access to, gathering chat records is a simple process that expedites the post-incident reporting processes.

Improves the Incident Response Plan

Overall OnPage will help to improve the incident response plan during a post-incident review. There are many times where response teams wish they had not missed an alert or did not check their email fast enough, which resulted in delayed response and increased damage. Which is why OnPage is a great tool to bring up in your next post-incident review. By implementing OnPage teams will see great improvement in their response times allowing them to focus on other incident response plan improvements in future reviews.