El primer paso de una revisión del problema es definir el alcance y los objetivos del análisis. Esto implica aclarar cuál fue el problema, cuándo y dónde ocurrió, quién se vio afectado y cuáles son los resultados esperados. El alcance y los objetivos deben estar alineados con las necesidades y prioridades del negocio, y deben ser acordados por las partes interesadas pertinentes. Un alcance y objetivos claros ayudarán a enfocar la revisión y evitar información innecesaria o irrelevante.
To conduct such this kind of review, a proper journal and recording must be in place to spell out all existed variables and factors that played one or two roles in the major incident. How it happened, when and the impact of the problem. This will suggest how to avoid this in the future by taking right organizational measures; - Allocation of resources to combat the problem catalysts. - Orientation on workforce and employees training. - Facilitate professional career development coaching programs to keep employees aware and updated.
Sometimes, the main objective of a problem review is not just to find out why the problem occurred, but in what circunstances it happened. Finding out the circunstances may lead us to a total different result of the root cause. A wrong code line was sent and we determine that it's an human error, but, we need to go deeper and investigate why it happened. Was it caused by a lack of documentation? Or, was it caused by a lack of attention? If it was caused by lack of attention, you may want to investigate why it happened, if the person is working on more than one project at the same time, if he/her is working over the capacity. A lot of things need to be investigated beyond the technical issues.
First, we need to establish a clear definition of incident severity and the SLA for review. This will help everyone understand their timeline for gathering and preparing the necessary information. If there is no designated incident owner, one should be appointed to ensure that all relevant parties are involved and aware of their responsibilities regarding the review. The incident owner will schedule the review meeting, invite relevant participants, and ensure they are aware of what they need to prepare in advance of the review.
Initially, it is crucial to assess the incident's priority, recurrence, and impact. Following this evaluation, a problem review can be conducted. In cases where incidents are of high priority and recurrent, the problem manager should prioritize them for the problem review. To streamline the management of problems and incidents, ServiceNow (ITSM Module), currently considered one of the most advanced tools, can be employed.
Conducting a problem review after a major incident involves several key steps and tools to ensure a thorough analysis and effective resolution. here are the steps and tools involved: Key Steps: 1. Assemble the Review Team: Gather a cross-functional team including representatives from IT operations, support, development, and other relevant stakeholders involved in managing the incident. 2. Document Incident Details: Compile detailed documentation of the major incident, including incident reports, timelines, actions taken, and communications exchanged during the incident response process.
1. Stabilize the situation. 2. Assemble a review team. 3. Collect data. 4. Analyze root causes. 5. Develop and implement solutions. 6. Document and share learnings.
At times, the primary goal of a problem review extends beyond simply identifying why the problem occurred; it's also crucial to understand the circumstances surrounding its occurrence. For such reviews, it's crucial to maintain a comprehensive journal and record of all pertinent variables and factors involved in the major incident. This includes documenting how, when, and the impact of the problem, aiding in future prevention through appropriate organizational measures: - Allocate resources to address problem catalysts. - Provide orientation and training for the workforce. - Offer professional career development coaching programs to keep employees informed and up-to-date.
Setting the stage would be the first step, here are some best practices/guidelines to follow for a problem review after a major incident: - kick of the workshop within 1/2 days after closing the incident (remember this is not a post mortem but rather after post mortem exercise) - set the stage for two main outcomes: 1. RCA - what caused the incident to occur, and here think from people/process/tools dimensions. We tend to forget that missing process, or lack of training/etc. could also be part of the RCA. 2. Corrective Actions - main objective is to ensure this incident does not occur again. Actions should be captured in the backlog with clear target and accountability
El siguiente paso es recopilar y organizar los datos relacionados con el incidente. Esto incluye los registros de incidentes, registros, informes, alertas, comentarios y cualquier otra evidencia que pueda ayudar a comprender qué sucedió y por qué. Los datos deben verificarse, validarse y clasificarse según el tipo, la fuente y la relevancia. Los datos también deben organizarse en un orden cronológico, mostrando la línea de tiempo de eventos, acciones y resultados. Una herramienta útil para organizar los datos es un diagrama de espina de pescado, que ayuda a visualizar las posibles causas y efectos del problema.
3. Conduct a Root Cause Analysis (RCA): Utilize RCA techniques such as the "5 Whys" method, fishbone diagrams, or fault tree analysis to identify the underlying causes and contributing factors that led to the major incident. 4. Identify Lessons Learned: Identify and document key lessons learned from the major incident, including areas for improvement in processes, procedures, technology, and communication. 5. Develop Corrective Actions: Based on the findings of the RCA and lessons learned, develop specific corrective actions and recommendations to address root causes, prevent recurrence, and improve incident response capabilities.
It's really important that the service manager working during the incident/problem resoltion to keep a detailed timeline of the events, people who worked for resolve the problem. It will make the investigation of the root cause easier and faster.
El tercer paso es analizar los datos e identificar la causa raíz del problema. Esto implica aplicar varias técnicas y métodos para examinar los datos, como 5 porqués, análisis de árbol de fallas, análisis de Pareto, análisis FODA, etc. El objetivo es descubrir los factores subyacentes y las condiciones que contribuyeron al problema, y eliminar cualquier suposición falsa o engañosa. La causa raíz debe ser específica, medible, procesable, realista y oportuna. Se debe preparar un informe de análisis de causa raíz para documentar los hallazgos y recomendaciones.
After you have a root cause, some action plans need to be included and implemented, so, don't forget to document it as well. It's important to have the actions documented, so it can be used in other similar situations and also to avoid them.
6. Assign Responsibilities: Assign responsibilities for implementing corrective actions to relevant individuals or teams, specifying timelines and expected outcomes for each action item. 7. Implement Changes: Implement the identified corrective actions and improvements, ensuring that changes are properly tested, documented, and communicated to relevant stakeholders. 8. Monitor and Review: Continuously monitor the effectiveness of implemented changes and conduct regular reviews to assess progress, identify any new issues or trends, and make further adjustments as needed.
A very important note for this step, especially if the review is conducted with several parties: the incident owner or the person leading the discussion needs to ensure that it does not become a "blame game." The discussion should not focus on assigning blame; instead, the focus should be on understanding why it happened and identifying necessary changes to prevent it from occurring again.
El cuarto paso es definir e implementar las acciones correctivas que abordarán la causa raíz y evitarán la recurrencia. Esto implica priorizar, planificar y ejecutar las acciones que resolverán el problema, restaurarán el servicio y mejorarán el rendimiento. Las acciones correctivas deben estar alineadas con los objetivos y el alcance de la revisión del problema, y deben ser aprobadas por las partes interesadas. Las acciones correctivas también deben ser monitoreadas y evaluadas por su efectividad y eficiencia. Se debe seguir un proceso de gestión del cambio para garantizar que las acciones correctivas se implementen sin problemas y de manera segura.
Key Tools: 1. Incident Management System: Utilize an incident management system or software tool to document and track major incidents, including incident details, response activities, and post-incident reviews. 2. Root Cause Analysis Tools: Use specialized software tools or templates for conducting root cause analysis, such as fishbone diagrams, fault tree analysis software, or RCA templates. 3. Lessons Learned Database: Maintain a centralized database or repository for capturing and documenting lessons learned from major incidents, including recommendations for improvement and corrective actions.
First, we need to decide on the action items, identify the owner for each action item (there can only be one owner), and determine the deadline for each action item. The list of required and agreed-upon action items, along with their respective owners and deadlines, should be communicated to the relevant parties and stakeholders at the end of the review. The incident owner should then follow up on these action items.
El quinto paso es comunicar y compartir los resultados de la revisión del problema con las partes interesadas y el público en general. Esto implica presentar la declaración del problema, el análisis de la causa raíz, las acciones correctivas y las lecciones aprendidas del incidente. La comunicación debe ser clara, concisa y consistente, y debe utilizar canales y formatos apropiados. La comunicación también debe solicitar comentarios y sugerencias para mejorar. Se debe crear un informe de revisión de problemas para resumir y archivar los resultados de la revisión del problema.
4. Action Tracking System: Implement an action tracking system or project management tool to assign, track, and monitor the progress of corrective actions and improvement initiatives identified during problem reviews. 5. Performance Metrics Dashboard: Develop a performance metrics dashboard or reporting tool to track key performance indicators (KPIs) related to incident management, including incident resolution times, recurrence rates, and effectiveness of corrective actions. By following these key steps and utilizing appropriate tools, organizations can conduct thorough problem reviews after major incidents, identify root causes, implement corrective actions, and continuously improve incident management capabilities.
El paso final es revisar y mejorar el proceso de revisión del problema en sí. Esto implica evaluar las fortalezas y debilidades del proceso, identificar las mejores prácticas y brechas, y aplicar las lecciones aprendidas para mejorar las futuras revisiones de problemas. La revisión y la mejora deben basarse en los comentarios, las métricas y los resultados de la revisión del problema. Se debe establecer un ciclo de mejora continua para garantizar que el proceso de revisión de problemas esté siempre alineado con las necesidades y objetivos del negocio.
Ensure that not only is everyone involved properly trained, but that they take ownership of training any of their backup people (assuming multiple people are involved in a large cross-functional team), and that you keep critical information readily available, easy to find, in a single source. Highly recommend 'The Checklist Manifesto' for keeping things simple if you can when it comes to crisis management.
The single biggest blocker to effective post incident problem reviews is a culture of blame. If someone has something to lose as a result of the investigation ( whether that be commercial, reputational or even plain old human ego/frailty ) then the Problem Manager must find a way to foster a feeling of safety for all. This enables proper learning from an incident and the ability to truly address underlying issues. If you have senior managers, business reps or even other silo heads with an axe to grind, it will make the process that much harder. Learning to deal with this and foster a no blame culture for reviews is probably one of the key skills for any problem manager.
Evaluate how effective was your configuration data base in supporting the major incident Evaluate how effective is your process compliance audits. Do you actually do audits for process compliance. Do you have good enough process policies (e.g. Why did not our process policies prevent this major incident)
After a major incident in renewable energy projects, conduct a thorough problem review by analyzing root causes using tools like fishbone diagrams or 5 Whys. Involve multidisciplinary teams to gather insights and implement corrective actions. For instance, in wind power, analyze turbine failure data to improve reliability. In battery storage, assess system malfunctions to enhance safety protocols. Regular reviews ensure continuous improvement and prevent future incidents.
I believe follow define objectives. Assemble a cross-functional team. Gather incident data and documentation. Create a timeline of events. Document the incident thoroughly. Conduct Root Cause Analysis (RCA). Perform risk assessment. Propose corrective actions. Communicate findings transparently. Update procedures based on lessons learned.
I would suggest adding a review of the issues that have been reported. We need to understand if there are any trends that can be identified from these incidents and see if we can find a common thread that might also be affecting the incident rate. This information can help us prevent or reduce the incident rate.