We own our response to major incidents on our platforms, including application downtime, major bugs, and third party service issues by following the Lessonly Incident Response Plan which lives here.
Expectations / Description
Success looks like:
When you are the owner for an incident:
- Everyone involved in remediation know what their role is
- No role is dropped during an incident
- The issue is fully and regularly communicated with the rest of the organization
- The issue is appropriately communicated outside the organization
- Successfully follow the Incident Response Plan
Feedback loop/deliverables
- Internal incident communication
- StatusPage updates
- Incident Retro Document
Details:
The Incident Remediation Lead is responsible for acting as the Incident Owner as defined by the Incident Response Plan during an incident response. This person is aware of appropriate communication channels for an incident and is able to organize everyone involved in remediation.
Key Results/Outcomes
- We own our response to major incidents on our platforms, including application downtime, major bugs, and third party service issues by following the Lessonly Incident Response Plan.
- We are responsible for making sure the following takes place (this individual doesn’t need to take on each of these tasks, but they need to make sure these tasks happen through delegation):
- All communication happens within the appropriate channels
- All necessary investigations happen
- The issue is resolved in a timely manner
- A Post-Mortem meeting takes place within a day or two of the incident
Requirements
- Incident Remediation Lead's must have a position with the reach of 3.1 or higher
- Incident Remediation Lead's must be milestone 2+,Rails
- Incident Remediation Lead's must be milestone 2+,Data
- Incident Remediation Lead's must be milestone 2+,Ruby
- Incident Remediation Lead's must be milestone 2+,Collaboration
- Incident Remediation Lead's are recommended to be milestone 2+,Infrastructure
- Incident Remediation Lead's must be milestone 2+,Engineering Communication
- Incident Remediation Lead's must be milestone 2+,Software Investigation
Configuration Health
- ✅ Has 7 Abilities
- ✅ Is a part of 10 Positions
- ✅ Has been referenced in 1 piece of public recognition
- ℹ️ No one has reacted to this Assignment
- ℹ️ No one has an official rating on this Assignment
- ⛔️ Last updated: over 5 years ago
- ℹ️ Never conversed about
Examples / Observations
Observation created over 5 years agoHere is the Slack message that inspired this write up
A copy of the post:
Every incident we have is an exercise in unplanned learning. I wanted to make sure we share the lessons from the incidents so that these lessons can benefit more than just those involved in the immediate incident.
https://about.lessonly.com/library/lesson/361443-incident-2020-04-20-access-exclusiveHoly Schnikes!! I feel most supported, most proud, and most elated when a teammate surprises me with something that is so clear, obvious, and valuable that I'm taken aback.
That happened this morning.
Stephen, who we all know is an absolute best when it comes to seeing a need and fixing that need. His Sense and Respond game is next level.
However, I haven't seen the "Systems-Building Stephen" as often, because Super-Hero Stephen is usually what we need.
Well, here is what I love about everything about this message:
- First, the sentiment... every bit of unplanned work, and especially incidents are learning opportunities... FACT 🎊
- Second, it would have been easy to fall back on our current policies and do the bare minimum, but he showed tremendous initiative by saying... "Nah, we are not only going to learn from this one, but we are going to share those learnings by putting them in the place where learning should live... Lessonly!" 🎉
- Third, it would have been easy for this to be a one-off thing. But he said... "Nah, this should be a part of who we are, a part of our culture, so I want to set us up to continue to do this in the future by putting this in a Path". ❤️
I've been pondering a few new abilities, and one of them is Systems-Building... if that ability existed I'd say this was an excellent showing of system building in that it creates a path of least resistance through small actions.
Well done sir, well done... now let's continue to iterate on this so that we can ensure we are continuously improving, continuously learning, and continuously doing better work. 🙌🏾 🙌🏾 🙌🏾
Official Incident Remediation Leads
This section is for Lessonly folks only. Sign your team up to find your Gruuv!
Teams needing an Incident Remediation Lead
This section is for Lessonly folks only. Sign your team up to find your Gruuv!
Positions that reference being an Incident Remediation Lead
This section is for Lessonly folks only. Sign your team up to find your Gruuv!
Conversations about Incident remediation lead
This section is for Lessonly folks only. Sign your team up to find your Gruuv!