Observation created about 4 years agoFeaturing:Stephen G.Engineering CommunicationInitiativeStakeholder / Feedback ManagementDevOps Enablement LeadUnexpected, concise, clear, communicated with both summaries and details, and action-oriented. As it pertains to emails about technical topics, it doesn't get much better than this.
I now feel that much more confident that Stephen isn't just thinking about doing the tasks he is assigned... he is thinking about the overall picture and health of Lessonly as we navigate this journey we are on.
Love it!!
Email #1:
HI everybody,
As you may or may not be aware there was a massive hack on solarwinds which provides infrastructure software to many companies. ( https://www.solarwinds.com/securityadvisory ).
I want to present to this group the (preliminary) findings as I investigated our exposure because industry-wide implications and more customers are going to be asking about this. It is going to be beneficial to have some sort of statement to respond with.
If you would like me to work with somebody on what that statement should be please let me know. And if you have any guidance on verbage before we craft such a statement I would appreciate it.
Executive Summary
We use a small unaffected Solarwinds product in non production environments. A couple of our vendors had similar exposure. Our biggest risk is Twilio which actually used an affected product, but they believe they were not impacted by the breach.
Our Direct Exposure
We do not use their software directly in our data centers.
PaperTrail
We use a product called PaperTrail in non-production environments for logging. No sensitive information should be sent there. This product
Vendors
Vendors known to have some exposure to SolarWinds Products
Twilio (Sendgrid, Twilio, Segment)
The company Twilio owns 3 services that we use. Twilio did use the affected software "in a limited fashion." They have been working with SolarWinds and believe they were ultimately unaffected by the breach
Twilio is our biggest risk, but at this point I believe it to be mitigated.
Harness
Harness used an unaffected SolarWinds product (Pingdom). Ultimately I see no risk here, but they did share this for transparency.
CloudBees (Codeship)
CloudBees owns the codeship product that we use. They, like Harness, used some unaffected SolarWinds products
Librato (scoped to our CodeShip product)
PaperTrail (scoped to our CodeShip product)
PingDom (basic external monitoring used by Ops)
Observation created over 4 years agoStephen publicly highlighted the work that Brittany and Matt B. did to keep queue size down when reindexing all Lessons: https://lessonly.slack.com/archives/G8Q5B0EVA/p1607543001201000 This shout-out not only encourages those on the receiving end to continue being good stewards of our systems, but gives all developers an example to emulate. I'm sure not every engineer would have considered the impact of enqueuing any number of jobs—perhaps now they will, and might even remember that keeping queue depth to within a few hundred is a good thing, as Stephen pointed out.
Observation created about 5 years agoHere is the Slack message that inspired this write up
A copy of the post:
Every incident we have is an exercise in unplanned learning. I wanted to make sure we share the lessons from the incidents so that these lessons can benefit more than just those involved in the immediate incident.
https://about.lessonly.com/library/lesson/361443-incident-2020-04-20-access-exclusiveHoly Schnikes!! I feel most supported, most proud, and most elated when a teammate surprises me with something that is so clear, obvious, and valuable that I'm taken aback.
That happened this morning.
Stephen, who we all know is an absolute best when it comes to seeing a need and fixing that need. His Sense and Respond game is next level.
However, I haven't seen the "Systems-Building Stephen" as often, because Super-Hero Stephen is usually what we need.
Well, here is what I love about everything about this message:
- First, the sentiment... every bit of unplanned work, and especially incidents are learning opportunities... FACT 🎊
- Second, it would have been easy to fall back on our current policies and do the bare minimum, but he showed tremendous initiative by saying... "Nah, we are not only going to learn from this one, but we are going to share those learnings by putting them in the place where learning should live... Lessonly!" 🎉
- Third, it would have been easy for this to be a one-off thing. But he said... "Nah, this should be a part of who we are, a part of our culture, so I want to set us up to continue to do this in the future by putting this in a Path". ❤️
I've been pondering a few new abilities, and one of them is Systems-Building... if that ability existed I'd say this was an excellent showing of system building in that it creates a path of least resistance through small actions.
Well done sir, well done... now let's continue to iterate on this so that we can ensure we are continuously improving, continuously learning, and continuously doing better work. 🙌🏾 🙌🏾 🙌🏾
Observation created over 5 years agoAs we've shared in previous Product All Hands and Team Shares, Ross and Josh implemented a new permission to replace our old role hierarchy. The new permission gives customers the flexibility to control which roles a role can edit and assign to people. Throughout the process Ross and Josh thoughtfully implemented the necessary changes behind the scenes. After all the pieces were in place, they removed this new permission from behind the Platform squad's feature flag this week and made it available to customers rather than holding it back until releasing Custom Roles.
Yesterday T-Mobile reported an issue because they noticed their Creators where unable to Edit Managers even though they had automatic management rights for all people and groups. What they reported was in line with how our old role hierarchy worked, but with the new permission they would be able to give Creators rights to edit Manager roles.
When they reported the issue, Stephen was already scheduling a deployment of the latest updates to T-Mobile's private instance. Allowing us to resolve their issue quickly and empowering them to configure the Creator role to meet their needs.
It is awesome to see new changes having such a positive impact, so quickly after being released! Way to go team!
Observation created almost 6 years agohttps://lessonly.slack.com/archives/C047M50C0/p1561058263013000
I absolutely love the empathy and the team-over-self-ness that is shown in this message.
❤️
It is small things like this that make me love coming to work with people who truly care. Not only about the mission, but that show their non-work side and share that with us.
Stephen man, I really really appreciate you!