Description
Data is at the center of a great business let alone a great product team. This ability is focusing on the full spectrum of helping our organization make better decisions.
Attributes of this ability:
- Creative skill to ask valuable questions
- Technical skill to get the data
- Creative & technical skill to convert data into insight / answers
- Creative skill to communicate your findings
Tools:
- Web analytics tools such as:
- Google Analytics
- Pendo
- HotJar
- DOMO
- Raw SQL
- Console-based Ruby scripting
Glossary:
- Group =
- Small = Squad or subset of your functional team (engineering, design, etc)
- Medium = Multiple squads, maybe the entire department or your entire functional team
- Large = Multiple departments, maybe the entire company
- Data domains =
- ATS transactional data
- Billing data
- Experience data such as Google Analytics and Pendo
- Internal operational data (data on scorecard)
Milestone 1
I have observed this person showing a consistent, comfortable, continuous, and clear positive impact to a squad when wielding this ability, and therefore I would put them in situations where they can employ this ability with only a small amount of guidance
| Attribute | Example |
|---|---|
| Creative skill to ask valuable questions | Reactively will offer to answer a question that only small groups will see |
| Technical skill to get the data | If using SQL, you understand SELECT, WHERE, and basic JOINs (coming soon: DOMO) ; If using Pendo, you've used Pages, Features, Track Events, and Product Areas to answer questions; 1-2 data domains |
| Creative & technical skill to convert data into insight / answers | If you are given a specific question, the answer you give is reliable/useful |
| Creative skill to communicate your findings | Your answers some times change the course of a decision |
Milestone 2
I have observed this person showing a consistent, comfortable, continuous, and clear positive impact to a squad when wielding this ability, and therefore I would put them in situations where they can employ this ability, with no assistance as well as being a trusted active or passive mentor to others
During shaping of a big bet, value/ops enhancement, or even a bug, if your squad needs some data to make a more informed decision, you are there. There are multiple examples of you giving analysis that has led to better shaping decisions for your squad.
| Attribute | Example |
|---|---|
| Creative skill to ask valuable questions | Sometimes will proactively take on answering a question that medium groups needs |
| Technical skill to get the data | If using SQL, JOINS and GROUP BY are tools you use with ease (coming soon: DOMO); If using Pendo, you've added to Pages, Features, Track Events, and Product Areas in order to be able to answer questions; 1-2 data domains |
| Creative & technical skill to convert data into insight / answers | Your answers show a strong understanding of a single domain (ie job performance, applicant flow, etc), but does not always take into account how one dataset could impact another (ie number of prescreen questions have an impact on applicant volume) |
| Creative skill to communicate your findings | Your answers are organized, clear, and may have some visualization... your answers have a direct impact on decision making at the small and medium group levels |
Milestone 3
I have observed this person showing a consistent, comfortable, continuous, and clear positive impact to multiple squads when wielding this ability, and therefore I would put them in situations where they can employ this ability as well as being considered an expert within this discipline
Doesn't matter if it is shaping solutions or prioritizing opportunities, when quantitative analysis is needed, you are the one that folks call. Within and outside of your squad. Your data analysis has had a lasting impact... for example your work had led to improvements to our company and departmental scorecards. You've also influenced major decisions such as pricing and packaging and/or company strategy.
You are an expert at multiple of the tools mentioned in the description, and when a quantitive survey needs to be conducted, you are consulted, because your surveys never fail to get to the essence of what we are trying to learn.
| Attribute | Example |
|---|---|
| Creative skill to ask valuable questions | You are known to be proactive in answering questions at the medium group level and are often pulled into large group questions to assist in large strategic data analysis needs |
| Technical skill to get the data | You are now able to pull data from disparate sources with ease... no query is too difficult for you; If using Pendo, if we are able to answer it via Pendo, you can get the answers... you also ensure we are making the right structural moves to ensure we can answer questions visa Pendo; 2+ data domains |
| Creative & technical skill to convert data into insight / answers | People come to you to understand the data... nearly every time you seek an answer, your analysis is spot on |
| Creative skill to communicate your findings | You have an array of visualization tools at your disposal (pie charts, spider diagrams, sankey diagrams, etc), and you use them well. People say your insights tells a story that resonates and has a direct impact in not only one time decision making, but are used as KPIs, scorecards, and other operational data. |
Milestone 4
I have observed this person showing a consistent, comfortable, continuous, and clear positive impact to a squad when wielding this ability, and therefore I would put them in situations where they can not only employ this ability but where they set the tone for this at the company level
You have introduced new ways to use the tools we have as well as new tools to our quiver. You are likely the one setting the definition for Milestones 1-3. No major data initiative is conducted without your involvement.
Everything in the earlier milestones +
- 3+ data domains
- Entrusted to be proactive with data needs
- Trusted to answer questions of large groups
Milestone 5
I have observed this person showing a consistent, comfortable, continuous, and clear positive impact to not just internal teams but the community/industry in general when wielding this ability, and they are recognized by the community/industry as an expert
Configuration Health
- ✅ Associated with 5 roles
- ✅ Has been referenced in 4 pieces of public recognition
- ℹ️ Fewer than five people (2) achieved a milestone on this ability. To ensure anonymity, analysis will only appear after at least five people have milestones.
- ⛔️ Last updated: about 2 years ago
- ℹ️ Last conversed about: over 3 years ago
Role & Position Requirements
- Data Analystsmust be milestone 2+
- Escalation Mitigatorsmust be milestone 2+
- Operations Reliability Agentsmust be milestone 1+
- Self Serve Scientistsmust be milestone 2+
- Value Shapersare recommended to be milestone 1+
Examples / Observations
Observation created about 3 years agohttps://careerplug.slack.com/archives/C01C4T6FYLB/p1662487684146559
I love that we are making decisions in a data-influenced way.
"Do our applicants use Dropbox"?
Seemingly innocuous feature right? Well, no... giving more options clutters up the UI and forces the applicant to think. We want them to apply, not to be thinking about if they have their resume in Dropbox or not.
Also 190k applications with resumes and 316 that used dropbox.... for those of you counting that is about 0.001%.
Love sunsetting features that don't serve us. Love doing that using quantitative analysis.
Observation created about 3 years agohttps://careerplug.slack.com/archives/C01SG5UDH7E/p1660753177339599
“If we have data, let’s look at data. If all we have are opinions, let’s go with mine.”
― Jim BarksdaleWithout data, you're just another person with an opinion.
― Deming
I love it when these quotes show up as true in daily product work!
Engage was debating whether or not the next iteration should include the ability to modify the job title. Then AW came swooping in with amazing data to help answer the question!
I did a little more digging on the usage for adding job title: It looks like about 55% of users are adding job title after creating a new user. Perhaps because it’s at the top of the current Add New User form and maybe they think it’s a required field. Only ~9% of users are editing the job title when editing an existing user. This is making me feel less strongly that this is something to worry about now.
She offered both sides of the argument giving reasons along the way. This is collaboration. This is the way to use data for decision making.
Conversations about Data analysis / synthesis
This section is for CareerPlug folks only. Sign your team up to find your Gruuv!