Product Quality's Ability: (QA) System Modeling

Definition

The ability to model systems, from the software system under test to client usage patterns to Lessonly business implications of product decisions all help a tester add ever-greater depth to investigations and the bug reports that result from them.

Why

Having models of systems (i.e. the software being tested, client workflows, even business impact at Lessonly) enables a tester to orient themselves to new software and develop test ideas and approaches without a lot of additional input, and in the absence of documentation. It also means developing models around parts of the platform that are higher risk and need more attention, while spending less time in areas that need less attention.

I have observed this person showing a consistent, comfortable, continuous, and clear positive impact to a squad when wielding this ability, and therefore I would put them in situations where they can employ this ability with only a small amount of guidance

Ask enough questions about the system under test to get started testing via the UI, API, or Rails console

Lessonly doesn’t write up-front specifications for new features, and frequently current system behavior is not documented. To operate effectively in the environment, it’s often necessary to get some clarification from teammates on basics but impractical to spend a great deal of time “fully” learning the system. Being able to ask enough for the test effort to deliver value is a key starting point.

Measurement: Feedback from stakeholders confirms that efficient interviews with product or engineering, synchronously or asynchronously, enables testing to make progress. Core concepts and basic behaviors are not repeated topics.

I have observed this person showing a consistent, comfortable, continuous, and clear positive impact to a squad when wielding this ability, and therefore I would put them in situations where they can employ this ability, with no assistance as well as being a trusted active or passive mentor to others

Build mental models of client usage and patterns to guide test scenarios

Testing software as delivered is necessary but not sufficient. Modeling end-user behavior and thinking about usability, discoverability, the client’s environment (e.g. Windows browsers or mobile), product consistency (with the product’s own history, the product’s image, marketing claims, user expectations, other areas of the product, or even comparable products) all go beyond the code as written and bring the user perspective into play.

Measurement: Testing should routinely uncover gaps (in the form of bugs or new stories) that go beyond whether the code works as written, identifying ways to make the platform better for users.

I have observed this person showing a consistent, comfortable, continuous, and clear positive impact to multiple squads when wielding this ability, and therefore I would put them in situations where they can employ this ability as well as being considered an expert within this discipline

Internalize core system models enough to test effectively with few questions, uncovering bugs when code is delivered

This level of testing requires mental models about the database, API calls, and Rails/React to inform risk and coverage decisions.

Measurement: Testing routinely progresses with just the information in the story and the story’s pull request, while still finding important (i.e. fix-before-we-ship) bugs. Bug reports include more technical evaluation of possible root causes of issues.

I have observed this person showing a consistent, comfortable, continuous, and clear positive impact to a squad when wielding this ability, and therefore I would put them in situations where they can not only employ this ability but where they set the tone for this at the company level

Build good enough models to raise questions before build that help inform the build

Using mental models of the platform and of how the client uses the platform enables the tester to anticipate problems in the design stage or even the preliminary discussion stage before any code has been written.

Measurement: Stakeholders report that both design and build are positively impacted by tester participation in planning meetings and discussions.

I have observed this person showing a consistent, comfortable, continuous, and clear positive impact to not just internal teams but the community/industry in general when wielding this ability, and they are recognized by the community/industry as an expert

Form good enough models to help shape P&E/Lessonly direction

Like the last one, but on steroids: tester participation can change the shape of entire business initiatives.

Measurement: This is hard to miss. Having this ability will mean inclusion in most key discussions about important features and team direction.