Protection Poker

Protection Poker is a security risk assessment technique for agile development teams proposed by professor Laurie Williams at NCSU.

The description below is slightly modified from the original.

How to play Protection Poker

Risk in Protection Poker – how is it determined?

Protection Poker uses a slight variation of the traditional computation of risk:

risk = (the total value of all assets that could be exploited with a successful attack) × (the exposure)

Risk is always related to the requirements that are to be implemented in the next iteration, often this will be some new, enhanced or corrected functionality. Exposure relates to how hard or easy this change in functionality makes it to attack the system, and in the evaluation of exposure, one should consider the possible ways in which attackers can attack the system (attack surface), what type of breaches they can perform (confidentiality, integrity, availability) and the skill level required. For asset value, the value of the asset for various groups should be considered: the value of the asset for an attacker is important for attacker motivation, whereas the value of the asset for customers, users, the business, etc. highly determines the consequences that a successful attack may have. Assets are typically considered to be database tables or system processes that the new functionality controls.

With Protection Poker, risk of a requirement is rated compared to other requirements of the same system. The goal is not to come to a "perfect" and "universal" risk value, but to rate the security risk of the requirements in order to be able to better prioritise security effort. Before starting to play Protection Poker for a system, it is thus recommended to perform calibration in order to arrive at a common understanding of the end-points of the scale, i.e. what does a <10 or a 100 mean for this product?

Calibration – what to do before playing Protection Poker for a new software system

Protection Poker uses the following numbers to determine asset value or system exposure: <10, 20, 30, 40, 50, 60, 70, 80, 90, 100. What these numbers mean may however vary between different development projects. To be able to prioritise between different requirements, it is important to be able to get a spread in the numbers assigned. Thus a 100 should be given to asset values and exposures that are high for this project, and similarly <10 should be given to asset values and exposures that are low for this project. This is to avoid that, e.g., high risk projects rate every requirement as high risk. That would make it very hard to make prioritizations within the project.

Before starting to play Protection Poker for a system, it is thus recommended to perform calibration. This is done the following way:

Asset value: The team asks itself: what assets are most important in this system, and what assets are of little value. They asset they can think of as most important is given a '100' and the asset they can think of with little value is given a '<10'.
Exposure: The team asks itself: what types of functional requirements can open up most for attacks, and which functional requirements can limit exposure, and assign a '100' and a '<10' accordingly.

In the evaluation of asset value and exposure, numbers should be assigned relative to these endpoints, as well as the values assigned for previously assessed requirements.

Playing the game

Protection Poker is played during an iteration planning meeting, and it is recommended that the full team (including developers, testers, product managers or business owners, project managers, usability engineers, security engineers, software security experts, and others) participates. One person should have the role as moderator, and this person will be responsible for leading the team through the game, and point the discussions in a good direction.

Introductory activities:

Focus is on the specific requirements the team will likely implement during the next iteration.

Step 1 – Common understanding of the requirements: The requirements to be implemented in the iteration are explained to the team (e.g., by the product manager or business owner) and the team members discuss or ask questions to clarify the requirements.
Step 2 – Initial discussion of security implications: The team performs a first discussion of the security implications of the requirements. The moderator can ask leading questions, e.g., "Who would want to attack this system?"; "What would an attacker do if he got access to this data?"; or "What damage could an insider do with this functionality?" The discussion goes on until it starts tapering off.

Assess security risk:

Before performing this step, the participants should have a basic understanding of how risk is determined with Protection Poker. The steps below are done for every requirement in an iterative fashion. During the discussions, the team is likely to come up with security problems and vulnerabilities, likely attacks, possible ways to mitigate attacks, etc. These should be noted, and someone (e.g., the moderator) should be given the responsibility for noting these.

Step 3 – Identify assets: Everybody together identify which assets are created or touched upon by the requirement under consideration. Some of these may have already been assigned a value, and then this value can be reused.
Step 4 – Assign value to assets: For identified assets that have not previously been assigned a value, the team:

Performs a brief discussion about the value of the asset for various actors (customers, users, the business, attackers), and consequences if the asset is compromised.
Each participant picks the Protection Poker card (individually and without telling anybody about which card has been picked) that best describes their understanding of the asset's value
All participants show their selected card to the whole team
The team discusses the rationale for selecting the cards; the team members with the highest and lowest cards explain them to the group, followed by an open discussion until the team is ready to revote.
All participants again select a Protection Poker card individually, that is then showed to the group. If necessary, more discussions may be needed, followed by new rounds of card selection.
When the team has reached a consensus on the asset value (or when there is no use in discussing any further – in these cases the moderator is responsible for making a suggestion for what value to assign to the asset), the moderator notes the asset value. The team now moves on to the next asset (if there are more left to assess) or to the exposure evaluation.

Step 5 – Evaluate exposure: As for asset value, the team performs a series of steps to evaluate to which extent the requirement increases the exposure of the system and the values to attack:

Brief discussion of how the requirement influences exposure, i.e., the ways the system can be attacked and the ease of performing these attacks. Take into account what aspects of the system asset that can be breached, e.g., whether attackers can get full access, only read access, or whether they can affect availability of assets.
Each participant picks a Protection Poker card.
Cards are presented to the whole group.
The values of the cards are discussed, starting with the highest and lowest card.
Revote if needed.
Note the exposure score.

Compute risk and prioritise security activities:

Step 6 – Calculate risk: The numbers assigned for asset values and exposure are used to calculate a risk value: Risk = (the sum of the value of all assets that could be exploited with a successful attack) * (the exposure)
Step 7 – Compare risk related to other requirements: The risk value for the requirement can be compared to the risk value of other requirements to see for which requirements security should be given specific priority.
Step 8 – Prioritise security activities: Based on the risk value and the discussion decision should be made on how to address security for this requirement. The decision should be documented. If there is a need for specific security activities or functionalities, these should be documented together with other requirements (e.g. in the backlog).

Print your own Protection Poker cards