In this study, we use survey to investigate software security usage, competence, and training needs in two agile organizations. We asked questions about the skill, usage and training needs of different development teams as follows:
What is your skill level (see scale below for skill level)?
Do you currently use this activity/tool?
Do you want to have training in this activity/tool?
The software security activities we use are drawn from Microsoft SDL, OWASP CLASP, Common Criteria, Cigital Touchpoints and divided into 6 categories:
1) Inception: Having a project security officer, Establishing security requirements, Writing abuse stories/cases, Addressing security logistics (e.g. creating relevant security fields)
2) Threat modeling: Threat Modeling, Attack surface analysis Countermeasure techniques Assets analysis, Risk analysis, Role matrix identification
3) Secure design & coding: Secure design (attack surface reduction, secure defaults), Secure coding (secure guidelines), Pair programming, Static code analysis
4) Security tools: Threat modelling tool, Dynamic code analysis tool, Static code analysis tool, Code review tool
5) Security Testing: Vulnerability assessment, Penetration testing, Red team testing, Fuzz testing, Dynamic testing, Risk-based testing, Security code review
6) Release: Incident response management
The software security activities are further divided into two categories:
Frequently Used Activities (FUA): Code review/Tool, Pair programming, Static code analysis/Tool
Core security activities: The rest of the activities.
We argue that FUA may and may not be used for security audit. As an example, static code analysis can be used for checking code styles, size complexities and various code smells but not necessarily for security audits.
It is also possible to focus code review solely on functionalities. Software security is about producing secure software. It is thus erroneous and a pitfall to assume that implementing security functionalities translates automatically to secure software. For instance, Authentication does not by default translate to implementation of strong authentication mechanism. Authorization does not by default translate to implementing complete mediation or avoiding security by obscurity.
We find the following results:
1. Usage of frequently used activities is on the average "high" (~60%) between the 2 organizations. However, in core security activities, usage is low (see Figure above).
2. Usage of core security activities in Org-1 is higher than Org-2. Security group in org-1 could be a factor for higher usage of security activities (see Figure above).
3. Developers are very similar with respect to their skills in FUA.
4. The 2 organizations are similar with respect to usage of FUA but different in other core security activities.
5. Developers have relatively similar training needs in core security activities (e.g. secure design, secure coding).
6. Security testing approaches are different in the 2 organizations.
7. Skill drives usage of activities. Developers use activities where they are skilled.
8. 80 - 100% of developers between 0-5 years of development experience say they have no security experience. Software Engineering educators need to integrate software security training in the curriculum.
Implications
A. Managements can rightly focus effort by:
1. Leveraging activities already integrated into developers daily practice to deliver secure features (Code review, static code analysis, pair programming, etc.)
2. Increasing developers' skills in important security activities as a strategy to increase usage.
3. Investigating need areas (e.g. Secure Design, Security Testing)
B. Implementing security activities in a meaningful way requires a driver such as:
1. Security group
2. Security research group
3. Government policy & funding
C. Research efforts can consider:
Areas of differences (e.g. security testing approaches differ between the two organizations). Expand on the works of Austin & Williams (2011) "One technique is not enough: A comparison of vulnerability discovery techniques". What approaches should be included in security testing? What approaches are efficient and within what context? e.g. Development phase, type of application, using 3rd party (consultant), etc.
You can download the full paper here: https://www.dropbox.com/s/p9x95mxfywzs23c/ARES_ASSD_2016_paper_164.pdf?dl=0
If you would like to run this study in your organization please contact us at:
Scale for skill level
Novice [1]: Have no experience working in this area
Basic [2]: You have the level of experience gained in a classroom and/or experimental scenarios or as a trainee on-the-job. You are expected to need help when performing in this area
Moderate [3]: You are able to successfully complete tasks in this area as requested. Help from an expert may be required from time to time, but you can usually perform the skill independently
High [4]: You can perform the actions associated in this area without assistance. You are certainly recognized within your immediate organization as "a person to ask" when difficult questions arise regarding this area.
Expert [5]: You are known as an expert in this area. You can provide guidance, troubleshoot and answer questions related to this area of expertise and the field where the skill is used.
