This is how you can secure your DevOps Cycle.
As a modern cybersecurity professional for a corporation, you may get many headaches when working together with the people responsible for developing applications, the DevOps team (and vice versa). This article tries to explain why this is the case and structure good communication for a fruitful together in the company. Plus, it outlines two concrete strategies for continuously creating more secure applications: security champions and tool integration.
Fig 1. Security needs to be involved from the start of an agile project.
Security Team’s Requirements
What are the things that you as a security professional need when interacting with the DevOps team?
- When an application is developed in-house, the most important requirement as a security professional is to get involved in the process. Often, developers do not think and/or care about security. Therefore, security is only thought of shortly before the release. I have seen projects developed for years and then cancelled shortly before the production release due to security considerations. In one case, a portal was designed to consolidate customer’s contracts. It was stopped as it would have been a “single big honeypot” for the company. This cost the company millions of Euros for the development of the system. It could have easily been prevented if the security teams were involved early during the software’s planning and development.
- “Security is always a blocker.” As long as this is the attitude that people responsible for a product have, you are in trouble. Only by instilling the attitude of “Security is an enabler” will you be involved earlier (or at all) and work together to create secure applications for your customers.
- You do not have a lot of time for comforting the DevOps department. You probably have enough to do even without taking the responsibility of the security for a new application. There are product owners who should take care of those kinds of things. The time that you invest in security should be spent as well as humanly possible. It does not make sense to explain basic security concepts to people creating and operating the application. You should be able to see the big picture of security in the company.
- If you invest time in checking a single application’s security, you want to concentrate on the important, difficult, and interesting part. Baseline security like a secure server configuration or up-to-date libraries should be the standard. The correct usage of frameworks to prevent vulnerabilities like SQL Injections and Cross-Site-Scripting should not be something you need to check. Instead, you can use OWASP ZAP or your favourite tool to check for flaws in the application’s business logic.
DevOps Team’s Requirements
Try to take your DevOps Team’s perspective now. What do they need when interacting with you?
- They probably hate to interact with you. Or at least they could think of thousands of different ways to use their time instead of talking to you. Their focus is to create meaningful applications for your customers. That is what they get paid for: Features.
- They do not care about security. In most cases, there will not be User Stories that relate to security issues. Things like “Authentication” are implemented as it is a basic product requirement to have an internal area, but then that’s it. The difference between authentication and authorisation is where the problems usually start…
- If security is a topic, time is usually the greatest issue. Either the developer has many more important things to do, which prevents him from focusing on security. Or there is something that needs to be resolved by the developer ASAP. This is usually the time when security becomes relevant: Only if the business need is critical. Either because a vulnerability was detected and can lead to serious troubles if it stays unresolved or a customer is asking for a security assessment, and suddenly, the most unimportant topic of all time becomes a blocker.
Security and DevOps
Why is this an issue now? Security has been a problem for developers for ages…
There was a planning phase before and a testing phase after the development in the waterfall development process. This is where security was involved. Setting up security requirements when creating the system specification and when it was checked before the operation started. Nowadays, there are fast iterations, and the planning is usually done in user stories. It is much harder to integrate security requirements when security needs to be involved every two weeks when new user stories are decided upon. Also, it is not feasible to do a manual penetration test every two weeks after a sprint. In contrast, manual tests were doable after a project period of multiple months or years.
Traditionally, security measures involved a lot of network and firewall rules. When networks were separated properly, a vulnerability in an application opened a much smaller attack surface than interconnected networks (See also our blog on the interconnection of web applications then and now). E.g. network access to an air-gapped network for industrial controls is much harder to gain than jumping to the same control system over an infected website. With web applications for monitoring and steering everything, their vulnerabilities are much more important than ever for attackers to gain access to networks, steal data or sabotage companies.
DevSecOps – The Solution?
What does DevSecOps mean, and is it the solution for all those problems?
DevSecOps is an attempt to bridge the gap between DevOps and Security. The same way that single people or teams take responsibility for development and operations, they are now also responsible for their applications’ security. This does not mean that you need to start developing applications now or that the DevOps engineers will do your work. It means that altogether, you take shared responsibility for the security of the application. As a security specialist, you will support the development of the application (instead of restraining it).
To be your voice in the team and as a relief for you, there is the concept of security champions, which are “active members of a team that may help to make decisions about when to engage the Security Team”. This helps you engage non-security people in security aspects, establish security culture, and scale through multiple (DevOps) teams. Often, product owners feel well in a security champion’s role if security is an issue for them and their product. But also, people with development, operations or testing background are good fits for security champions. However, they need to have a standing in the team to be heard (and not overheard). Finally, these security champions should receive some training through courses or coaching to professionalise their teams’ work.
The security champions will be your “security voice of reason” during the regular sprint planning. When a user story is defined, they can add misuse-cases to it. For example, a user-story may define that a user of an application can download his invoices. The abuse-case may be to download different users’ invoices. If this is forbidden within the acceptance criteria, there will be no doubt whether this security requirement has been implemented. It also serves as a guideline for testing the implementation of the feature.
Your job is to provide the necessary guidelines and tools to the teams (and your security champions). This usually will be a mixture of the following:
Fig 2. Security Measures in the DevSecOps Cycle.
DevSecOps Teams and a secure DevOps Cycle
As shown in Figure 2, there are multiple ways to integrate security into the DevOps cycle. Below is a DevSecOps tool overview in a table format:
|DevOps Phase||Possible Security Tools for this Stage|
Best Practices / Guidelines
Static Analysis (SAST)
|Test||Dynamic Analysis (DAST)
Interactive Analysis (IAST)
|Operate||Infrastructure as Code (IaC)
Logging & Alerting
Below, we dive deeper into some of the topics:
- Policies: What are the overall security policies for the company? The security champions and product owners can use threat models for their application in combination with your policies to create an overall security plan for the application. They will also be used for validation once features are released.
- Coding Guidelines: What is allowed technologies or libraries? How are the frameworks used securely? The DevOps teams then use this information to follow best practices of secure development. During peer reviews, they check whether the guidelines are followed.
- Vulnerability Scanner: Which scanners are used within the company to detect vulnerabilities early? Tools such as SonarQube for Static Analysis or Crashtest Security for Dynamic Analysis greatly help to identify any flaws or issues before the deployment – even in an agile environment. If the scanners are set-up and ready to use, it is easy for the teams to get started with them, immediately get feedback, and benefit from continuous checks for insecure software.
- Infrastructure-as-Code (IaC) Modules: How can a server be configured securely? How does a firewall need to be configured? Using tools like terraform, it is easy to create reproducible infrastructure. You can create modules, e.g. for server set up in your preferred cloud. The module contains not only the server but also network configuration, a firewall and access restrictions. This way, the team can easily spawn their servers with a secure configuration. If you change the module due to new security requirements, all the existing infrastructure is automatically updated with the next release.
- Secret management: How can secrets be stored securely? It is always hard to store your secrets securely. You do not want to end up revealing secrets when they are leaked in code repositories. If you provide a secret management service including guidelines and documentation on how to use it, you can control the application’s secrets, rotate them in case of an incident and monitor their usage. The teams have control of which secrets they need and have an easy way to handle them.
- Threat Intelligence: Are there any new developments regarding the threat landscape? Important attacks on your infrastructure? This is one of the main tasks that you will keep in your hands. In case of incidents or a change in the attacks, you can take over and support the teams on topics where you are most needed.
- Logging & Alerting infrastructure: How do you keep track of the running application? Provide a logging infrastructure for all the teams. That way, you can correlate events and detect incidents that would not have been visible otherwise. If you pre-configure alerts, the teams will be notified if there is an incident within their application. They will be responsible for it and only need to call you if the incident has security implications and cannot handle it themselves.
- Vulnerability Disclosure Policy or Bug Bounty Program: What happens if an external researcher detects a vulnerability? You will be responsible for set-up and lead the vulnerability disclosure within the whole company. If you are setting up such a program, you can pre-screen the issues and create comprehensible chunks of action. Then discuss the issues with the security champions and slowly transfer the responsibility for the issues to them. In the long run, they will be capable of screening the reported issues for their team and only consulting you in critical cases.
By using the tools mentioned above, you will end up with much more secure software. Every software is created by your guidelines that are manifested in the tool implementation. Additionally, you have a team of security champions that are spread throughout the company. If there are serious issues, you will be involved. Security is thought about and integrated from the beginning of software projects till their operation and maintenance. Instead of being the DevOps people’s enemy, you are their friend, and they are happy to ask for your help whenever they need your support. One step closer on the way to continuous security.