I’m not a security guy. I haven’t done much hands-on software development for awhile either. I’m a development manager, project manager, and CTO, having spent much of my career building technology for stock exchanges and central banks. About six years ago I helped to launch an online institutional trading platform in the US, where I serve as the CTO today. The reliability and integrity of our technology and operations are critically important, so we worked with some very smart people in the info sec community to make sure that we designed and built security into our systems from the start.
Through this work I got to know about OWASP, SAFECode and WASC, and met many of the people involved in trying to make the software world secure. Over the last couple of years I’ve helped out on some OWASP projects, most recently the OWASP Top 10 Proactive Controls (a Top 10 list for developers instead of security auditors) and at the SANS Institute in their SANS Analyst program. My special areas of interest right now are secure development with Agile and Lean methods, and integrating security into DevOps.
I’ve learned that preaching about security principles, “economy of mechanism” and “complete mediation” and “least common mechanism” (and whatever else Saltzer and Schroeder talked about 40 years ago) and publishing manifestos are a waste of time. There’s nothing actionable; nothing that developers can see or use or test.
I’ve also learned that you can’t legislate “security-in”. Regulatory compliance constraints and formal security policies don’t make software more secure. The best that they can do is act as leverage to get security on the table.
I am only interested in things that work, especially things that work for small teams. Because a lot of software is built by small teams and small companies like mine, and because what works for small teams can be scaled up: look at the success of Agile as it makes its way from teams to the enterprise. I want things that developers can do on their own, without the help of a security team – because small companies don’t have security teams (or even a “security guy”), and because the only way that appsec can succeed at scale is if developers make it part of their jobs.
Here’s what I’ve found works so far:
In design: trust, but verify
When designing, or changing the design, make sure that you understand your dependencies and system contracts, the systems and services and data you depend on – or that depend on you. What happens if a service or system fails or is slow or occasionally times out? Can you trust the quality of the data that you get from them? Can you trust that they will take good care of your data? How can you be sure?
All input is evil
Michael Howard at Microsoft has “one rule” for developers: if there’s one thing that developers can do to make applications more secure, it’s to recognize that
All input is evil unless proven otherwise
and it’s up to developers to defend against evil through careful data checking. Get to know – and love – type-checking and regex.
This is simple to explain and easy to understand. It’s something that developers can think about in design and in coding. There are testing tools that can help check on it (through taint analysis or fuzzing). It will pay back in making the system more reliable and more resilient, catching and preventing run-time errors and failures, as well as making the system more secure.
Write good code
Security builds on top of code quality. Bad code, code with serious bugs, code that is too hard to understand and unsafe to change, will never be secure. Look at some of the most serious recent security problems: the Apple iOS SSL bug and the Heartbleed OpenSSL bug. These weren’t failures in understanding appsec principles or gaps in design – they were caused by sloppy coding.
Identify your high-risk code: security plumbing and other plumbing, network-facing APIs, code that deals with money or control functions or sensitive data, core algorithms. Make sure that your best developers work on this code, that they have extra time to review it carefully to make sure that the code does what it is supposed to do, and that it “won’t boink” if it hits bad data or some other exception occurs. Step through the logic, check API contracts, data validation, error handling and exception handling. Spend extra time testing it.
Never stop testing
Agile and XP and TDD/ATDD and Continuous Integration have made developers responsible for testing their own software to verify that it works by writing automated tests as much as possible. This needs to be extended to security. Developers can write security unit tests and run them in Continuous Integration.
Find a good security testing tool – a dynamic scanner or a static analysis tool – that is affordable and fast and generates minimal noise, and make it part of your automated testing pipeline. Run it as often as possible. Bugs that are caught faster will be fixed faster and cost less to fix. Make security testing something that developers don’t have to think about – it’s always “just there.”
Automated testing is an important part of a testing program, but it isn’t enough to build good software. Like some other shops, we also do a lot of manual exploratory testing. A skilled tester working through key features of the app, pushing the boundaries, trying to understand and break things will find important bugs and tell you if the software really is ready. A good pen test is exploratory testing on steroids. For every bug they find, ask: Why is it broken? How did we miss it? What else could be wrong? Pen tests as a security gate are too little too late. But as a health check on your SDLC? Priceless.
Vulnerabilities are bugs – so fix them
Money spent on training and tools and testing is wasted if you don’t fix the bugs that you find. Security vulnerabilities are bugs – you may have to track them separately for compliance reasons, but to developers they should just be bugs, handled and prioritized and fixed like any other bug. If you have a development culture where bugs are taken seriously (Zero Bug Tolerance), and security vulnerabilities are bugs, then security vulnerabilities will get fixed. If developers aren’t fixing bugs, then you have a serious management problem that you need to take care of.
Training works – to a point
I agree that every developer and tester (and their managers) should get some training in secure development, so that everyone understands the fundamentals. But not everybody will “get it” when it comes to secure development – and not everybody has to. Adobe’s karate belt approach to appsec training makes good sense. Everybody should get white belt level awareness training. Every team should have somebody the rest of the team looks to for technical leadership and mentoring. These people are the ones who need in-depth black belt appsec training, and who can take best advantage of it. They can do most of the heavy-lifting required, help the rest of the team on security questions, and keep the team honest.
Simple is not so simple
These are the things that I’ve found that work: simple rules, simple tools, simple good practices. Make sure that at least your best developers are trained in application security. Understand trust and layering-in design. Write solid, defensive code. Test your code for security like you test it for correctness and usability. If you find security bugs, fix them.
Sounds easy. But what I’ve learned from working in software development for a long time is that the easy things are often the hardest to get right.
About Jim Bird
Jim is a development manager and CTO who has worked in software engineering for more than 25 years. He is currently the co-founder and CTO of a major US-based institutional trading service, where he manages the company’s technology organization and information security programs. Jim has worked as a consultant to IBM and to stock exchanges and banks globally. He was also the CTO of a technology firm (now part of NASDAQ OMX) that built custom IT solutions for stock exchanges and central banks in more than 30 countries. You can follow him on his blog at Building Real Software.
About our “Unsung Hero Program”
Every day app sec professionals tirelessly protect the Web, and we recognize that this is largely owed to a series of small victories. These represent untold stories. We want to help share your story. To learn more click here.