Security-First Culture

I’ve been working in security now for almost 20 years (one of these days I’ll jot down some colorful stories from my time in this field), and I know of only one way for organizations to excel in this space: The importance of security must be communicated by executive leadership and must be a fundamental part of the technical culture. This means there needs to be strong collaboration between security and engineering leaders, AND technical personnel at all levels must work jointly to produce secure code, infrastructure, and product.

It’s important to me that both our engineers and candidates understand our security philosophy and buy into the practical implications. That’s what this post is for — to detail how we work, give some practical examples, and outline what we’re hoping to achieve.

Shared Ownership

As one of the early enterprise software companies building an industry solution on the public cloud and pushing the boundaries of newer technologies (a serverless computing architecture for example), it’d be easy to focus our engineering resources solely on product and architecture. Ridgeline, however, has made a conscious decision to incorporate mature, robust security as a technical pillar from the very beginning — not as a bolt-on initiative later. We want to be a security leader in our space. Making an early investment is a key part of our plan to get there.

The first step is partnership and shared ownership. Partnership must first exist at the very top — and it must be real. Security, privacy and compliance (and their associated strategies and goals) must be jointly owned, agreed upon, understood, and communicated by the leadership team. Additionally, time to delivery of product and technologies should be a natural part of decision making and planning.

At Ridgeline, these priorities are represented and discussed during initial strategy planning sessions. The goals for security are also represented in our company and team objectives and key results (OKRs). It’s refreshing to me that our leadership team, from our founder on down, can articulate where we are with respect to security goals and why they’re essential to our success. When those around you take the time to explain why security, privacy, and compliance are important — and offer ideas and resources to push them forward — your engineering org is on the right track.

Technical Credibility (Do the Work!)

I strongly advocate that our Security team earns technical credibility with the teams doing the work, especially Development and Engineering. I tell my team to meet people where they are. That is, articulate compliance and security practices in a tangible, every-day context and then switch gears to work directly with engineers for technical design sessions and implementation work. At Ridgeline, for example, this includes not just explaining the importance of AWS security to DevOps and Engineering but working directly with them to properly secure a Lambda instance, configure identity and access management, implement infrastructure as code, employ strong tenant isolation, and so on.

This puts a certain burden on the Security team. Application security engineers must be able to understand and contribute to the codebase, while the operations security engineers need to work alongside DevOps to implement automated security controls. Both roles are responsible for the security frameworks and automation required to ensure making secure engineering decisions is easier. Ridgeline took this into account when making our first Security team hires. They can all “do the work”.

Make Security Easier

I want to expand on the idea that Security’s job is to make it easier for developers to practice good security hygiene. This is important for three reasons. The first is scale. We all know that Security will never scale headcount 1:1 with Development and Engineering. The way I see it, the only ways to reliably scale are…

• Through automation, frameworks, and design patterns;
• By having a Security team with the chops to understand the applications, technology, and associated compliance requirements.

The second reason we value security hygiene is that it produces results quickly. At Ridgeline, we’re sticklers for the basics (patch and configuration management for example). To make it easier to follow secure design patterns, we’ve opted to automate security best practices whenever possible. When our engineers use an API to spin up a new Lambda for example, a firewall is configured automatically, as is monitoring, logging, and alerting. It’s our job to ensure security is an embedded component of “infrastructure as code” and environment automation.

The third reason my team focuses on making secure engineering easier: It reinforces the idea of partnership. Security requires a large upfront investment. Automation is one way to achieve return on that investment, and it’s a concept that resonates with software engineers. If you can execute on automation, it means you have a Security organization that can empathize with the challenges other teams face. When security hygiene becomes burdensome, engineers know they can partner with their security counterparts to create and maintain a scalable solution. This idea — partnership — is a cornerstone of healthy technical culture.

Technical Debt

It’s safe to say that every company has technical debt. It’s a cost of doing business, and it should and does keep all good engineering leaders up at night. The operative question is how to minimize it. When it comes to security, debt accrues fast. Slowing down a little bit to get security and automation right in the beginning pays massive dividends later (a concept our future customers know well).

Take identity and access management (IAM). AWS’s IAM has over 2000 controls, making possible highly configurable, granular permissions. Getting permissions right is essential. Mistakes can break applications, or worse, lead to unauthorized access. Obviously, defaulting to broad, permissive controls is a liability, especially for a fintech company that believes in the principle of least privilege, but the less appreciated complication is the difficulty of fixing permissions post-hoc.

Imagine auditing 2500+ controls for hundreds of engineers across multiple commercial and homegrown authentication mechanisms. That is what we mean by technical debt. At Ridgeline, we insist on automating the implementation of such rules wherever possible. With respect to IAM, we’re not only automating the initial provisioning of a user’s permissions based on role, but we’re also leveraging innovative solutions such as Netflix’s Repokid along with internally built tools to automatically revoke unused privileges over time. Implementing automation is nontrivial, but compared to a failed permissions audit, it’s an uncontroversial investment.

Here’s another example that highlights debt less likely to be squashed without a technical chief security officer and team. When deploying services in a cloud environment, it’s considered best practice to tag the code pipeline with basic metadata: ownership, resources, functionality, etc. It is no surprise that this doesn’t happen with regularity given a manual process, as the benefits might not be apparent to the developer whose priority is to quickly deliver code and services. However, to connect all the dots and add metadata after the fact can require a staggering amount of manual work. Why is this tagging important? Having reliable “what”, “who”, “where” and criticality will prevent huge issues later. Ensuring this metadata exists for your services will aid in numerous operational and security tasks including operational troubleshooting, categorization, asset criticality measurement, ownership attribution, firewall automation, and tracking for auditing and compliance.

The list of benefits goes on, yet this is still an overlooked component of cloud security. For this reason, the Security and DevOps teams at Ridgeline have automated metadata tagging to remove much of this burden from the developers. This means we are less likely to be pulling our hair out during an audit months from now or chasing down owners in an incident response scenario.

Security as Competitive Advantage

If you’re at a tech startup and you’re storing customer data, or if your app reliability is business-critical, or if your customers’ compliance is legally required, you have a philosophical question to answer: Is it worth the effort to ensure security is a differentiator for your company and product?

Choosing to make security a competitive advantage means making an investment in talent, process, automation, and time. Is this difficult? Of course, but it’s easier than the alternative. Given our mission and our market, Ridgeline must be a leader in security. So building out the Security team from the beginning and integrating the team with Engineering were simple decisions.

People tell me I’m a pretty cool dude to work for (I am a motorsports enthusiast — from road racing to rock crawling, an avid gamer, custom computer builder, a fan of all things 80s, plus I occasionally dabble in security). If you’re a talented security engineer and share our perspective on culture and team-building, you might be one of us. I hope you’ll get in touch.

‍