DevOps Best Practices for FinTechs

[michaelbogan]

The nature of FinTech makes customized DevOps a necessity. Most modern, sophisticated companies recognize the value of developing a DevOps culture that removes organizational silos, changes how they do business to deliver value rapidly, and creates iterative feedback loops to enable continuous improvement.

However, there are several unique aspects of the FinTech industry that make a tailored approach to DevOps vital. Delivering innovative value to the customer while navigating an ever-changing regulatory and security scene is challenging, but a FinTech-centric approach to DevOps will help you succeed.

In this article, we’ll explore how DevOps fits into a FinTech world. We’ll look at the approach you should take in developing your unique DevOps culture and engineering practices, and we’ll touch upon successful DevOps practices used by top FinTech companies.

This article assumes you’re familiar with the basic concepts of DevOps, including the idea that DevOps stresses cultural change, not simply the adoption and implementation of technology tools.

What Makes FinTech Different?

While most tech companies share many common concerns, several aspects of the FinTech space make it substantially different from many other industries. If you want to develop an effective, personalized DevOps culture in your FinTech, then consider these unique factors.

Although the traditional software industry already changes rapidly, the FinTech landscape is evolving more rapidly. FinTech is a relatively new domain even though it operates in one of the oldest domains of human enterprise: financial services. However, recent advances in technology combined with changing consumer expectations and comfort result in a segment of the global economy that is constantly responding to market changes.

Consumer-driven market changes

Some of these changes are driven by consumers, who only recently have come to trust technological solutions for their basic financial needs. People are beginning to demand increasingly sophisticated FinTech solutions for financial tasks that have been performed manually for decades or even centuries.

For example, FinTech services are taking the real estate industry by storm. Of the companies listed in Forbes FinTech 50 for 2022, five FinTech companies are in the real estate sector. From offering new ways to invest in rental properties to disrupting the mortgage experience for today’s homebuyers, FinTechs in real estate are driven by a younger, technology-savvy demographic. This next generation of consumers demands the same ease of use in the property investment and mortgage experience that they have come to expect in routine point-of-sale and personal banking services.

Legislation-driven regulatory demands

Other changes in the FinTech space are driven by a volatile regulatory environment. Traditional financial institutions have been known for their frustratingly slow response to changes in digital technology. Much of the legislation in place around the planet was crafted for a financial industry that bears little resemblance to the modern FinTech industry.

As governments attempt to reconcile legacy finance law with the innovative financial products and services provided by FinTech companies, they’re constantly studying new products. For example, the Consumer Financial Protection Bureau (CFPB) analyzed buy-now-pay-later (BNPL) in its new report.

Therefore, FinTech companies often find themselves needing to balance their pace of innovation against regulatory constraints.

A company in the FinTech space is not like any old software company. Its business landscape is different, so it’s sure to have unique processes. This includes the area of DevOps. With its use of automation and feedback to achieve rapid delivery of value to the customer, DevOps is essential to the strategy of any successful software company. FinTechs, however, have additional DevOps considerations that are unique to their space.

Crafting Your DevOps Culture

Adopting a DevOps methodology impacts a company’s developer culture as well as its practices. In FinTech companies, certain cultural elements of DevOps should receive special attention.

Decentralized ownership

One of the most important cultural elements to consider is the idea of “decentralized ownership”. In order to have the agility to respond effectively to changing consumer demands and regulatory constraints, ownership of specific product features or infrastructure requirements cannot be walled off in siloes. Any competent employee—or team—who identifies a critical need or task should be empowered and encouraged to work towards its resolution, independent of organizational structure or rigid job descriptions.

Valuing self-criticism

A related element to integrate into your DevOps approach is to stress the value of being self-critical. As operators in the vanguard of consumer financial technology, FinTechs will inevitably encounter things going wrong. If your culture doesn't make it acceptable (and encouraged) for teams to examine what role they played in how something went wrong, you won't be able to innovate or respond quickly enough.

Readiness for breach

As FinTech services become more vital for the modern consumer, FinTech companies will increasingly become the target of security threats. Mature teams take an approach to security that prepares for breach, assuming that a significant security incident will occur at some point. This posture will impact their approach to the software development life cycle (SDLC) and DevOps strategy.

Empowering Your Engineers

As DevOps has so much to do with the culture we create in a company, it can be helpful for FinTech leaders to consider their engineering culture specifically.

High communication regarding requirements

In FinTech, it’s especially important to communicate business goals and constraints to your engineers. Of course, every engineering team would like this; but in FinTech, we often have to modify our product and its features not just to create value for our customers but also to satisfy regulatory requirements. For an engineer who takes their craft seriously, understanding technical requirements without the accompanying rationale can be a challenge.

To be realistic, however, even the best efforts at high communication may fall short. Sufficient communication across the organization or the engineering team is not always achievable. Where there may be gaps or the possibility of gaps, put “policy as code” in place to act as preventative controls for what humans might get wrong.

Strategic innovation amidst a constantly changing landscape

Another critical area of engineering culture that you should give thought to is how your engineers should spend their “innovation energy”. In an industry with so much change influenced by external factors, your engineering resources are extremely valuable. Any time your engineers can free up for innovation work should be used with judgment. As an organization, discuss when and how to use your resources toward innovation.

Also, keep in mind that innovation is not always about doing more. Sometimes, resources can be put to answer the question: Can we do less? When innovating in this space, the answer may often be, “Yes, we can save some energy here.”

Balancing Security and Agility

Security is, of course, a vital concern for any tech company. In FinTech companies that deal with sensitive personal and financial data, the consequences of a security breach can be devastating. A key influence on your DevOps culture is the tension between the desire to move fast and the necessity to build secure services.

FinTech companies work to create a culture where this conflict isn't resolved with an either-or solution. Which element should we sacrifice—security or innovation? Instead, these top companies push their teams to always look for both-and solutions. How do we create an innovative feature AND meet our challenging (and legally mandated!) SLA and security targets?

As part of your DevOps approach, you’ll certainly use automation to improve the throughput of your software delivery machine. Because of this, it’s important to give a lot of thought to the targets you choose for automation. You’ll need an evaluation process in place that helps you quickly determine what to automate. This way, your automation work will produce the biggest improvements in delivering value to the customer while addressing legal and regulatory requirements.

Best Practices: Insights from an Industry Leader

On the subject of DevOps, I had a conversation Jeffrey Hamblin. Hamblin is an engineering lead at Marqeta, a FinTech company focused on innovative payment solutions through modern card issuing. In 2021 alone, Marqeta processed over $110 billion in payment transaction volume. As a leader in the FinTech space, Marqeta has had many opportunities to explore the DevOps space and develop a successful DevOps strategy. Here are some of the specific best practices that Marqeta found useful in their DevOps culture:

Automation of source code chain of custody

In their code repositories, Marqeta automates the enforcement of branch protection rules. This isn’t because they don’t trust people (although this practice does help reduce error-prone and time-consuming manual work). Instead, this enables Marqeta to assert to auditors that reasonable dual-human control exists for production changes.

This type of enforcement needs to be accompanied by education to build a culture of understanding. Putting up guardrails can cause friction, especially if not everybody in the organization is on the same page about compliance requirements. In a highly-regulated industry like finance, every member of the team needs to understand that compliance regulations are non-negotiables that can get your business heavily fined or shut down if you don’t give them heed.

On this topic, Hamblin commented, “Discovering ways to make the necessary assertions of compliance that simultaneously benefit your culture or your quality is a rewarding way to creatively engineer the processes for your team.”

Continuous measurement of DevOps performance

No company can know ahead of time how a change to the SDLC process might deliver more value to the customer (for example, through faster releases of features or better code quality). However, by continuously measuring DevOps performance, a company can test the effectiveness of any changes and respond early to this feedback.

For Marqeta, their most important metric is commit-to-deploy latency, or the amount of time between when a change is “done” and when that change is adding value in production. They found that an increase in this time indicated sub-par functioning of their engineering systems, while a decrease usually accompanied higher confidence in the process and the development process.

Whether it’s a DevOps team adopting chaos testing or going through incident management training, measurements help Marqeta track how those initiatives translate to business outcomes. By tracking metrics related to sprint health or DORA metrics like mean time to recovery (MTTR), an organization has data to decide how best to use its resources.

Provide motivated teams with great tools

Hamblin shared the following:

One of the most fulfilling events occurred when we integrated a code scanning and analysis tool into our CI pipelines. Some teams saw that and really took the initiative to raise the bar. Discussions of “what should be a blocking gate” on merges began to take place. Shining the light on that mindset and celebrating those teams is the name of the game.

What’s the takeaway? Choose tools that can bring to light what DevOps teams are doing well and concentrate on that, balancing out the near-constant focus on what isn't going well. When a team is motivated, provide them with tools to help them lean into their strengths, and you might see even higher capacity and performance.

Conclusion

The unique character of the FinTech world requires a thoughtful, forward-thinking approach to your DevOps strategy.

Striking the right balance between security and agility is key. You’re responsible for creating a culture. Why not create a culture of possibility where members of your DevOps team ask themselves, “Instead of security or agility, why can't we have security and agility?”

Taking the time upfront to discuss the DevOps culture that makes sense for your organization is a worthwhile endeavor. After all, it's your culture and your people that will guarantee your survival and prosperity in the wild world of FinTech.

Want to connect with and learn from fellow FinTech developers? Join the Marqeta Developer Community.