At Border, we specialize in UX/UI development. Our team of expert UX designers understand how to make great products that consumers intuitively understand and maximize consumer satisfaction. This is typically a weaker area for our clients which is why they seek us out. On the other hand, our clients thoroughly understand their business and typically have large, complex systems that contain the real logic and processes of their world. Being able to expose and interact with these systems in a secure and intuitive way is always a struggle with our clients and it's what we do at Border day-in and day-out. Our "secret" to properly managing this with our clients is:

1. Create/define a clean API separation layer

2. Install an API mocking infrastructure and process

When properly done, it allows the Border engineering team to work independently and remotely with maximum efficiency. It also allows fully remote and hybrid working models for our clients, which they love. Finally, it opens up new possibilities for lower cost development all with almost zero compromise to our clients' security.

The Friction

Maybe you've been developing software with your team for a while, or maybe your small software team has started to grow and you've been noticing the "friction." You know what we're talking about. Things aren't going smoothly anymore. Bugs are starting to pile up and features just seem to take forever to compete. Do any of these quips sound familiar?

1. "Its hard to hire new people. I want full stack developers, but they are hard to find in my mix of technology and they are really expensive"

2. "Ya, outside firms are nice because they bring new tech and great new look, but they struggle to integrate into our complex software stack and run it efficiently"

3. "Using outside resources is hard because of (granting network access, security, custom software, dedicated machines)"

What you need is a way to develop your cool, new application without having to grant special security clearance, train the team and teach everyone all about your business systems' internal workings.

But how can this be done?

It's All About Isolation

When working on complex systems, it's very important to isolate parts of the system so they can be worked on independently. It is also important to minimize the amount of dependencies and interaction between the various isolated parts to ensure they will not be blocked by each other in order to maximize efficiency. In web application development, the most basic and well known architecture that demonstrates isolation is client-server, typically referencing the "Front End (FE)" and the "Back End (BE)."

The FE focuses on user interaction, behavior, look and feel. This is where understanding user behavior, cognition and interaction patterns are important, but less understood by most clients.

The BE focuses on business process, work flows, data storage, synchronization and, most importantly, security. It is the gate keeper to your business data and intellection property and is typically built up over a long time. It contains tons of undocumented tribal knowledge. Training external terms on how this works makes no sense and only a few people in your organization know all the details. Also, access is typically limited due to security including private company networks.

Two of the biggest bottlenecks that we run into with our clients are:

1. Access to specialized BE knowledge

2. Access to actual BE equipment and networks

In order for our teams to work efficiently, especially as consultants coming in, we need to remove these bottlenecks. In our experience, the best way to achieve this is to agree with the client on the following tenants:

1. There is no security in the web browser. No matter what you do, you cannot ensure the security in the client/browser environment.

2. Although your UI is unique and interesting, it does not have to be a secret. Since it's not secure, anyone can access your assets and FE code to pull it apart.

3. A clean, well documented API layer minimizes the amount of interaction/communication needed between teams and allows for a well defined layer where your tribal knowledge can be documented and tested.

If these core tenants are accepted, it then opens up a whole new world of distributed and parallel FE UI development. Different technologies can be used, developers can work from anywhere, and there is no need for specialized development machines or networks. This maximizes your businesses thorough-put since development can be done anywhere, by anyone, at any time. Your only limitation is defining the requirements and ensuring compatibility with your API interface.

But It's Too Hard to Create/Clean-up Our API

Many clients don't have clean API layers or it's mixed between FE and BE. Not a problem. There is no better time than the present to start one. Pick your favorite API framework (GraphQL, REST, SOAP, etc.) and start small. Ultimately, having a well defined separation will pay off because it clearly defines the boundary and context between the two parts of your software as well as the two parts of your organization: BE-to-FE, team-to-team, client-to-customer. The biggest hurdle is trying to build such a system later on since it seems like such a monumental effort and there is no apparent visible gain. If it's done in small bits over time, it naturally grows with your software and becomes an integral part of the development experience.

The key decision is to commit to creating the API and have the correct tools and process in place when creating it so it can grow easily and seamlessly. The most important things are:

1. Common code standard and practice for writing APIs

2. Proper procedures for documenting them and requiring the documentation

3. Tools for testing the APIs once written

4. And finally, a proper mocking system

What is Mocking?

Once your API layer is well defined, the next important step is to create an emulator that can be used for development and testing. This emulation should be easily runnable by developers and not require special access to servers or networks in order to run them. The key is that it is easily extensible and fast. You don't need it to run all aspects of your software as developers are typically working on small/isolated components of the system that only access small parts. The most important thing is that it's easy to use and it's built in from the beginning.

This emulation is referred to as mocking, or to emulate the API call with a mock result. For example, this is typically done by creating a simple server that receives the network requests from your FE application and sends back JSON responses that have been saved as files on the emulation server. Depending on what the developer is testing, it can send back a specific type of response to test a particular condition, or send back a failure to test how the application handles errors. There are many benefits to this, but we think the most important and generally overlooked are:

1. Your saved responses are excellent documentation. No matter how well developers try to document an API, they typically fail to document all the possible unique conditions, especially failure conditions. It's much easier to review a saved response than to try to run an API and review the output.

2. Developer turn-around-time (TAT), or the change-build-run cycle, is much faster. Not having to wait for a BE system to build or respond saves seconds or even minutes. Add this up for every time a developer makes a change and it can be saving hours a day. Your developers are more productive.

3. You need to demo your feature during a scrum review, or even better, show it off to your customer. Don't use the real system as any number of things can go wrong. Use a system that you know exactly how it will respond: a mock system.

4. As mentioned before, developers don't need access to your BE or your network. This pays off in speed as you no longer have to use slow VPNs or build machines. Developers and testers don't need access to your network at all which no longer raises any security concern with your IT team.

How We Mock

If you're not aware, we at Border tech are big fans of component based design and development. In fact, it's the only way we work. Our UI designs are broken down into individual/small components using Figma and our developers work on each component in isolation. We then build larger components out of composites of the smaller ones. In order to do this, we need to define all component behavior as well as be able to test them to ensure they were implemented properly. The two primary reasons we do this are the following:

1. We build ADA compliance in at the component level. This makes it easier to use the component in a larger view and not be concerned that it behaves strangely or does not meet compliance requirements.

2. All possible user interactions are defined and tested including all UX edge cases that are not always seen in the larger application. It's much easier at the component level as our brains are quickly overwhelmed with the complexities at the view level.

The two primary tools we use in our component based development are:

Unit Tests (Functional Testing)

Our unit tests are focused mainly on pure logic/service testing, not UI testing. Things like utility functions and business logic are much easier and faster to test. If they are pure (meaning no inherited state) they are very straight forward to test. Most UI test frameworks are brittle because they are coupled with internal state of the component or the DOM. They ultimately struggle to test what the user sees. Our preferred test running is Jest (https://jestjs.io) as it's fully Javascript, has excellent community support and is very easy to use.

Storybook (Visual Testing)

Storybook JS (https://storybookjs.org) is an excellent tool to both develop and demo components in isolation. Our designers do all their reviews and feedback to developers via Storybook builds/demos prior to integrating it into the application. In order to do this, we need individual stories that capture all the different UX use cases so the developer, designer and QA can easily reproduce a condition and verify its visual styling matches the design.

The key requirements for all of this is that it needs to be fast. Our unit tests are run with every build to ensure that our code changes are correct. Our Storybook stories need to be quickly pulled up and verified without waiting for the entire application to build and for long, complicated reproduction steps to be accurately followed.

When we work with our clients, our process is to reproduce the bug in a unit test or in Storybook before we work on it. If we can't, we write a new unit test to reproduce it and generate mock server responses that simulate the error conditions. This way we can more efficiently develop a fix. Now we have added a new automated test that runs to catch a future regression. We only need one person who has access to the client's application/system to create the failing unit test conditions, which is then assigned out to a developer that does not have access.

Why we mock

Over time we have used a number of tools in our quest to find the best tools and processes for mocking. The key areas we evaluate in a good mocking system are:

1. Usability: The fewer steps to configure and the fewer concepts to understand the better.

2. Integration: How cleanly the mocking system integrates into the code base and build infrastructure.

3. Speed: This is the most important. Mocking needs to be fast, fast, fast. They need to build in seconds rather, not minutes, and need to run in milliseconds for each test. The faster the developer TAT, the greater the increase in dev productivity.

Mock Servers

When we first got started we used actual servers for our mocking. The mock-server framework worked quite well and is an excellent solution. The main problem is the additional configuration steps it requires to set them up; it poses a learning curve for new developers as well as complications when creating stories/unit tests. The separation in the code base of the unit test code vs. the mock response becomes burdensome and confusing. The biggest benefit we found with mock servers is having QA teams take over maintaining them. Mock servers provide a huge benefit for the QA team in terms of testing and reproduction of issues as well as for demos.

Mock Service Workers

The Mock Service Workers (MSW) library can be integrated into your unit tests and stories that captures the API request at the network level in the client. The response is also defined in the client. MSWs are extremely fast and easy to create (much easier than mock server responses) and the test/response conditions are right there in your unit test rather than in some other file location. We've used MSWs quite extensively and have been very happy with how they integrate into our developer workflow.

The API Layer

The above solutions are great and have their pros and cons, but ultimately we found another solution that addressed some other issues that typically come up in development:

1. APIs need to be strict. They are a protocol definition that needs to be followed. We shouldn't send a string when a boolean is required. We also want to make sure our APIs are responding with what we expect. To ensure this in code, we need two things: type-safe definitions of the API request and response, and actual checking of the API response to make sure it meets our expectation.

2. It's good practice to create an abstraction layer above the actual API call mechanics so that responses can be altered to fit our needs or additional work can be done. For example, maybe two different APIs need to be called in order to get the full response and we can wrap them both in a single function to simplify the interaction with the higher layers.

3. Redefining or overriding an API response/behavior should be very easy in client unit test code and with minimal boilerplates.

4. Overrides and responses should be visible right in the same unit test code/story so it's clear what is being overridden and why. Developers should not need to look in another location in the code base to figure out what the behavior will be when calling the mock.

The solution we primarily use now is creating a separate layer of code called the api-layer. Its sole purpose is to create a layer of abstraction above the actual API calls that enforces type-safety and provides documentation. It alters the input-output of the call to meet our needs. It's very easily overridden by allowing developers to hook the actual function call during a unit test. Finally, it organizes APIs so that they are easily identified in the code base. BE teams can jump into the FE code and understand how an API is called and from where. If adjustments need to be made it's in just one location in the code base.

We still use mock servers occasionally mainly because of the benefit of usage with QA teams. If there are multiple ways to mock and one method is much easier and faster for developers, the other way will be forgotten and languish quickly.

Conclusion

Hopefully you now understand why mocking and emulation is a critically important feature of your software architecture. We're sure this was something you already understood, but hopefully this article provided some additional insight as well as some ideas on how to do it. We feel the most important benefit is that it frees up your development team to allow outside resources to easily step in and augment your staff. Additionally, it provides more flexible remote work possibilities for your existing software team members. Ultimately, we feel the minimal additional cost to setting up a mocking system and process will pay off ten-fold when you are hiring new devs or need to augment your staff.