Accelerating API Delivery Through REST API Generation

The Challenges of Scaling API Design and Delivery

As we discovered earlier, over half of all organizations plan to invest more in their API program. But that isn’t the full story. Organizations are also encountering obstacles to designing and delivering APIs. Here is a bit more insight from the 2023 Postman State of the API Report:

• 43% of organizations cite lack of time makes it difficult to deliver APIs.
• 37% of organizations cite lack of people limits their API production.
• 32% of organizations cite lack of API design skills.

Most likely, your organization falls into one or more of those categories. So, how are organizations supposed to deliver more APIs with limited time, people, and skills? Let’s look at three common approaches for scaling your API design and delivery:

Approach #1: Designate teams for API delivery. This option frees up teams that are more familiar with API design and development to continue to produce APIs. By dedicating them to producing APIs, they are prevented from being allocated to workstreams that do not involve strategic delivery of APIs. While this doesn’t resolve the issue fully, it may optimize the delivery process when API design skills are limited in an organization.

Approach #2: Upskill your staff in API design skills. Training your staff is a great option for APIs that need to tailor the API design on how consumers will interact with the API.  Teams gain the skills that they need and are able to produce APIs that they otherwise would not have been prepared. API design and delivery focuses on the jobs-to-be-done (JTBD), ensuring that the delivery team can empathize with the needs of the end users and design an API based upon these insights. This is referred to as user-driven design and is used to address high-value and/or high-visibility APIs for public, partner, or internal consumption. 

Approach #3: Generate APIs from data models. For APIs that deliver lower-value capabilities, such as those used for system-to-system integration, generating APIs directly from data models may be a better option. This approach addresses the lack of time, lack of people, and lack of API design skills by accelerating or completely removing the need for writing code to power the API. The emergence of API generation software is making this a more feasible option for enterprises, as we will learn a bit later.

Most organizations start by first identifying individuals or teams familiar with API design and concentrating their efforts on high-value, high-priority initiatives. Over time, API training is delivered to upskill product leaders and development teams across the organization. A federated API coaching model may be used to scale the effort across the organization as well.

With the introduction of API generation software, teams are now able to generate APIs directly from data models when user-centric design is not necessary. But can API code generation be a bad thing? Let’s dig into this topic next by looking at a brief history of API code generation and how API generation software is changing the face of API delivery.

Is API Code Generation Really A Bad Thing?

As APIs became more popular, attempts were made to use code generation to expose data directly over the network. API code generation offers some significant benefits for organizations, beyond the ability to overcome the time, people, and skill limitations we discovered earlier. 

Speeding up integrations as part of an EIP strategy – Code generation expedites the creation of connectors and APIs needed for enterprise integration platforms (EIPs), reducing the time and effort needed to implement complex integration scenarios.

Rapid prototyping – Developing APIs could be a time-consuming and complex process, requiring extensive coding and testing. With API code generation, organizations can now create API prototypes quickly for proof-of-concepts or to identify technical risks earlier in the software development lifecycle.

Unlocking siloed data and legacy systems – System APIs provide access to legacy systems that are used for a variety of needs across lines of business. Leveraging API code generation speed up the process of connecting newer systems to legacy systems and exposing previously siloed data. In the 2023 Postman State of the API Report, 82% of organizations identified private or system APIs as the primary use case for their need for APIs.

Graph 3: Over 80% of organizations use APIs to drive internal integration efforts (Postman State of API report)

Early code generation solutions focused on raw SQL-to-JSON code generation. While these solutions can be effective in specific cases, they lack several critical elements for robust production use, including:

• Role-based access control (RBAC) – Traditional code-generated APIs return all data without any filtering based on permissions. This is fine for internal, specific use cases but most situations require at least basic RBAC support.
• Performance optimizations – Code-generated APIs traditionally lacked pagination support, meaning that all results were sent at once. This is fine for a small table, but for some data sources this might mean returning millions of rows, which can impact database performance and require considerable bandwidth and memory requirements for the API consumer. Additionally, a lack of rate limiting means that clients could be making these kinds of requests often, further degrading database and API performance. 
• Robust customization – Most API code generators lacked the ability to extend or customize the behavior of the generated API. The generated code might support some custom SQL expressions but did not support the inclusion of custom transformation or business logic. This severely limits the usefulness of the generated code to a smaller set of use cases. 
• Lack of API governance support – API code generators were not designed to integrate well with existing API governance efforts, forcing specific naming conventions based upon the existing data model design. 
• Lack of software development lifecycle integration - Limited customization and lifecycle management caused traditional API code generators to require their own delivery lifecycle rather than leveraging existing CI/CD infrastructure. 

Many of these challenges have been overcome by a new category of API code generation tools called API Generation Software. 

Introducing API Generation Software

Recently, G2 introduced a new G2 category called “API Generation Software”, which combines elements of code generation with API management. These tools present opportunities to unlock data from existing systems, drive the generation of API prototypes, and accelerate enterprise integration strategies commonly found within traditional IT departments. 
‍
As we discovered earlier, tools to generate API code have existed for some time. What differentiates this category of API tooling is their support for customization using a low-code approach for customized scripting, fine-grained security, and automated deployment. Essentially, API generation software abstracts the intricacies of API design, making it possible to achieve instant and secure API creation.

How much will it cost your organization to build your next API?

Did you know that it could cost your organization $20,000-$75,000 USD or more to build your next API? This includes work beyond just coding the API against a data source. The API must be properly configured, documented, secured, tested, and deployed. Plus, the more operations it will need to support and the more complexity involved will increase the cost further. 

Check our out API Calculator to find out what your APIs are costing you

When API Generation Software Isn't A Fit

API generation software offer many benefits, but they also come with their set of challenges and limitations that may not make them a good fit for your use case. It's essential to consider these challenges and evaluate whether API generation tools are the right choice for a specific solution. Here are some challenges of API generation software:
‍
Limited customization: While API generation software offers scripting support for customization, it may not always support the complexity required for some APIs. Businesses with highly specialized or complex needs may find these tools too restrictive. Be sure to take some time to evaluate available tools to ensure that they meet your needs.
‍
You have complex data transformations: If an API requires intricate data transformations, such as aggregating data from multiple sources, performing complex calculations, or implementing specific business logic, API generation software may not handle these scenarios well.
‍
When developer experience is paramount: By default, poor quality data models may be surfaced directly through a generated API, resulting in confusion by API consumers trying to work with your API. This can sometimes be mitigated through custom queries that transform internal data models into a more approachable API resource design. However, complex data transformations, as discussed above, are generally outside the scope of API generation software. 

Complex security or compliance requirements: While API generation software often includes role-based access capabilities, businesses with strict security and compliance requirements may need to implement additional security measures or customize API behavior, which can be challenging with pre-built tools. This is common in industries that require data entitlements to be enforced in addition to RBAC. 

Legacy system integration contraints: API generation tools may not seamlessly integrate with legacy systems or technologies that have unique integration challenges. Compatibility issues may arise, necessitating custom solutions.

High-performance requirements: For applications with high-performance requirements, fine-tuning and optimizing API performance may be challenging with API generation software. Custom code and optimizations may be necessary, requiring the involvement of more senior developers as part of the delivery process.

Vendor lock-in: Some API generation tools may lock businesses into a particular vendor's ecosystem or proprietary formats, limiting future flexibility and scalability. While this risk may be acceptable in a variety of circumstances, in some cases it may be better to consider alternatives to avoid lock-in for core APIs that are meant to be differentiators in the market. 

Complex use cases and workflows: Complex use cases that involve multi-step workflows, asynchronous processing, or stateful interactions may not be well-suited for API generation tools, which often focus on simpler request-response APIs.

For most integration and private APIs, API generation software will accelerate your delivery schedule. However, requirements for some APIs may require excluding these tools from consideration. As always, proper due diligence should be performed to validate that these tools will deliver on your expected results and unique constraints.

Incorporating API Generation into the API Delivery Lifecycle 

No longer is API code generation a bespoke solution that is isolated to one team or business unit in your organization. API Generation Software offers new opportunities to enhance the traditional API delivery lifecycle. It is designed to accelerate the API delivery process while integrating into your API delivery lifecycle and tools. 
‍
Let’s dive deeper into how API generation software can be incorporated into your existing API delivery lifecycle. We will do this by looking at each phase of the API delivery lifecycle, comparing API generation software with traditional, user-centric API design and delivery processes. This is what we call “flipping the model” of traditional API delivery: 

1. Plan: Identifying the Need for Speed and Efficiency

Phase Overview: Identify the need, purpose, and scope of a new or enhancing an existing API.

Using API generation tools, a data-driven API design approach may be used to find the necessary data sources to support the API. Customizations to the API may be made using low-code capabilities in the tooling to go beyond the data to support specific consumer needs.

2. Design: Mapping the Data to the API the Need for Speed and Efficiency

‍Phase Overview: The API's architecture, endpoints, resource models, and user interfaces are carefully planned and documented. It is crucial to create a clear design that aligns with the intended use cases and business objectives.

API generation software stands out here by offering an interface that lists the available data sources, tables, collections, and columns/fields that will be required. These tools enable designers to visualize and document the API's structure efficiently, ensuring that the design aligns with both technical and business objectives.

3. Prototype: Visualizing with Agility

Phase Overview: Prototyping, a stage critical for validation, benefits immensely from API generation software. 

API generation software provide immediate feedback without the need to compose functional mock-ups. Stakeholders are able to see live data returned by the API with little-or-no code, enabling immediate feedback from API consumers. This rapid prototyping not only saves time but also increases API’s ability to meet user needs.

4. Implement: Streamlined Development

‍Phase Overview: In the implementation phase, developers write the code to build the API based on the design specifications. This phase includes coding, database integration, and the development of any necessary data mapping and business logic.

API generation tools automate the coding and integration tasks, turning weeks of development time into an almost immediate delivery process. This automation reduces human error and accelerates the development process, allowing developers to focus on more complex and creative aspects of API development.

Graph 5: API Generation model starts with mapping data schemas then auto-generating live APIs & interactive docs

5. Secure: Built-in Security Measures

Phase Overview: Security is a critical aspect of APIs that should be shifted left into the design phase. Security measures such as authentication, authorization, encryption, and access controls are identified and implemented early to protect the API and its data. In most cases, this involves defining configuration rules that will be applied when publishing to an API gateway.

API generation software provides a low-code method of defining role-based access controls (RBAC), ensuring that security is integrated into the API from the outset. This is a significant improvement from simple code generators that apply no authorization rules without significant effort. Teams can be confident that their API is secured properly prior to deployment, with no coding effort required.

6. Publish and Test: Efficiency in Deployment and Testing

Phase Overview: Makes the API available for use by developers before being rolled out to production to gain feedback from consumers. The phases are repeated to incorporate changes until the API is ready for promotion to a production environment.

Publishing the API allows the lifecycle to enter the crucial testing process. While developers building APIs without code generation may be testing their code already, generated APIs are able to jump straight into comprehensive testing. This includes functional, load, and security testing – all without waiting for developers to write the code.

7. Document: Auto-Generated Documentation

Phase Overview: API documentation is crucial for users to understand how to interact with the API effectively. Typically, this phase involves writing comprehensive documentation that includes usage instructions, endpoints, parameters, and examples. 

Documentation is generated by the tooling, reducing the manual effort necessary to produce reference documentation for developers. Typically, the documentation is generated using the OpenAPI Specification format. 

8. Deploy, Monitor, and Monetize: Continuous Improvement and Revenue Generation

Phase Overview: Deployment involves making the API accessible to users and systems in a production environment. Steps often include setting up servers, configuring network settings, and ensuring scalability and reliability. Continuous monitoring is then applied to track API performance, usage, and health in real-time. 

API generation software offer rapid deployment and rate limiting to support quick deployment without the need for existing API management infrastructure. Deployments often occur in only a few seconds, without the need to stand up a full CI/CD pipeline. 

If desired, organizations may leverage their existing investments in their CI/CD processes and API management infrastructure to properly deploy, secure, monitor, and monetize their APIs.

9. Maintain: Adapting to Change

Phase Overview: The maintenance phase is an ongoing process that involves regular updates, bug fixes, security patches, and feature enhancements. Maintaining API quality and performance is critical to meet evolving user needs. This often involves going through the same lifecycle once again, applying the enhancements into a new API release. 

In the maintenance phase, API generation software simplifies updates and adaptations, enabling APIs to evolve with changing business needs and technology landscapes. Caution must be applied, however, to prevent introducing breaking changes in the API design that could result in API consumers scrambling to adapt to changes. Some tools are capable of warning of breaking changes to prevent causing an outage with API consumers. 

In Conclusion

In conclusion, as organizations navigate the intricate digital landscape, the adoption of API generation software emerges not merely as an option but as a necessity for ensuring agility, security, and efficiency in API development. DreamFactory's API generation capabilities exemplify a forward-thinking approach to overcoming traditional barriers in API development, enabling rapid deployment, enhanced security measures, and seamless integration across various data sources.

We encourage technology leaders and organizations to explore further how DreamFactory can revolutionize their API strategies. For more insights and best practices, delve into our selection of articles:

• (blog) Unleashing the Power of API Generation for SQL Server Databases: An Exciting Innovation
• (blog) Best Practices for Naming REST API Endpoints
• (whitepaper) Governmental Agencies Are Rapidly Adopting DreamFactory