Extending REST interface with Custom OutSystems Code

Overview

Agile.Now Accelerator equips you with a robust and ready-made interface to expedite your OutSystems development journey. Developers can seamlessly integrate custom OutSystems code to enhance the functionalities provided by Agile.Now Accelerator, which include Authentication, Authorization (RBAC & ABAC), Data Validation, and OpenAPI documentation.

Procedure:

1. Create a New Interface:

   • Start by creating a new interface from the OutSystems database table.
   • Define all the necessary settings and requirements according to your project needs.

2. Use the Integration Template:

   • Utilize the provided Integration template, which contains ready-made functionalities, to streamline the integration process.
   • The template ensures service internal login and always returns a complete list or record that can be directly stored in the OutSystems database, eliminating issues with partial data storage.

3. Develop Custom Code:

   • Write your own OutSystems code to integrate additional functionalities, history logs, and other business processes as per your project requirements.
   • Ensure that your custom code seamlessly communicates with Agile.Now Accelerator.

4. Communicate with Agile.Now Accelerator Service:

   • Return any errors or success messages to the Agile.Now Accelerator service.
   • The service processes the requests and returns the appropriate response to the caller.

Benefits:

• Efficiency in Development:

  • Develop Enterprise-level REST services without extensive coding, allowing you to focus more on programming the business layer and creating value for the end customer.

• Cost and Resource Savings:

  • Save on OutSystems application objects and associated costs by leveraging the functionalities provided by Agile.Now Accelerator.

• Microservice Architecture:

  • Build and deploy authentic microservice architecture, enhancing the scalability and maintainability of your applications.

• Focus on Business Layer:

  • With ready-generated services, developers can concentrate on implementing business logic, thereby accelerating the delivery of benefits to the end customer.

Conclusion:
Agile.Now Accelerator empowers developers to extend its capabilities with custom OutSystems code, facilitating the development of secure, efficient, and scalable applications. By focusing on the business layer and leveraging the ready-made functionalities of the Accelerator, developers can realize authentic microservice architectures and deliver enhanced value to end-users.

Customizing REST Interface

The Dynamic REST Interface introduces the custom callback endpoint, an innovative feature that permits OutSystems developers to embed bespoke functionalities within standard API interactions. By activating this customization for particular Endpoint methods, developers can seamlessly route data through specialized logic, ensuring data remains secure, authenticated, and adheres to stringent validation standards.

Workflow

1. Authentication, Authorization, and Validation: As a precursor to any customized function, each call undergoes stringent scrutiny:

   • Authentication: This step ascertains the identity of the caller, whether it's a user or a system.
   • Authorization: This process validates that the caller possesses the requisite permissions to execute the request.
   • Validation: Ensures the data is not just compliant with expected formats but is also complete. This eliminates the need to rectify partial data from the database, streamlining the entire data handling process.

2. Direction to Custom Endpoint: Post verification, the API call advances to a user-defined REST interface. This pathway ensures that the data relayed to the custom endpoint is comprehensive and mirrors every attribute from its corresponding database table. Crucially, this step encompasses an internal authentication mechanism, bolstering call security.

3. Configuration: Operationalizing this feature demands the precise configuration of the CustomUrl parameter. This parameter should house the callback address of the tailor-made code/logic that the developer aims to deploy. Furthermore, one must select all methods slated for customization.

4. Dynamic REST Interface: On invocation, this formidable interface reaches out to the custom callback endpoint (specified by CustomUrl). Importantly, this call is laden with the structured data discussed, enabling hassle-free data processing and manipulation. Given that a single endpoint path can accommodate one custom URL, all API methods traversing this path utilize the same custom interface. This architecture allows developers to efficiently manage all methods using a unified callback input structure, optimizing OutSystems application objects.

5. Custom Authentication Flow: Developers must integrate the OpenAPI.LoginByInternalToken method into their custom OutSystems authentication mechanism. This step fortifies the internal token validation process, ensuring endpoint security.

6. Crafting Custom Code:

   • Save and Delete Callbacks: Here, the objective is to insert, update, or delete records in the database. The dynamic interface abstains from direct data manipulation, offering developers complete control over transaction chains.
   • List Callback: This mandates the construction of a SQL query tailored to fetch accurate data.
   • Return Protocols:
     • For Get, Save, and Delete callbacks, one must return the database entity identifier. The service then autonomously retrieves and relays the appropriate structure to the API caller.
     • Patch method callbacks should return a text array of entity identifiers.
     • Depending on the List API method, return the pertinent entity or structure list, corroborated with generated OpenAPI documentation. Regardless of the scenario, ensure all returned variables are converted to a JSON string.

7. Error Management: In situations where custom code triggers errors, it's essential to revert with an Errors object. This structure resides in the OpenAPI module. Developers can conveniently leverage the pre-configured SetApiErrorState action to append custom errors, ensuring each is paired with an accurate error code and message.

Key Components of the Callback Structure:

These callbacks adhere to specific OutSystems structures, albeit with variations contingent on the API method. Refer to the Data variable section for detailed insights.

• Callback: This signifies the method in action, providing clarity on the operation being executed. Supported methods span Get, List, Delete, Save, and Patch.

• EntityKey: Serving as the unique identifier for entities, this key becomes instrumental for intricate data manipulations. Developers can harness this key to validate the actual OutSystems database entity, particularly when leveraging multiple entities within a single callback endpoint.

• ActionId: This component unveils the operation type on a record. Ranging from Create, Delete, Read, Update, to Upsert, it's a pivotal element when processing Save callbacks.

• SpecialFields: These are predefined query fields with specific functionalities that cater to more nuanced data operations.
  Here's a breakdown of the SpecialFields:

  • DeleteNotExists: Enables the removal of orphaned values from a table. This field can work in conjunction with multiple other fields for comprehensive cleaning.
  • Unique: Extracts distinct values from a table. It can amalgamate multiple fields to generate a composite unique list.
  • UniqueIgnore: Retrieves unique values but overlooks rows that are blank, ensuring data relevance.
  • PassiveNotExists: Passivates or deactivates orphan values. This works similarly to "DeleteNotExists" but instead of removal, it sets them to a passive state. This can be combined with multiple fields.
  • Passive: It updates boolean values of orphaned records to indicate they are passive or inactive. Can be combined with multiple fields for a broader update.
  • CreatedBy: Updates the field with the user identifier at the time a record is created. Useful for audit trails. It can affect multiple fields if required.
  • CreatedOn: Inserts the timestamp indicating when a record was created. This can be applied to multiple datetime fields for comprehensive record-keeping.
  • ModifiedBy: Reflects the user identifier at the time of the latest update on a record. It can influence multiple fields as necessary.
  • ModifiedOn: Logs the timestamp of the latest record modification. This can be used across multiple datetime fields.

• Data: The Data parameter comes in the form of a JSON-formatted string.

  • The JSON string can be parsed and adapted to align with OutSystems entities or entity lists, depending on the API method type in use.
  • The adaptability of this structure serves diverse application requirements. A distinctive advantage is its capability to manage the entire dataset, avoiding the complexities associated with handling fragmented records or lists.
  • Delete, Get, Save Callback:
    • Envisages or returns a distinct JSON structure in the Data parameter.
    • After decoding this string, it can be tailored for compatibility with OutSystems entities, guided by the particular EntityKey.
  • List Callback:
    • Anticipates or returns a JSON structure encapsulated within the Data parameter.
    • This string incorporates the AgileNow_DataAccelerator.Search structure, or a similar layout.
    • This configuration determines search conditions, filter specifications, and other vital parameters for the dynamic function of these methods.
  • Patch Callback:
    • Expects or produces a defined JSON structure via the Data parameter.
    • Post decoding, this string is malleable to fit with OutSystems entity lists, steered by the designated EntityKey.

Custom REST Interface in OutSystems

In the modern development landscape, dynamic REST interfaces offer unmatched flexibility and scalability. Here's a structured guide on customizing a dynamic REST interface in OutSystems:

1. Setting Up the REST API in OutSystems

a. Create a New REST API

Follow these steps in OutSystems Service Studio:

1. Navigate to the Logic tab.
2. Right-click on Integrations and opt for Consume REST API.
3. Choose 'Expose REST API' and label it as Integration.
4. Within this new API, introduce a method called Callback.

b. Configure Callback Method Parameters

1. Introduce an input parameter and set its data type to the structure AgileNow_Endpoint.ICallback.

2. Depending on the purpose of the callback, set your output:

   • Get, Save, and Delete Callbacks: Ensure the output returns the ID of the relevant database entity. The service will then automatically fetch and relay the correct structure to the API requester.

   • Patch Callbacks: Your output should be a text array of entity identifiers.

   • List Callbacks: The output should be an entity or structure list that matches the OpenAPI documentation.

   Note: All output variables should be converted to a JSON string format.
   

2. Implementing Valid Internal Token Check (OnAuthentication)

a. Integrate Custom Authentication

1. Within the properties of the Callback method in the REST API, set the Authentication to Custom.
2. OutSystems will then automatically append an OnAuthentication action.

b. Implement Token Validation

1. Navigate to the OnAuthentication action. Here, incorporate the function OpenAPI.LoginByInternalToken. This function is pivotal as it oversees user authentication and authorization for internal usage.
2. The method retrieves the access token present in the HTTP request headers and decodes it, extracting the relevant claims.
   

3. Handling Save and Delete Callbacks

Given the distinct nature of Save and Delete operations, it's advisable to address them separately.

• For Save: Construct logic that discerns whether to insert a new record or update an existing one based on the data incoming and the database's current state. OutSystems has the built-in CreateOrUpdate<EntityName> action for this. Alternatively, the Agile.Now module provides the AgileNow_DataAccelerator.EntityRecordProcess function, which offers extensive support for various entities within OutSystems.
  

• For Delete: Use either the inbuilt Delete<EntityName> action or the aforementioned AgileNow_DataAccelerator.EntityRecordProcess function from the Agile.Now module.
  

4. Handling List Callback

For APIs designed to return a list of entities or structured data, the List Callback is crucial. Here’s how to employ it:

a. Defining the Output for List Callback

1. Within your Callback method in the REST API, make sure the output is configured to return a list of the appropriate entity or structure.
2. Cross-reference this output structure with the generated OpenAPI documentation to ensure the list's format and fields align with the expected schema.

b. Implementing the List Logic

1. Depending on the input parameters, such as filters or search queries from the AgileNow_Endpoint.ICallback structure, set up your logic to procure the necessary data.

2. For data retrieval, either OutSystems aggregates or SQL queries can be employed. For simple data fetch tasks, aggregates are ideal, while SQL is more suitable for intricate queries or when there's a need for optimized performance.

   • Moreover, the AgileNow_DataAccelerator module provides two potent functions, namely DynamicEntityQuery and DynamicEntityWhere:

     • DynamicEntityWhere (⚠️Rest service is not supported!) returns the SQL where and order by statement of an entity using a dynamic SQL statement. It doesn’t verify the SQL table and fields and can't be accessed directly through the rest service.
     The function does not verify the SQL table and fields and is inaccessible directly through the rest service.

     • DynamicEntityQuery (☑️Rest service support) permits the construction of a dynamic SQL filter seamlessly via the rest service. It safeguards by validating the syntax and values, and effectively prevents SQL injections. Additionally, it supports "order by" clauses and pagination, giving developers more flexibility in returning ordered and paginated results.
     DynamicEntityQuery provides the capability to formulate a dynamic SQL filter freely through the rest service. It not only validates syntax and values but also safeguards against SQL injections, all while supporting order by and paginations.

   • Both functions can operate with the 'Search' structure obtained from the 'Data' parameter. To morph the Data JSON into an OutSystems structure, employ the JSONDeserialize action provided by OutSystems.

3. If pagination is required (which is typical for list APIs), integrate logic to fetch a specific subset of records based on input metrics like page number and page size.

4. Transform the fetched list of records into the pertinent output structure format. This might encompass field mapping, data conversions, or the addition of extra attributes as needed.

5. Before relaying it in the callback response, make sure the output list is transformed into a JSON string format.

With these directives, the List Callback will adeptly provide a list of records based on the API requester's criteria, ensuring alignment with the OpenAPI documentation and delivery of essential data in the prescribed format.

5. Handling the Patch Callback

The Patch callback method in OutSystems is primarily designed for updating specific attributes of records, ensuring a precise data manipulation process:

• Data Expectations:

  • Anticipates a specific JSON structure through the Data parameter, tailored to resonate with the requirements of OutSystems entities.

• Decoding and Adaptation:

  • After receiving, the JSON string undergoes decoding to match OutSystems data formats.
  • This data, post-decoding, is adjusted to synchronize with OutSystems entity lists, using the specified EntityKey. This involves translating the JSON data into an entity records list, furnished with all necessary details.

• Implementation:

  • The Patch operation updates certain record attributes based on the processed data while retaining the originality of other fields. This ensures that only designated fields are modified without altering others.
  • For optimal results, employ the 'EntityListProcess' server action from AgileNow_DataAccelerator. This function inspects the provided record list for errors, returning an error list if found, and saves the entity list to the database. It works seamlessly with static, standard entities/tenants, and external entities within OutSystems.

• Response Framework:

  • Once the patching concludes, the callback method returns a textual array of the entity identifiers, indicating the updated records. This facilitates subsequent operations or verifications by keeping the invoker informed about the specific altered entities.

6. Handling the Get Callback

The Get callback in OutSystems primarily deals with data retrieval, and its correct implementation ensures data consistency and integrity.

• Returning Entity Identifiers:

  • For the Get operation, it's pivotal to return the relevant database entity identifier. After the callback is executed, the service autonomously retrieves this identifier, formulating and dispatching the appropriate structure back to the API caller.

• Augmenting with Audit Logging:

  • While the core functionality of the Get callback revolves around data retrieval, there's an opportunity to layer in audit logging for enhanced traceability. Such logs can offer a window into user activity, request patterns, and more. Even though it might seem superfluous for simple retrieval actions, maintaining a consistent audit trail can be beneficial in the long run, especially for systems that value historical records and operational insights.

Remember, the goal is to fetch the data accurately and quickly while ensuring the returned structure aligns with what the API caller expects. Incorporating audit logging can add an extra layer of reliability, security, and insight into the operations.

7. Advanced Response Handling

1. Integrate the Advanced OnResponse action within the Callback method.
2. Within this action, employ the SetCustomizedResponse function from the OpenApi module. This action grants you the ability to adjust the response – this could be to change headers, modify the response body, or even alter the status code based on particular conditions.
   

Conclusion

Crafting a customized dynamic REST interface in OutSystems merges the power of dynamic APIs with the flexibility of the OutSystems platform. While this guide offers a fundamental framework, developers are encouraged to modify and expand upon these procedures, aligning them with their unique project specifications and use-cases.

By routing data via specific logic and ensuring unwavering security and validation, this feature underscores the promise of bespoke solutions within robust frameworks.