“Good fences make good neighbors.”
- Robert Frost
We love to tell true stories, especially when they help us make a point. This one is about a hapless rookie reports developer. His IT department invested in a low-budget reporting platform for their ERP and didn’t do anything to optimize the integration. The young developer tried to extract a massive data set covering several years of transactions. Hundreds of people screamed when the ERP slowed to a crawl.
Fortunately, this developer was not in a multi-tenant environment, so he upset only a few dozen people, not thousands.
Multi-tenant SaaS providers like Workday® have put controls in place to prevent tenants from running large, poorly designed integrations that use too much overhead. The safeguards include terminating processes that exceed these limitations:
While these “fences” help us be good neighbors in a multi-tenant environment, there is more we need to do to prevent glitches and support issues in our Workday® Studio integrations. Fortunately, Workday® engineers and customers have already experienced the pain and shared their experience. You can follow a few best practices to make sure your integrations run quickly and smoothly.
You can handle most of your integrations with Workday® using packaged integrations like connectors, but situations do occur where a connector doesn’t exist or we have a unique requirement that requires the skills of a developer. For this purpose, Workday® provides Studio, an IDE for integration developers.
The assembly framework in Studio provides all the components you need such as splitting, transforming, aggregating, and streaming. It also provides all the standard transports in addition to the Workday®-In and Workday®-Out transports.
Using these components doesn’t optimize your integration for you. You will also need to follow these best practices.
Service limits. In addition to the restrictions mentioned above, you also need to consider these:
Scalability. The simple way to deal with scalability is to make all your integrations scalable. By assuming your data sets will grow to size that will be unmanageable as a single document you will prevent future issues. Workday® recommends you always use the splitter before you process your data, then aggregate it or invoke a paged web service.
Streaming. Some assembly components have streaming turn off by default. If you use them, turn streaming on. For instance, the stream-xpath property provides support for simple XPath expressions to improve performance.
Repetitive Web Services Calls. There are many ways to deal with repetitive service calls that slow down traffic.
XSL and XLST load messages into memory in a Document Object Model (DOM) object. Avoid problems by using Streaming Transformation for XML .
To prevent future issues, we recommend you use these best practices on every custom integration, no matter how large or small. These general guides will help you know what to look for when you are planning a custom Workday® Studio integration. You can visit the Workday® Community to get the in-depth technical documentation you will need to optimize your integrations.
One of the biggest concerns of organizations that use enterprise software platforms is connectivity. If a company’s business applications don’t work together, there is little chance it can be competitive at today’s speed of business.
To maintain its commitment to business integration for its customers, Workday® provides pre-built connectors to many vendors, toolkits for integration types such as payroll, and a simple integration builder for business users. For more complex integrations, it provides a complete developer tool.
Workday®’s most powerful integration platform is Workday® Studio, a graphical developer tool for building sophisticated data integrations and transformations. It gives developers full control over integrations, making it possible to connect to any application using any of the commonly used Web transports and frameworks.
Workday® provides pre-built components which developers can manipulate in integration flows. As a developer, you can assemble these components to split, transform, loop, route, validate, wrap, or perform any other operation. The platform displays components in a graphical representation of the integration flow to make it easier to understand, configure, and troubleshoot.
Workday® recommends using XSL, XSLT, and XPath operations to transform data. These tools are powerful, but to use them effectively you need to understand their limitations. Workday® loads message into memory as DOM objects, so large documents will create bottlenecks or cause a server to abort the operation. We can recommend some best practices to avoid the problems that large documents cause.
Use Streaming. STX transforms documents without building a document tree in memory. It works well if the sequence of items or elements in the output is similar to the input document. Complex transformations, however, require using STX buffers or Java Bean objects to temporarily store data.
Split Large Documents. For large, complex documents, use a split-transform-aggregate sequence to reduce the size of DOM objects in memory. To ensure scalability, assume your documents will grow over time.
Use Simple Expressions. Complex MVEL expressions can have high performance overhead. Break down complex or heavily nested XPath evaluation statements into a sequence of simpler expressions. If you use a complex expression, consider using XSLT, static methods, or custom Beans instead of MVEL.
MVEL and XPath. xpathF.text has a 1,000,000 character limit. It is best not to use it. Considering the performance overhead of large DOM objects, it is best not to use xpath and xpathB in MVEL template @if conditions.
Avoid Recursive Operations. Rootless XPath queries will evaluate every node in an object, and recursive calls in custom functions will tend to create never-ending loops.
Use XSLT 2.0. To maintain compatibility, use XSLT 2.0 with the Saxon Transformation Factory in XSLT steps. The best practice is to use the tools present in the developer interface since they have been proved compatible.
Use Text Schema instead of XSLT or STX for formatting number or date-time values. In additional defining the structure of a text document, Text Schema defines formatting rules for numeric and date-time values. It is much cleaner.
XSLT Functions are better for complex logic than templates when you need to return these values:
Use Decimals instead of Double. In XML Schema, the IEEE 64-bit double is subject to precision and rounding errors. Also, the XPath fn:number(node) returns a double.
Use Strongly Typed Variables so you don’t have to explicitly convert values every time you evaluate them. They also simplify logic in XSLT in conditional and choice operations. It also helps to define the data type when you are sorting to avoid unexpected sort behavior.
Always check the latest information on the Workday® Community if you are unsure about how to use the components in Workday® Studio. Careful planning up front will save a lot of headaches downstream.
Workday® integration service has pre-built connectors, standardized integration tool kits, and a graphical integration builder designed for business users and analysts in its cloud integration platform. Error-handling in these services is pre-configured and cannot be changed by the user.
The platform also provides Workday® Studio, a graphical developer tool for building sophisticated data integrations and transformations. Workday® studio gives developers control over error-handling. The assembly components in the Studio graphical interface have builtin properties for error handling.
One of the most frustrating things in the life of payroll manager, benefits administrator, or reports analyst is to get an error message that doesn’t tell them what went wrong. Give them all the information they need to understand it. In some cases, if the end user has enough information, they will not require support to rectify the problem. You could even tell them whether you want them to resubmit the process or resend only records with errors. Don’t write a novel, but be verbose enough to give them a clear idea of the issue.
Make error messages readable to an ordinary business user, with a clear description of the problem. Where appropriate, identify the data that caused the error. The ExtractRootCause component will help you create meaningful error messages. Be sure to provide the context for your messages so an ordinary business user can communicate with troubleshooters.
For the Workday® support team, set the overall Integration Event status to Failed. Uncaught exceptions automatically set the Integration Event status to Failed.
In the Workday®-Out Transport, the failure-message property gives you the ability to augment the provided error message if the web service call fails. The original is appended to the failure message you provide.
When you set error handlers for asynchronous and synchronous mediations, use the Handle Downstream Errors property:
The handler will report it to the user in a PutIntegrationMessage and trap or rethrow the error, whichever is appropriate. Use the MVEL expression context.getError() to extract the details.
All Integrations should have a global error handler. Link the global error handler to a PutIntegrationMessage step, to post an Integration Message with a severity of “CRITICAL.” (Set the is.message.severity parameter to “CRITICAL.”)
Steps configured with one or more MVEL expressions that will cause the assembly to throw an error if they evaluate to false. Customize the Validation Message to provide a meaningful message.
Use XML Validation against an XSD schema, but avoid using it for large messages. For large messages, you should always use a splitter before you process your data, then aggregate it or invoke a paged web service. Configure XML validation message using the Failure Message and Replace Message properties.
XPath Validation
XPath can be used to validate a message by applying an XPath expression to it. XPath operations are limited to a 1MB message size. Use streaming for large files.
You can use the context helper variable in Java or MVEL to throw an error:
context.setError(
Description of Error
, 20001);
Do not fail integrations because of a single record error unless there is an explicit requirement. Report single-record issues as an error or warning and continue processing.
Use the Validate Exp step to create readable error messages. These are useful as a way to validate parameters passed to a sub-assembly.
For inbound integrations, there is a possibility that an integration event may not have been created. The error handler should check if an integration event exists and log the error message instead. Wherever possible, the integration should return an error message to the caller.
An integration that fails (catches and then rethrows and error) alerts the Environments team who will investigate the error. An error that can be resolved by an end user, the Environments team will let the user know. It may be sufficient to update the Integration Event with details of the error and to fail it.
Use the XML Schema, MVEL and XPath Validation steps in assemblies to ensure that messages are valid and errors are reported as early as possible. Text Schema also validates messages as part of the transformation.
Following these practices will save you time and trouble. Your users will appreciate it, and your fellow developers will appreciate, too.
