It’s not unusual for smaller companies — and even some pretty large ones — that do on-premise data integration to have inadequate testing processes for new and modified translations maps and software upgrades. Changing a map or putting a new one straight into the production environment, or upgrading an application that uses maps without testing the consequences, can be costly. If it doesn’t work correctly, you have three choices: figure out how to fix the map, roll back the upgrade, or call tech support — all of which take time, effort and can seriously impact operations.
Inadequate testing for translation maps or application upgrades can wreak havoc on your production environment. The solution to the problem is to set up a User Acceptance Testing (UAT) environment and create a sound strategy for integration testing.
The pitfalls of ad hoc testing for integration maps
If you’re doing ad hoc testing, which is where you’re testing just a few maps for a quick verification, or if you’re testing an existing map with only a few data samples, you’re bound to miss something. Ad hoc testing is sometimes called “smoke” testing in reference to the old adage “where there’s smoke, there’s fire.” Except that’s not always the case. Maps that get tested once or on just one set of data, instead of being run through a full battery of tests, are ripe for causing problems. Yet fully testing a map that takes five minutes to write could mean running a hundred pieces of data through it — not an easy or quick process.
Because of this complexity and time constraints, it’s understandable that many companies do smoke testing, despite the pitfalls. Unfortunately, there are plenty of “smokeless” fires in mapping and the only way these can be discovered before they hit the production environment is by setting up a formal UAT environment.
Pre-production application testing
In addition to testing maps, you also need to test new versions of software with your maps. You can hope the vendor issuing the software update is fully testing its product before releasing it, but do you really want to bet your business on that assumption? The problem that arises once the new version makes its way into your production environment may not be due to a bug, but rather because the vendor has made behavioral changes or fixes to the software. If your process is dependent on the way that function used to work, your maps may break in the new environment.
Experience shows that these unexpected consequences of a software upgrade happen often enough to make it worth running the software upgrade in your test environment with your maps before putting it into your production environment.
The best testing environment for on-premise data integration is comprehensive, continuous and automated with strong version control and a rollback strategy.
Five best practices for on-premise data integration testing
1. Have a UAT environment — Having a User Acceptance Testing environment to test integration maps and to test application upgrades with established maps before they go into the production environment, is critical for preventing business process problems.
2. Establish acceptance criteria — Acceptance criteria help prevent problems in the production environment by establishing a standard that maps and software must meet before they are deployed.
3. Develop a comprehensive set of tests — This test suite should test every possible permutation of a map across applications to have the best chance of catching problems before the map or a software upgrade goes into production.
4. Adopt automated continuous testing — If your organization makes frequent changes to maps or runs a high volume of tests, consider an automated testing approach that runs continuously and issues alerts when a test fails. In this type of testing environment, tests are always running instead of being manually triggered for a particular event such as a new map or a software upgrade. In an automated continuous testing environment, mapping analysts will be alerted to anomalies as soon as they create a problem. There are several third-party tools available for automated continuous testing.
5. Use version control — Putting into place a change control system will make it much easier to manage changes to maps by keeping a record of every change. You’ll know who did it, when, and what changes were made. This is especially useful when the map doesn’t work correctly. It gives you visibility into exactly what version of a map is running and where, making it much easier to fix the problem, whether it’s in the test or production environment. There also should be a rollback mechanism that allows you to revert to the previous version.
If your integration tool doesn’t support version control, there are plenty of third-party tools or, alternatively, you can set up a manual process to keep track of changes. The best version control tools automatically audit changes, allow you to easily roll back to a stable version until the new version is ready to re-test, and deploy entire solutions. A good change control system will also work across all environments — UAT, production and even development.
Setting up a UAT environment for on-premise data integration and establishing a testing strategy based on best practices help to ensure that map changes and new software versions won’t interrupt business processes. UAT testing makes it easier to identify and diagnose problems with transformation maps and software upgrades, and experiment with fixes before introducing changes to your production environment.
Robert Fox is the senior director for Software Development at Liaison Technologies, a global provider of secure cloud-based integration and data management services and solutions based in Atlanta. An original contributor to the ebXML 1.0 specification, the former Chair of Marketing and Business Development for ASC ANSI X12, and a co-founder and co-chair of the Connectivity Caucus, he can be reached at rfox@liaison.com.