Archive for the ‘Methodology’ Category

ICM Test Planning, Scenarios and Templates – Part 3

May 16, 2008

In my last post about testing I said that ICM / SPM systems should be tested in phases; the reason for this is that discovering issues late in the development life-cycle could add unexpected delays and ultimately make the budget run over-budget.

It is surprising how often I have seen the business users and stakeholders not agree with the results and the development team exclaim, “Oh, that’s how it’s supposed to work!” or “That’s what you meant!”. Without proper planning, it is possible that during UAT the users will try to perform various activities and “break” the system – to which the development team usually answers “You’re not supposed to do that”, or “That’s not how it’s supposed to work”. Often, shortly before the payroll date, business users often ask questions such as “What about the draws?” or “What if orders have negative amounts” (good requirement documentation is also important!). Proper planning should eliminate all those nasty surprises.
There is an excellent article on Wikipedia regarding creating a test plan based on the IEEE 829 format. The test plan describes what will be tested, how it will be tested, what will be the deliverables, who will be responsible for what, etc.

Creating Test Scenarios for a Sales Performance Management System

I believe the most important aspect of testing is the test scenario preparation. I briefly mentioned how creating good test scenarios was particularly difficult with an ICM application because of the volume of test cases it will typically generate. This is unavoidable, but proper planning is required to ensure that tests are not testing the same conditions twice (wasting time) and that all conditions are being tested (not cutting corners).

A test scenario should have a name, a scenario ID and a description. This will help quickly refer to them during meetings. The test scenario should include the initial conditions, input (such as the order type), and the outcome or expected results. Finally, it is a good practice to list the business requirement ID that the scenario is testing. It is important for the test scenarios cover each plan, each rule, and each formula used within the rules. To test an ICM system, I like to group the scenarios, by Plan and by Rule:

1. Plan A
1.1 Rule A
1.1.1 Scenario 1
1.1.2 Scenario 2
1.1.3 Scenario 3

1.2 Rule B

Creating Test Data
The test data is the data that will be “staged” to test the scenarios. Typically, an order or a combination of orders will be required to test different scenarios. These orders should be created in the appropriate format to be staged and it should be documented which orders test which scenarios. Because of the different testing phases and because test data is often altered or corrupted during testing, it is important for the test data files to be kept together and be readily available to be re-staged when required.

After processing all the test data, all the test scenarios should be tested. That should cover the entire system, and in theory, after completing these steps, the system should have no outstanding defect.