Overcoming Testing Challenges in SAP Upgrade Projects

Organizations having SAP-based business solutions are involved in an SAP upgrade project at one time or another. At the present time, many of them are involved in or are planning upgrades to SAP ERP 6, which will form the stable version until 2017. The periodic improvements would be delivered through enhancement packages, which would be installed and activated as per the needs of the organization.

The success of SAP upgrade projects relies on efficient and comprehensive planning of testing activities that start from the project kick-off itself and last until a few weeks before the project go live date. Organizations involved in SAP upgrade projects face a dilemma over the testing of the upgraded and the unicoded SAP-based business solution.

The question that each organization faces is over the quantum of testing: how much to test, the scope and the effort involved. It becomes important that the optimum testing approach be defined upfront for the SAP upgrade project -- and then followed. Testing involves substantial effort and has a direct impact on the project plan and the project budget. It forms the crucial parameter which, if not managed well, may result in serious doubts over the acceptability of the upgrade project by the business users. This article covers the inputs to plan and manage efficiently the testing scope and effort in an upgrade project.

Sandbox Testing – Pyramid Approach

The objective of sandbox testing is to ascertain the potential scope of changes and issues in the upgraded and unicoded SAP system so that the testing approach can be suitably framed/revised for the project. This will also act as an input to the project effort planning and duration.

The scope preparation starts in parallel while the sandbox is being prepared and is finished when it is ready. This helps to initiate sandbox testing without any delay.The important aspect is to test the critical and important transactions of the business processes by the various functional teams to ascertain the extent of changes in the new system. A representative set of testing scope (also called “Top of the Pyramid”) covering the following is prepared based on these factors:

• Critical SAP standard and custom transactions – to make sure that frequently used SAP standard and custom transactions are behaving as expected. Transaction usage statistics from last year or a given period provide important inputs. Additionally, less used but critical transactions (e.g., month-end or year-end reporting) should be covered.
• Critical reports in different languages – Both custom developed and standard reports should be covered in different languages used in the system.
• Representative set of critical interfaces – All the critical interfaces should be identified and their connectivity should be established with corresponding test systems. It is also of importance that all kinds of interfaces are covered: e.g., inbound, outbound, file-based, database connects. A matrix of business and IT contacts for interfaces should be maintained for faster contact and issue resolution.
• Representative set of printouts in different languages – All the important printouts that go to outside entities either electronically or in manual form should be covered. Printouts in different languages should be considered to assure the correctness of layout and characters in the output. 

Complex enhancements (e.g., custom developed functionality, user exit) should be included in the scope.

If the number and nature of issues is high, the testing scope should be suitably increased in the “quality testing” so as to increase the confidence in the quality and stability of the upgraded and unicoded SAP system. In this situation, sandbox testing scope should also be revised to cover medium impact transactions so as to gain a suitable confidence level on the new system behavior.

The possible nature of issues encountered in sandbox testing is as follows: 

Nature of Issues	Causes	Corrections
Program Dumps	Unicode incompatibility, Reference SAP standard object is changed	Development correction, SAP notes
Wrong Characters in the Output (report, printouts)	Unicode incompatibility, incorrect vocabulary	Development corrections, vocabulary corrections
Interfaces	Program errors, legacy system compatibility, file systems errors, connectivity errors	Development corrections, legacy system upgrade/corrections, basis corrections
Impacted Objects	Changed technical components in new SAP version	Development corrections

The functional experts will perform the sandbox testing of the identified critical transactions and log the issues encountered. The solution providing team -- i.e., development, authorization, basis and in some cases the functional team (identification and application of OSS Notes) -- provide the solution to the issues. The issue log is monitored for faster resolution. It is analyzed by project management for the possible impacts/corrective actions on subsequent project phases and deliverables.

If the types of issues point to dumps in transaction execution, then issues are related to upgrade and unicode causes and require relevant correction by the development team. A high number of such issues will also mean that there is potential likelihood that more such issues would be observed if the scope of testing were expanded. This observation should be used by project management to alter the testing scope in the upgraded and unicoded SAP quality system.

The corrective transports will be applied to upgraded and unicoded SAP development systems as testing duration in sandbox is of limited time and thereafter the changes/bug-fix will originate from the new SAP Development System and will move to the SAP Quality system of upgrade landscape. This regulation of changes follows the transport strategy laid out in the organization.

The sandbox testing documentation will involve a) scope list b) result list and c) issue list. The sandbox testing is peformed in screening mode -- it is not documented comprehensively in the actual test scripts that may be in use in the organization. This saves time for the functional teams as they can focus on performing testing and verifing the system behavior, which is the main objective of the sandbox testing.

Next page: Testing in the SAP Development System