Assessing the Real Costs of a Major IT System Change

The widespread assumption of the late 1980s that downsizing from mainframes would reduce IS costs for just about any business sparked what has come to be an increasingly confusing debate about the costs of computing. Today, evidence from a wide range of sources challenges this assumption. Users have become increasingly disturbed about the high costs of decentralizing IS resources. There is also a growing realization that the cost structures of mission-critical systems have remarkably little to do with underlying technologies.

As user experiences with major platform changes have shown, no one system is inherently more or less expensive than another. Real costs depend greatly on the applications and workload profiles of individual organizations. Cost profiles vary by industry, by type and size of business, as well as by geographical location.

Underlying hardware and software technologies are also less important than the way the IS department is organized, the quality of IS management and staff, and the efficiency with which IS resources are used, including the effectiveness of cost accounting and control procedures. IS managers who are considering a major system change involving new computing platforms need a better understanding of IS cost structures and how these structures change over time. Lack of understanding of these areas can result in a business applying the wrong solutions to the wrong problems — and wasting a great deal of money.

This article discusses the key factors affecting IS costs in any business. It offers IS managers suggestions on how to avoid common mistakes in forecasting the costs of computing and guidelines for using comparisons with other companies to determine real computing costs more effectively.

COMPOSITION OF IS BUDGETS

The cost of providing IS services to a business involves a wide range of items. These include not only hardware and software acquisitions, but also personnel and their associated costs, along with the c osts of maintenance, telecommunications, facilities, supplies, and various other related items and outside services.

A 1993 survey of corporate IS budgets in the United States conducted by Computer Economics, Inc., of Carlsbad, California, found that the largest single IS cost item was personnel (42.1 percent), followed by hardware (28.9 percent), software (11.9 percent), and “other” (17.1 percent). The category of “other” included telecommunications, outside services, supplies, facilities, and miscellaneous items. IS managers should pay particular attention to this category because although the individual items in it may each represent only a fraction of total IS costs, their percentage is far from insignificant. Most of these costs remain the same or increase during major system changes.

Businesses often make the mistake of focusing solely on one-time investment outlays, primarily hardware acquisitions, rather than on longer-term operating costs. This approach inevitably misses the main components of the total IS costs of the company. Organizations that make this mistake are likely to experience a U-curve effect. Costs may follow original projections while a new system is in the start-up and test stages, but as soon as the system enters production and handles real workloads, IS spending escalates rapidly as new costs are incurred.

QUANTIFYING PRODUCTION WORKLOADS

In any production environment, a minimum six sets of workload parameters affect the capacity requirements of a system and hence its costs:

1. Numbers of active users

2. Volumes of data generated and used by applications 3. Type, size, and volume of transactions 4. Type, size, and volume of database queries

5. Type, size, and number of documents generated and printed

6. Batch workloads, including data consolidations, backup operations, and production printing

These parameters can vary widely within a company, as well as between different types and sizes of businesses in different industries. As a result, use of standardized, generic measurements for quantifying system performance is one of the most common causes of cost underestimates.

Because there is no such thing as a generic business, any valid computing cost comparisons must be specific to the needs of an individual business organization.

Most standardized measurement techniques have little relevance to the performance that will actually be experienced by users in any given production environment.

For example, much of the industry debate about millions of instructions per second (MIPS) is based on a fundamental error. Instruction sets, as well as the complexity and size of instructions, and the system processes that are instructed vary widely between systems. MIPS has no validity in a cross-architecture comparison. In addition, MIPS represents only a measure of CPU performance. In any real production environment, the performance actually experienced by users is influenced by hundreds of different parameters.

Measurements of transaction performance, such as TPC/A, TPC/B, or TPC/C, are based on stylized suites of software and workloads. These rarely correspond to the applications portfolios, transaction volumes, or workload characteristics of a specific business. Variations between different types of transactions have a major effect on capacity requirements, and hence on costs.

The use of any single benchmark is inherently misleading because few production systems handle a single type of workload. Transactions and queries may vary widely from application to application, and the overall processing mix is likely to include batch as well as other types of system operations.

Thus, any cost comparison between different systems should be based on quantified workloads that correspond to the current and future requirements of the business.

Database Query Volumes

Particular attention should be paid to query volumes. Although the effects of transaction and batch workloads are well documented for IS environments, large volumes of user-initiated database queries are a more recent phenomenon. Their effects are still poorly understood in most organizations.

Businesses that move to client/server computing commonly experience annual increases in query volumes of from 30 to 40 percent. Moreover, these types of increases may continue for at least five years. Although most transactions can be measured in kilobytes, queries easily run to megabytes of data. Long, sequential queries generate particularly heavy loading. Thus, once client/server computing comes into large-scale use within an organization, heavier demands are placed on processor, database, and storage capacity.

In many organizations, queries can rapidly dominate computing workloads, with costs exceeding those for supporting online transaction processing applications. IS costs in this type of situation normally go up, not down. It is better to plan for this growth ahead of time.

SERVICE LEVELS

Service levels include response time, availability, hours of operation, and disaster recovery coverage. They are not addressed by generic performance indicators such as MIPS or TPC metrics and are not always visible in workload calculations. However, service levels have a major impact on costs.

Many businesses do not factor in service-level requirements when evaluating different types of systems. The requirements need to be explicitly quantified. All platforms under evaluation should include costs for configurations that will meet the required levels.

Response Time

Response time is the time it takes a system to respond to a user-initiated request for an application or data resource. Decreasing response time generally requires additional investments in processors, storage, I/O, or communications capacity, or (more likely) all of these.

In traditional mainframe -based online transaction processing applications, response time equates to the time required to perform a transaction or display data on a terminal. For more complex IS environments, response time is more likely to be the time required to process a database query, locate and retrieve a file, and deliver a document in electronic or hard-copy form. Delivering fast response time according to these criteria is both more difficult and more expensive than it is for traditional mainframe applications.

Availability

Availability is the absence of outages. Standard performance benchmarks, and even detailed measurements of system performance based on specific workloads, provide no direct insight into availability levels. Such benchmarks do not indicate how prone a system will be to outages, nor what it will cost to prevent outages.

Even in a relatively protected data center environment, outages have a wide range of common causes. These include bugs in system and applications software, hardware and network failures, as well as operator errors. When computing resources are moved closer to end users, user error also becomes a major source of disruptions.

Production environments contain large numbers of interdependent hardware and software components, any of which represents a potential point of failure. Even the most reliable system experiences some failures.

Thus, maintaining high availability levels may require specialized equipment and software, along with procedures to mask the effects of outages from users and enable service to be resumed as rapidly as possible with minimum disruption to applications and loss of data. The less reliable the core system, the more such measures will be necessary.

Availability can be realized at several levels. Subsystem duplexing and resilient system designs have cost premiums. For example, to move from the ability to restart a system within 30 minutes to the ability to restart within a few minutes can increase costs by orders of magnitude.

Hours of Operation

Running multiple shifts or otherwise extending the hours of operation increases staffing requirements. Even if automated operations tools are used, it will usually be necessary to maintain personnel on-site to deal with emergencies.

Disaster Recovery

Disaster recovery coverage requires specialized facilities and procedures to allow service to be resumed for critical applications and data in the event of a catastrophic outage. Depending on the level of coverage, standby processor and storage capacity may be necessary or an external service may be used. Costs can be substantial, even for a relatively small IS installation.

SOFTWARE LOADING

The cost of any system depends greatly on the type of software it runs. In this respect, again, there is no such thing as a generic configuration or cost for any platform. Apart from licenses and software maintenance or support fees, software selections have major implications for system capacity. It is possible for two systems running similar workloads, but equipped with different sets of applications and systems software, to have radically different costs.

For example, large, highly integrated applications can consume substantially more computing resources than a comparable set of individual applications. Complex linkages between and within applications can generate a great deal of overhead. Similarly, certain types of development tools, databases, file systems, and operating systems also generate higher levels of processor, storage, and I/O consumption.

EFFICIENCY OF IS RESOURCE USE

Capacity Utilization Most computing systems operate at less than maximum capacity most of the time. However, allowance must be made for loading during peak periods. Margins are usually built into capacity planning to prevent performance degradation or data loss when hardware and software facilities begin to be pushed to their limits. If the system is properly managed, unused capacity can be minimized.

When planning costs, IS managers must distinguish between the theoretical and used capacity of a system. Failure to account for this is one of the more frequent causes of cost overruns among users moving to new systems. It may be necessary to add additional capacity to handle peak workloads. For example, properly managed disk storage subsystems may have high levels of occupancy (85 percent and over is the norm in efficient installations). There is a close relationship between data volumes used in applications and actual disk capacity. Inactive data is more frequently dumped to tape, thus reducing disk capacity requirements and corresponding hardware costs. If a system operates less efficiently, capacity requirements, and hence costs, can be substantially higher even if workloads are the same.

Consolidation, Rationalization, and Automation

Properly applied, the principles of consolidation, rationalization, and automation almost invariably reduce IS costs. Conversely, an organization characterized by diseconomies of scale, unnecessary overlaps and duplications of IS resources, and a prevalence of manual operating procedures will experience significantly higher IS costs than one that is efficiently managed.

For example, in many organizations, numerous applications perform more or less the same function. These applications have few users relative to their CPU and storage capacity utilization, as well as to their license fee costs. Proliferation of databases, networks, and other facilities, along with underutilized operating systems and subsystems, also unnecessarily increase IS costs.

Requirements for hardware capacity can also be inflated by software versions that contain aged and inefficiently structured code. System loading will be significantly less if these older versions are reengineered or replaced with more efficient alternatives or if system, database, and application tuning procedures are used. Automation tools can reduce staffing levels, usually by eliminating manual tasks. Properly used, these tools also deliver higher levels of CPU capacity utilization and disk occupancy than would be possible with more labor-intensive scheduling and tuning techniques. A 1993 study commissioned by the U.S. Department of Defense compared key cost items for more efficient best-practice data centers with industry averages. Its results, summarized in Exhibit 3, are consistent with the findings of similar benchmarking studies worldwide.

Cost Disparities between Best-Practice Data Centers and Industry Averages

Source: Defense Information Systems Agency, U.S. Department of Defense, 1993.

It should be emphasized that these figures (which are based on used rather than theoretical capacity) compare best -practice organizations with industry averages. Many organizations have cost structures much higher than the averages cited in this study. Capacity utilization, along with the effects of consolidation, rationalization, and automation, suggest that efficiency is in fact the single most important variable in IS costs. Clearly, the best way to reduce IS costs for any platform is to increase the efficiency with which IS resources are used.

APPLICATION LIFE CYCLE

One of the major long-term IS cost items for any business is the staff needed to maintain applications. In this context, maintenance means ongoing changes and enhancements to applications in response to changing user requirements. Even organizations that use packaged software will need personnel to perform these tasks.

The typical software application experiences a distinct U-curve pattern of demand for changes and enhancements over time. Demand is relatively high early in the cycle as the application is shaken down. Change and enhancement frequency then decline, before increasing again at a later stage as the application becomes progressively less appropriate for user requirements.

The frequency of application change and enhancement is affected by change in such factors as organizational structures and work patterns. The level may remain low if the business operates in a reasonably stable manner. Because all applications age and eventually become obsolete, increases are inevitable. The main variable is how long this takes, not whether it occurs.

The application life cycle has important implications for IS costs. Once the shake down phase is completed, a new application usually requires comparatively little application maintenance overhead. However, at some point maintenance requirements usually escalate, and so will the required staffing level.

Measuring application maintenance requirements for limited periods gives a highly misleading impression of long-term costs. Application maintenance costs eventually become excessive if organizations do not redevelop or replace applications on an ongoing basis. Moreover, where most or all of the applications portfolio is aged (which is the case in many less-sophisticated mainframe and minicomputer installations), the IS staff will be dedicated predominantly to maintenance rather than to developing new applications.

As an applications portfolio ages, IS managers face a straightforward choice between spending to develop or reengineer applications or accepting user dissatisfaction with existing applications. Failure to make this choice means an implicit decision in favor of high maintenance costs, user dissatisfaction, and eventually a more radical — and expensive — solution to the problem.

APPLICATION DEVELOPMENT VARIABLES

The cost of applications development in any business is a function of two variables:

1. Demand for new applications

2. Productivity of the application developers

Costs will decrease only if there is low demand and high productivity. In most organizations, demand for applications is elastic. As the quality of IS solutions increases, users’ demands for applications also increase. This is particularly the case for interactive, user-oriented computing applications. Early in the cycle after a major system change, user demand for these applications can easily reach exponential proportions.

If this effect is not anticipated, unexpected backlogs are likely to occur. More than a few IS managers who committed to reducing IS costs have been forced to explain to users that it is not possible to meet their requirements. Similarly, the productivity of applications development can vary widely.

Factors Affecting Applications Development Productivity:

- Proper Definition of Requirements

- Application/Systems Design

- Applications Characteristics Applications Structure/Size

- Underlying Applications Technologies Applications Complexity

- Functionality of Tools Development Methodology Match of Tools to Applications Degree of Customization Training/Documentation Programmer Skills/Motivation Project Management

Management Effectiveness

Development tools are an important part of the equation. Normally, third-generation languages (3GLs) yield the lowest levels of productivity, fourth-generation languages (4GLs) offer incremental improvements, and computer-aided software engineering tools, particularly those using rapid application development methodologies, perform best. Visual programming interfaces and the use of object-oriented architecture can also have significant effects. Productivity, in this context, is normally measured in terms of function points per programmer over time.

Productivity gains are not automatic, however. Tools designed for relatively small, query-intensive applications may not work well for large, mission-critical online transaction processing systems, and vice versa. Matching the proper tool to the application is thus an important factor in productivity. However, IS managers should be wary of vendor claims that the tools will improve productivity and reduce staff costs, unless it can be shown that this has occurred for applications comparable to their own requirements. Increases in productivity can be offset by increases in underlying software complexities (i.e., increases in the number of variables that must be handled during the programming process) and in degrees of customization.

Sophisticated user interfaces, complex distributed computing architectures, and extensive functionality at the workstation level have become common user requirements. However, multiuser applications with these characteristics are relatively difficult to implement and require realistic expectations, careful planning, and strong project management skills. Failure to take these factors into account is the reason for most of the delays and cost overruns associated with client/server initiatives.

New development methodologies and tools may alleviate, but not remove, problem areas. Regardless of the tools and methodologies used, effective requirements definition and management of the applications development process are more likely to be the critical factors in productivity.

TRANSITION COSTS

Costs involved in switching applications and databases from one system to another are commonly underestimated. Many businesses treat transition costs as a secondary issue and give them less scrutiny than capital investment or ongoing

operating costs. Even organizations that handle other aspects of the costing process with diligence often tolerate a great deal of vagueness regarding the time and expense required for transition.

This imprecision also extends to many claims of cost savings. Most of the figures quoted by vendors, consultants, and the media refer to purported savings in operating costs, not to net gains after transition outlays. Moreover, operating costs may be artificially low early in the few years following a change precisely because major one-time investments have been made in new hardware and software, with applications that are relatively new and require little maintenance.

Initial Installation of New Hardware and Software

Costs of the initial installation of new hardware and software are comparatively easy to quantify, provided that capacity has been properly estimated using workloads, service levels, and other criteria. If this has not been done, the organization may experience a sharp increase in costs above projected levels during the first year as the new system comes into production and actual requirements become apparent.

One-Time Services Outlays

Some one-time services outlays are usually required as well. These may range from large-scale conversions of applications and data, to the recabling of data centers and end-user environments, to retraining of IS and user personnel, along with the installation and assurance of system and applications software.

Many organizations use generic cost data supplied by consultants or vendors. This data can be highly inaccurate for the specific situation. Hard costs should be obtained to provide more precise data for planning purposes.

Length of the Transition Period

The length of the transition period has a major impact on costs. The actual time taken depends greatly on several factors. Organizations moving to new, relatively untried platforms and technologies need to allow for longer periods to shake down the new system and test its stability in the production environment. Depending on the size and requirements of the organization, as well as application workloads, transitions can take five years or more. A protracted transition period means that operating costs (e.g., software, hardware and software maintenance, personnel, cost of capital) are substantial even before a new system goes into normal operations. Parallel operations costs (i.e., maintaining the existing system in production) can also increase if the transition period is extended.

All these factors make it important that IS managers set precise dates for the transition process, beginning with the start-up of a new system in test mode and ending with the shutdown of the old system.

COMPANY-TO-COMPANY COMPARISONS

Reliability of Case Studies For any business considering a major comp uting systems change, the experiences of others who have made similar changes should, in principle, provide useful input. However, few well-documented case studies exist, and those that do are not always representative. The lack of reliable information is most obvious for mainframe migration patterns.

Organizations that have replaced and removed mainframes entirely fit a distinct profile. In the majority of the cases, these organizations possessed older equipment and aging applications portfolios. Their IS organizations were characterized by lack of previous capital investment, poor management practices, inefficient manual coding techniques for applications development and maintenance, lack of automation, and just about every other factor that leads to excessive IS costs.

All this raises some major questions about comparisons based on case studies. Even where overall savings in IS costs are realized, the savings occur only under specific circumstances, usually because IS costs were abnormally high to begin with. Organizations with higher-quality applications, current hardware, efficient software, and different workloads will have an entirely different cost structure. Cost savings are particularly unlikely in an organization that uses system resources and personnel effectively.

Mainframe replacements are a relatively small percentage compared with the total volume of mainframe procurements.

The extent of real mainframe replacement appears to be relatively small. The majority of downsizing actions involve moving specific applications from mainframes to smaller platforms, not actually replacing mainframes. This affects the validity of cost -savings claims. Although the cost of individual applications ma y be lower on new platforms, this does not necessarily mean that overall IS costs were reduced. In many cases, applications are either relatively small or exploit mainframe databases, or both. In addition, transition costs are seldom included in calculations.

CONCLUSION

One of the enduring ironies of the computer industry is that those who most uncritically and aggressively target IS cost savings are the least likely to achieve their goal. That is because unrealistic perceptions about automatic cost savings or inexpensive platforms often lead to inadequately managed planning and procurement. Preoccupation with technology may mean that major opportunities to achieve real, substantial cost savings through increased operation efficiencies are neglected. Even if real business benefits are achieved, the cost of realizing them is likely to be unnecessarily high.

Only by understanding IS cost structures, and targeting all the variables affecting these structures, can businesses ensure that they will obtain maximum cost effectiveness from their IS expenditures.