AJM 17(4) Web_Master.pdf 70 American Journal of Management Vol. 17(4) 2017 EVM2�A New Look at the Earned Value Management Model Gary L. Richardson University of Houston The project management world of Earned Value (EV) analysis is due for a major shakeup. We are now entering the broad acceptance stage for this topic and the availability of computer software to spit out misunderstood or meaningless EV parameter values threatens to ruin the validity of this model. There is an old adage that says �To err is human, but to really screw up requires a computer.� That statement matches the thesis of this paper. The goal of this paper is to outline some of the key issues that will distort the proper interpretation of computed EV parameter values. **Author Note: Contribution credit is recognized for Charu Behl and Rafael Ribeiro (Federal University of Vicosa, Brazil) who provided significant research and computational support related to this paper. BRIEF EV HISTORY Historical acceptance and usage of EV in commercial projects has been rocky since its most formal publication in the mid-1960s via the Cost/Schedule Control Systems Criteria (C/SCSC) as part of the DODI 5000.1 project management specification. (Richardson, 2015, p.435) This early contractor requirement was focused on improving various aspects of project management; however at this point in time government contractors essentially rejected the concept based on the operational maturity level that was required to produce these status parameters. The next thirty years were essentially marked by evolutionary maturation of organizational project management processes and by the 1990s mature organizations, primarily DoD related, had improved their operational infrastructure and management processes sufficient to produce valid EV status parameters. Christensen�s research showed that the EV parameters being produced provided a robust tool set for evaluating both current status, as well as forecast cost and schedule values (Christensen, 1998, p. 13). As a result of this type validation the Project Management Institute (PMI) included a brief overview of Earned Value mechanics in the 1987 edition of its Project Management Body of Knowledge (PMBOK ®Guide). Also, private industry in 1998, through the sponsorship of the National Defense Industrial Association (NDIA) formally recognized acceptance of EV through the publication of ANSI/EIA-748 Standard for Earned Value Management (Fleming and Koppelman, 2006). These milestone steps along with improvement in desktop project management software has stimulated broader acceptance of EV along with its promised analytical value. Following this long conceptual formulation and acceptance phase a second evolutionary wave occurred. This stage was primarily associated with explaining the underlying computational processes in understandable terms using various vendor products to produce the parameters. One of the main implementation support procedural steps came from Fleming and Koppelman who described ten understandable operational steps that would satisfy parameter production for most commercial American Journal of Management Vol. 17(4) 2017 71 applications (Carstens et al, 2013, p.261-262). Also, various software vendors such as Microsoft and Oracle added popular software utilities that purported to contain the ability to produce all of the traditional EV parameters. At this point the stage was set for broader usage of the EV parameters to describe current and forecast project status. As with most silver bullet (perfect answer) items this one also comes with a hidden hazard that has not yet become visible in popular industry literature. EV in Use Today Song�s excellent overview of Earned Value usage internationally outlines a complex user profile (Song 2009). Based on this reported data it is difficult to provide a single usage profile since actual levels vary greatly across industry and geographical boundaries. However, there is one trend that seems consistent. That is, use of Earned Value is increasing as computer software has eased the calculation complexity. At the same time, there is an old adage that says �To err is human, but to really screw up requires a computer.� That statement matches the thesis of this paper. We see this phenomenon in action through software utilities such as Microsoft Project, which is in use by about most commercial project environments, with much of the remaining market supported by Oracle�s Primavera. In both cases EV parameters can be automatically generated essentially with a click of the mouse--Problem solved! While its output capability is mechanically accurate, the actual operational accuracy of the output is now in question. It is the goal of this paper to highlight some essentially hidden issues in the use of these parameters, such that traditional parameter values do not represent necessarily what the current literature leads one to believe. On one positive side, EV parameters offer the most robust status metrics available to the project manager, but conversely these can be misleading if not understood. To that end the current computerized output taken at face value is potentially misleading and may actually indicate an erroneous status. That is a strong statement and one that needs to be justified. Used properly, the EV model does in fact offer the best status tracking capability of all known methods. The user challenge then is how to make best use of the model and subsequent output. As the organizational EV operational infrastructure and associated methodologies have evolved through the past forty or so years a great deal of maturity has been added to the project manager�s process. For instance, the EV model has introduced several three letter parameter acronyms such as SPI, CPI, EAC and VAC. Each of these are touted as a predictor of present or future project performance. However, as user experience has evolved the calculated results often do not properly answer the promised measures for either current of forecast outcomes. As a result of this there is danger that the EV process will be dropped as a poor tool. The data presented here will show that this process needs to be better understood as to what it represents and adjust the calculations accordingly. One might interpret these statements as rejecting the traditional EV model, but we prefer to state this situation as �We come not to bury Caesar, but to praise him.� By this we mean that the existing popular literature has not focused at the electron level of EV usage, but rather more on macro level parameter calculations that do not offer the level of granularity needed for root cause type project management decision support. In order to add this capability we now need to understand how the lower level driver components actually impact the output interpretation. One very recognized public example of this has been pioneered by Walt Lipke in his derivation of Earned Schedule (ES), which was derived to correctly interpret errors in the SPI parameter calculation (Lipke, 2009). This correction factor is now well understood and will not be focused on here, but it does show how the model needs to be tweaked for operational accuracy. Interested readers should review other sources related to the ES topic. For this paper the main focus is on the cost analysis side of EV parameter calculations. EV Parameter Calculations As indicated earlier, typical EV computer generated parameters may well produce suspect management oriented values for both current and forecast calculations. Realize that the project manager�s control focus is on the status of various resource areas and each of these need to help in evaluating corresponding elements of current baseline deviations and forecast values for the project. Research results 72 American Journal of Management Vol. 17(4) 2017 reported here will show how the traditionally calculated metric can create significant distortions through failure to recognize this level of granularity. Assuming this conclusion is valid, one might well also conclude that EV may become judged as a poor management measure and fall into misuse. Given this recognition the time is right to dedicate focus on this situation in the same manner that Lipke�s ES-based schedule correction has logically shown a correction without destroying the overall approach. The goal here is to duplicate that strategy for the cost side of the parameter set, as well as highlighting the resource granularity issues. Additionally, it is also important to recognize that at this point there is no visible sign that either Microsoft or Primavera are working to deal with any of these issues and in fact are guilty of perpetuating the problem. So it is going to be up to the user community to understand how to extract the appropriate data views until more robust commercial solutions emerge. Some of the more obvious EV parameter data and mechanical issues involved will be described here and the remainder of this paper will outline various characteristics of these components as they relate to the EV cost status environment. Earned Value Infrastructure The stated project management role for EV is to produce parameter values that will quantify current and forecast cost and schedule status. Space here does not allow a full review of EV mechanics, but a brief summary will be offered to show the fundamental parameters involved. Essentially, a suite of EV model parameters are defined for current cost and schedule performance variance (CV and SV), run-rate indices (CPI and SPI) and completion forecast parameters (e.g., EAC, VAC and ETC). The basic role of each parameter group can be summarized as follows: CV and SV�Current baseline status variance measures for cost and schedule CPI and SPI�Current index measures to show relative index performance for cost and schedule. For example, a 0.90 index value would indicate that performance is 90% of planned baseline value EAC, VAC and ETC�these parameters represent forecast estimates for the project at completion; EAC is cost Estimate at Completion, VAC is cost variance at completion compared to the baseline budget and ETC is the estimated dollar amount required to complete the project. Popular literature clearly defines what each of these parameters represents. The challenge here is to highlight whether that definition is valid and under what circumstances. Fleming and Koppelman (2006) studied the required operational infrastructure required to produce valid EV (traditional) parameters and from this they derived ten process oriented steps necessary for the model to work as advertised. Seven of these process items fit well into this discussion and are summarized here with a brief interpretation as follows: (Richardson, Chapter 15, 2013) 1. Define the project scope. One common method for doing this is to use a work breakdown structure defining the project scope with well defined, small work packages. 2. Define the performing organization responsible for each work package and an integrated detail plan specifying items such as task sequence and make versus buy options. 3. An estimate of the resources required to support the defined work packages. 4. Develop an approved and baselined work schedule that integrates planned time, budget, and work resources for the various work units. 5. Define how actual work accomplishment will be measured for each work package. 6. Establish a formal Performance Measurement Baseline (PMB) that incorporates the items above. This essentially defines a time-phased cost plan for the project. 7. Record actual costs incurred at a status control point for each work package (or more accurately a control point in the structure). The remaining three definitional items outlined relate to management of the ongoing process in regard to monitoring, forecasting, and scope control. American Journal of Management Vol. 17(4) 2017 73 Developing the Examples A base point for this discussion starts with figure 1 showing a simplistic abstract Work Breakdown Structure (WBS) that is used to represent the project scope. At the lowest levels of this structure are Work Packages (WPs) that collectively represent the total scope of work planned for a project. The role of the WBS is to compartmentalize the overall work required into manageable work units that are then time- phased and used to evaluate status. FIGURE 1 PROJECT WORK BREAKDOWN STRUCTURE (WBS) Proper scope definition lies at the heart of these calculations and are fundamental to any measurement process. Failure to achieve this level of work definition sabotages any subsequence EV parameter calculation regardless of the methodology used. In order to produce EV parameters for each work package, three data items are required as illustrated in figure 2. The driving measurement data values are actual costs (AC), baseline planned cost (PV) and a measure of actual work accomplished or earned value (EV). For this conceptual example we will ignore deviations created by resource collection errors and scope changes. But recognize that both of these items could be added to the causal list of interpretation errors. They are omitted hereto simplify the fundamental comparison analysis. AC and PV values are deterministic from the original approved plan. For example, AC values are taken from the formal resource accounting system and PV is the baselined cost value for the associated WBS work units. EV values are then calculated by multiplying the work unit PV by the estimated level of completion for that unit. Using these three driving values all of the related EV parameter values outlined above can be computed. Traditional EV literature says that the suite of EV parameters define the status of the project. This in fact is accepted as a global indicator of status; however, from a management viewpoint there are certainly other aspects of analytical concern. FIGURE 2 EV DATA ITEMS For the moment we might accept the fact that a composite EV calculation does in fact define the overall project status and does represent a worthwhile set of Key Performance Indicators (KPIs). However, beyond this high level view a manager is most often concerned with identifying the source of variances at a more granular level. With this goal in mind this research effort has identified seven common �parameter distortion� situations. These are: 74 American Journal of Management Vol. 17(4) 2017 1. Unit labor rates of the project team human resources 2. Third party vendor contractual roles 3. Level of effort (LOE) resources�typically service agreement type assets 4. Dollar expenses (non HR and non-product; i.e., travel expenses) 5. Material cost at the work package level 6. Labor rate variances (compared to plan values) 7. Padding of planned work package estimates In order to use EV successfully for management and forecasting purposes these groupings need to be isolated for analysis and then synthetically combined based for an interpretive review. As a starting point recognize that the project manager most controllable element is his internal team productivity, but combining the various other resource groupings with this group can hide the team�s actual performance. Likewise, mixing all of the groups into a single assessment hides away the management decision analysis value of the EV parameters. As noted, none of the major software packages separate project data into groupings of this type, so that type of granularity is lost with current utilities. Yet another analysis aspect of the traditional parameter calculation is to assume the current trends will continue through the life cycle. This assumption may be true, may not be true, or may vary across the various groups. In any case there is no reason to assume that all of these groups trend the same way. For example, LOE resources tend to bill on a constant preset basis regardless of actual work level. Obviously this may not be the same for the internal team, contractors, expense type dollars, and material charges? Even more complex to analyze is the labor rate variation impact on parameter values. In this category, EV assumes a rate value used by initial PV estimates and that is highly likely to be inaccurate. Forecasts for EAC and VAC are then based on this static assumption. As an example of this assume that a higher priced resource is used? This may result in the work getting done quicker, or it may have little effect. In any case, actual labor rate variances will distort the interpretation of the calculated cost parameters. A project assessment process needs to understand the effect of actual rates versus the planned rate used in calculating PV and not just the fact that the resulting cost parameter is different from the baseline plan. For detailed analysis, deeper understanding of the underlying drivers is needed for improved understanding. Also, the forecasting model computation must also be redone as a synthesis of the various resource types and not a singular computation as viewed today. This is the essence of the more granular view that must be developed to keep EV as a valid analysis and forecasting tool. In order for this required level of granularity to be achieved it will be necessary to separate the project planned and actual data into proper groups and then reconstitute the parameters into a more holistic grouping for improved forecast projections. Thus, the desired goal for EV is to not only to accurately quantify the current status of the project but to help identify where the real variances lie across defined resource groups such as those shown above. Simple Example In order to help make the point that resource types can impact EV parameters a high level Excel- based phase grouped example is shown here in figure 3. In this simplified model the various resource types have been segregated and performance status maintained for each group. American Journal of Management Vol. 17(4) 2017 75 FIGURE 3 SIMPLE GROUPED RESOURCE EXAMPLE To help decipher the EV calculation logic shown in this simplistic example the January task elements shown in figure 3 are exploded into a more visible granular format in figure 4. Each of the simulated work packages in this plan contains a separate PV and AC for each of the six resource groups. As total status is calculated for baselined activities through July (see figure 3) the various EV parameters can be computed for each resource grouping as shown in Table 1. A review of these results shows how the various resource groups can produce different EV performance characteristics for the overall project as well as for individual work packages. This same phenomenon is observed in various projects, but note how using just the overall EV values produced in traditional software models will lead to the wrong interpretation for the lower level resource units. In this example case, it appears that the root performance issue lies primarily in the material and contractor resource group variances and not team productivity. Figure 5 highlights the resource group variability for resource level CPI values in graphical format. 76 American Journal of Management Vol. 17(4) 2017 FIGURE 4 JANUARY ELEMENTS EXPLODED VIEW Activity Jan PV Feasibility Analysis 400 Contracts 0 LOE 20 Direct Dollars 60 Internal Team 300 Materials 20 TABLE 1 JULY STATUS RESULTS Group AC EV PV CPI Overall 5,400 4,340 4,700 0.80 Contracts 1,471 1,058 1,140 0.72 LOE 545 545 545 1.00 Direct Dollars 420 348 380 0.83 Internal 2,000 1,809 1,940 0.90 Materials 984 599 695 0.61 ** CPI values are computed as EV/AC, while SPI. A graphical view such as shown in figure 5 this makes a nice presentation format for use in cost status performance discussions. From this resource data view collection, forecast estimates to completion can be derived by calculating an EAC parameter for each resource group and combining that into a total project view. From this point an independent assessment of trend continuity is needed to decide how to forecast that resource. Once this review is completed the total project EAC would be the sum of the individual EAC components, or EACgroups. A more sophisticated example project plan can be produced, but previous research has shown the same characteristics as exhibited here for the resulting EV cost parameters. American Journal of Management Vol. 17(4) 2017 77 FIGURE 5 CPI VARIABILITY ACROSS RESOURCE GROUPS From this example we have numerically demonstrated that the traditional cost EV calculations are flawed at least by the following scenarios: a. Project internal team cost performance is hidden by values of other resource groups, thus distorting actual performance results b. Project internal team cost performance is distorted by values of other resource groups, thus leading to an erroneous conclusion as to actual team performance. c. There is no reason to believe that each of the resource types has the same variability characteristic and this in turn will create different EV values for each. CALCULATION ENGINE Early research determined that these new EV calculations would be overly cumbersome without a supporting calculation engine. It is also important to point out that existing project modeling utilities such as Microsoft Project or Oracle Primavera will adequately handle aggregate traditional EV parameter calculations assuming the user is properly disciplined in data and underlying process. Also, these modeling tools provide needed interim schedule and tracking logic that should not be duplicated externally. So the design challenge is to use portions of the existing utilities and use some form of external process to supplement them with these new calculations. An initial design strategy explored ways to embed the new calculation logic inside of a modified Microsoft Project (MSP) view using dummy variable fields, macros, etc. However, the internal design of MSP makes it difficult to organize the data as needed, although successful tests were made in splitting out labor, material and dollars within a work package. Also, contractor, LOE and labor rate issues were found to be more numerically complicated to deal with and no viable internal based solution was found. Based on these results a second design phase effort was made oriented towards keeping all of the granular data external and feeding it into Project to handle status tracking and producing traditional EV parameters. From this base point the raw status data would be extracted and moved to some external calculation process. The needed granular data would then be combined in the external utility model and formatted according to the new rules. Required data flows between Project and the external EV calculation engine, plus a data base design to support this process have now been completed. Excel has been used to verify the calculation algorithms. Work has also been completed on the physical design of a prototype analytical engine connected to Microsoft Project. This engine is designed to support the level of data granularity required to properly produce the modified parameters. Eventually, a production version would serve the interface role of 78 American Journal of Management Vol. 17(4) 2017 moving required data into and out of MS Project. In other words it would feed the same planning data used today, but would house offline lower level detail data required to produce the new EV parameters. All of the resource groups outlined in the test examples described would be contained in the engine�s data base. In addition to the planning data, actual resource consumption data would be captured at the required level of analysis. This means that actual low level resource consumption by work package or cost control accounts would be needed. In operation, the calculation engine would feed MS Project needed data to generate the traditional project plan (i.e., WBS, task, duration, and predecessor). These input data are sufficient to create the project baseline and ongoing schedule, but would not produce cost values. At status time MSP values would be sent back to the calculation engine for decomposition and parameter generation. Interpretation and project status analysis would occur based on the values created in the calculation engine. At this point the research design effort hit a philosophical decision point. That is, will MSP be the system of record for all status reporting, or is that to be moved to the calculation engine? The conclusion reached is that EV calculations must be handled external given the logic flaws evident in the commercial model. Conversely, MSP contains valuable work unit status calculated that should not be duplicated, so for the first phase all cost data will be external. Conceptually, the calculation engine has the advantage of being more flexible that the commercial vendor products. From the phase one design the future expansion should be in the directions of easier user interface and interpretative support of the output data. At least three sets of data values would need to be extracted from MSP. These are: a. WBS ID b. Task names c. Baseline performance for active work packages (PV,% Complete, etc.) The EV engine would then use this collection of data along with its stored granular supporting data to produce the modified values of EV for each work package (or control account), as well as producing the aggregate and forecast parameter views for both cost and schedule. Analysis Engine Prototype Figure 6 shows a design schematic of the analysis engine prototype currently under development by the author. FIGURE 6 EV ANALYSIS UTLITY The role of this prototype utility is to collect necessary planning data for insertion into Microsoft Project, then extract project status data for manipulation according to the groupings outlined here. Trend assumptions will be made for each resource group and used in the later status interpretation. EV parameter calculations will be driven by lower level data resource values and assumed trend directions for each resource group. As an example, what is the anticipated future material costs trend for the remainder American Journal of Management Vol. 17(4) 2017 79 of the project? This assumption will then be used to calculate the EAC for material. Similar assumptions would be made for the other resource groups. From this process the utility would produce a cost at completion forecast for each group and then aggregate this into a forecasts for the total project. CONCLUSION From the early research there is high confidence that the calculation thesis as stated is valid. EV parameters have high potential to be distorted when only looking at the macro resource level as outlined in traditional literature. Additionally, current computer software is useful (with proper operational discipline) for tracking overall plan versus actual work status, but is suspect in accurately evaluating internal project performance for the various resource types. At this research stage it seems logical to conclude that producing an erroneous EV parameter using the traditional techniques will make the model less effective operationally and potentially erode trust in the technique itself. Logic also suggests that if EV calculations do not accurately validate current or forecast status project managers will lose confidence and seek out other alternatives. A summary of these research conclusions follows: 1. Project status is best understood by increased granularity of resource data within a work package. It is important to recognize that some resource variables are outside the control of the internal project and these external resources can distort the actual project internal status. 2. There is no reason to suspect that each resource type has the same forward trend projection and this variability needs to be recognized in the parameter calculation. 3. Access to lower level data granularity lies at the heart of the solution. Work package estimates have to be made in such a way that individual resource trends can be assessed at that level, then combined for an overall view. 4. There is research evidence that project culture is established somewhat early, but that assumption does not hold for all variables. Failure to account for this can easily distort the calculation interpretation. 5. A more recognized evaluation step for each resource type seems to be required for effective forecasting. This means that specific focus on trend assumptions for each resource becomes more important in the process. Current EV literature suggests that project performance becomes somewhat static relatively early in the life cycle. That assumption may well be weak when looking at the lower level resource issues. 6. The level of complexity involved in this type analysis requires computational support that cannot be handled within traditional utilities such as MSP and the process illustrated is to labor intensive to handle with spreadsheet manual processing. A broader summary set of conclusions found during this research effort is summarized below: 1. Blindly extracting EV parameters from a computer model is worse than not using them at all (false indicators) and will erode credibility of the EV metric as a meaningful indicator for performance or forecast. 2. Using zero variance or a 1.0 EV index parameter values as indicators of on-plan performance is an erroneous indicator for the reasons outlined here. 3. A WBS Dictionary oriented data store is needed to provide flexible work package data views for various EV oriented analysis. 4. Percent (%) complete is a fundamental component of the EV parameter calculation, yet this measurement approach is one of the most error prone process elements. The most significant conclusion uncovered in this analysis is that poor performance by a single resource area can drag down an overall EV cost performance metric and hide away the root cause of this situation. For example, if the internal team has a 0.99 CPI value but the overall project CPI measure is 0.80, the team�s performance is not properly reflected by the 0.80 value. It is important to recognize that in many situations other resource group�s performance is external to the control of the project manager and certainly does not reflect the actual performance level of the team. Would it not be more useful to know that the material or contractor overruns were causing the poor result and the project team was doing 80 American Journal of Management Vol. 17(4) 2017 great? If one were trying to analyze a corrective action strategy this level of status visibility is needed. Failure to understand this and stay at the traditional macro level makes such analysis difficult if not impossible to derive. The traditional macro-level parameter calculations limit the proper usage of EV as a project level performance interpretation and are not of great value in root causal analysis. In order to be the robust analytical tool it promises, EV calculations must go to the lower resource view. If this can be achieved the model should evolve into the promised best-of-class project performance metric. From a prescriptive view, the following items are offered as advice for the project manager attempting to use EV: 1. Recognize the need for low level analysis of WP performance to evaluate various aspects of project current status and forecasting. 2. Work Package time and cost padding will affect accuracy of the calculated project critical path and result in an inaccurate plan, as well as distorting the subsequent EV parameters computed from this data. 3. EV parameter calculation is dependent on the current status date. Work performed in advance of the plan is not used in the calculation, but it can also distort the EV calculations. 4. Actual cost of a task is independent of task duration, so cost data must be externally collected and not derived from an �effort driven� calculation based solely on duration. 5. Project resource labor resource rate analysis is a key project management productivity analysis consideration and this requires a lower level granularity of data analysis. 6. Lipke�s research has highlighted the flaws in SV and SPI formulas after the 70th life cycle percentile. His Earned Schedule (ES) modified calculations should be used throughout as a schedule status measure. 7. Recognition of the role of TCPI (likelihood to complete parameter) calculations are now being recognized as another aspect of EV and this can also be distorted by using wrongly produced values. More research is needed in this area as well. This paper has highlighted sample ways in which traditional EV parameter calculations can yield erroneous conclusions when compared to traditional methods outlined in the published literature. Proper analysis requires a more granular resource data view. The examples shown here have illustrated selected samples to how the current popular computer models produce values that do not represent what the traditional literature implies. Also, the traditional parameter calculations have been shown to not accurately support management status analysis requirements. REFERENCES Carstens, Deborah, et al, (2013), Project Management Tools and Techniques, ISBN 978-1-4665-1562-8, Boca Raton, FL, CRC Press. Christensen, D.S., (1998). The Cost and Benefits of the Earned Value Management Process, Defense Acquisition University, http://www.dau.mil/pubs/arq/98arq/chrisevm.pdf (accessed October 17, 2008). Fleming, Q.W. and J.M. Koppelman, (June 16, 2006), Start With �Simple� Earned Value on All Your Projects, Crosstalk, www.stsc.hill.af.mil/crosstalk (accessed October 17, 2008). Lipke, W.H., (2009). Earned Schedule, ISBN 978-0-557-17738-7, North Charleston, SC, Lulu Publishing,. Richardson, Gary L. (2015), Project Management Theory and Practice, 2nd Ed., ISBN 978-1-4822-5495- 2, Press, Boca Raton, FL, CRC Press. Song Lingguang, (2010), Earned Value Management�A Global and Cross-Industry Perspective on Current EVM Practice. ISBN 978-1-935589-8, Project Management Institute, Newtown Square, PA.