“Strive not to be a success, but rather to be of value.”
– Albert Einstein

When tasked with measuring the success of a project, scope, time and cost are three typical forms of measurement. Implementing a Manufacturing Execution System (MES) or other IT system is no different where success is often measured by the time, resources and cost spent to achieve a pre-specified result (scope). This approach to determining success is analogous to evaluating a trip from point X to X’ by how much time and gasoline was consumed.

This might be too simplified of an approach.

Based on this methodology, 61% of Enterprise Resource Planning (ERP) implementations failed, when evaluated by a time-only constraint (taking longer than planned); 74% failed when based on a cost-only constraint (cost exceeded budget). These figures were obtained from market research conducted by Panorama Consulting, which issued a study in February 2011 that was reported on here. I was not able to locate similar research on MES deployments, so let’s just assume that the figures are comparable. If this many IT projects are “failing”, then why do so many companies continue to invest time and resources in this activity? Clearly, some sort of business value is being achieved.

Let’s see if we can understand better by first digging deeper into what the criteria are for a successful MES project.

Scope or Goals?

For manufacturers, an MES implementation is not about delivering IT functionality. Instead, it is about gaining certain (preferably tangible) benefits like lead time reduction, reduced scrap rates or improved efficiency. In this context, goal achievement is the real success criteria. This confusion is actually a common mistake when a project is focused on delivering a pre-agreed upon scope of work (typical with a “fixed bid” project) rather than efficiently addressing a business need. In this sense, MES projects are destined to fail – final scope does not necessary match the initial vision. Scope should be aligned with goals, and business goals should dictate intermediate project priorities or even justify deviation from initial scope.

Cost or ROI?

Does increased cost over budget mean a project failed if twice as many benefits were achieved? Most project sponsors would accept a budget overrun if they could see some intermediate benefits. This means that being “on initial budget” is not necessarily a requirement for project success. Just to be clear, cost overruns to accomplish the original project objectives should indeed be considered a failure. Under either of these scenarios, surprises are to be avoided at all costs. If budgets are being exceeded then communications are needed to evaluate business success achieved thus far. Flexibility of implementation is also needed to achieve necessary validation prior to the project continuing, enabling a divergent path that is mutually agreed upon by all stakeholders.

It is not easy to implement a typical MES as an iterative process with frequent changes. MES solutions that are built on a BPM (Business Process Management) platform, however, can be adapted quickly and easily. Quick wins are therefore readily achievable, which then allow sponsors to justify further budget extensions and extract proportionally more business value over the long term, even if the initial project cost exceeded budget.

Time or Cycles?

Time is money, so projects that take longer than planned might be considered as failure. To this point I would suggest you expand your time horizon for a more accurate measurement of value, beyond the duration of your initial implementation. An MES project is not really a single “event,” but is part of a never ending journey towards manufacturing nirvana. Manufacturing Execution Systems must continue to adapt to meet the changing needs and processes of the business. Must goals and objectives be met? Absolutely. But, these objectives might be better measured by phases, such as initial implementation, sites 2-5, global collaboration effectiveness, etc. These objectives should be periodically reviewed as successive iterations, with accompanying Return on Investment and continuing impact on the strategic goals of your company. In some cases, short term goals may be missed in exchange for a company’s long term survival. Alternatively, quick “off-the-shelf” implementations might efficiently address some local needs but drive unnecessary costs associated with the long term maintenance of multiple MES systems on a global scale.

Conclusion

One might say that this is “creative project accounting” and that any project success or failure is if we follow this way of measuring. And I agree with that! Therefore, in order to keep some discipline, you still must set clear boundaries that will motivate a project team to achieve tangible success metrics (on time, on scope, on budget). But, at the same time, I encourage all manufacturers to also have periodical reviews of your projects and benchmark them by their ROI and how well they support your strategic goals for months and years following your implementation.