Model Auditing & Assessment

Do you use your forecasting model to make important decisions for your company, decisions that are potentially worth millions or billions in revenue or investment? Given these stakes, it is prudent to seek an independent audit of your model.

We review your proprietary Excel-based forecasting model to assess its quality and veracity. This allows you to proceed with confidence or to fix the model before errors or deficiencies open the way for poor decision-making.

iceberg and mountain
Is your forecasting model a mountain (everything is visible) or an iceberg (hidden dangers you don’t know about)?

Why This Matters

An erroneous or flawed forecast model can pose risks to your company, such as reputational risks (where the model is shared with investors or potential partners) or financial risks (errors may affect decisions about product development or manufacturing capacity, deal valuation, or guidance to investors).

Forecasting models can be subject to several potential sources of error:

  • Formula errors
  • Assumptions (i.e., numbers) that are hard-coded into formulas, making them invisible
  • Statistical errors
  • Pivottable errors (e.g., tables may still show previous data when the underlying data have changed)
  • Time-series (trending) errors
  • Different parts of the model show different results for what should be the same quantity
  • Incorrect use of units (e.g., applying a conversion factor of 1,000 instead of 1,000,000)
  • Non-standard calculations that could break in future versions of Microsoft Excel
  • Macro/VBA (coded software) errors

Even if a forecasting model is free of overt errors, it can still suffer from limitations that greatly affect its reliability, such as:

  • Model is overly simplistic or unnecessarily complicated
  • Model has a flawed design that doesn’t address the model objectives
  • Non-logical calculation stream
  • Assumptions that are unrealistic, based on questionable sources, or are entirely undocumented. These affect future usability and updates and limit the explanatory value of the model.
  • The model has a poor layout, making it difficult to use
  • The model size is much bigger than it should be or the model calculates slowly
  • Unclear formatting or labeling of units (e.g., is that $2.8 in a cell denominated in thousands, millions, or billions? If currency formatting not used, how do you know if the model results in a particular section are the product volume or the revenues if the section isn’t labeled)?

Our Experience

Objective Insights has deep expertise in designing, using, and evaluating forecasting models:

  • We have worked with Microsoft Excel since it was originally released in the late 1980’s
  • We have decades of experience building and using Excel-based forecasting and financial models in life science applications
  • We have been assessing client forecasting models for three decades, both in industry positions and as consultants
  • We are familiar with forecasting both marketed and pipeline products

As shown elsewhere on this website, we have worked with epidemiology/patient-based forecasting models (including sequential segmentation and patient flow designs), unit and prescription trend-based time-series models, and source-of-business patient switch models. We have also used and developed a variety of financial models (not necessarily forecasting), as well as production planning models.

What We Provide

Objective Insights provides a thorough and independent assessment of your forecasting model. We combine an automated analysis using auditing software with a domain expert review of model structure, logic, assumptions, and documentation. We offer quick turnaround and complete confidentiality.

We supply the following deliverables in a written review of your model:

  • Assessment of model complexity, structure, and assumptions/documentation
  • List of any errors identified (location, severity assessment)
  • Sensitivity analysis (i.e., does your model behave as expected with given changes to inputs?)
  • Recommendations for model corrections and improvements

Please contact us today to discuss your needs!