The Fuzzy Art of Solution Evaluation: 7 Key Questions for Building a Solid Evaluation Methodology


So you’ve been assigned to evaluate a new technology or application for your organization. Yikes! What next?

First of all, remember that you’re not alone. Many growing organizations are faced with the task of selecting a increasingly wide variety of sophisticated solutions for endpoint security, cloud initiatives, systems management and more. Accordingly, the codification of the fuzzy art surrounding the evaluation process itself has become increasingly important. It’s no long enough to cobble together an ad hoc committee, draft a list of requirements and then let the vendor pitches commence. Planning for and defining the components of the evaluation process are now an essential piece of the bigger project, and critical to its success.

Developing a comprehensive, reusable solution evaluation methodology

Some of these planning steps may seem obvious, while others take a bit more concentrated brain power to articulate. This list of seven key questions will help you develop a more comprehensive solution evaluation methodology:

  1. Who are the stakeholders? Take the time to identify the appropriate members for the selection team and to understand both their roles and their interest in the project. You’ll likely want a different team for evaluating endpoint management or endpoint protection tools than for broader information systems management Solutions.
  2. Who has the decision-making power? Distinguish recommenders from decision makers, and clearly articulate roles and final approval authority.
  3. What resources are required? Consider how long the evaluation process is likely to take and make sure that all participants have bandwidth to participate throughout. You need management buy-in for both the project itself and the required allocation of employee time.
  4. Have you formulated your list of questions and solution criteria? Be sure to involve all affected constituencies as you determine what you need from the solution you choose. In addition to thinking about specific features, be sure to include broad, strategic questions, such as whether the solution will enhance your competitive advantage and how much time it will free up for innovation, especially for the IT team.
  5. What’s the process for determining prospective vendor participation? The proper research legwork is necessary to narrow the pool of possible providers based on a mapping of your solution requirements to the functionality provided by available solutions.
  6. What kind of supporting vendor activities do you require? Do you need an RFP, demo, trial or proof of concept? References? Legal approvals?
  7. Who will do the final analysis? Determine who will be charged with the final analysis and presentation of objective conclusions.

There’s no doubt that solution evaluations require work — and lots of it. But as technology evolves and new offerings enter the market, the evaluation process itself becomes an even more critical foundation for future success.

Applying the evaluation methodology to systems management solutions

If you’re in the market for a systems management solution, we’ve done some of the work for you. Our new Endpoint Systems Management Evaluation Guide lays out a comprehensive list of features and functionality to look for in any candidate solution, helping to make the systems management solution evaluation process as inclusive and straightforward as possible, In fact, even if you’re not currently looking for systems management software at the moment, you might still find it useful to take a look at the guide and perhaps even use it as a template for your own list of questions and solution criteria.

About the Author