Chapter 5

Evaluating an Embedded Analytics Solution

Learn how to pick the right technology and the right vendor for your use case.

Assessment of Top Analytics Solutions

Great! You’ve decided to invest in embedded analytics. Now what?

Choosing the right solution involves a great deal of research. This includes assessing the technology, understanding the expertise of the vendor, and putting together an A-to-Z plan for success.

First, let’s examine the list of criteria that are critical to the assessment. These include the common technical and non-technical requirements.

  • Self-Service Capabilities
    These are the core capabilities that you will make available to your users. These may include dashboards and reports as well as the interactive and analytics functions.
  • Data Environment
    The solution you choose will connect to your current data environment in order to meet your security requirements. In an ever-changing environment, it should be flexible enough to meet increased demand.
  • Embeddability, Customization, and Integration
    One of the ways in which embedded projects differ from those of standalone is the ability to integrate with the app environment. This means providing a look and feel aligned with your brand as well as the extensibility to meet evolving business requirements.
  • Development and Deployment
    Since time-to-value is so critical to the success of the project, having a well-run development environment is key. Meaning, one in which you can create, style, embed, deploy, and iterate on embedded analytics. The end result is that your group delivers the functions that the business needs.
  • Licensing, Services, and Company Expertise
    Choosing the right partner is not simply about the technology. It’s also about finding the level of expertise you require for training, support, and services, as well as agreeing on the business terms that ensure shared success.

First, we will detail the common evaluation criteria in each of these categories. Then, we will walk through the entire process for a successful evaluation.

Self-Service Capabilities

These are the core capabilities for all the end users of your application. During your evaluation, make sure that the capabilities important to your project are showcased. Plan how you will deliver and iterate upon these capabilities within your application.

Strategic Objective
Requirement
The Spectrum of Self-Service Personas

Increase the adoption rate of embedded analytics. This can be accomplished by providing a broad range of users with a tailored experience that matches their needs and skills. Users will typically fall into one or more self-service personas.

Information Consumers
Preference of a defined experience where they can access core business metrics through dashboards and reports that have been prepared for them.

Content Creators
More knowledgeable workers who respond to ad hoc requests for new dashboards and reports.

Analysts
The need for an exploratory environment to discover insights and create new metrics that drive the business forward.

Presentation and Information Delivery

Empower everyone to leverage visualizations. This will help to monitor KPIs and get a complete view of the business.

Data Visualizations
Inclusive of bar charts, gauges, heat maps, spark lines, and geographic maps.

Dashboards
In formats that are both static and interactive, these showcase multiple visualizations in a single view.

Reports
In formats that are both static and interactive, these showcase tabular views of data.

Provide an optimal user experience regardless of where and how users prefer to access information. Evaluate the compatibility of solutions across different devices and formats.

Web Browser
Users should be able to access all content and capabilities on standard web browsers.

Mobile
Users should also be able to easily access and interact with analytics on mobile devices and utilize mobile features such as touch input.

Exports
Content should be available in non-web formats for printing and offline access, such as PDF and Excel spreadsheets.

Interactivity and Automation

Create an engaging experience in which users can explore and interact with their data.

Filtering
Users can choose the data that is important to them and get more specific in their analysis.

Drilling
Users can dig deeper and gain greater insights into the underlying data.

Personalization
Users choose the visualizations and reports most important to them. They can also re-arrange content into their preferred view.

Grow user adoption through the usage of analytics into everyday work.

Workflow Actions
Users can act on specific data sets, by initiating a workflow process on select records or making updates without having to leave the application.

Alerts
Users receive automated notifications when certain actions are carried out or thresholds are met.

Scheduling
Content can be scheduled for delivery on a one-time or recurring basis.

Analysis and Authoring

Empower users by giving them greater flexibility in their analysis. Give them the ability to create and format the desired content on their own.

Data Query
Users choose the data sources, tables, and columns in which they are interested – without having to write SQL.

Data Analysis and Visualization
This is an experience that is intuitive. Users can see, understand and visualize the data. This is supported through the filtering of data and the creation of new calculations and visualizations.

Dashboard and Report Authoring
Users lay out dashboards and reports. They share what they’ve created with colleagues.

Extend the value of the data in your app. Provide deeper insights into business trends.

Benchmarking
Users can compare their performance against that of the industry. They can then pinpoint areas for improvement.

Advanced Analytics
Provide the unique benefit of advanced (and often proprietary) statistical models in your app.

Data Environment

First off, the solutions you consider should be compatible with your current data architecture. Second, these should be flexible enough to meet the changing demands of users. We have outlined the requirements that most providers ask for:

Strategic Objective
Requirement
Data Sources

Use native connectivity optimized for the data source. Ideally, your primary data source should belong in this group.

Databases
Included are SQL Server, Oracle, MySQL, and DB2.

Modern Data Sources
Painlessly connect with modern data such as streaming, search, big data, NoSQL, cloud, document-based sources. Quickly link all your data from Amazon Redshift, MongoDB, Hadoop, Snowflake, Apache Solr, Elasticsearch, Impala, and more.

Cloud Applications
Included are Salesforce, Carbonite, Forcepoint, DigitalOcean, AWS, Dropbox, and Civis Analytics.

OLAP cubes
Used for multi-dimensional analysis

When a vendor-specific connector is not available, generic connectors provide flexibility with data.

ODBC/JDBC
Used for connectivity.

Web Services
Included are REST APIs.

Files
Included are XLS, CSV, and XML.

Enjoy the ultimate flexibility in data sourcing through APIs or plug-ins. These connect to uncommon or proprietary data sources.

Data APIs and Plug-Ins
Coded in your language of choice, these provide customized data access.

Data Management

A solution that enables you to (1) connect directly to underlying data sources and (2) cache data from transactional systems has two key benefits. It provides real-time reporting and interactive self-service analysis.

Direct Connect
Query directly to the data source for reporting in real-time. Tap into the capabilities of the data source that is underlying.

Data Caching
Unlike “direct connect,” data is extracted from the underlying sources into a high-performance data store. This makes the most out of reporting and analysis from systems that are transactional.

Create a complete, user-friendly view of the data by preparing it for analysis.

Multi-Source Data Blending
Data from multiple sources is compiled and the output is a single view, metric, or visualization.

Data Transformation and Enrichment
Data can be enriched for analysis. Examples include new metrics and calculated values that are frequently used, standardization of dates, aggregations, and manipulation of multi-part text (e.g., addresses).

Metadata
Self-service analysis is made easy with user-friendly naming conventions for tables and columns.

Create an efficient user experience that allows users to immediately act on insights.

Bi-Directional Data Flow
For data updates and workflows, inputs can flow directly back to the source systems.

Incorporate data from external sources into a single consolidated view. Transform your app into a vital hub of information.

External Data
This could be in the form of third-party industry benchmarks, data feeds (such as weather and social media), and customers’ data stores.

By consuming data services from the analytics solution, you can offer flexibility in presentation style.

Data Services
The analytic solution provides both the raw data and functionality. These data services produce outputs to be used by jQuery components, third-party charting, and other application functions.

Embeddability, Customization, and Integration

Implementations of embedded analytics vary from that of traditional business intelligence. The former place a greater emphasis on integration capabilities and customizations. Application providers typically want to offer a seamless user experience within the context of their existing app and brand. Focus on enhancing the value of your application while minimizing the cost of development.

Strategic Benefit
Requirement
Security

Making sure that security controls are in place is critical. It should be easy to transfer the security from your application to the analytics content. Scrutinize vendors on the flexibility of their security models. Check out how they launch single sign-on. Ask whether or not data needs to be synchronized or replicated between apps.

Authentication
Single sign-on should leverage the authentication of the parent app. This should be taken care of without having to replicate and synchronize user profiles.

Authorization
Roles and rights established in the parent application are passed to the analytics application. This ensures end users are granted the appropriate levels of access.

Application Security
Fine-grained permissions can be applied to end-user visualizations and functionality. These include charts, reports, and dashboards as well as input controls and user functions.

Data Security
Security can be applied to data sources, tables, columns, and rows. This is crucial for multi-tenant applications.

Multi-Tenancy

Accelerate development with a solution that has built-in multi-tenancy support. This allows you to create a report once and deploy for multiple customers.

Multi-Tenancy
A single application has the ability to share data access between multiple customers, whether data is stored in the same database and/or in individual databases per customer. Look for the ability to parameterize and tokenize. Look for those that do not require data replication or advanced data modeling. These support multi-tenancy.

User Experience

Create an improved UX by embedding analytics as a natural part of your application.

White-Labeling
The look and feel of embedded analytics should match your brand and application. The logo of your analytics provider should not be visible.

Embedding API
Content is usually embedded via a JavaScript API. Parameters can then be passed from the parent application to ensure visualizations are rendered in the correct context.

Application Linking
Users can navigate from analytic content to the parent application and vice versa. A common example is clicking on a part of a chart to go to the specific record in the application.

Workflow

Create the most efficient user experience. Meaning, one in which users can immediately act on what they see, be it visualization or report.

Workflow Processes
Users can initiate API calls to your application from a report or dashboard. They can perform data operations or process transactions the moment they see the data. For example, a user could select a region of a chart and perform an action on the selected records without having to leave the visualization.

Extensibility

Competing through analytics often means delivering unique functionality. Ensure you’ll be able to meet any future requirement with a solution that can be expanded in scope.

Custom Code
For specific presentation needs, see how custom HTML, CSS, and JavaScript can be incorporated. For specialized functionality requirements, understand how custom-compiled code can be integrated into the solution.

Third-Party Charts
For unique charting requirements, understand how third-party charting libraries and components can be utilized and embedded alongside “out-of-the-box” visualizations.

Development and Deployment

Time-to-value is so critical to the success of the project. Having an environment where you can create, style, embed, and deploy analytics is crucial. These will enable your team to deliver the functionality that your business demands.

Strategic Benefit
Requirement
Development

Empower your development team with the tools to quickly create and iterate on embedded analytics capabilities.

Rapid Development
Assess the tools. Determine how quickly you can create content, fine-tune how the content looks and behaves, and make changes to what you’ve done. Understand how to make both small changes to functionality as well as large-scale ones that affect the entire app.

Out-of-the-Box Functionality
A rich set of capabilities – visualizations, self-service analysis, input controls, and UI themes – will accelerate your product development.

Sample Applications
Access to sandbox applications will accelerate both the learning process and adoption of best practices.

Collaborative Development
Embedded analytics should integrate with your source control systems. It supports version control and collaborative development.

Deployment

Quickly deploy and scale an implementation that is aligned with your current technology stack. Make sure that you have the flexibility to shift as your technical environment evolves.

Web Architecture
The best solution fits well into your web architecture. It reduces the need to deploy technology that is proprietary. Furthermore, it uses techniques that are known for scaling the implementation.

Deployment Style
The greatest flexibility comes from solutions that can be easily deployed on-premise at customer sites, hosted in your data center, and made available in the cloud through such data platforms as Amazon Web Services and Microsoft Azure.

Licensing, Services, and Company Expertise

Choosing the right partner is not just about the technology. It’s about finding the right level of expertise and team commitment to get you to the finish line (and beyond).

Licensing

Software licensing terms should align the vendor with the value that you provide to your customers.

Licensing
Terms of the license can depend on a variety of factors, i.e., number of users/customers, servers, and usage. These may be further impacted by whether you are embedding into a commercial product. Be sure the terms make business sense for the short and long term.

Services

Completing your project on time and in the right way can require resources outside your team. Take comfort from a full range of service options even if you do not employ them.

Pre-Sales Technical Support
Leverage pre-sales resources to fully evaluate solutions. This experience will give you an indication of the vendor’s commitment to you as a customer.

Professional Services
Whether you simply need to augment your staff with a consultant or require a whole team to complete a large scope of work, assess the range of professional services offered. Think about the extent of the partner network.

Training
Virtual and instructor-led training options will bring your development team up to speed quickly. It will help them gain a firm understanding of best practices.

Customer Success

Vendors should supply a process that maps your path to success. They should provide a wide array of resources to address any issues along the way.

Onboarding
Look for a process that quickly engages your team in the solution. This will align resources so you have a clear path. Set milestones for completing each phase.

Account Management
Expect dedicated resources that proactively manage your account. They will update you on trends and can be relied on to handle your questions.

Documentation
The quality of documentation is another sign of a vendor’s commitment to your success. Read carefully.

Support
A combination of live and self-service support options plus backed by professionals should be readily available. They will walk you through any technical issues. Service-level agreements (SLAs) should clearly set expectations for response times.

Community
An active user community can lend peer support. They can share best practices so you can benefit from the experiences of others.

Expertise

Leverage your vendor’s experiences to make you and others like you successful.

Company Expertise
Inquire about the vendor’s history with embedded analytics. Learn about the resources dedicated to partnering with software providers (OEMs).

Product Roadmap
Inquire about future product releases that will be of benefit to you and your customers. Participate in annual user conferences for up-to-date information and informative content.

References
Ask to speak to existing customers in similar verticals.

The Evaluation Process

Now that we’ve established the criteria for evaluating vendors, let’s look at how to choose the best one for your business.

1. Determine Your Goals

To get where you want to go, write it down. Statistically speaking, you increase your likelihood of success simply by putting your goals on paper.

Draw from the strategic benefits we discussed earlier in Chapter 2 (Embedded Analytics: No Longer a Want but a Need).

  • Quantifiable metrics may include an increase in revenue, higher user adoption rates, and/or improved customer retention.
  • Soft metrics may include improved UX, creation of a competitive edge, and/or higher customer sat.

2. Establish the Timeline

Identify the steps you’ll take to reach your goals. Ask yourself, “When do I want to…”

  • Begin the selection process?
  • Have detailed vendor presentations and demos?
  • Finish a proof-of-concept?
  • Make my final decision?
  • Start development?
  • Release product?

3. Assemble the Team

Pick the stakeholders who need to be involved. Who is going to care about embedded analytics inside the business (your executive team, product management, lead developers)? And outside (your key customers, customer advisory board)? Build your business case as a team to secure buy-in.

4. Identify Requirements

Review your technical and non-technical requirements. Use the previous pages as a guide to rank and weigh the importance of these requirements. Research the competition. Talk to your customers in order to develop a firm understanding of the capabilities you want to add to your application.

First, describe the functional scenarios in which end users will use embedded analytics. Then, map out their goals for each scenario, and turn these into technical requirements. Consider who will use the third-party products inside the company. Understand their skill sets and identify any potential resource gaps as you move into the evaluation phase.

5. Research Potential Vendors

Assign a point person to research each vendor. Assess which vendors’ functionality matches your requirements. Utilize independent industry resources, such as the 2021 Wisdom of Crowds® Business Intelligence Market Study report, to create your initial list. Pay close attention to vendors that specialize in the OEM market for software providers.

Attend product demonstrations by each vendor in order to confirm a basic fit. Discuss your requirements and ask each one to demonstrate how they would deliver your processes and scenarios. Ask tough questions and make sure the vendors show you the functionality they promise. Confirm ballpark pricing to move forward.

Evaluate each vendor’s ability to make you successful. Ask them to demonstrate how their best practices, support, and training will benefit you throughout the implementation.

Avoid a feature bake-off. Meaning, focus on the requirements you identified in step 4 above, and try not to be dazzled by features that don’t deliver on your criteria. Of course, during your search process, you may update your goals as you learn what’s possible. Just remember to stick to the features that will provide value to your customers and that you can really envision yourself embedding into your app.

Embedded Analytics

More Than Just Pretty Pictures

During your evaluation process, it will be easy to get lost among a dizzying array of charts and graphs. Don’t forget everything we have discussed in this guide. Ultimately, you want to bring the most value to your application, your organization, and your users.

  • Embeddability is how tightly you plan to integrate analytics into the user experience. This extends to the existing security and workflow of the app.
  • Customization is your ability to white-label and control the look and feel of the app to make it your own. Tailor the functionality so that every user has access to the capabilities they need.
  • Extensibility gives you the flexibility to create a unique app experience so you will stand out from the crowd. The goal is to future-proof your solution so you can tackle any new requirement.

6. Complete Technical Evaluations with a Select Few

Narrow down your list to the top two or three vendors. Begin a structured evaluation process with each one. This is where you’ll define proof of concept. Establish clear-cut guidelines for what you want to accomplish within a reasonable timeframe of, say, thirty days.

The amount of interaction you have with each vendor is based on your preference. This can be an assisted trial, where support is generally available if you run into issues. Or, a true structured evaluation, where you and the vendor are building a proof-of-concept together.

Always implement this proof-of-concept in a technical environment that is as close to the production environment as possible. That means it should be connected to your data sources, integrated with your security, and be embedded into your app. If you host a SaaS application in the cloud, do not simply assess desktop tools or run analysis off a cleansed spreadsheet. Do what you expect your customers to do.

At the end of the evaluation, share the output back with your stakeholders. Get prompt feedback and validate your direction.

7. Talk to References

Now it’s time to find out if your vendor can actually make customers like you successful.

Ask your vendors for references. Solicit feedback from others in your personal and social networks. Look for references that are similar (in terms of size, industry, use case, etc.) to your organization.

Find out whether your situation is similar to theirs. Don’t just ask whether they’re happy with the vendor. Really drill into the functionality the vendor has delivered. Ask about the nature of vendor support and training, the duration of implementation, and any roadblocks they’ve encountered. Examine how the vendor handled any problems or issues.

8. Select a Vendor and Get Started

It’s go time! Choose the vendor you feel most confident in as a partner to reach your goals. Of course, you’ll have to compare and negotiate terms and conditions. Look beyond software and focus on the vendor who gives you the highest chance of success.

Make sure your vendor has the resources to help you, even if you don’t need the help today. Later on, you’ll appreciate being able to test ideas and leverage best practices as your needs evolve.

Get training for those who will be using the platform to create analytics. Build your first set of reports. Work with your vendor’s enablement and consulting teams for best practices.

9. Monitor, Adapt, and Optimize

There’s a lot that can be said here, given the endless possibilities that come from using embedded analytics. For the purpose of time and space, here are a few tips for this phase of your process:

  • Invest in the training you need to be successful.
  • After three to six months, do a check-up and consider reengaging with your vendor’s services. Assess additional services that could take you to the next level.
  • Engage with your vendor’s community to learn and share best practices. Suggest ideas for new features while you’re at it.

Questions to Ask During a Reference Call

Success Criteria & Selection

  • What were the key business processes and goals you set for the embedded analytics project? How well has the system delivered on these goals?
  • Were you the key decision maker in purchasing this solution?
  • What made you choose the solution you selected?

Implementation & Ramp-up

  • Tell me about your implementation. What was better than expected and where did you run into challenges?
  • How long did it take you to learn basic functions, like creating a dashboard or report?
  • How tight is the integration between the analytics and your core application (including security, white labeling, etc.)?
  • How hard was it to first set up and then maintain?
  • What has your experience with training and support been like?
  • How proactive has the vendor been to make sure you are successful?

Results

  • Have you deployed your analytics solution yet? If so, what was the reaction like from your customers and prospects?
  • Have you seen any specific benefits? Time to market? Competitive differentiation? Maybe better sales demos or a surge in customers? Have you observed an increase in revenue?
  • What do you love about this platform, and what do you hate?
  • Beyond the licensing, what other costs did you incur during implementation?
  • If you were to do things over again, would you choose the same? Would you do anything else differently?