Measuring with HEART: Evaluating the Quality of the VINE User Experience

Last year, Appriss unveiled a significant redesign of VINE, including a new layout, improved features, and a more intuitive design. The update was the culmination of a year-long research and design initiative led by our Product team, built from feedback received through user surveys and feedback, interviews, and focus groups to provide a more personalized, victim-centric experience.

That direct feedback continues to inform our updates to the product, as we continually look to improve or grow the VINE service and ensure it remains a trusted safety net for victims and survivors of crime nationwide. We want to know how the new design is working for those people who use our service. Is it having the impact we hoped? Are people quickly finding the information and resources they need?

As we previously highlighted, Appriss’ Product and Design teams consistently involve the perspective of the people who use VINE in their approach to improvements and enhancements of the service. To help us measure the success of the service and its features, we have adopted Google’s HEART framework as a human-centered analytical tool for evaluating VINE.

To see how we’re measuring the success of VINE, we recently sat down with two of our VINE Product Managers. Angela Art works on both VINE and health-related products, delivering meaningful solutions that help people feel safe and stay healthy. Vanessa Reichelmann is dedicated to delivering our VINE solution through a human-centered development process and is passionate about creating the best experience for all people.

For those who might not be familiar, what is HEART?

The HEART framework was designed by user design (UX) researchers at Google, who laid out five human-centered factors to measure user experience at scale:

  • Happiness – how people feel about and respond to the product (i.e., are they happy that theyHeart-Diagram-02 can access a product like VINE in their time of need and locate important information);
  • Engagement – how people take time to explore and customize their experience with the website;
  • Adoption – how people come to the site for the first time, as well as how they interact with and use new features;
  • Retention – how often people come back or continue to use features as part of their interaction with the site;
  • Task Success – the ease with which people are able to complete actions.

Each metric helps us evaluate the quality of the user experience, providing us with powerful insights to track, improve, and perfect our VINE offering.

Why did we want to use HEART as a framework for determining success on VINE?

HEART aligns with our human-centered product development approach, which focuses on the people that use and engage with VINE every day. We understand that people come to VINE in a moment of high stress, so we want to provide a calming experience while helping them to find the information they need. HEART really allows us to measure the quality and efficiency of their experience, which is so important to us as a victim services technology provider.

For example, we know people who use our service may be victims and survivors of intimate partner violence. When that victim or survivor uses VINE, we know they may be using it as part of their safety plan. So, we not only want them to be able to quickly and easily register, but we also want to be sure that they feel safe and empowered, and we want to be sure they understand the information they see in a notification. Using the HEART framework helps us measure if we are achieving those objectives that go beyond the registration process itself.

How have we been able to apply these HEART metrics so far?

Applying HEART in our human-centered design approach has already proved crucial in helping us help our users. Some examples of this can be seen through our ongoing Happiness and Task Success measurements.

One of the more common ways we measure Happiness is through surveys, which help us focus on the features or improvements that make our users the happiest. Many of our product initiatives begin with extensive research and user surveys. One recent survey around notifications is helping us to better understand how our users feel about their experiences with notifications. We view notifications as a crucial feature in helping users become empowered and safe. If our users do not feel the same way, we want to change that, and in turn, if they are happy, we want to keep that going.

Screen Shot 2021-02-09 at 3.59.58 PMWhen it comes to measuring Task Success, we like to proactively get in front of this as an integral part of our design process by using technology that integrates with our design system allowing us to collaborate with users on testing, validation, and iteration. As an example, during research and redesign of our search and registration process, we used a one-click prototype to test various designs with users. Subsequently, we were able to get quantifiable insight that led us to quicker and more efficient task completion. Task Success goes beyond just being able to complete something. We want our users to successfully complete tasks without encountering errors, on their own, and as fast as possible.

How is HEART informing what we are hoping to do next?

HEART is more of a journey than a destination, so we are actively working on broadening our capabilities and scope. We are currently considering layering in additional technology that will help us gain a more real-time understanding of the VINE user experience. We also have plans to use HEART in measuring the success of features after they are delivered.

If any of our readers would like to contribute to our efforts, how can they help?

There are several vital ways our readers or people who use VINE can contribute to helping us apply our HEART metrics.

  • A major part of our communication to the people who use VINE is our “New in VINE” change log. This log displays improvements, new features, announcements, and upcoming updates and can be found on both each state’s landing page. Each post in the log allows our users to share direct feedback on the specific information in the post. We review all the feedback received through “New in VINE” and share it with internal teams for potential action.
  • User participation in surveys is another key component of understanding our users and if we are meeting desired outcomes. We have a permanent survey available within VINE that encourages users to tell us how they feel about their experience and why. Users can complete a survey every time they use VINE while remaining anonymous or can choose to provide contact information. The survey can be found on each state’s site under “Help” or, if a user registers for notifications, the survey selection will appear on the registration confirmation page.
  • We also encourage our state program managers and other state agency users to provide feedback to their Client Relationship Manager (CRM). Product Management works closely with the CRM team on reviewing feedback and requests for potential action.
  • Our CustomerFirst Center (CFC) is available 24/7 to offer support to users. Those users can in turn provide feedback to our support staff in the CFC. Information received through the CFC is then regularly reviewed by Product Management to determine opportunities for improvement.
Karen Adams


Karen Adams

Karen Adams serves as Training Manager with Appriss Insights, where she educates crime victims, service providers, advocates, law enforcement, and criminal justice professionals about technology solutions including VINE (Victim Information and Notification Everyday) and other issues related to victim safety. In addition to a combined 30 years of experience as an administrator, trainer, facilitator, and mediator, Adams is certified as an ATD Master Trainer and holds a master’s in Management and Leadership from Webster University School of Business and Technology, and undergraduate degrees in Management and Applied Science from the University of Louisville School of Business. As a proud Louisvillian, Karen resides in Kentucky, the Bluegrass state, and has a heartfelt connection to family and friends.

Other Posts By This Author