Overview:

Problem: Due to high ticket volume, the operations team was unable to triage and resolve time-sensitive support tickets.

Team: CTO, Customer Success Manager, Program Manager, Front-end Engineer

Role: Product Manager supporting the Operations Team

Timeline: 1 month

Understand the Problem:

Possible Finance is a mission-driven financial technology company that provides small-dollar, credit-building loans.

As the number of new customers increased after the company expanded to several new states, the operations team began having trouble managing the ticket volume, especially resolving time-sensitive support tickets. The ticket backlog could number in the thousands of tickets per day.

Because of delays in resolving the time-sensitive tickets, some customers would experience delays with submitting payments on-time (which could negatively effect their credit score), face delays receiving their loan deposits, submit additional tickets, and subsequently leave negative reviews on social media and app stores. These factors impacted cash collections, the customer experience, and the public reputation of the company.

After investigating the issue more deeply, I also learned that many tickets were 'self-help' tickets, or tickets related to issues that the customers should have been able to resolve themselves without the intervention of an agent (changing their payment card or updating their email address, for example).

This case study describes the product management process to solve the above issue.

Solution Selection:

Due to recent state launches, an aggressive growth plan, and the small size of the company, minimizing engineering time was a high priority for this product. Rating the potential solutions for value vs. effort, I determined that embedding a Zendesk-based CRM contact form into the app would require the least effort for the greatest impact. This solution would also minimize engineering effort to build and to maintain, as once the form was embedded in the app, it could be modified and updated by the product team.

The additional advantage of the contact form solution was that it could display blocks of informational text before the customer submitted the ticket, potentially giving the customer the help they need and delecting the ticket.

Assumptions:

After deciding move forward on an MVP for a CRM-based contact form, I produced a list of assumptions that would be tested with this solution:

1. Customers will be willing to take the extra time to navigate a menu to select the issue they are experiencing.

2. Customers will select the correct issue from the menu (rather than make an arbitary selection).

3. Customers will take the time to read a short text block describing how to resolve their issue.

Building a Hypothesis:

Using the 'we believe _____ will _____ because _____' hypothesis model, I outlined the following hypothesis that the MVP would test (blocks of informational text would be added after the initial MVP was validated):

"We believe that if customers can select their issue when submitting a ticket, the operations team will be able to resolve time-sensitive tickets within 24 hours because tickets for time-sensitive issues can be filtered separately and responded to first."

Defining Success for the MVP:

Before beginning the MVP, I worked with the stakeholders to define a minimum criteria for success (MCS). We decided that a successful MVP would need to meet the following criteria:

1. The ops team is able to immediately identify all time-sensitive tickets after the ticket is submitted.

2. Tickets are correctly categorized by the customer at least 90% of the time.

3. Reduce self-help ticket volume by 10%.

MVP Execution Plan:

I divided the MVP into tasks and created an execution plan:

1. List all potential issues a customer could face (this list was large: there were around 150 known issues a customer could experience)

2. Group issues into 'time-sensitive' and 'non-time sensitive' categories

3. Group issues into 'self-help' and 'non-self help' categories

4. Organize potential issues into subject groups and create a menu hierarchy using customer-facing terms

5. Create the menu system in the contact form using Zendesk's Contact Form builder

6. Create an internal binary ticket tag for the ops team to mark whether or not the customer selected the correct category when they submitted the ticket

7. Create a filter view in Zendesk that separates 'time-sensitive' and 'non-time sensitive' tickets

8. Deploy the contact form on the website and the app; test the form to make sure tickets submit correctly

9. Monitor ticket volumes to ensure that tickets can enter the backlog appropriately; spot check individual tickets, and search for keywords like 'can't submit' to catch potential issues with the new form

The issue list was created in an Google Sheets document based on input from members of the ops team, customer success team, technical QA team, and product team; internal ticket category data; and bug reports.

Each issue was placed in a category menu that was up to seven levels deep. The menu would give between one and five options per level, where an initial option would be something like, "Issue with my payment card", or "Issue with my credit report", followed by a more detail selection such as, "My loan is not marked as paid off on my credit report", until the exact issue could be selected. The menu was created with a focus on clarity and ease for the customer, using non-technical, customer-facing wording such as 'debit card payment' rather than 'interchange transaction'.

Evaluating the Results:

One week after the form was deployed, I reviewed the ticket data against the previously determined MCS.

The form proved to be effective in filtering time-sensitive tickets into a high-priority ticket queue. The ops team was able to respond to these tickets immediately after they entered the queue, so the first criteria was passed!

The second MCS concerned whether or not customers would select the menu option that corresponded with their issue. To gage success for this metric, I look at the percentage of tickets that the ops team tagged as 'categorized correctly' using the internal flag. According to the flag, 92% of tickets were categorized correctly, which passed the criteria required for success by 2%.

With the first two MCS validated, we were ready to iterate on the product to validate the third.

Results:

Overall, the product was greatly successful. The ops team was able to identify and respond to time-sensitive tickets, and the overall number of tickets per week dropped by 10% on average. But this metric did not tell the whole story: because the number of active customers grew significantly week-over-week, I used a ratio to determine how much the new form impacted the number of tickets relative to the number of active customers. With the week-over-week ticket counts as the numerator and the total number of active weekly customers as the denominator, the relative ticket volume dropped from 5% to 2.5%, effectively cutting ticket volume in half.

Case study links: