Can you share an experience when your analysis produced unexpected results, and what did you learn from it?

Certainly! Reflecting on my experience at Better.com, there was a particularly memorable instance where my analysis yielded unexpected results, leading to valuable insights.

Context of the Situation: During a quarterly review, our team was tasked with analyzing the customer journey to identify any bottlenecks in our loan application process. The goal was to streamline the process to improve conversion rates and overall customer satisfaction.

Steps Taken During Analysis:

1. Data Collection: I began by gathering a comprehensive dataset that included customer interactions, application timeframes, drop-off points, and survey feedback.
2. Data Cleaning: Ensured the data was clean and relevant, removing any duplicates or irrelevant information.
3. Segmentation: Segmented the data based on different customer demographics, loan types, and stages in the application process.
4. Statistical Analysis: Conducted a series of analyses, including regression analysis and hypothesis testing, to identify patterns and correlations.
5. Visualization: Used data visualization tools to create clear and informative charts and graphs, making it easier to spot trends and anomalies.

Nature of the Unexpected Outcomes: Contrary to our initial hypothesis that technical issues were the primary cause of drop-offs, the analysis revealed a significant, unexpected correlation: customers were dropping off at a higher rate during the income verification stage, particularly those applying for certain loan types. This was surprising because the platform's user experience assessments were positive, and technical drop-off reasons had been assumed to be the main issue.

Insights and Lessons Learned:

1. Customer Experience Focus: We learned that while technical factors are important, understanding the nuances of customer experience is crucial. The discomfort or delay perceived during sensitive stages like income verification can significantly impact decisions.
2. Iterative Process Improvement: The analysis highlighted the importance of revisiting and refining our hypotheses. It reminded us that data can tell a different story, and being open to adjusting our expectations is key.
3. Cross-department Collaboration: Insights into the unexpected outcomes encouraged cross-department collaboration, particularly between tech, customer service, and compliance teams, to refine the income verification process.
4. Prioritizing Transparency and Communication: We discovered the need for clearer communication with customers about what to expect during the verification process, which could alleviate their concerns and reduce drop-offs.

This experience underscored the value of leveraging data to challenge assumptions and the importance of approaching analyses with an open mind to unlock actionable insights.