Written by Bamidele Suarau

At Valon, we’re more than just a mortgage servicing company—we’re on a mission to empower homeowners and investors by creating value at every step. Our unique position allows us to offer lending-related products like purchase, refinance, and home equity loans, while also providing additional home-related consumer offerings like insurance policies and property tax appeal services.

ValonX is the R&D team that drives this mission forward. We focus on discovering new consumer products, identifying the right audience for them, and supporting a top-class sales team. To do this, we create tools and advertising frameworks that make these efforts more efficient and effective, ensuring our sales and marketing teams are equipped to thrive.

Today, we’re focusing on one of the newly implemented cornerstones of our infrastructure—a “Sales Lead Pipeline” for our Lending business—and detailing how it came to fruition.

Terminology

• CRM: Customer Relationship Management (Valon uses a third-party}
• MLO: Mortgage Lending Officers (our Lending Sales team)
• Sales Prospect: An individual or entity we’re looking to sell a product to.
• BizOps – Our Business Operations team.

The Problem

The initial version of our Lead Pipeline was relatively simple. It focused on two core tasks: gathering data signals from various sources that indicated a homeowner was potentially interested in a new mortgage  and presenting them to sales agents.

• Gathering Signals/Events: Our pipeline gathered data signals from user actions, third parties, analytics models, and more. These signals were then forwarded to our CRM. But in practice, things were more complicated. Users engaged with our services across multiple platforms – visiting our website, responding to emails, calls, etc. Each one of these actions were treated as individual events, leading to a fragmented view of the prospect and their recent actions. 

• Assigning to Sales Agents: The next step was to assign and show potential leads to sales agents. Once a lead was assigned, the agent would decide how to reach out—whether by phone, email, or another method—and record the outcome. However, this process became increasingly complex as more systems and tools were introduced.

Problems Introduced

Here’s where things started to get even more tricky. ValonX interacts with numerous internal and external systems, and each system maintains its own state of affairs without a central source of truth. Figuring out the current aggregated status of the lead across all these systems became a challenge, and causing us to hit a few big problems:

• Disjointed Systems: Each system served a purpose, but together, they created a fragmented ecosystem. There was no feedback loop on how a call went, how an email conversation was playing out, what stage in the pipeline a user was in, etc. Here are a few of the pieces:
  • A CRM to handle telephony (i.e., inbound and outbound calls) and call tasks presented to our agents
  • Two email-related applications for bulk email marketing and managing communication between MLO and Sales Prospects
  • Lending Applications System: Software for Mortgage application intakes from Sales Prospects
  • Loan Origination System (LOS): manages the lending application/funding workflow end-to-end, ensuring compliance with laws and regulations.
• Missing Lead Aggregation: All the individual user actions were modeled as standalone actions in our infrastructure. For example, if you called us 5 times regarding a refinance, we’d treat them as 5 different leads rather than a single lead with 5 incoming calls from the prospect.
• Missing Opportunities: Our infrastructure was limited to only powering Sales Prospects who were already Valon homeowners. This led to dropped sales opportunities when non-Valon homeowners reached out to us. They could be co-applicants with existing Valon homeowners or interested in  consumer offerings. 
• Complex Customer Relationships: The relationships between loans, borrowers, leads, contacts, phone numbers, email addresses, and MLOs formed a complex web. This had the potential to leads to all sorts of issues at scale, including:
  • Users with multiple loans or phone numbers could receive more calls than Valon intended.
  • Inbound communication mishaps, like calls and emails not immediately finding the intended recipient.
  • Lack of a clear history of interactions with a given individual, making follow-ups difficult.
• Lead/Task Management & Prioritization: We also struggled with prioritizing leads. Since we had no centralized source of truth for the state or priority of a lead, MLOs often worked on whatever was in front of them—sometimes leading to missed opportunities with inconsistent follow-ups and efforts. There was no mechanism to ensure that the most critical leads were being addressed first.
• No Code Journeys: On top of that, we relied on our CRM to manage actions triggered by different types of events: for instance, checking if a user was callable, creating follow-up tasks over a specific time period based on successful or unsuccessful attempts to reach the homeowner, and more. While this no-code tool provided flexibility, it introduced additional complexity in orchestrating and managing these various workflows.

This combination of challenges made it clear that as we scaled, we needed to invest more into our approach to the Lead Pipeline. We needed a solution that would unify these systems and streamline the process to make it more efficient and scalable.

THE STRATEGY/APPROACH

Buy vs Build

One of our early considerations was whether to buy this functionality from another partner instead of our current CRM. After exploring the market, we realized that while some products offered parts of what we needed, most didn’t provide the level of customization we required without potentially repeating the same problems we were facing. So, we decided to build a solution in-house that could meet our specific needs.

Event-Based Infrastructure

To manage the various signals and actions coming from different systems, we shifted to an event-based infrastructure. This change allowed us to process events in real-time, ensuring that data was consistently updated across the board. As a result, we could present a more complete picture of each lead, enabling better decision-making for our sales agents.

Flexible Design with Solid Foundation

Given the scale of the project, we anticipated that some product requirements might shift over time. Therefore, it was crucial to avoid making decisions that would be difficult to pivot on later. The system was designed with flexibility in mind, allowing us to make follow-up improvements once it was up and running. A quick feedback loop with our agents and BizOps team played a key role in refining the system based on real-life impacts.

Level of Customization

In our previous system, our CRM offered a high level of customization—arguably too much—which introduced complexity. In our new system, we struck a balance by codifying core logic at the right abstraction layers while ensuring that our engineers could easily and quickly implement changes. For instance, adding a new event type or adjusting how an event is assigned to agents now takes less than a day. This approach allows us to support ever-changing business requirements without needing an overly complex customization system from the outset. As we learn more about the specific needs of BizOps, we can build targeted customizations rather than creating an all-powerful system prematurely.

Lead State Machine

To tackle the challenges of lead prioritization and tracking, we introduced a Lead State Machine. This system allows us to define and manage the various states a lead could be in, providing MLOs with a clear understanding of the lead’s status and the next steps to take. We implemented two key state machines:

• Lead State: This high-level, one-directional state tracks where a lead is in the pipeline, helping prioritize its importance. Possible states include NEW, ALERT_GENERATED, INCOMPLETE_APPLICATION, SUBMITTED_APPLICATION, PROCESSING, FUNDED, NOT_INTERESTED, DOES_NOT_QUALIFY, STALE, and MERGED. For example, a lead in INCOMPLETE_APPLICATION indicates that an agent should reach out to the homeowner to push it toward SUBMITTED_APPLICATION.
• Conversation State: The conversation between an agent and a Sales Prospect is a continuous back-and-forth process. To ensure timely follow-ups, we modeled this with states such as PRE_CONTACT (default state), CONTACT_ATTEMPTED, CONTACT_REQUESTED, and CONTACTED. For example, a missed call from a prospect or an incoming SMS/Email moves the state to CONTACT_REQUESTED, prompting the agent to reach out.

The combination of these states allows us to prioritize the leads we present to agents throughout their workday effectively.

UI for Agents

Finally, we wanted to develop a user-friendly interface for our sales agents. To ensure we were building the right tool, our designer conducted UXR sessions with the agents to understand their needs thoroughly. This process guided the development of a UI that provides a clear view of their tasks, prioritizes leads effectively, and streamlines decision-making.

For the initial version, we opted to use Retool instead of Valon’s web infrastructure. This choice allowed us to move much faster and make UI changes based on feedback from the agents before migrating the UI into Valon’s web infrastructure. 

THE SOLUTION

Sales Lead Models

In our new system, we introduced a single model called “SalesLead”, which serves as the source of truth for all Sales Prospects in the pipeline. This model is central to tracking the state of each lead and managing the flow of information across the system. Let’s break down the core components of this model and pipeline. This model has 1-many relationships with multiple other models and tables. 

Rough Representation of a Lead

High-level overview of the DB modeling

Sales Lead Pipeline

The Sales Lead Pipeline is the end-to-end flow of how events are processed, and how they update leads. As mentioned earlier, this is built using an event-based infrastructure. Let’s break down some of the core components of the pipeline.

High level view of Sales Lead pipeline

SalesEvents

SalesEvents are the backbone of the pipeline, created and triggered by various actions across all our systems—whether internal signals, user actions, agent interactions, web events, communications (calls, texts, SMS), or third-party integrations. Each event carries an Event Type and an Event Name:

• Event Types group similar events together, such as all events from our CRM under a CRMEvents class.
• Event Names specify the individual event, allowing us to configure behaviors at both the type and name levels. For instance:
  • The idempotency key, crucial for avoiding duplicate events, is usually defined at the Event Type level.
  • Assignment algorithms, which determine how events are assigned to agents, are typically set at the Event Name level.

We wanted SalesEvent to be very structured because it touches so many parts of the system, and a badly structured event can have downstream implications. Every SalesEvent must:

• Define the “ContactAssociations” related to it (more on this later).
• Generate a unique idempotency key at runtime for database storage to manage duplicates.
• Implement logic to serialize and deserialize from the database.
• Include relevant validations specific to the event’s use case.

Upon entering the pipeline, each event undergoes validation, including its Contact Associations, and is stored in an event-specific database. This setup ensures we can retry processing in case of errors.

Example of a SalesEvent being initiated.

Contact Associations

A major challenge we identified was the complex web of relationships between loans, borrowers, leads, contacts, phone numbers, and email addresses. To manage this, every lead is linked to one or more Contact Associations, which represent key entities we care about, such as:

• Phone Numbers
• Email Addresses
• Borrower IDs
• Loan IDs
• Lead IDs
• Property Address IDs

During lead validation, we resolve this information to find at least one valid user for each Contact Association. Sometimes, this information doesn’t match an existing borrower within our system because:

• They use a phone number or email address we don’t have on record.
• The individual isn’t a Valon borrower.

To address this, we created the concept of a “Sales Contact,” a table that sits above Borrowers and stores both borrowers and non-borrowers. This allows us to create leads centered around non-borrowers as long as we have a valid contact method, ensuring every event and lead is associated with at least one Sales Contact.

Lead Identification & Aggregation

A core requirement of the system is correctly aggregating events related to the same entities into a single lead. This aggregation allows us to view a lead’s entire interaction history in one place. Additionally, our agent tools support merging leads manually when our system doesn’t automatically do so, since there are some core business use cases for intentionally keeping leads separate.

Executing Subscribers

Finally, we introduced the concept of “Subscribers” to handle different actions triggered by events, such as assigning leads to new MLOs, sending data to our CRM, or transitioning lead states. Each event name defines a list of Subscribers that execute in order during lead processing. This also lays the groundwork for future customization, allowing our BizOps team to define their own Subscriber lists for different scenarios.

CHALLENGES FACED

Lead Locking

When events act on the same lead or related entities simultaneously, the stateful nature of these 

events can lead to conflicts. To prevent this, we needed an effective locking mechanism. At Valon, we primarily use a SQL database where SQL locks are typically ideal for locking specific rows within the same session. However, this approach wasn’t suitable for our case, as we required the ability to lock multiple entities simultaneously during the lead identification phase—covering borrower, contact, loan, lead, and other related entities.

To solve this, we implemented a standalone lock table. This table locks all relevant entities as soon as an event enters the Lead identification phase. If any other events with similar IDs enter the pipeline concurrently, they are automatically blocked and set to retry after five minutes. This approach ensures that no conflicting operations occur on the same lead, preserving data integrity and consistency.

Data Migration

Given the significant transformation in our data models for leads, the new structure didn’t map easily to our old system’s models. However, it was crucial to ensure that data migration was still possible because records of old leads are vital for compliance and completeness.

We approached this by carefully mapping the old data to the new models, preserving the integrity and history of each lead. The migration process was meticulously planned to avoid data loss and ensure that all historical records remained accessible and accurate. This allowed us to seamlessly transition to the new system without compromising on regulatory requirements.

Lead Merging

As part of the transformation, we also had to address the issue of duplicate leads, which can be a common challenge in any CRM system. The new system needed to accurately merge leads to prevent duplication while preserving the history and activities associated with each lead.

We developed a robust lead merging process that automatically identifies and merges duplicate leads based on a set of predefined criteria. This ensures that the data remains clean and that all interactions with a lead are consolidated into a single record, providing a comprehensive view of each lead.

Rollout

Rolling out such a massive project posed its own challenges. Our goal was to implement these changes without disrupting the day-to-day operations of our agents or introducing significant changes to their workflows all at once. We opted for an incremental rollout strategy, which allowed us to make changes under the hood while ensuring the system worked as expected before exposing agents to the new workflows.

To facilitate this, our BizOps team conducted educational sessions with the agents, helping them understand how to use the new system. Throughout the rollout, we gathered feedback from the agents, which was instrumental in refining and improving the system further. This gradual, feedback-driven approach allowed us to integrate the new system smoothly, ensuring that the transition was as seamless as possible for our agents.

Metrics

One of the key metrics we were looking to drive down was “Speed to Lead,” which is the time it takes an agent to respond to an inbound lead (in hours). Since this implementation, Speed to Lead has dropped 10x to less than a few business hours. This is a huge win for the business. Our Sales Prospects are getting very quick turnaround times, which is often the difference between successfully converting a lead and losing a lead to a competitor. 

The business now has the power to control what leads are prioritized for our Sales Agents based on the general economic climate.

Conclusion

Building a robust and scalable lead sales pipeline is a challenging but rewarding endeavor. We embarked on this journey with the clear objective of enhancing our sales process by making it more efficient, data-driven, and tailored to our unique business needs. From the early stages of understanding the complexity of our lead ecosystem to the final rollout of the system, every step required careful planning, collaboration, and iteration.

Introducing the SalesLead model allowed us to create a unified source of truth on our Sales Prospects, centralizing data and enabling better decision-making. We addressed the intricacies of event processing, contact association, and lead identification by developing innovative solutions such as custom event handling, a robust lead locking mechanism, and a sophisticated merging process. 

Furthermore, our approach to the rollout, including agent education and gradual adoption of the new systems, minimized disruptions and facilitated a smoother transition. By prioritizing user feedback and iterative improvements, we were able to build a system that truly meets the needs of our sales agents, empowering them to focus on engaging with prospects and closing deals.

The scale of this project showcases Valon’s commitment to leveraging technology to solve complex business challenges, and it highlights the importance of cross-functional collaboration in achieving success. As we continue to refine and expand our pipeline, the lessons learned from this experience will guide us, driving even greater outcomes for our team and our customers.

If you’re interested in these problems and joining Bami as part of Valon’s team, check out our open career opportunities.