How we rebuilt recurring bill detection at Cleo

A good financial assistant needs an accurate picture of what enters and leaves a user’s account. For Cleo, that picture is most clearly represented in Bills Hub, where users see upcoming recurring payments with predicted dates, amounts, and cadences. It also powers Cleo’s “safe-to-spend” estimate of how much room a user has for discretionary spending for the rest of the month.

When we set out to rebuild bill detection, timing seemed like a natural organizing principle, but real transaction data complicated that assumption. For example, our system flagged Transport for London payments as a recurring bill due to merchant consistency and steady cadence, but the user considered it normal commute spending. Conversely, the system didn’t detect a sports club membership paid in the same amount but at irregular intervals, although the user considered it a bill.

This false positive and false negative share a root cause: Regularity is neither necessary nor sufficient for something to be a bill. Evaluation and testing led us to a bill detector that inverts the intuitive order, grouping transactions by merchant and amount first and only then inferring cadence.

Our initial approach: Schedule-based detection

Our original algorithm ran a cascade from high- to low-frequency patterns: weekly, biweekly, monthly, quarterly, and yearly. Within each frequency bucket, the algorithm treated each candidate transaction as an anchor, then searched backward in time for matches in windows centered on expected recurrence dates. Description matching was fuzzy and amount tolerance was adaptive, starting broad for groups with few transactions and tightening as more were matched. It retained a candidate group only if every expected period produced a match.

After processing each frequency bucket, the algorithm merged overlapping groups and removed matched transactions from the pool so that nothing was double-counted. Once all buckets were processed, a final stitching step reconnected groups separated by short gaps (e.g., a missed payment or paused subscription) using progressively relaxed matching tolerances.

Evaluating the algorithm required ground truth that didn’t exist yet. To create it, we worked with a third-party annotation partner to label the outgoing transactions of roughly 100 users as bills or non-bills. Because these labels applied to individual transactions, we then grouped those transactions into recurring bill series, for an initial evaluation set of 55 users and 561 annotated bill groups.

Against this data set, the cascade recovered 79% of bill groups, a promising result for a first version. Latency, however, was a different story; processing a user’s transaction history took long enough that requests often timed out (P95 latency = 122 s).

What we learned from real transaction data

The cascade’s design encoded three assumptions about recurring bills that real transaction data forced us to rethink:

1. Bills always keep to a schedule. Some do, of course; an auto-deducted rent or insurance payment might arrive on the same date in the same amount every month. But plenty of others shift dates, amounts, or both. If a detection algorithm demands a match in every expected period, it will incorrectly read ordinary variation as evidence against recurrence.

2. Anything arriving on a schedule is a bill. A user might buy groceries every Saturday or pick up pet supplies at the beginning of the month, but neither belongs on their list of bills.

3. Merchant names can be trusted. Some niche merchants appear too rarely to standardize reliably, so merchant identity alone is not a stable key. One recurring bill we examined, for example, was split across two apparent merchants because our enrichment pipeline assigned the same counterparty different names. They were sufficiently similar that a human would read them as the same merchant, but diverged enough to fall below our fuzzy-matching threshold.

Rethinking our approach

Before shifting our approach, we made incremental improvements to the original cascade algorithm. The most significant targeted latency.

Instead of comparing every transaction against every other, we introduced a pre-clustering step that grouped each user’s transactions by counterparty or description, then ran the cascade independently within each cluster. Together with a handful of further optimizations, this cut P95 latency from 122 seconds to single-digit seconds and brought the service within its production latency budget.

Most of the remaining improvements were corrections to assumptions we had baked into the algorithm. A minimum-amount threshold intended to filter out noise turned out to reject legitimate low-amount bills, such as $0.99 app subscriptions and small monthly bank fees. Lowering it added six percentage points of group-level recall. Dedicated logic for semimonthly patterns captured the twice-monthly bills common among users paid on, for instance, the 15th and the last day of the month.

Repeated rounds of dogfooding, in which Cleo colleagues ran the detector against their own accounts, found failure modes underrepresented in our annotated evaluation set, including subscription price changes, one-off amount spikes, and inconsistently formatted transaction descriptions. Fixes for these, together with the corrections above and several smaller changes, lifted group-level recall on our evaluation set from 79% to 94%.

The remaining problem was structural. The cascade assumed that a bill follows a schedule, which made irregularly recurring bills undetectable. The clearest example was the sports club membership mentioned above, where the amount was identical every time, but the varied gaps between payments didn’t fit a frequency bucket.

To capture these cases, we added a final pass after the cascade and stitching were complete. It collected the transactions in each cluster that no schedule had claimed, grouped them by exact amount, and retained any group of at least three transactions as an irregular recurring bill. This addition pushed group-level recall from 94% to above 99%.

This handler, initially designed as a fallback, ended up capturing the insight our final algorithm is built on: A bill’s stable identity is the combination of merchant and amount, each matched with some tolerance, and cadence is a property to infer afterward. If amount-based matching catches everything the cascade misses, why not make it the centerpiece rather than the fallback?

Our solution: Amount-first detection

Within the broad clusters created during pre-clustering, our shipped algorithm groups transactions whose amounts fall within a small tolerance of one another. Each candidate group is then checked for merchant description consistency. The similarity thresholds adapt to the group’s size and structure, separating legitimate variation in how a single merchant appears from unrelated merchants that happen to share a price point.

Only once a group has passed both checks does the algorithm consider timing, attempting to fit the group to a standard schedule and labeling it irregular if none fits. In effect, the algorithm treats every candidate bill as irregular until a schedule is established, instead of assuming a schedule and searching for transactions to fit it.

A set of follow-up passes rounds out detection. Two are mechanical: Detected groups are extended backward and forward in time under a relaxed amount tolerance to absorb price changes, and interrupted patterns are stitched back together.

The other two address issues that arose in our first attempt. Because merchant names can be unreliable, a second, more permissive clustering pass merges near-duplicate names, but only when the clusters also share repeated amounts. The detector consults the transaction categories assigned by our enrichment pipeline and excludes those that recur without being bills, such as groceries and pay-as-you-go transit.¹

On the evaluation set, the amount-first algorithm edged out the improved cascade. To test whether either approach would generalize, we compared the two on the held-out set of 39 annotated users that neither had encountered during development. The amount-first algorithm came out ahead on every accuracy metric.

The largest gains were at the transaction membership level, where detected bills were more complete and picked up fewer unrelated transactions. Group-level precision rose to the point where false positives of the kind that opened this post became rare. Most tellingly for the user experience, the share of users for whom we correctly detected every single bill rose from 28% to 74%, with latency essentially unchanged. On that basis, it became our in-service detector.

Held-out test set of 39 annotated users unseen during development or tuning. “Detected perfectly” means every one of a bill’s constituent transactions was captured and nothing extra was included. Group-level recall uses a stricter definition than the evaluation-set figures above; a bill counts as recovered only if at least three of its annotated transactions are grouped together.

Choosing the right tool for the job

An obvious question at this point is why we shipped a rule-based algorithm rather than a machine learning model, LLM or otherwise. The short answer is that it wasn’t something we decided in advance, nor does it reflect a standing preference for rules over learned models. Instead, it was a series of engineering judgment calls made against the task, the timeline, and our accuracy targets.

Data availability ruled out using a machine learning model due to timing, as training would require labeled examples at a scale we didn’t have. Our golden source of truth, the annotations described earlier, covered roughly 95 users. This high-quality data set was sufficient to evaluate an algorithm, but far too small to train on.

The realistic route to more data would be an LLM pipeline generating (noisier) labels calibrated against the golden set. However, this is a substantial project in its own right and would not have landed in time. The real contest was therefore between an LLM and a rule-based system.

The constraints of a live service helped us decide between those two options. For this context, results are needed in seconds. When we trialed an LLM during an internal hackathon using a prompt that distilled everything we had learned about the problem, we found that it performed respectably but didn’t outperform the rule-based algorithm. More importantly, its latency was an order of magnitude larger. 

Determinism was also important. Bill detection feeds budgeting features and Cleo’s safe-to-spend calculation, so the same transaction history should result in the same identified bills every time. When a rule-based system misses or misgroups a bill, we can also more easily debug by tracing the exact check that rejected it, as in the dogfooding fixes described earlier.

Had accuracy plateaued short of our targets, we would have switched approaches, whether to an LLM or to a trained model backed by broader annotation, but it never did. The metrics kept improving, dogfooding during the Bills Hub rollout confirmed the gains out of sample, and the first version shipped without QA bug reports.

None of this argues against LLMs generally. The question was never whether they would be involved, but where: in the shipped artifact or in the process that produced it. In this project, that place was development. Choosing rules did not mean the problem was simple. Our shipped algorithm is far more complex than a single prompt, encoding the accumulated logic of pre-clustering, adaptive tolerances, stitching passes, and category exclusions. We could only build and debug a system of this complexity on this timeline because we leaned on LLMs as coding assistants throughout development, which let us write and evaluate far more candidate implementations than we could have produced by hand.

The next phase where we plan to use LLMs is related to the data itself. Human labeling was among the slowest and most expensive parts of this project. The annotation pipeline we set aside during this project due to timing remains planned as future work, with LLM-generated labels produced at scale and validated against a golden batch from our internal annotators.

What’s next

The amount-first detector is live in production today, but our work continues in three main areas:

• The first would close a loop with our enrichment pipeline. Bill detection currently uses the merchant names that the transaction enricher assigns. When a group of transactions shares a stable amount and cadence, that pattern points to a single merchant behind them. We are exploring how to feed that signal back so that the detector can improve the enricher’s ability to establish merchant identity.

• The second is broader coverage of irregular bills. Today, an irregular group’s amount must be nearly constant for the detector to find it, so bills that vary in both timing and amount remain the hardest cases.

• The third involves users themselves, as whether something is a bill is inherently subjective. One person might consider their monthly transit pass a bill, for example, while another sees it as everyday spending. Ground truth therefore has a fuzzy boundary that no detection algorithm, however good, can fully resolve, and the most direct way to settle cases on that boundary is to give users the final say. Today, a user can confirm a bill from a list of candidates or remove a transaction that was incorrectly detected as recurring; we are exploring letting users mark any transaction as a recurring bill.

Bills are only half of the recurring transaction picture. Predicting when income will arrive is the other half. For income prediction, fitting the tool to the task led us to instead implement a machine learning model. We plan to cover that system, and why it required a different approach, in an upcoming post.

¹ This filter is deliberately blunt and currently misses some possible legitimate bills. Refining it to correctly suppress ad hoc purchases without losing valid recurring bills is ongoing work.