In good time.

01 / Context

The problem

Large settlement groups could take longer to process than the queue's one-hour visibility timeout. Messages became visible again while work was still underway, leading to duplicate processing. The consumer made two database queries for every meter point: one for settlement status and one to reconstruct its event-sourced state.

02 / Responsibility

My contribution

I traced repeated processing to an N+1 query bottleneck, then introduced bulk queries and batches of 1,000 meter points.

03 / Approach

The decisions

I introduced bulk queries for settlement status and aggregate snapshots, then changed the consumer to work through chunks of 1,000 meter points. This bounds the amount of data handled at once while replacing per-meter database round-trips with two queries per batch. The query pattern changes from 1 + 2N to 1 + 2 × ceil(N / 1,000).

1. 01

   Trace the retry

   Connect repeated queue processing to work exceeding the visibility timeout.

2. 02

   Find the repeated work

   Identify two database queries for every meter point.

3. 03

   Batch the reads

   Fetch settlement status and event-sourced snapshots for 1,000 meter points at a time.

04 / Result

The outcome

For a group of 100,000 meter points, the revised query pattern requires 201 queries instead of 200,001. The change addresses the database bottleneck behind the visibility-timeout issue; the query reduction is not an end-to-end runtime measurement.