How We Find Out Something Broke Before You Tell Us

For a while, RFQTrace had a bug where invitation emails didn’t send.

Not “sent to spam”. Not “sent late”. The code that built the email referred to a variable that didn’t exist, threw before it reached the mail server, and the invitation quietly went nowhere. The person inviting a supplier saw the invite appear in their team list. The supplier saw nothing, because there was nothing to see.

That bug is fixed. What’s more interesting is how long it could have lived there, and what we changed so the next one can’t.

The failures that don’t announce themselves

When something breaks while you’re clicking, you find out. The page shows an error, the button spins forever, you email us.

But a lot of what procurement software does happens after you’ve closed the tab. An invitation gets sent. A supplier gets notified their quote was received. A verification code goes out so someone can finish creating their account. Nobody is watching those. If one fails, there’s no red box — there’s an absence, and an absence looks exactly like nothing having happened yet.

The failure mode that matters here is specific: a silent failure is indistinguishable from a supplier ignoring you. You invite three shops to quote. Two reply. You assume the third wasn’t interested. If that third invitation never actually left the building, you’d never know — and you’d quietly drop a supplier who never heard from you.

What we changed

Every background job in RFQTrace now reports its own failures. When one throws, we get a message within seconds containing the error, who it happened to, and a link to the logs for that exact job.

The “who it happened to” part is the half that actually matters. An alert saying verification emails are failing tells you something is wrong. An alert saying this address, this purpose, this error tells you what to do about it. We learned that the boring way — chasing a failure that looked like users entering wrong codes and turned out to be two requests racing each other. The symptom pointed one direction; the details pointed somewhere else entirely.

We also made the alerts hold their tongue. Identical failures collapse into one message every few minutes. An alerting channel that fires fifty times in a minute is a channel people mute, and a muted alert is worse than no alert, because it feels like coverage.

What this actually buys you

Not fewer bugs. Software breaks; anyone who tells you their product doesn’t is telling you they don’t know when it does.

What changes is the gap between broken and known to be broken. Before, that gap was however long it took a customer to notice something they’d expected hadn’t arrived — which for a silent failure could be forever, because the thing you’re waiting for looks the same as the thing that never got sent. Now it’s seconds, and the person who finds out first is us.

If you’re evaluating any tool that sends things on your behalf — invitations, reminders, notifications to suppliers — that’s a fair question to ask of it. Not “does it break”, but “when it breaks quietly, who finds out, and how long does that take?”