Why automations fail in production: five causes

Automations fail silently because the platform's default alert is an email to individual users, set by each person's notification preferences, and a scenario that keeps erroring can be switched off by the platform itself. On Make, for example, a scheduled scenario is deactivated after three consecutive errors by default (the threshold is a setting, and instant-trigger scenarios are handled differently), so an unread email may be the only signal that the process has stopped. Check your own platform's current documentation for its defaults. Often nobody notices until a downstream consequence appears, such as a customer asking where an order is.

On the fleet order automation, the clock on an installation job existed only in somebody's memory, so a breach of the two-business-day commitment was something the team found out about from the customer. That is a silent failure with a person in place of an automation. The fix was an SLA engine that runs the clock against a business-day calendar and escalates by itself when it expires.

What to build instead

• Alert into the channel the team actually uses, not an inbox. A failed run should interrupt somebody.
• Attach the run log to the alert: the trigger, each step's input and output, and the error.
• Add a heartbeat for anything scheduled. A scenario that should run hourly and has not run in three hours has failed even though nothing errored.
• Tell a run that failed apart from a run that never started. They have different causes, and only one of them appears in the platform's error list.

A retry creates duplicates when the automation's write is unconditional. A webhook delivered twice, a scheduled run that overlaps the last one, or someone re-running a failed execution each repeat the create, and you now have two records. The fix is idempotency: running the automation twice leaves the same result as running it once.

Duplicates erode trust faster than almost any other fault, because everyone can see them and they imply the whole system is unreliable.

What to build instead

• Derive a stable key from the payload, such as an order number or document identifier, and search before creating. On the fleet order automation, inbound orders are matched against existing rows, so a resent order corrects the row instead of adding a second.
• Prefer upsert semantics wherever the platform offers them.
• Where the destination is a database you control, put a unique constraint on that key. A constraint is a guarantee. A search-then-create is a race.
• For one-off loads, make them re-runnable. The trading ERP's history load was idempotent, so running it again corrected records rather than duplicating them.
• If you cannot say of a scenario that running it twice is safe, it is not finished.

A token pasted into a module's headers becomes part of the workflow body. It travels with every export and appears in every screenshot, and it can only be rotated by editing every place it was pasted, so in practice it is never rotated.

What to build instead

• Move tokens into the platform's connection or variable store and reference them by name.
• Audit existing scenarios for hardcoded keys. Blueprint Runner does this with a single scan-and-extract action that finds bearer tokens and keys in a pasted Make blueprint and rewrites it to reference variables. The same sweep is worth doing by hand on any platform.
• Where a token is account-wide and therefore dangerous, keep it in one server process. On the fleet order automation the monday.com token reaches commission columns, so a Python service owns it, and the module that calls that service imports server-only, which turns a client component reaching for it into a build failure rather than a review comment.
• Never let an export containing credentials reach a repository, a ticket or a chat message.

Bills grow faster than volume when the structure multiplies billable units: polling more often than the data changes, doing per-row work that could be batched, or iterating a whole table where a filter would do. When a bill outgrows volume, look at structure before price: each of the three causes below can be fixed without changing what the automation does.

1. 01A polling trigger checking far more often than the data changes. On a platform that bills every check, checking every 5 minutes is 288 checks a day, 288 x 30 = 8,640 a month. If the data changes twice a day, that is about 60 changes a month. Move to a webhook or widen the interval. The FormCraft build uses a database webhook on insert, with polling only as the fallback.
2. 02Per-row work that could be batched. Fetching a hundred records and acting on each individually, where one filtered query and a single write would do.
3. 03An iterator over an entire table because a filter was easier to omit than to write. The filter belongs in the query, not in a condition downstream of it.

Whether an empty poll is billed depends on the platform and the trigger type, so check how yours counts before you estimate the saving.

Rules whose violation would damage the business belong in the database schema, because every code path, automation and manual edit has to pass through it. A check in one function disappears when that function is refactored, or when a second path writes straight to the table.

The constraints that protect a business, such as this cannot send twice, this address is not verified, or this record cannot be deleted, are often implemented as one check in one place. Application code gets refactored. Database constraints do not.

• Unique constraints for what must not duplicate. In Skofi Outreach, a step sends once per enrolment.
• Check constraints for states that must never coexist. There, an address cannot be marked verified without a verification source and timestamp, so a guessed address never enters the sendable pool.
• Triggers for cascades that must always happen. A suppression stops every enrolment for that address at once.
• Row-level security, so a query from any client returns only what that user may see. The school ERP enforces branch and role scope this way, so a mistake in the interface cannot leak another branch's data.
• Append-only tables for audit trails. In the trading ERP, staff can select and insert but there is no update or delete policy, so the trail cannot be rewritten through the API.

Test the constraints, not only the application. Skofi Outreach ships functional tests that run against a scratch database, and they are the acceptance gate: all of them must pass before the system goes near a real sending domain.

Run it in shadow mode first: alongside the existing process, logging what it would have done without doing it, then compare against real work before cutting over. Comparing predicted against actual on genuine data is the only test that counts.

The fleet order automation shipped its allocation and SLA layer behind a shadow-mode flag and a go-live cutoff. It ran next to the manual process, and comparing the two on real orders is what made the cutover uneventful.