Your internal dashboards can look healthy while a customer’s request returns last month’s figures from a cache, a stale replica, or an endpoint that missed an update. Internal telemetry may not show any of those.

I check your API from outside, with the access a customer has, on a schedule built around your releases. The review needs no access to your internal systems.

Each finding can be reproduced

Each finding names the request, the times it was checked, the responses that support it, and how to reproduce it. A hypothetical example: a monthly series due at 08:30 was missing from its observations endpoint in the 08:45 and 09:00 checks and present at 09:15, with all three responses attached.

Findings usually take one of these forms:

  • An expected release that hadn’t appeared by its deadline, with the last check that lacked it and the first check that returned it.
  • Records missing from the expected set, repeated across pages, or inconsistent between related endpoints.
  • Revisions the documented queries don’t return.
  • Time fields, units, or response shapes that disagree with your API documentation.
  • Checks that failed or couldn’t run.

The limits of polling

A check sees the API at one moment. The last check without a release and the first check with it bound when customers could retrieve it; the exact moment stays unknown. A check that didn’t run is a gap in the record, and the report shows it.

I report the scheduled publication time, the confirmed source publication, and the first observation through your API as separate facts. A late publisher can be told apart from late delivery only when the source’s release time is known independently. I label a suspected anomaly as suspected, and keep any guess at an internal cause apart from the customer impact the checks observed.

The findings cover the requests, credentials, location, and period tested. They don’t show that every endpoint or every customer received the same data.

Setting up the review

The review starts by choosing requests that match how your customers use the API, including their filters, pagination, historical queries, and credentials. One call to each endpoint would miss what a paginated or historical query returns.

Each expectation comes from outside the API, such as your release calendar, a documented expected set, or another reference source. An expectation built from the API’s own responses could never include a record the API failed to return.

You provide customer-level credentials, your delivery expectations, the request limits I have to stay within, and someone who can answer questions about expected behavior. We agree on how the credentials are stored and which response data the report may keep.

Before the observation period starts, I run the checks against cases with known answers, such as a past release, to confirm they catch what they should. The review is complete when the checks have run across the period, every gap is recorded, and every finding has responses behind it. A review can be complete without finding a discrepancy. You receive the report, an exportable table of every observation, the check configuration for rerunning the review, and a walkthrough of the findings with your team.

The checks can keep running after the review, with scheduled reports, and with alerts once you’ve decided who receives them and how quickly they should respond. Investigating your ingestion and serving systems from the inside is a different engagement, since it needs access to those systems.

To scope a review, get in touch on LinkedIn with the API, your release schedule, and the requests your customers make most. Delivery problems customers have already reported are especially useful.