Trade compliance verification sounds straightforward in theory: take data from a document, check it against authoritative government databases, report the results. In practice, those "authoritative government databases" are a fragmented patchwork of aging systems, inconsistent APIs, unreliable uptime, and wildly different data formats.
At Atlas Verified, we connect to dozens of government and regulatory data sources to verify trade documents in real time. Every integration has been a lesson in defensive engineering. This post shares what we've learned about building reliable systems on top of unreliable government infrastructure.
The Landscape
International trade compliance touches multiple U.S. government agencies, each maintaining its own databases with its own access patterns:
USDA Organic Integrity Database. The canonical source for organic certification status in the United States. Contains every certified organic operation, their certifying agent, certification scope, and current status. There is no real-time API. The USDA publishes a CSV export — a single, massive file — that must be downloaded, parsed, and loaded into your own database. Updates happen on the USDA's schedule, not yours.
OFAC Specially Designated Nationals (SDN) List. Maintained by the U.S. Treasury Department's Office of Foreign Assets Control. Every business involved in international trade is legally required to screen counterparties against this list. Available as XML downloads and through third-party API services. The list changes frequently as geopolitical situations evolve.
FDA Import Alerts. The Food and Drug Administration publishes import alerts that flag specific products, companies, or countries for increased inspection or detention without physical examination. Available through the OpenFDA API as JSON, but the data model is complex and the relationship between alerts, firms, and products is not always obvious.
Consolidated Screening List (CSL). Maintained by Trade.gov, this aggregates 11 different screening lists from multiple agencies — including OFAC SDN, BIS Entity List, BIS Denied Persons, and ITAR Debarred. Available as a JSON API, but each underlying list has different field structures and matching criteria.
WTO Tariff Data. The World Trade Organization's Tariff Download Facility provides Most Favored Nation tariff rates by country and HS code. Useful for validating declared duty rates on commercial invoices. Available as downloadable datasets, not a real-time API.
Each of these sources uses different data formats, updates on different schedules, has different reliability characteristics, and requires different query strategies. There is no unified government compliance API. You build one yourself, or you don't have one.
The Freshness Problem
When a compliance officer checks whether a company is on a sanctions list, they need to know the answer is current. Not current as of last week — current as of right now.
This creates an engineering tension. Some databases, like the USDA Organic Integrity Database, are only available as periodic exports. Others, like OFAC, have third-party API services that claim real-time access, but "real-time" actually means "we sync every few hours."
Our approach is a tiered freshness strategy:
Tier 1: Live API verification. For critical compliance checks like sanctions screening and denied party lists, we call external APIs at query time. The result is as fresh as the API provider's data. We cache results briefly to handle repeated queries within the same verification session, but we never serve stale sanctions data across sessions.
Tier 2: Synchronized local data. For large reference datasets like USDA organic operations, Consolidated Screening List, and port indexes, we maintain synchronized copies in our own database. Automated sync jobs run on configurable schedules, pulling the latest data from source. Local queries are fast and don't depend on the source's availability.
Tier 3: Static reference data. For datasets that change infrequently, like WTO tariff schedules, HS code definitions, and port characteristics, we load reference data and update it on a longer cycle. These datasets change quarterly or annually, so daily freshness is overkill.
Each query result includes metadata about data freshness: when the underlying source was last synced, and what the maximum staleness window is. Downstream systems — and users — can make informed decisions based on this metadata.
The Fuzzy Matching Challenge
Government databases don't know how your documents spell company names. And your documents don't know how the government database spells them.
This is not a trivial problem. A single company might appear as "Pearl White International Limited" on a bill of lading, "PEARL WHITE INTERNATIONAL LIMITED" in an OFAC filing, "Pearl White Intl. Ltd." on a certificate of origin, and "Pearl White Int'l" handwritten on a packing list. These are all the same entity.
Effective entity matching in compliance requires multiple strategies working together:
Normalization. Before any comparison, both the query and the database entries are normalized: case folding, punctuation removal, abbreviation expansion, whitespace standardization. This handles the easy cases.
Phonetic matching. Company names that sound alike but are spelled differently — common with transliterated names from non-Latin scripts — need phonetic algorithms. A company name transliterated from Chinese or Arabic characters will often have multiple valid English spellings.
Token-based similarity. Rather than comparing full strings, we break names into tokens and measure overlap. "Pearl White International Limited" and "International Pearl White Ltd" are the same tokens, just reordered. Token-based approaches handle word order variation gracefully.
Contextual disambiguation. When fuzzy matching produces multiple candidates, context from the document helps disambiguate. If the document says the company is in Mumbai, India, and only one of three "Pearl White" matches is based in India, that's the one.
The matching pipeline scores each candidate on multiple dimensions. High-confidence matches proceed to verification automatically. Low-confidence matches are presented to the user with the evidence for each candidate.
Building Adapters for Unreliable Sources
Government APIs are not built for high-availability commercial use. They go down for maintenance without notice. They rate-limit aggressively. They change response formats in minor version bumps. They return HTML error pages instead of proper error responses.
Every external data source in our system is accessed through an adapter layer that handles these realities:
Circuit breakers. If a source fails repeatedly, the adapter stops sending requests for a cooldown period. This prevents cascading failures — if one API is down, we don't want every concurrent verification to hang waiting for timeouts.
Graceful degradation. When a data source is unavailable, the verification continues with what is available. The result clearly indicates which checks completed and which were skipped due to source unavailability.
Response validation. We validate every response against expected schemas before processing. Government APIs occasionally return malformed data, empty bodies with 200 status codes, or valid JSON with unexpected structures.
Retry with backoff. Transient failures trigger automatic retries with exponential backoff. The retry strategy is tuned per source — some government APIs recover quickly, others stay down for extended periods.
The Verification Chain
A single trade document can trigger verification checks across five or more independent data sources. A bill of lading might require OFAC sanctions screening for all parties, FDA import alert checks, Consolidated Screening List checks, vessel position verification, container tracking, and tariff validation.
These checks are independent and run in parallel. But they all need to complete — or fail gracefully — before the verification result is presented. Our system dispatches all applicable checks concurrently, tracks their completion status, and assembles the final verification result as checks complete. Fast checks are delivered to the user immediately, and slower checks appear as they finish.
The orchestration layer also handles dependencies between checks. If a sanctions screening returns a high-confidence match, the system flags the entire verification as requiring human review — regardless of what other checks find.
From Raw Data to Actionable Intelligence
The rawest form of a compliance check result is unhelpful: "API returned a JSON object with 3 potential matches." A compliance officer needs to understand what those matches mean, how confident they should be, and what action to take.
Our result processing layer transforms raw API responses into structured, actionable findings:
Scoring. Each match is scored based on name similarity, geographic correlation, and entity type match. A 97% name match against a sanctioned entity in the same country is very different from a 72% match against an entity on a different continent.
Contextualization. Results are presented in the context of the specific document and verification. Findings include the specific party being checked, the matching entity details, and why the match is relevant.
Prioritization. Not all findings are equal. A positive sanctions match is critical. An expired organic certification is important. A minor tariff discrepancy is informational. The system categorizes findings by severity.
Recommended actions. Where possible, the system suggests next steps: "Consider requesting an updated certificate," "Verify company registration number directly," "Route for enhanced due diligence review." This transforms a data lookup into a workflow.
Engineering for Trust
The fundamental challenge of connecting AI to government databases isn't technical — it's about trust. Compliance professionals are legally responsible for their verification decisions. They need to trust that the data is current, the matching is accurate, the results are complete, and the system is transparent about its limitations.
Every design decision in our compliance data layer serves this principle. We show data freshness timestamps. We explain match scores. We disclose when sources are unavailable. We never present incomplete results as complete.
Building that trust requires engineering rigor that goes beyond getting the right answer most of the time. It requires getting the right answer every time — and being honest about the times when the answer is uncertain.
Atlas Verified connects to dozens of government and regulatory databases to power real-time trade compliance verification. Our compliance data layer handles the complexity of fragmented government infrastructure so compliance professionals can focus on decisions, not data gathering.