Home · Martins Ogiekpolor
Bank Health US bank early warning
Data as of ––––––– · updated ––– ––, –––– Model v– Refreshed weekly from FDIC call reports
Pipeline and data

Every number on this site, traced to its source

A weekly pipeline pulls the FDIC and CFPB data, cleans it in layers, scores every bank and checks the result before anything is published.

ScheduleMondays, 06:00 CT
Last successful run–
Freshness targetReady by Monday 12:00 CT
Latest run took–

How the data flows

  1. 01SourcesFDIC call reports, institutions and failures; CFPB complaintsPublic APIs
  2. 02IngestFetch only new quarters and dates, check each file against its data contract, uploadApache Airflow
  3. 03BronzeRaw data exactly as delivered, with lineage on every rowDatabricks · Delta
  4. 04SilverTyped, de-duplicated, complaints matched to banksDatabricks · PySpark
  5. 05Gold and modelOne row per bank per quarter; score, explain and monitorDatabricks · MLflow
  6. 06WarehouseLoaded and modelled into tested marts, published only after the auditSnowflake · dbt
  7. 07PublishTop-20 release check, then the site's data and the reports refreshWebsite · Power BI

Row counts through every layer

Bronze keeps data exactly as delivered, re-deliveries included. Silver removes duplicates. Gold keeps one analysis-ready row per bank per quarter. Counts from the latest successful run.

Checks before anything is published

Data contractsA renamed, missing or mistyped source field stops the run with a clear message, before anything is loaded.
Freshness and volumesThe newest quarter must be recent, and each source must add a sensible number of rows.
Warehouse testsKeys, ranges, accepted values and relationships on every mart; no capital ratio may read 0 where it was not reported.
Write, audit, publishThe marts are built and tested out of sight, and swapped in only when every critical test passes.
Top-20 release checkFails the run if a new bank, a foreign-bank office or a 0% capital ratio reaches the top 20.
Champion ruleA new model goes live only if it matches the current one on the same test rows, with a person approving.

Recent runs

RunResultDurationDatabricks

From the pipeline's run log (the last runs on the production server). A run that failed and was resumed after a fix shows as one run, timed from its first start, with the failed attempt underneath.

Sources: FDIC BankFind Suite (public domain) and the CFPB Consumer Complaint Database (CC0). dbt docs: every model, test and its lineage · Code, tests and decision records on GitHub.