All projects

Mortgage document review

The screen where a mortgage lender's team checks what Multimodal's AI read from each loan document, fixes it, and sends it on. I built it on my own in my first months at Multimodal.

Client
A US mortgage lender, through Multimodal
Role
Frontend engineer, on my own
When
May – Aug 2024
A loan document beside the fields the AI read from it
  • React
  • TypeScript
  • Vite
  • TanStack Query
  • TanStack Table
  • react-pdf
  • shadcn/ui
  • Tailwind

What it is

A US mortgage lender sends the documents in each loan file, pay stubs, bank statements, employment verifications, through Multimodal's AI, which works out what each document is and reads the fields the loan needs.

A person still checks that work before it reaches the loan system. Every correction they make is also worth something to the model that made the mistake.

My part

I built the review app on my own, from the first commit in May 2024 to the hand-over in August, when it moved into the lender's own repository and I helped fit it into their setup.

A queue to work through

The problem. Reviewers face a steady stream of documents across many loans, and need the next one that is waiting, fast.

What I did.
  • Pending and Completed queues with a search that forgives typos, date filters, sorting and CSV export.
  • Moved paging to the server in July, so the queue loads one page at a time.
  • Next and Previous on the review page follow the table's own order and filters, so a reviewer never has to go back to the list.

What changed. A reviewer works straight down the queue, one document after another.

The review queue: documents waiting, with their loan, type and dates
The pending queue. Made-up data.

The page beside its data

The problem. Checking means comparing every value the AI pulled out with the spot on the page it came from.

What I did.
  • The document in a viewer with zoom and rotate, next to the fields read from it.
  • Focusing a field outlines where the AI found it on the page, from the coordinates it returns.
  • The document type with the AI's confidence, which the reviewer can change from a list.
  • Forms for the harder documents: employment verifications with several employers, bank statements with several accounts.

What changed. Each value is checked against its source at a glance, not hunted for.

A loan document beside its extracted fields, one field highlighted on the page
A field and the spot on the page it came from. Made-up data.

Corrections that count twice

The problem. A fixed value has to land in the loan system, and the model should learn from it too.

What I did.
  • Save writes the checked fields to the client's loan system and records the review.
  • Every correction also goes back to Multimodal as feedback for the model.
  • A history of what the AI returned and of every edit since, plus similar documents from the same loan.
  • Resend a document to OCR, take it off the queue, or add a borrower, bank or employer to a pick list when the loan system doesn't have it yet.

What changed. With the tool in use, annotation was 50% faster, and the client's loan processing costs fell by 80%.

50%

faster annotation

Reviewed documents with the AI's classifier and extraction accuracy and the time each review took
Completed reviews, scored for the model. Made-up data.

Next project

SIDAKUThesis · Android app

Have a project in mind?

Tell me what you need built, web or mobile, design to backend. I will come back with a plan.

Hiring a software engineer?

I am open to full-time roles. Let’s talk about your team and the work.

Let’s Talk