Bachelor’s Thesis: Contextual LLM UI Assistant

Page Guide was my bachelor’s thesis in Media Technology at KTH Royal Institute of Technology. It explored how an LLM-based assistant could help people understand dense financial interfaces by explaining the part of the interface they were looking at. The thesis was carried out with Mrava.ai as the external industry partner after I independently contacted the company and arranged the thesis collaboration.

Bachelor’s thesis context

The full academic title was Design and Evaluation of an LLM-Based Contextual UI Explanation Assistant for a Financial Operations Platform. I designed, implemented and evaluated Page Guide inside an existing financial operations platform; I did not build or own the wider platform itself.

The problem: understanding dense financial interfaces

Enterprise software can show specialised terms, statuses and workflows without making their practical meaning clear. On an Accounts Payable Supplier Invoices dashboard, a person might see a blocked approval, a failed invoice or reconciliation status without knowing why it matters or what should happen next. Page Guide explored whether contextual assistance could make that moment easier to understand.

Page Guide: contextual UI exploration

The prototype used a small exploration bot that a user could drag over a part of the interface. That selected context was sent with an explanation request. Page Guide then described what the selected region meant, why it mattered and possible next actions. The scope was deliberately narrow: the Accounts Payable Supplier Invoices dashboard, not product-wide support.

From a quick MVP to a grounded architecture

The first working version reused an existing streaming LLM flow and built much of the prompt on the frontend. It made a quick MVP possible, but the explanations could stay too close to surface-level UI text. I then moved prompt construction to the backend and introduced a structured flow:

• UI interaction and selected context became a structured frontend request.
• The backend constructed the prompt and enriched it with verified workflow information where available.
• The LLM generated the explanation, which streamed back to the interface through Server-Sent Events.

In plain terms, the system kept deterministic platform data as the source of truth. The LLM’s role was primarily to turn that context into a useful explanation, rather than to guess how a financial workflow worked.

Keeping system data as the source of truth

Deterministic backend support was implemented for a small set of workflow labels, particularly Blocked Approvals and Failed Invoices. For those cases, Page Guide could use verified counts, workflow state, issue context and possible next-step information before generating language for the user.

This support did not cover every explanation. Other interface regions relied more heavily on contextual prompting, which made the boundary of the prototype clear: grounded explanations were strongest where the backend could supply specific system context.

User evaluation

I ran a formative evaluation with eight participants, most of whom had limited or no experience of ERP or professional finance software. The study combined observation during use, a post-test questionnaire, Likert-scale ratings and open-ended feedback. These are small-sample formative results, not statistically generalisable findings.

• 8 evaluation participants
• 4.25 / 5 overall Page Guide rating
• 4.88 / 5 explanation relevance
• 4.50 / 5 help understanding possible next actions
• 4.50 / 5 usefulness in complex business or finance systems
• 4.38 / 5 trustworthiness
• 3.00 / 5 initial dashboard understanding before using Page Guide

What the evaluation showed

The clearest direction was that data-specific explanations felt more useful than generic ones. Blocked Approvals was particularly helpful when Page Guide could explain what was blocked, why, and what could happen next. That supports a focused design principle from this prototype: LLM assistance is more useful in complex operational software when the language is grounded in verified system context.

Limitations and what I would improve next

The evaluation did not only validate the idea; it also showed exactly where the interaction model needed improvement. Several first-time users did not immediately understand drag-and-drop, some expected a standard click or chat interaction, and the generate step was unclear. Participants also asked for clearer feedback, follow-up questions, shorter explanations and more action-oriented guidance.

The prototype was limited to the Supplier Invoices dashboard, only a small number of labels received deterministic backend enrichment, and all questionnaire respondents were students rather than professional Accounts Payable users. Those constraints define the next research and product steps rather than undermining the prototype’s value.

Technical Overview

• React and TypeScript frontend prototype integrated into an existing platform
• Python-backed service integration
• Contextual UI-state capture from the drag-based interaction
• LLM prompt orchestration with structured frontend-to-backend context
• Deterministic backend enrichment for selected workflow labels
• Server-Sent Events (SSE) for streamed explanations in the UI
• Authenticated existing-platform APIs for verified workflow context