The performance of LLMs at modularising financial models

An examination of how well LLMs modularise a financial model in Excel when instructed to modularise according to business domain, not model structure. This paper was submitted, accepted and presented to EuSpRIG 2026

Publication · 15 Jun 2026 · John Yeldham, Dan Wilcockson

Abstract

This paper investigates the performance of Large Language Models (LLMs) in abstracting a complex Excel financial model into modules aligned with commercial business functions, challenging the prevalent practice of structure-led financial modeling. We tested three commercially available LLM agents against a project finance model under three conditions with progressively fewer structural cues (sheet titles, cell labels).

We found that all agents exhibited significant structural bias, anchoring modularization to the model’s physical layout rather than commercial knowledge, though removing structural anchors improved the quality of business decomposition. While LLMs competently detailed mechanical logic within modules, they provided incomplete and often erroneous analysis of implicit assumptions, highlighting a potential inability to traverse multi-level formula trees and interpret non-explicit cues like cell formatting.

We conclude that for LLMs to effectively support a modular, domain-expert-driven financial modeling workflow, new human-led standards are necessary to mitigate existing structural biases and leverage the LLMs’ analytical strengths.

Research & Contributions

Research Question

How well can various LLM agents decompose a large spreadsheet-based financial model into modules that represent the interest of teams or commercial functions within the business (“business modules”).

Contributions

1. An empirical comparison of three commercially available LLM agents (Copilot, Claude in Excel, Cellori) on the same financial models
2. A controlled three-condition subset of the same financial model that progressively removes structural cues such as titles, labels, and sheets, removing the amount of semantic information the agents have to decompose the model logic
3. Measured evidence of “structural anchoring”, whereby agents are shown to be heavily biased by structural cues and physical layout in the source material, and that this bias can degrade the agent’s performance on business tasks
4. Characterisation of where LLMs performed well and badly across the tasks, discussion of potential causes of bad performance (such as formula-tree traversal) and a commentary on the potential implications for modelling standards.

Introduction

This paper addresses the topic of LLM tools abstracting a model into business modules, and their providing relevant analysis on each module.

We have broken the paper down into the following sections:

1. purpose
2. methodology
3. interpretation
4. summary results
5. further thoughts.

We are testing the skill of the LLMs at abstraction and review, with their output in text form only, and this does not say anything about their ability to create or edit the spreadsheets themselves.

Purpose

Despite financial modellers being technically minded people, our experience is that they do not have the same fundamental data science understanding that a typical IT developer would have. In particular, and relevant to this article, they do not properly appreciate modularisation and functional thinking.

Methodologies such as FAST give modellers a framework that in effects decomposes models into the smallest modules possible, with each module being single calculation - in nature a single function, with a set of inputs gathered together, and a single set of outputs, usually a cell, a row of cells or a block of cells.

When models are built this way the “writing” of a model flows like the writing of a book, with each small semantic module acting like a sentence in a book. This workflow is reflected in the language and principles often quoted in financial modelling advice, although often framed as a benefit and the right way to model, for readability. For instance, the ICAEW Financial Modelling Code [ICAEW Financial Modelling Code, 2024] says:

Set up your model to read like a book: with logic flowing from top to bottom within each worksheet, and from left to right both within each worksheet and across sheets within a workbook.

There has not been a great deal of thought about whether a different size of module, in a different order, may be a better fit for financial modelling workflows.

Our paper looks at larger modules, at the size of a business department or team.

Modularisation of financial models at this level has a number of key effects:

1. It diffuses ownership of the model away from a centralised bottleneck financial modeller to separate teams, each of which has greater domain expertise than a financial modeller, albeit with less financial modelling expertise.
2. It separates the model into semantic units that map to business and commercial knowledge, not model logic frameworks. This helps with communication with non-modeller business people, as it moves the language away from the technical modeller words and into a business language that non-modellers are more familiar with.
3. It also encourages LLMs who are used to analyse or build models to utilise their business and commercial knowledge to design modules, instead of falling into the pattern of simply replicating the model designs that humans have been using up to now. As we will see, it is quite difficult to drag LLMs from dwelling on modelling knowledge instead of business knowledge.
4. It allows for easier templating, with a library of templates for each module. We can move from recreating the detailed logic each time we build a model and instead construct the model we need by selecting fairly large modules from libraries. It is analogous to modularisation in computer programming, which has similar benefits. At a certain point, a creator of an individual model will just need to understand the business domain and the library options to choose the best fit library module, and then the “architect” (who could be automation, AI, or human) only has to piece the modules together.
5. As well as the building efficiency benefits, the libraries also form permanent stores of knowledge, with business and modelling knowledge encapsulated in their template modules. This democratizes financial modelling knowledge and reduces reliance on very specialised financial modelling individuals.
6. The libraries also allow for much better documentation, including the parameters of modules as well as implicit logic and assumptions. This documentation can be reused, again and again, and fills a gap in many modern financial models, which fail to properly document their own implicit assumptions and logic. A “correct” model that does not communicate its assumptions often still causes error [Pryor, 2003]. The quintessential example of implicit assumptions is “Cash flows are assumed to occur at period end”. Without something like this written inside a model, the model will not properly communicate what its inputs and outputs mean to the end user.

This paper seeks to understand how well LLMs can decompose models into the sort of module we discuss above, with the focus on business departments, and not model sheets or sections.

A note on business modules

We define a business module as a particular knowledge domain aligned with teams or departments in a business. It is not the same - though often coincident with - a functional module in a financial model.

For instance, revenue is often a model module, and maps nicely to the business domain of sales. Fixed assets are more complicated. Capital expenditure is managed by project development teams, whereas the accounting treatment of that capital expenditure is dealt with by accounting and reporting teams. In a model, these can sometimes be separate but sometimes merged into one functional module.

A clear differentiator between business and models is timing. Model timing is almost always a distinct module in a model, but of course there is no team in a business that deals with time passing - it is implicit in the planning of any business department. Along the same lines, inputs (or “assumptions”) can often be defined as a standalone functional module in a model, but the assumptions of a business are embedded within each business team’s knowledge - there is no centralised team that manages the assumptions of the entire business, except perhaps if they have a financial modelling team!

We do not count a financial modelling team as a business module for this purpose, as the whole model would simply be the module in that case, and we don’t get the mapping to specific business - rather than forecasting - knowledge that we are looking for.

Methodology development

Design choices

For our research, we needed careful design and choosing of:

1. The list of LLMs to assess
2. The financial models to analyse
3. The LLM prompt to use
4. The environment and workflow for entering the prompt
5. Appraisal criteria.

First iteration methodology

We started by looking at two models, a Lodeum template model and a Cellori template model, which were both project finance models for renewable energy. These models are large (compared to most financial models in general), and contain deep logic and forecast multiple aspects of business (including revenues, costs, debt, tax) and outputting full 3 statements (Profit and Loss, Balance Sheet, and Cash Flow Statement) over a long timeline.

We begun with 4 LLM tools for assessment: Copilot, Claude, Cellori, and Shortcut. We asked them to modularise the financial models into business modules, explaining for each module the complexity, relevant business department, implicit assumptions, relevant data sources and connections to other modules.

First iteration assessment

From initial results, two things were readily apparent:

1. The agents produced a very large volume of text - of the order of 30 pages per test.
2. The three agents were clearly doing a poor job of modularising the model according to business domain, and had clearly stuck closely to the model structure and language rather than reflecting the business structure. For instance, modules were almost perfectly mapped to sheet names in the model.

Methodology improvements

To get a set of results that would highlight the key contrasts and behaviours without overloading us with materials to read we made the following improvements:

1. The prompt was adjusted to even more clearly stress commercial modularisation and not model structure modularisation. At the same time, we didn’t want to prompt engineer a better answer, but chose wording that made this clear in plain English without extensive examples or scenario-based guidance for the LLMs. We included a glossary in the prompt to give clear meaning to the analysis we wanted carrying out.
2. The prompt was adjusted to try to reduce the number of words, stressing the importance of being concise.
3. We expanded the set of test models to cover a range from well structured and well labelled, to monolithic and poorly labelled, to see whether the tools behaved any better or worse with a lot of model structure to latch onto.
4. We reduced the number of LLM tools to three: Copilot, Claude and Cellori, to reduce the volume of text to review and also reflecting that the differences between the LLMs were not actually the significant discovery of this research.

Final methodology

LLM test subjects

Three commercially available AI agents were tested against the same problems. The selection spans generalist mass-market to highly specialised tools to generate a range of outcomes and focus on systemic behaviours driven by the underlying language models.

1. Microsoft Copilot in Excel with GPT 5.4. A generalist AI assistant integrated across the Microsoft Office suite, designed to support a broad range of tasks across many industries.
2. Claude in Excel with Claude Opus 4.7. An integration of Anthropic’s general-purpose Claude Opus 4.7 model with Excel. A generalist agent, although with some focus and specialist tools surrounding financial modelling.
3. Cellori with Claude Opus 4.7. A specialist financial modelling framework designed specifically for the analysis, audit and construction of project finance and infrastructure models.

The same agent versions were used across all nine test runs. All tests were conducted in March 2026.

Financial models

Three versions of a project finance model were used.

The core test model was a GBP-denominated project finance model of a 20MW solar power plant. It contains construction and operational periods, and spans 30 years. Revenue streams are generated through the sale of electricity, both through a fixed power purchase agreement and at merchant market pricing. It contains three funding sources (debt, shareholder loans and share capital), sculpted debt repayments, and UK-based tax calculations. The model contains three statements (income statement, balance sheet and cash flows) including a cash waterfall.

The model contains 15 sheets with 80,175 formula cells and c. 400 unique calculations.

Tests were conducted across three conditions, with a decreasing amount of structural information in each subsequent condition.

Model 1 is split into several sheets typical of a project finance model (time, construction, economics, operations, debt, equity, fixed assets, tax). Model 2 and 3 have all calculations consolidated onto a single sheet (“Calcs”), with section headers removed. In Model 2, all cell labels were retained. In Model 3, all row labels were also removed from the inputs and calculations, but left on the financial statements.

Prompt

IMPORTANT: do not edit the file; this is a research exercise only.

Break the model down into modules, with each module corresponding to business and commercial functions as would typically be understood by project finance specialists (e.g. Revenue, Operating Costs, Fixed Assets, Working Capital, Debt, Tax). Modularise according to business purpose, not spreadsheet or Excel implementation structure (although spreadsheet structure may give clues to purpose).

Terminology (use these meanings consistently):

• Model input cells: user-entered assumption cells anywhere in the model.
• Module parameters: assumption values used internally within a module (these may be model input cells).
• Module inputs (inbound flows): values that enter a module from another module’s outputs.
• Module outputs: values produced by a module that are used elsewhere.
• Avoid the unqualified term “inputs”.

Example: Capex assumptions used inside a Fixed Assets module are module parameters, not module inputs, unless they originate as outputs from another module.

Then, for each module, provide a single consolidated report covering the following:

1. Summary A one-sentence description of the module’s business purpose.

2. Module inputs and outputs Describe inter-module data flows only:

• What values enter this module from other modules (module inputs).
• What values this module produces for use elsewhere (module outputs). Do not list internal parameters or assumption cells unless they originate from another module.

3. Level of detail Describe the depth, granularity, and structural complexity of the business logic in this module (e.g. aggregation level, timing detail, conditionality), not the Excel techniques used.

4. Connections to other modules Identify which other modules this module exchanges information with. For each connection, describe:

• Absolute volume of the link: the relative amount of distinct inter-module flows (low / medium / high).
• Relative density of the link: how concentrated those flows are within the module, relative to its overall internal logic (low / medium / high). Be explicit about what you mean in each case.

5. Responsible team What type of business or commercial team would typically own or maintain this module in practice?

6. Real-world data sources Identify good real-world data sources relevant to this module. Explain how such data would be used (e.g. parameter setting, validation, negotiation support) and how it could provide reasonable assurance to stakeholders.

7. Implicit assumptions List the key implicit assumptions embedded in the logic of this module (e.g. structural conventions, timing rules, thresholds, allocation logic). Do not restate explicit assumptions that appear directly as input values. Highlight where unstated assumptions could reasonably cause user misunderstanding or material risk.

After the per-module reports, provide a summary describing the overall flow of logic and information between modules, from initial model input cells through to final model outputs, treating modules as nodes and inter-module flows as links.

Prompt environment

1. For each tool, the chat was in an environment without any precedent history. For instance, it was not launched from within a “project”, “agent” or in CoPilot’s “Work” environment, which draws on previous chats and data to work.
2. The prompt was the first and only turn in the chat.
3. The file was uploaded as an attachment into the chat.

Appraisal criteria and scoring process

Each of the authors appraised each LLM for each model, assigning a score between 0 and 5 for each criteria, and providing a brief explanation of the score given. We then took stock of the results as a whole and summarised in qualitative terms. The work draws a narrative from the analysis rather than detailed quantitative assessment.

The criteria were:

1. Ability to modularise according to business function
2. Level of detail
3. The ability to assess connections between modules, including quantifying the volume and density of connections
4. The number and quality of implicit assumptions listed for each module.

Note on stochasticity

LLMs are stochastic by nature. As the measurements in this study required manual study and characterisation of a large volume of LLM-generated output, each agent to model sample has been measured only once. It is acknowledged that this methodology will overlook variances caused by the stochastic nature of LLM outputs, even when all input variables are controlled. The conclusions of this study are accordingly directional and observational.

Interpretation

Modularisation

The agents were tasked with mapping the model into commercial units most closely tied to real business functions, rather than into the financial modelling modules already present in the model layout. Sections such as inflation, time, and working capital do not have business units behind them, they are model scaffolding set up to improve the organisational flow of the business logic. High-quality answers would ground each module in real-world commercial ownership.

Every output in the study fell short of this bar. We found that heavy bias driven by the structural anchors inside the model (i.e. sheet names, headers, row labels, calculation ordering) impacted the agents’ thinking so heavily that they stopped drawing from their wider world knowledge. The structuring of the financial model within each agent’s context window was found to consistently reduce the quality of outputs on this task, a behaviour conceptually consistent with framing disparity cited in LLM literature [Lim et al., 2026].

We highlight two examples to illustrate this dynamic:

Example 1 — Copilot anchoring to sheet names

Copilot, working on model 1, opens its module list with an explicit disclaimer:

Proposed business-purpose modules (not Excel-structure-led)

The ten modules that follow then map exactly one-for-one onto the workbook’s sheets:

Although the agent has seemingly understood the task, and (as implied from later tests), appears to have the correct information within its training data to solve this problem, the structural bias driven by the sheet names collapsed its judgement and it proceeded to violate its own strategy. This pattern is consistent with documented LLM failures in which models tend to “err predictably based on how the input prompt is framed [and adjust] outputs towards anchors” [Jones and Steinhardt, 2022].

Example 2 — Claude anchoring to row ranges

In Model 3, without sheet names or section titles to lean on, Claude shifts the anchor to row positions, explicitly in the opening lines of its analysis:

I have enough to produce the analysis. The Calcs sheet has roughly these zones (read from row labels and units):

• Rows 1-145: Time/timeline scaffolding
• Rows 146-170: Construction cost build
• Rows 171-220: Generation/production
• Rows 221-260: FX & inflation indexation factors
• Rows 261-295: Revenue (merchant + PPA), VAT
• Rows 296-590: Operating costs and revenue VAT/working-capital postings
• Rows 591-665: Funding cascade (drawdowns, fees, IDC)
• Rows 666-725: Construction cost allocation to FA categories & tax categories
• …

Now I’ll write the analysis.

The twelve modules that follow are recognisably those row ranges with business-sounding names attached. The primary structural anchor shifted from sheet names to row positions, but the process was similar: line up the model’s physical layout first, label each contiguous block as a module, then justify each module as a business unit.

Scoring

Each of the nine outputs were scored on a 0 to 5 scale with consideration to how well the named modules correspond to real business units rather than directly against model structure. The “baseline” (0 points) was set on Copilot’s results against Model 1, the most extreme example of structural bias in the corpus.

The highest score given (2.5) reflects that none of the results effectively decomposed the models into business units. Even the strongest outputs showed bias towards the existing structuring in the model, most notably in timing, fixed assets and financial statements. A score of 5 would have required these elements to be redistributed into the business units with careful consideration to typical commercial structure of the teams operating the project.

The items between the lowest and highest scores were measured by what additional decompositions were found in each case. For example, splitting the operations into separate generation, revenue and operating expenses modules earned credit, whilst labelling “Time” as a business module lost credit, or combining many business areas (e.g. senior debt and equity distributions) into a single module lost credit.

Knowledge versus bias

Removing sheet structure, titles and row labels makes the model harder to read on paper, but it also reduces the density of structural anchors that bias the agent’s output, forcing it to draw more heavily on its world knowledge. As the cues are removed, the quality of business decomposition rises.

The mean score almost triples between Model 1 and Model 3, with the largest jump coming from removing sheets and section titles. Removing the row labels again also drove a small improvement in the quality of the outputs.

These results indicate that all structural anchors introduce bias, but sheet and header structure has a larger impact than row-level labelling. We speculate that this aligns with the bulk of training material fed into the underlying LLMs, in the typical hierarchical organisation of written prose (articles, sections, paragraphs, sentences etc.) with semantic weighting concentrated at the higher levels. There exists ample evidence that training pipelines explicitly expose document structure to the model [Ponkshe et al., 2024] [Chang et al., 2019]. In this hypothesis a sheet title is the spreadsheet analogue of a section heading, where a row label is closer to a sentence-level marker.

The more fundamental observed behaviour, where specific prompts instructing the agents not to follow contextual structure does not eliminate anchoring bias, is consistent with broader findings that anchoring bias resists “ignore the anchor” instructions, and “initial information disproportionately influences judgement” [Lou and Sun, 2024].

Detail

The agents described the depth, granularity, and structural complexity of the business logic inside of each of their chosen modules. They gave details on the calculation concepts and business mechanics that determine how each module works.

Scoring

Each agent output was scored on the count, detail and correctness of the business concepts and complexity identified. We considered which concepts were surfaced, which were missed, and how well the agent evaluated the overall complexity of the calculation logic.

We first compiled a list of all “concepts” across the model, a concept being a distinct mechanic, parameter rule, assumption, or sequencing rule. For example, “PPA volume as fixed percentage of net generation”, “repayment sculpted to flat ADSCR target”, and “tax paid same quarter as accrued” are all individual business concepts. Concepts that relate directly to Excel mechanics (i.e. formula selection), were not allowed. To measure the scores, we compared the agents’ analysis to our list, and then overlayed our own judgement as to the appropriateness of the agent’s findings in the context of its own chosen modules.

Overall the agents performed well at this task. We found that once problems were scoped to an individual module, the conceptual analysis within such module was generally well performed and often close to exhaustive. We observed a drop off in quality as the structural information within the models was removed due to the increased complexity of the problem. In Model 3 in particular, the agents spent considerably more tokens unpicking the structure and logical flow of the models without calculation labels present, and the quality of the granular observations dropped noticeably.

We point to two examples of losses to business logic understanding when the row labels were removed.

Two examples of detail lost when labels are removed

The first example is VAT. With cell labels intact, the outputs collectively identified five distinct VAT mechanics. With row labels stripped, the same VAT logic survives only as a generic timing lag.

In the VAT calculations for Model 2, all three agents were able to fully break down and describe the depth of five distinct mechanics related to the commercial logic. With the row labels removed in Model 3, the coverage of these calculations became sparse across the agents, and the quality of the analysis that was present dropped.

We observed the same pattern in the tax calculation. With row labels, the agents were all able to surface complex tax mechanics. Without labels, the coverage dropped.

Labels and the limits of formula reading

We conclude that row labels act as semantic cues within the model that can notably increase an agent’s understanding of the raw formula logic driving an individual mechanic. These characteristics are consistent with broader LLM prompting literature, which argues that semantic cues can effectively direct a model to the relevant region of its learned capability [Reynolds and McDonell, 2021].

Sheet and section titles, and individual row labels are changing agent behaviour differently. Removing titles had a much lower cost on the agent’s mechanical formula-level understanding, and the agents were mostly able to accurately infer the section’s purpose based on its contents. Removing the row labels forced the agents to trace through formula logic to a deeper level before accessing semantic cues, which decreased the volume of detail surfaced.

Connections

When we look at how well the tools assessed the volume and density of connections between modules, we find that it is broadly correct, with caveats. To understand the output, it is best to look at an example. Below is a grid showing the connection volume between modules for the full model (model 1), assessed by Claude. Outputs from all 9 tool and model combinations are in Appendix A.

Connection Grid — Volume | Model: 1. Lodeum (full labels) | Tool: Claude

Scale: 0 = not mentioned, 1 = Low, 2 = Medium, 3 = High [blank diagonal = self-connection, intentionally empty]

The grid is a graphic representation of what the tool outputted as text, but it helps highlight the common patterns. The rows show the source of the connection (where the link comes from) and the columns are the destination (where the link goes to).

Firstly, because the modules chosen are closely tied to sheet and section structure, there is a natural triangle pattern in the top right. The models - like most professional financial models - build up from core workings (such as time) and then step through in sections or sheets that are the next step of the logic flow. So time flows into everything, revenue flows into profits, which flows into tax, all of which flows into debt (via cash shortfalls and surpluses), and everything flows into checks. So we expect logic to flow from modules of lower number to higher number, and this produces the triangle of zeros in the bottom left corner.

Secondly, we can see some other obvious row and column patterns, such as:

1. The diagonal is empty, as expected - modules can not be inputs or outputs to or from themselves.
2. Timing flows into almost everything, as expected. Time isn’t a business module, but to the extent that the tools have chosen it as a module it certainly does flow into everything else.
3. Almost everything flows into the financial statements. Again, financial statements really should not be a business module, but to the extent the tools have chosen them as a module, then almost all results really do flow into them.
4. Integrity is not well linked. This is an error. While model integrity is not a business module, if it is deemed to be a module it should be heavily linked to all sections. What Claude has missed here is that there are lots of checks in the model and they flow via an aggregation mechanism to the checks sheet. We suggest that this has been missed due to the tool’s inability to see the Excel tree and dig down into logic multiple layers deep (unless signposted with lots of language).

There is no meaningful difference in performance between the analyses of the different models that we could discern from manual review of the outputs. While the choice of modules is obviously important, once that choice is made, the tools were all reasonably proficient at assessing the strength of connections between those chosen modules.

It is worth noting that by definition a module is only connected to another module through direct links. There is no benefit to this analysis from being able to trace links deeper, or to see the Excel tree of cell links. This explains why the tools - who only dig down 1 or 2 layers at most in tracing links - can perform well in this task but fail in other link-based activities.

Implicit assumptions

The request

We asked the tools to list the implicit assumptions in each module and highlight risks.

We asked for all implicit assumptions, not just those that were not otherwise stated. This created an even playing field across the 3 models, since otherwise models 1 and 2 already contain numerous stated implicit assumptions (usually written simply as explanatory text in a cell) whereas those statements are absent from model 3.

Fundamentally, all 3 models had the same implicit assumptions. They also had the same explicit assumptions, as values (numerical or text) written into input cells. None of the changes we made between the models changed the implicit or explicit assumptions, and as a result all three models produced identical outputs and results. Differences in implicit assumptions listed therefore indicated differences in tool performance, when analysing to different models containing different cues and clues, but which should in theory yield identical results.

Performance

The performance can be stated in general terms as:

1. All three agents listed many useful implicit assumptions that would aid a financial modeller and model user in understanding the model and assessing risks.
2. All three agents in this study were wrong in some of their statements, and would have the potential to mislead a modeller or user of the model.
3. All lists were incomplete - none comprehensively identified all implicit assumptions.

This can be characterised as the tools giving some useful advice, that would help identify some risks, and help moving towards full model documentation, but all of which advice would need to be checked, line by line, before being formalised in a document, and certainly would not give a complete picture. In essence, they provide inspiration for further investigation, but can not be used to vet completeness or accuracy of assumptions - at least in this form with this kind of prompt.

Example errors

There were repeated errors in the implicit assumption statements, which are potentially revealing about the shortcomings of these LLMs when examining spreadsheets. Some examples include:

1. All the LLMs assumed distributions were based on profits in the previous quarter. In fact, the profits in the previous quarter were just the start of that calculation, with the current period profits being added on top. The LLMs saw the prior column reference and jumped to a conclusion without properly assessing the subsequent parts of the calculation. We speculate that this is because they do not have a deep understanding and representation of the Excel formula tree.
2. Text inputs - like the name of the inflation profile used for a particular cost item - were mistaken by these LLMs as hardwired assumptions. They complained that particular lines had fixed inflation treatment when in fact the inflation profile was freely choosable from a drop down list (using data validation in Excel), formatted as an “input”. From this we can gather that these LLMs can’t - or are unwilling to - read cell formatting and map it to the purpose of the cell, or see or examine the data validation options, which would give a clue as to cell purpose.
3. They struggled to understand whether distributions were based on profits, cash flows or both. In fact the model was based on both, but these LLMs often jumped on one or the other - each of which would be in a contiguous block - rather than properly following the calculation logic through in multiple blocks. We know from talking to these LLMs that they cannot delve into the tree of links hidden within an Excel file, so they are left imputing links through looking directly at formulas, and clearly they did not do more than 1 or maybe 2 levels deep in that analysis. As a result, any calculations with multiple links to multiple blocks are easily misunderstood or overlooked.
4. They all correctly identified that opex followed the 30/360 days convention, but all then complained that that didn’t align with the interest calculation days convention. The two calculations are completely separate and uncorrelated in reality, and trying to compare the two is spurious. Perhaps because day conventions are more commonly used in interest calculations, these LLMs may have a strong semantic connection between any day convention (for whatever purpose) and finance calculations.
5. Very often they concluded that drawdown timing was at period start. In fact, the model is strictly consistent that all cash flows are at period end. That drawdowns occur at period end could be derived from the logic of the interest calculations, but that would require a rather deep analysis of calculations, and cannot be derived from a surface level examination such as row labels.
6. They also struggled to understand the full range of internal model checks that were implemented, seemingly able to see only checks that were explicitly labelled as such. The model contains a lot of checks but to properly identify them one needs to drill down several layers into formula trees. As mentioned, this is something that the LLMs are not good at doing. In effect, the extent of the checks system in the model was seriously underestimated.

Comparison between LLMs

Cellori found by far the most true implicit assumptions, followed by Claude, and then Copilot a long way behind. Cellori breaks down a model into semantic chunks that makes it easier for the LLM to analyse, so this result is not surprising.

Cellori and Claude both performed worse when labels were removed, showing that at least some of their insight came from the language used in the model, rather than model logic.

Conversely Copilot actually performed better without labels. It may be interesting as a future project why this may be the case.

Grid 1 — count of useful implicit assumptions

The relative accuracy of the list of assumptions was measured by a % of useful (and true) implicit assumptions listed from all implicit assumptions listed (including false and spurious items). Cellori and Claude both had much higher accuracy than Copilot. Note that the accuracy is above 50% but well below 100%. The LLMs have a way to go before we can habitually use their implicit assumptions without significant review.

Grid 2 — % of comments that are useful

Summary results

Modularisation

The agents failed to properly decompose the models into business modules. We observed significant bias driven by structural anchors (sheet names, titles, row labels) that prevented these LLMs from drawing on their wider commercial knowledge. When removing such anchors, we observed an increase in the commercial knowledge present in the agent’s responses, with sheet and section titles driving considerably more bias than row labels.

Detail

When scoped to an individual module, the agents performed well at unpicking and surfacing underlying mechanical detail and complexity. We observed row labels acting as semantic cues that improved the agents’ interpretation of formula logic, while the presence of sheet and section titles had relatively low impact.

Connections

The agents were broadly correct in their assessment of inter-module connections, and produced the expected triangular flow pattern from the upstream assumptions and workings through to the financial statements. There was no meaningful performance difference between the conditions, and we noted that the agents were not relying on structural anchors for this task (albeit being based on a choice of modules that did depend on structural anchors).

Implicit assumptions

The results in this category were mixed, with all agents surfacing useful insights but none generating an exhaustive or fully accurate picture. Recurring errors such as missing validation inputs and making incorrect assumptions on periodicity revealed shared limitations on the agent’s facility to traverse the formula tree, or consistently derive implied assumptions from formatting and sheet structure.

Further thoughts

Today’s most prominent financial modelling approaches are optimised for mechanical consistency and auditability over semantic descriptors [The FAST Standard Organisation Limited, 2019]. When AI-powered agents are tasked with interpreting these models in a real-world context, bias driven by structural anchoring reduces the world knowledge present in analysis. For a more LLM-optimised modelling approach, we believe there likely exists a tension between the inclusion of structural anchoring, driving LLMs to the relevant regions of their training corpus, and the removal of structural anchoring, reducing the framing bias this injects into an LLM’s context window, and leaving room for better holistic commercial analysis.

Recent technological advancements in spreadsheets, such as data analysis (e.g. Power Query in Excel), dynamic arrays, and LAMBDA functions, which open the door to a more modular, library based approach to building models. That approach could democratise the financial modelling workflow, putting it in the hands of domain experts instead of financial modelling specialists. Current financial modelling methodologies are still catching up to this, and are still currently focused on financial modelling carried out by individual financial modellers using long-standing Excel functionality.

If the financial modelling industry wishes to embrace modularisation, they will need an awareness of how to structure models and libraries semantically to utilise LLMs in that workflow, accounting for the current failures LLMs exhibit in thinking about modularisation. This will take strong, human leadership in generating new standards that LLMs can subsequently learn and apply. Right now, LLMs are too wedded to the existing body of knowledge of financial modelling, and lack a depth to their logical thinking skills, to play a strong role in creating those new structures. However, with the right guidance from humans, they would certainly have a huge role in applying those standards and modelling libraries to build accurate, well-documented models quickly and efficiently.

Bibliography

Works Cited

Chang, Ming-Wei, et al. Language Model Pre-training for Hierarchical Document Representations. 26 January 2019. arxiv, https://arxiv.org/abs/1901.09128. Accessed 17 May 2026.

The FAST Standard Organisation Limited. “The FAST Standard.” The FAST Standard Organisation, July 2019, https://fast-standard.org/the-fast-standard/. Accessed 14 May 2026.

“ICAEW Financial Modelling Code.” ICAEW, 2024, https://www.icaew.com/-/media/corporate/files/technical/technology/excel/financial-modelling-code.ashx. Accessed 13 May 2026.

Jones, Erik, and Jacob Steinhardt. “Capturing Failures of Large Language Models via Human Cognitive Biases.” NeurIPS, vol. 1, no. 2, 2022, p. 22. arxiv, https://arxiv.org/abs/2202.12299. Accessed 17 May 2026.

Lim, Kahee, et al. DeFrame: Debiasing Large Language Models Against Framing Effects. 1, 4 February 2026. arxiv, https://arxiv.org/abs/2602.04306. Accessed 17 May 2026.

Lou, Jiaxu, and Yifan Sun. Anchoring Bias in Large Language Models: An Experimental Study. no. 2, 18 December 2024. arxiv, https://arxiv.org/abs/2412.06593. Accessed 17 May 2026.

Ponkshe, Kaustubh, et al. StructFormer: Document Structure-based Masked Attention and its Impact on Language Model Pre-Training. 25 November 2024. arxiv, https://arxiv.org/abs/2411.16618. Accessed 17 May 2026.

Pryor, Louise. “Correctness is not enough.” Proc. European Spreadsheet Risks Int. Grp. (EuSpRIG), vol. 1, no. 1, 2003, pp. 117-122. ArXiv, https://arxiv.org/abs/0808.2045. Accessed 17 May 2026.

Reynolds, Laria, and Kyle McDonell. Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm. 15 February 2021, https://arxiv.org/abs/2102.07350. Accessed 14 May 2026.

Appendix A: connections tables

Connection Grid — Volume | Model: 1. Lodeum (full labels) | Tool: Cellori

Scale: 0 = not mentioned, 1 = Low, 2 = Medium, 3 = High [blank diagonal = self-connection, intentionally empty]

Connection Grid — Volume | Model: 1. Lodeum (full labels) | Tool: Claude

Scale: 0 = not mentioned, 1 = Low, 2 = Medium, 3 = High [blank diagonal = self-connection, intentionally empty]

Connection Grid — Volume | Model: 1. Lodeum (full labels) | Tool: CoPilot

Scale: 0 = not mentioned, 1 = Low, 2 = Medium, 3 = High [blank diagonal = self-connection, intentionally empty]

Connection Grid — Volume | Model: 2. Lodeum one sheet no titles | Tool: Cellori

Scale: 0 = not mentioned, 1 = Low, 2 = Medium, 3 = High [blank diagonal = self-connection, intentionally empty]

Connection Grid — Volume | Model: 2. Lodeum one sheet no titles | Tool: Claude

Scale: 0 = not mentioned, 1 = Low, 2 = Medium, 3 = High [blank diagonal = self-connection, intentionally empty]

Connection Grid — Volume | Model: 2. Lodeum one sheet no titles | Tool: CoPilot

Scale: 0 = not mentioned, 1 = Low, 2 = Medium, 3 = High [blank diagonal = self-connection, intentionally empty]

Connection Grid — Volume | Model: 3. Lodeum no labels | Tool: Cellori

Scale: 0 = not mentioned, 1 = Low, 2 = Medium, 3 = High [blank diagonal = self-connection, intentionally empty]

Connection Grid — Volume | Model: 3. Lodeum no labels | Tool: Claude

Scale: 0 = not mentioned, 1 = Low, 2 = Medium, 3 = High [blank diagonal = self-connection, intentionally empty]

Connection Grid — Volume | Model: 3. Lodeum no labels | Tool: Copilot

Scale: 0 = not mentioned, 1 = Low, 2 = Medium, 3 = High [blank diagonal = self-connection, intentionally empty]

Appendix B: pics of example models

Image 1: example of part of the model 1 (the full model)

Image 2: example of part of the model 3 (no labels)

← Back to Writing