Knowledge & Content.

Industry: Financial services My role: Functional lead Delivered: Content optimiser launched and handed over; knowledge assistant built, QA-tested and live in production

A consultant on a live call needs an answer they can use. Searching several knowledge pages, interpreting unfamiliar shorthand and checking whether a passage must be read exactly as written all compete with the conversation happening in front of them.

For this organisation, we built two connected tools: a content optimiser to help knowledge owners prepare their pages, and a knowledge assistant to retrieve answers for frontline staff.

I was the functional lead, responsible for the user workflows, business rules and functional design, with QA and user-acceptance testing support. The technical lead owned the architecture and integration patterns, and engineers built the solutions.

My work included a decision to put suggested changes directly into page drafts, the approach for handling mandatory scripts, and the client discussions that reshaped the content workflow during testing.

Start with the content owner's job

The optimiser registered a knowledge site, queued pages for analysis and produced suggested changes. Those changes went into a draft, accompanied by a change log. The content owner reviewed the meaning and decided whether to publish.

I made the decision to move from leaving comments to editing the draft directly. The platform did not support the comments approach cleanly, and a draft gave reviewers a concrete version to inspect while retaining control of publication.

The separation of responsibilities was important: the tool assessed how content was written. It did not establish whether the underlying policy or guidance was correct.

That last responsibility became more important as testing progressed.

UAT exposed the cost of reviewing AI edits

I led the user-acceptance demonstration with the content team. Testing surfaced incorrect acronym expansions, duplicated content and a rewrite that changed meaning. There were also technical issues, including a queue stall during the demonstration.

An edit can look tidy and still be wrong. That is a particularly awkward problem when the purpose of the tool is to help someone work through a large content backlog.

Early review of a single page could take much of an hour. The time spent generating a suggestion was only one part of the workflow; someone still had to examine what had changed and whether it was safe to publish.

These findings changed the scope of the optimiser.

Adapt the design as the technology changes

The initial approach used a broader set of readability rules. During the engagement, contextual chunking became a practical alternative for some of the problems those rules were addressing: retaining context around pieces of content used in retrieval.

We narrowed the optimiser to undefined jargon and acronyms rather than continuing to tune every rule. The engineers implemented the technical changes; I helped frame the revised functional approach in the client discussions.

The terminology workflow also moved towards shared configuration. Instead of repeatedly correcting the same term on individual pages, the client could maintain protected terms and approved expansions centrally. I set out the storage options and positioned maintenance as a client-owned decision.

The retrieval sample was too small to establish a reliable improvement in answer quality. The concrete change was to the solution and review workflow; we did not have evidence to turn it into a performance claim.

Build the assistant around different kinds of answers

The assistant combined keyword and semantic search to retrieve relevant knowledge and produce grounded answers with source links. Its functional design distinguished between an answer supported by the content, uncertain results and a question without a useful match.

Mandatory scripts needed a different approach altogether.

The chat front end could not guarantee exact reproduction of wording. I proposed directing staff to the source instead of generating the script in the conversation, and took that approach to the client. The content stakeholder agreed during the UAT discussion.

QA recorded the scripted-content redirect working as intended. The assistant's choice to step back was a tested behaviour, not simply a principle in a design document.

Deliverables and handover

The content optimiser was launched into the client's environment and handed over with its review workflow and configuration guidance.

The knowledge assistant was built, QA-tested and handed over, and is now live in production.

Testing and handover work included user-acceptance materials, guidance for content owners and analyst-led QA. My contribution connected the functional decisions with the people who would review content, maintain terminology and use the answers.

We did not receive the usage data needed to establish adoption, handling-time savings or query-resolution improvements. Those are separate from the delivery milestones above.

What I would bring into the next engagement earlier

This work reinforced the need to test with a representative content set as soon as there is something testable. A handful of pages cannot expose the variety of structures, terminology and exceptions found across a real knowledge base.

I would also build formal model evaluation and client-side value measurement into the engagement from the start, with post-delivery checkpoints to examine how the solution is actually being used.

The technology changed during delivery. So did our understanding of the human work around it. The design had to respond to both.