Skip to content

4.1 Architectural Approaches for LLM-powered Test Infrastructure

Using ChatGPT in a browser is useful for an isolated tester. But for an organization like MagicFridge, an integrated architecture is needed where the AI can access internal specifications and interact with test tools.

4.1.1 Key Architectural Components

An LLM-powered test infrastructure is not just the model itself. It includes:

  1. The Front-end: the user interface for testers.
  2. The Back-end: the orchestrator managing requests.
  3. Data sources: vector databases (for context) or relational databases.
  4. The LLM: the brain (hosted locally or via API).

4.1.2 Retrieval-Augmented Generation (RAG)

LLMs have a flaw: they do not know your corporate secrets and their knowledge stops at their training cutoff date. RAG is the solution to connect the AI to your private documents.

How does it work?

  1. Retrieval: The system searches for relevant documents in your knowledge base (e.g., your User Stories) using vector search (embeddings).
  2. Generation: It sends these documents to the LLM along with your question.
  3. Response: The AI answers based on YOUR data, not just its memory.

Red thread: MagicFridge

The problem: The tester asks GUS: "Generate tests for the new 'Shared Cart' feature." GUS answers with generalities because it does not know the specific rules of this feature which was written yesterday.

The RAG solution: The infrastructure automatically retrieves the "Shared Cart" User Story in Confluence. It injects it into the prompt: "Using the attached documentation [Rules: only the cart creator can validate payment], generate the tests."

Result: GUS generates accurate tests aligned with MagicFridge's current business rules.

4.1.3 The Role of LLM-powered Agents

An agent is an AI capable not only of answering but of acting. It can use tools (calculator, web access, code execution) to accomplish a mission.

We distinguish:

  • Autonomous agents: they plan and execute without human intervention.
  • Semi-autonomous agents: they ask for validation from a human at key steps.

When several specialized agents collaborate (for example, a "Tester" agent and a "Developer" agent), we speak of orchestration. It is the mechanism that coordinates their exchanges to solve a complex problem.

Red thread: MagicFridge

Mission: "Verify that all website links work."

The autonomous agent GUS in action:

  1. It generates a Python script to crawl the site.
  2. It executes the script (it has access to a terminal).
  3. It analyzes the 404 errors found.
  4. It connects to Jira and creates a bug ticket for every dead link.

All this happened without the tester typing a single line of code or clicking a button.

🎓 Syllabus point (key takeaways)

  • Architecture: Requires a Back-end to orchestrate the flow between the user, the data, and the model.
  • RAG: Essential technique to provide the LLM with private, fresh, and company-specific data, thereby reducing hallucinations.
  • Agents: Systems capable of using external tools to automate complex action chains (plan -> code -> execute -> report).


Is this course useful?

This content is 100% free. If this section on MagicFridge helped you:

Buy Me a Coffee
It's 0% fees for me, and 100% fuel for the next chapter! ☕