Long-form perspectives on legal-tech, benchmarks, and building verifiable AI systems.
Blog
LLM Response Validation with LLMs (LLM-as-a-Judge)
LLM response validation is the use of an LLM to evaluate AI-generated texts based on user-specified criteria stated in the evaluation prompt.
Dr Victor Cheng · January 22, 2026 · 5 min read
Introduction
In short, LLM response validation is the use of an LLM to evaluate AI-generated texts based on user-specified criteria stated in the evaluation prompt. Since AI-generated texts may not always align with facts or the user-provided context, issues of trustworthiness, traceability, quality, and even safety can arise. There is a need to detect false or hallucinated responses before delivering them to users.
Validation is not limited to factual alignment; it can also assess other aspects of responses, such as:
Bias: prejudice against certain group
Tone: formal or casual
Format: JSON or custom format
Sentiment: positive, negative, or neutral
Faithfulness to source: only include information appearing in the source
Correctness: May need tools to access external information source
Helpfulness: if the response fully answers the query?
To validate, the AI-generated text together with the evaluation prompt (with criteria stated) is sent to the LLM. The LLM then provides comments or a score to indicate the evaluation results.
Prompt Example
“Given the user query/question and context information: {{QUERY +CONTEXT}} and the response: {{RESPONSE}}, examine whether the response information is solely based on the context. Ensure that all facts described in the response can be found in the context. Return your result as a label: ‘Yes’ or ‘No,’ followed by a concise reason to support your decision.”
You may wonder why an LLM is used to assess the responses of another LLM — or even itself. The point is that generating a response to a user query is more difficult than evaluating the quality of that response, since the latter focuses only on quality factors. Content critique is easier than content generation. Moreover, we can even use the same LLM for evaluation because this leverages different capabilities of the model.
Blog
Query Expansion for Retrieving Relevant Documents from Vector Database
When performing document retrieval, the quality of the results usually depends on the query string.
Dr Victor Cheng · January 22, 2026 · 5 min read
Introduction
When performing document retrieval, the quality of the results usually depends on the query string. If the query is informative, containing relevant keywords and sufficient context, precision and recall will be high. On the other hand, if the query is ambiguous, contains only one or two keywords, and lacks context, the retrieval results may be poor.
With the help of large language models (LLMs), a short query can be expanded by appending context or even examples that help retrieve more relevant documents. Query expansion can:
Increase the recall
Reduce query ambiguity
Give better result ranking
2. Approach
With the help of LLMs, we can generate query-relevant context or examples using prompts such as:
“The user has the query: {{query}} and wants to retrieve relevant documents from a vector database. Please generate a short, relevant context to augment the query so that more precise documents can be retrieved.”
OR
“Given the query: {{query}}, please generate a short text that may be found in a document containing the answer to the query.”
3. Remarks
Although query expansion can enrich the query content, it may also over-expand the query and lead to the retrieval of irrelevant documents (i.e., reduced precision). Therefore, the approach should be used carefully, especially when the query already contains sufficient context or information. If too many documents are retrieved, it is better to use reranking models, such as cross-encoders or ColBERT, to reorder the documents so that more relevant ones appear at the top of the results.
