AI大模型实战篇：Basic Reflection，AI Agent的左右互搏之术

风叔 2024-09-11 17:43:06

在前面几篇文章中，风叔依次介绍了REWOO、Plan-and-Execute和LLM Compiler三种更侧重规划能力的AI Agent设计模式。

从最初的ReAct模式出发，加入规划能力即演变成REWOO；再加上Replan能力即演变成Plan-and-Execute；最后再加上DAG和并行处理能力，即演变成LLM Compiler。

从这篇文章开始，我们将转向另外几种侧重反思的AI Agent模式，我们首先从Basic Reflection开始。

一、Basic Reflection的概念

Basic Reflection可以类比于左右互博。左手是Generator，负责根据用户指令生成结果；右手是Reflector，来审查Generator的生成结果并给出建议。在左右互搏的情况下，Generator生成的结果越来越好，Reflector的检查越来越严格，输出的结果也越来越有效。

下图是Basic Reflection的原理，非常简单。

Generator接收来自用户的输入，输出initial response
Reflector接收来自Generator的response，根据开发者设置的要求，给出Reflections，即评语、特征、建议
Generator再根据Reflector给出的反馈进行修改和优化，输出下一轮response
循环往复，直到循环次数

二、Basic Reflection的实现过程

下面，风叔通过实际的源码，详细介绍Basic Reflection模式的实现方法，具体的源代码地址在文章结尾处获取。

第一步构建Generator

在下面的例子中，我们先构建一个能够生成5段话的文章生成器。

首先，我们给Generator约定了Prompt，告诉Generator具体的角色和目标，并且要求如果用户给出建议，需要给出修改后的版本

然后选择LLM模型，并构建Generator

最后要求输出Generator生成的内容

from langchain_core.messages import AIMessage, BaseMessage, HumanMessage
from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
from langchain_fireworks import ChatFireworks

prompt = ChatPromptTemplate.from_messages(
[
(
“system”,
“You are an essay assistant tasked with writing excellent 5-paragraph essays.”
” Generate the best essay possible for the user’s request.”
” If the user provides critique, respond with a revised version of your previous attempts.”,
),
MessagesPlaceholder(variable_name=”messages”),
]
)
llm = ChatFireworks(
model=”accounts/fireworks/models/mixtral-8x7b-instruct”,
model_kwargs={“max_tokens”: 32768},
)

generate = prompt | llm
essay = “”
request = HumanMessage(
content=”Write an essay on why the little prince is relevant in modern childhood”
)

for chunk in generate.stream({“messages”: [request]}):
print(chunk.content, end=””)
essay += chunk.content

下面是Generator的Initial Response，可以看出Generator虽然按照我们的要求生成了初始回复，但是回复的整体质量还有很大的提升空间。

Title: The Relevance of The Little Prince in Modern Childhood

The Little Prince, a novella by Antoine de Saint-Exupéry, has been a childhood favorite for generations. Despite being published over seven decades ago, its timeless themes continue to resonate with modern children, making it highly relevant in contemporary childhood.

Firstly, the story explores the complex nature of human relationships, which is particularly relevant for modern children growing up in an increasingly connected yet impersonal world. Through the little prince’s encounters with various grown-ups on different planets, the book highlights the importance of genuine connections and understanding. In an age where digital communication often replaces face-to-face interaction, this message is more pertinent than ever. The Little Prince encourages children to look beyond superficial relationships and seek deeper connections, fostering empathy and emotional intelligence.

Secondly, the book deals with the concept of responsibility and self-discovery, elements that are integral to a child’s growth. The little prince’s journey is essentially a quest for self-discovery, leading him to realize his responsibility towards his beloved rose. This narrative encourages modern children to embrace their individuality while understanding the significance of their actions. In a society that often overlooks the emotional well-being of children, The Little Prince offers a refreshing perspective on personal growth and responsibility.

Thirdly, the book addresses the challenging theme of loss and bereavement. The little prince’s departure from his asteroid and his subsequent encounters with the fox and the snake are profound reflections on the inevitability of loss and the importance of cherishing relationships. In a time when children are exposed to various forms of loss, from the death of loved ones to environmental degradation, The Little Prince provides a gentle yet powerful way to understand and cope with these experiences.

However, some critics argue that the book’s pace and abstract concepts might be challenging for modern children with short attention spans. To address this, a revised version could incorporate more visual elements and interactive activities to engage young readers better. Additionally, supplementary materials explaining the book’s themes in simpler terms could be provided for parents and educators to use in discussions with children.

In conclusion, The Little Prince remains relevant in modern childhood due to its exploration of human relationships, self-discovery, and loss. These themes, wrapped in a captivating narrative, offer valuable lessons for modern children. While some adaptations may be necessary to cater to the preferences of today’s children, the essence of the story remains a powerful tool for teaching emotional intelligence, personal growth, and resilience.

第二步构建Reflector

针对Generator，我们构建一个专门的Reflector。

同样的，我们要先给Reflector约定Prompt，告诉Reflector“你是一名正在给论文评分的老师。请对用户的论文提出批评和建议，包括长度、深度、风格等要求。”

然后选择LLM模型，并构建Reflector

reflection_prompt = ChatPromptTemplate.from_messages(
[
(
“system”,
“You are a teacher grading an essay submission. Generate critique and recommendations for the user’s submission.”
” Provide detailed recommendations, including requests for length, depth, style, etc.”,
),
MessagesPlaceholder(variable_name=”messages”),
]
)
reflect = reflection_prompt | llm

下面是针对Generator的Initial Response，Reflector反馈的评语和建议。Reflector给出了B+的评级，并且按照我们的要求，从文章的长度、深度、风格等维度给出了具体的建议和总结。

Essay Grade: B+

The essay you submitted provides a clear and well-structured argument about the relevance of The Little Prince in modern childhood. You have demonstrated a strong understanding of the text and its themes, and have effectively applied them to the context of contemporary childhood. However, there are some areas where improvement could be made to enhance the depth, style, and overall flow of your essay.

1. Length: While your essay is well-written and informative, it is relatively brief. Expanding on each point with more detailed analysis and examples would strengthen your argument and demonstrate a more comprehensive understanding of the text. Aim for a minimum of 500 words to allow for a more in-depth exploration of your ideas.

2. Depth: Although you have touched upon the relevance of the novel’s themes, further analysis is needed to truly establish its significance in modern childhood. For example, when discussing the complex nature of human relationships, delve into how the digital age affects children’s communication skills, and how The Little Prince addresses this issue. Providing concrete examples from the text and connecting them to real-world scenarios will make your argument more compelling.

3. Style: To engage your readers more effectively, consider varying your sentence structure and length. Using a mix of simple, compound, and complex sentences will improve the flow of your essay and make it more engaging to read. Additionally, watch your tense consistency. Ensure that you maintain the same tense throughout your essay to avoid confusion.

4. Recommendations: While your suggestions for adaptation are a good start, they could be expanded upon to provide more comprehensive recommendations. For example, you may want to discuss different methods of incorporating visual elements and interactive activities, such as illustrations, quizzes, or discussion questions. This will demonstrate that you have thoughtfully considered the needs of modern children and have developed strategies to address these challenges.

5. Conclusion: Your conclusion could benefit from a stronger summarization of your key points and an assertive final statement about the relevance of The Little Prince in modern childhood. Tying all your arguments together in a concise and powerful manner will leave a lasting impression on your readers and solidify your position.

Overall, your essay is well-researched and provides a solid foundation for a compelling argument about the relevance of The Little Prince in modern childhood. With some expansion, deeper analysis, and stylistic improvements, your essay can achieve an even higher level of excellence.

第三步循环执行

接下来，我们就可以循环执行这个“生成 – 检查”的过程，重复规定的次数，或者约定当Generator的生成结果达到多少分时，停止循环。

大家可以看到，在循环过程中，我们也加入了人类的反馈建议，有助于Generator和Reflector学习迭代。

for chunk in generate.stream(
{“messages”: [request, AIMessage(content=essay), HumanMessage(content=reflection)]}
):
print(chunk.content, end=””)

第四步构建流程图

下面，我们构建流程图，将Generator、Reflector等节点添加进来，循环执行并输出结果。

from typing import Annotated, List, Sequence
from langgraph.graph import END, StateGraph, START
from langgraph.graph.message import add_messages
from typing_extensions import TypedDict

class State(TypedDict):
messages: Annotated[list, add_messages]

async def generation_node(state: Sequence[BaseMessage]):
return await generate.ainvoke({“messages”: state})

async def reflection_node(messages: Sequence[BaseMessage]) -> List[BaseMessage]:
# Other messages we need to adjust
cls_map = {“ai”: HumanMessage, “human”: AIMessage}
# First message is the original user request. We hold it the same for all nodes
translated = [messages[0]] + [
cls_map[msg.type](content=msg.content) for msg in messages[1:]
]
res = await reflect.ainvoke({“messages”: translated})
# We treat the output of this as human feedback for the generator
return HumanMessage(content=res.content)

builder = StateGraph(State)
builder.add_node(“generate”, generation_node)
builder.add_node(“reflect”, reflection_node)
builder.add_edge(START, “generate”)

def should_continue(state: List[BaseMessage]):
if len(state) > 6:
# End after 3 iterations
return END
return “reflect”

builder.add_conditional_edges(“generate”, should_continue)
builder.add_edge(“reflect”, “generate”)
graph = builder.compile()

async for event in graph.astream(
[
HumanMessage(
content=”Generate an essay on the topicality of The Little Prince and its message in modern life”
)
],
):
print(event)
print(“—“)