wow-agent-day02 Handcrafted an agent that's so rustic it crumbles.

Preparation Stage#

Create a virtual environment named agent, and run pip install openai python-dotenv.
To connect the openai library with domestic large models, we need to prepare the following for each manufacturer:

An api_key, which needs to be applied for on their open platform.
A base_url, which needs to be copied from their open platform.
The name of their dialogue model.

The latter two are public.
Create a new txt file in the root directory of the project and rename it to .env. Fill in a line of string: ZISHU_API_KEY=your_api_key. Here, use the default API provided by Zishuku.
Zishuku:

base_url = "http://43.200.7.56:8008/v1"
chat_model = "glm-4-flash"

Load environment variable code:

import os
from dotenv import load_dotenv

# Load environment variables
load_dotenv()
# Read api_key from environment variables
api_key = os.getenv('ZISHU_API_KEY')
base_url = "http://43.200.7.56:8008/v1"
chat_model = "glm-4-flash"

ERROR: api_key could not be successfully extracted.
fixed: load_dotenv('.env.txt')
4. Construct client

from openai import OpenAI
client = OpenAI(
    api_key = api_key,
    base_url = base_url
)

Prompt#

What is a Structured Prompt?#

Reference: https://github.com/EmbraceAGI/LangGPT
The idea of a structured prompt is to write prompts as if you are writing an article. Structured prompts have various high-quality templates that help you write prompts more easily and perform better. Therefore, writing structured prompts can involve various templates, and you can choose or create your favorite template just like using a PPT template.

LangGPT Variables#

ChatGPT can recognize various well-marked hierarchical content.
Present the content of the prompt in a structured way and set titles for easy reference, modification, and setting of prompt content. You can directly use paragraph titles to refer to large sections of content, or tell ChatGPT to modify specific content. This is similar to variables in programming, so we can treat these titles as variables.
The Markdown syntax hierarchy is excellent and suitable for writing prompts, so LangGPT variables are based on Markdown syntax. In fact, various formats that can achieve marking functions, such as JSON, YAML, or even well-formatted text, can also be used.

LangGPT Templates#

Using the Role template, you only need to fill in the corresponding content according to the template.
In addition to variables and templates, LangGPT also provides commands, memory, conditional statements, and other syntax settings.

Concept of Structured Prompts#

Identifiers: Symbols like #, <> (also -, []), these two symbols sequentially identify titles, variables, and control content hierarchy, used to mark hierarchical structures.

Attribute Words: Role, Profile, Initialization, etc., attribute words contain semantics and summarize and prompt the content under the module, used to identify semantic structures.

Use delimiters to clearly mark different parts of the input, such as triple quotes, XML tags, section titles, etc., can help mark text parts that need to be processed differently.

For GPT models, the hierarchical structure identified by identifiers achieves the effect of gathering the same semantics and organizing semantics, reducing the model's difficulty in understanding the prompt, making it easier for the model to understand the prompt's semantics.

Attribute words provide semantic prompts and summaries for prompt content, alleviating the interference of inappropriate content in the prompt. Combining attribute words with prompt content achieves a local hierarchical structure, making it easier for the model to grasp the overall semantics of the prompt.

A good structured prompt template, in a sense, constructs a good global thinking chain. For example, the template design shown in LangGPT considers the following thinking chain:

Role -> Profile -> Skills under Profile -> Rules -> Workflow (the workflow of roles that meet the above conditions) -> Initialization (preparation for formally starting work) -> Start actual use

When constructing prompts, it is advisable to refer to the global thinking chain of high-quality templates. Once mastered, you can completely adjust it to create a template suitable for your use. For example, when you need to control the output format, especially when you need formatted output, you can add modules like Output or OutputFormat.

Maintain Consistency of Contextual Semantics

This includes two aspects: one is format semantic consistency, and the other is content semantic consistency.

Format semantic consistency refers to the consistent identification function of identifiers. It is best not to mix them; for example, using # to identify both titles and variables creates inconsistency, which can interfere with the model's recognition of the prompt's hierarchical structure.
Content semantic consistency refers to the appropriateness of the semantics of attribute words in the thinking chain. For example, the Profile attribute word in LangGPT was originally Features, but after practice and reflection, I changed it to Profile to clarify its function: that is, the resume of the role. After the structured prompt idea has been widely used by many friends, many templates have emerged, but most have retained many designs of Profile, indicating that its design is successful and effective. Content semantic consistency also includes the semantic consistency of attribute words and the corresponding module content. For example, if the Rules section is about the rules the role needs to follow, it is not appropriate to pile up a lot of role skills and descriptions here.

Role Template in LangGPT#

Role: Your_Role_Name
Profile

Author: YZFly
Version: 0.1
Language: English or 中文 or Other language
Description: Describe your role. Give an overview of the character's characteristics and skills.

Skill-1

Skill description 1
Skill description 2

Skill-2

Skill description 1
Skill description 2

Rules

Don't break character under any circumstance.
Don't talk nonsense and make up facts.

Workflow

First, xxx
Then, xxx
Finally, xxx

Initialization
As a/an < Role >, you must follow the < Rules >, you must talk to the user in default < Language >, you must greet the user. Then introduce yourself and introduce the < Workflow >.

Prompt Chain decomposes the original requirements and uses multiple small prompts to connect/parallelly solve a complex task.

Prompts collaboration can also be a prompt tree, which continuously breaks down sub-tasks through a top-down design approach, forming a task tree, obtaining multiple model outputs, and combining these outputs through custom rules (permutations and combinations, filtering, integration, etc.) to get the final result. The API version of Prompt Chain can make the entire process more automated when combined with programming.

Prompt Design Methodology#

Data preparation. Collect high-quality case data as the basis for subsequent analysis.
Model selection. Choose an appropriate large language model based on specific creative purposes.
Prompt word design. Design the initial version of the prompt words based on case data; pay attention to role settings, background descriptions, goal definitions, constraints, and other key points.
Testing and iteration. Input the prompt words into GPT for testing, analyze the results; communicate with GPT through follow-up questions, in-depth discussions, and pointing out issues to obtain optimization suggestions.
Revise prompt words. Adjust various parts of the prompt words based on feedback from GPT, strengthen effective factors, and eliminate ineffective factors.
Repeat testing. Re-test the revised prompt words, compare results, continue to ask GPT, and adjust prompt words.
Cycle iteration. Repeat the above testing-communication-revision process until the results are satisfactory.
Summarize and refine. Summarize the valuable experiences gained during the optimization process of prompt words, forming best practices for designing prompt words.
Application expansion. Apply the mastered methodology to the design of other creative content, continuously enriching the skills of prompt word design.

sys_prompt = """You are a smart customer service representative. You will be able to assign different tasks to different people based on user questions. You have the following business lines:
1. User registration. If a user wants to perform this operation, you should send a special token with "registered workers". And inform the user that you are calling it.
2. User data inquiry. If a user wants to perform this operation, you should send a special token with "query workers". And inform the user that you are calling it.
3. Delete user data. If a user wants to perform this type of operation, you should send a special token with "delete workers". And inform the user that you are calling it.
"""
registered_prompt = """
Your task is to store data based on user information. You need to obtain the following information from the user:
1. Username, gender, age
2. Password set by the user
3. User's email address
If the user does not provide this information, you need to prompt the user to provide it. If the user provides this information, you need to store it in the database and inform the user of successful registration.
The storage method is to use SQL statements. You can write insert statements using SQL and need to generate a user ID and return it to the user.
If the user has no new questions, you should reply with a special token containing "customer service" to end the task.
"""
query_prompt = """
Your task is to inquire about user information. You need to obtain the following information from the user:
1. User ID
2. Password set by the user
If the user does not provide this information, you need to prompt the user to provide it. If the user provides this information, you need to query the database. If the user ID and password match, you need to return the user's information.
If the user has no new questions, you should reply with a special token containing "customer service" to end the task.
"""
delete_prompt = """
Your task is to delete user information. You need to obtain the following information from the user:
1. User ID
2. Password set by the user
3. User's email address
If the user does not provide this information, you need to prompt the user to provide it.
If the user provides this information, you need to query the database. If the user ID and password match, you need to inform the user that a verification code has been sent to their email and needs to be verified.
If the user has no new questions, you should reply with a special token containing "customer service" to end the task.
"""

Intelligent Customer Service Agent#

Define an intelligent customer service agent. class SmartAssistant:

def __init__(self):
  self.client = client # Define client
  # Define different parts of the prompt
  self.system_prompt = sys_prompt
  self.registered_prompt = registered_prompt
  self.query_prompt = query_prompt
  self.delete_prompt = delete_prompt
  # Use a dictionary to store messages from different sets
  self.messages = {
            "system": [{"role": "system", "content": self.system_prompt}],
            "registered": [{"role": "system", "content": self.registered_prompt}],
            "query": [{"role": "system", "content": self.query_prompt}],
            "delete": [{"role": "system", "content": self.delete_prompt}]
        }
  # Current assignment for handling messages
  self.current_assignment = "system"

Define method:

def get_response(self, user_input):
        self.messages[self.current_assignment].append({"role": "user", "content": user_input})# Record each user reply
        while True:
            response = self.client.chat.completions.create(
                model=chat_model,
                messages=self.messages[self.current_assignment],
                temperature=0.9,
                stream=False,
                max_tokens=2000,
            )

            ai_response = response.choices[0].message.content
            if "registered workers" in ai_response:
                self.current_assignment = "registered"
                print("Intent recognition:", ai_response)
                print("switch to <registered>")
                self.messages[self.current_assignment].append({"role": "user", "content": user_input})
            elif "query workers" in ai_response:
                self.current_assignment = "query"
                print("Intent recognition:", ai_response)
                print("switch to <query>")
                self.messages[self.current_assignment].append({"role": "user", "content": user_input})
            elif "delete workers" in ai_response:
                self.current_assignment = "delete"
                print("Intent recognition:", ai_response)
                print("switch to <delete>")
                self.messages[self.current_assignment].append({"role": "user", "content": user_input})
            elif "customer service" in ai_response:
                print("Intent recognition:", ai_response)
                print("switch to <customer service>")
                self.messages["system"] += self.messages[self.current_assignment]
                self.current_assignment = "system"
                return ai_response
            else:
                self.messages[self.current_assignment].append({"role": "assistant", "content": ai_response})
                return ai_response

Analysis:
self.client.chat.completions.create: This is a method call used to generate a response to the dialogue. It uses a chat model to process the input messages and generate corresponding output.
ai_response = response.choices[0].message.content: choices is an attribute of the response object, which is a list containing multiple possible reply options. [0] indicates selecting the first option. Typically, the API returns multiple possible replies, but here the first one is chosen. message is an attribute of choices[0], representing the message content of that option. content is an attribute of message, representing the actual text content, i.e., the reply generated by the AI assistant. Therefore, this line of code extracts the text content of the first option from the API response and stores it in the ai_response variable.
If the AI response contains "registered workers", the current task switches to "registered", and the user input is added to the message list for that task.
If the AI response contains "query workers", the current task switches to "query", and the user input is added to the message list for that task.
If the AI response contains "delete workers", the current task switches to "delete", and the user input is added to the message list for that task.
If the AI response contains "customer service", the current task switches back to "system", and the messages from the current task are merged into the system messages, ending the current task.
Start conversation loop:

def start_conversation(self):
        while True:
            user_input = input("User: ")
            if user_input.lower() in ['exit', 'quit']:
                print("Exiting conversation.")
                break
            response = self.get_response(user_input)
            print("Assistant:", response)

Code analysis: The user can input information, and the system will call the get_response method to generate a response based on the input and display the response to the user. The user can end the conversation by entering "exit" or "quit".

Effect Display#

Assistant: Hello! User with ID 1001, what operation would you like to perform?

User registration - If you want to register a new user, please let me know.
User data inquiry - If you need to inquire about user data, please let me know.
Delete user data - If you need to delete user data, please let me know.

Please choose the corresponding operation based on your needs, and I will call the corresponding workers to assist you.
Intent recognition: Okay, you need to inquire about the user data for ID 1001. I will send a special token with "query workers" to call the workers handling user data inquiries. Please wait a moment, and I will notify you of the inquiry results.
switch to
Assistant: Please provide your password.
Intent recognition: Inquiring, please wait. (Simulating inquiry)

User ID 1001 information is as follows:

Username: Zhang San
Email: [email protected]
Registration Date: 2021-05-15

Inquiry completed.

Do you have any other questions? If not, please reply with "customer service" to end the task.
switch to
Assistant: Inquiring, please wait. (Simulating inquiry)

User ID 1001 information is as follows:

Username: Zhang San
...
Do you have any other questions? If not, please reply with "customer service" to end the task.
Intent recognition: You have provided the user ID and password for the inquiry, and now I will send a special token with "query workers" to call the workers to query the database. Please wait a moment.