Our initial chatbot (v0.1) delivered mixed results, prompting us to experiment with improvements.
One puzzling challenge emerged when we asked questions like:
“Give me the US election related summary from the media for the last week?”
The chatbot responded with an explanation that it didn’t have real-time access to news or browsing capability. Subsequent queries—like asking for today’s date or the date range for last week—revealed inconsistencies.
For example, when asked, “What day is it today?”, it replied, “Today is October 4, 2023.” Then, upon requesting the date range for the previous week, it stated, “The date range for the last week, considering today is October 10, 2023, would be from October 3, 2023, to October 9, 2023.”
This discrepancy wasn’t a flaw; it was simply a consequence of the GPT-4o-mini model’s knowledge cutoff, which is fixed to October 2023.
Understanding AI Agents in Our Workflow
Before diving into our solution, it’s helpful to define what an AI Agent is. According to an Amazon post:
AI Agent: A software program that can interact with its environment, collect data, and use that data to perform self-determined tasks to meet predetermined goals.
A relatable everyday example is a robotic vacuum cleaner. Much like an AI agent, it:
- Perceives its surroundings by scanning the environment,
- Decides on an optimal cleaning route while avoiding obstacles,
- Acts by vacuuming the floor—all autonomously.
Building on this concept, we began to view parts of our application as isolated workflows—mini AI agents that handle specific tasks. In our case, the challenge was dealing with timeframes in user prompts.
The Challenge: Handling Timeframes in prompts
Our chatbot struggled with prompts containing temporal references. Since the LLM wasn’t aware of the current date, any timeframe-based query produced inaccurate results.
We decided to break down the problem into a multi-step workflow, each step handled by a dedicated prompt.
Step 1: Identifying Timeframes
The first step was determining if the user’s question contained any explicit or implicit timeframes.
To do this, we crafted a prompt that checks for various types of time references—relative dates, specific dates, date ranges, and time periods.
Prompt:
private static final String TIME_FRAME_FINDER_PROMPT_TEMPLATE = """
Find all explicit or implicit timeframes mentioned in the user’s question. Identify types of timeframes such as:
- Relative dates (e.g., "today," "yesterday")
- Specific dates (e.g., "November 21, 2024")
- Date ranges (e.g., "from January to March 2023")
- Time periods (e.g., "last week," "this year")
If actionable timeframes (specific dates, ranges, or relative time periods used in a calculative context) are found, return `true`. Otherwise, return `false`.
User Question: {question}
Provide the response, with no additional text or labels.
""";At first glance, the response from the LLM was exactly what we expected—a simple true or false based on whether any timeframes were detected.
Step 2: Extracting and Converting Timeframes
Once the model indicated the presence of timeframes, we needed it to convert these references into explicit date ranges. We injected today’s date into the prompt and provided detailed instructions on how to extract, convert, and replace timeframes with exact date ranges. We also instructed the model to remove any timeframes from the original query, outputting a structured JSON object that could be used for Vector Store similarity searches.
Prompt:
private static final String TIME_FRAME_TO_DATE_RANGE_PROMPT_TEMPLATE = """
Identify and replace all explicit or implicit timeframes mentioned in the user’s question. The timeframes can include:
- Relative dates (e.g., "today," "yesterday")
- Specific dates (e.g., "November 21, 2024")
- Date ranges (e.g., "from January to March 2023")
- Time periods (e.g., "last week," "this year")
User Question: {question}
**Instructions:**
1. As a reference for today always use {now}
2. Convert any identified timeframes into exact date ranges using the format "yyyy-MM-dd'T'HH:mm:ss.SSSZ", where Z represents the timezone offset from {now}.
3. Replace the user’s timeframes with their corresponding date ranges. Include "today" as the end of the range if it makes sense in the context.
4. Additionally remove all timeframes from the original question.
5. Return the result in the following JSON structure:
\{
"expression": \{
"type": "logical",
"expressions": [
\{
"type": "comparison",
"leftOperand": "publicationDate",
"rightOperand": "START_DATE", // Replace with calculated "from" date
"operator": "GTE"
\},
\{
"type": "comparison",
"leftOperand": "publicationDate",
"rightOperand": "END_DATE", // Replace with calculated "to" date
"operator": "LTE"
\}
],
"operator": "AND"
\},
"searchQuery": "ORIGINAL_QUESTION_WITHOUT_TIMEFRAMES", // replace with user's question having no timeframes
\}
Provide the response, with no additional text or labels.
""";Result:
The result we got was quite impressive. The model returned JSON output with valid date ranges in the correct format and a clean search query free of any timeframes. For instance, the JSON output looked like this:
{
"expression": {
"type": "logical",
"expressions": [
{
"type": "comparison",
"leftOperand": "publicationDate",
"rightOperand": "2024-10-28T00:00:00.000+0000",
"operator": "GTE"
},
{
"type": "comparison",
"leftOperand": "publicationDate",
"rightOperand": "2024-10-31T23:59:59.999+0000",
"operator": "LTE"
}
],
"operator": "AND"
},
"searchQuery": "Give me the US election related summary?"
}Step 3: Converting Dates to UTC
The final step was ensuring all date values were standardized to UTC. For this purpose, we introduced one more prompt in our workflow to convert any date stored in our JSON response from the user’s timezone to UTC. The resulting dates were in the precise format we required for further filtering in our Vector Store.
Prompt:
private static final String DATE_TO_UTC_PROMPT_TEMPLATE = """
Convert all the occurrences of Date, stored as rightOperand value
in the JSON response, from the user's timezone to UTC.
JSON Response: {response}
When doing conversion, use the following Date format: yyyy-MM-dd'T'HH:mm:ss.SSSZ
Provide the response, with no additional text or labels.
""";After processing, the final JSON object featured dates like these:
Result:
{
"expression": {
"type": "logical",
"expressions": [
{
"type": "comparison",
"leftOperand": "publicationDate",
"rightOperand": "2024-10-28T00:00:00.000Z",
"operator": "GTE"
},
{
"type": "comparison",
"leftOperand": "publicationDate",
"rightOperand": "2024-10-31T23:59:59.999Z",
"operator": "LTE"
}
],
"operator": "AND"
},
"searchQuery": "Give me the US election related summary?"
}Overcoming Holiday Hiccups: Refining Our Timeframe Extraction
As our workflow evolved, we encountered an unexpected challenge. When we thought our timeframe logic was flawless, we ran into issues with prompts that contained specific holidays. For example, consider this query:
“Can you give me suggestions for travel destinations during Thanksgiving? Extract the destinations as a list.”
Our timeframe extraction prompt chain misinterpreted the phrase “during Thanksgiving” as an actionable timeframe. This was totally misleading making similarity search and afterwords the LLM response totally irrelevant.
Realizing our mistake, we went back to the drawing board. After some prompt tweaking, we updated our timeframe finder prompt to explicitly ignore descriptive or situational references related to holidays. Here’s how we refined it:
FROM:
private static final String TIME_FRAME_FINDER_PROMPT_TEMPLATE = """
Find all explicit or implicit timeframes mentioned in the user’s question. Identify types of timeframes such as:
- Relative dates (e.g., "today," "yesterday")
- Specific dates (e.g., "November 21, 2024")
- Date ranges (e.g., "from January to March 2023")
- Time periods (e.g., "last week," "this year")
If actionable timeframes (specific dates, ranges, or relative time periods used in a calculative context) are found, return `true`. Otherwise, return `false`.
User Question: {question}
Provide the response, with no additional text or labels.
""";It evolved TO:
private static final String TIME_FRAME_FINDER_PROMPT_TEMPLATE = """
Find all explicit or implicit timeframes mentioned in the user’s question. Identify types of timeframes such as:
- Relative dates (e.g., "today," "yesterday")
- Specific dates (e.g., "November 21, 2024")
- Date ranges (e.g., "from January to March 2023")
- Time periods (e.g., "last week," "this year")
Do not consider descriptive or situational references (e.g., "during Thanksgiving" in a query asking for suggestions or opinions) as actionable timeframes. Examples of such non-actionable references include:
- "Can you suggest travel destinations during Thanksgiving?"
- "What are the best dishes to prepare for Christmas?"
If actionable timeframes (specific dates, ranges, or relative time periods used in a calculative context) are found, return `true`. Otherwise, return `false`.
User Question: {question}
Provide the response, with no additional text or labels.
""";After implementing this change, our system’s response improved dramatically. For instance, if you’re considering travel during Thanksgiving, you might receive a list of enticing destinations along with actionable travel tips. One sample output was:
If you’re considering travel during Thanksgiving, you might want to explore some international destinations, as domestic flight deals tend to diminish around this time. According to the article titled “The Best Place for a Last-Minute Thanksgiving? Abroad.” published by the New York Times, November is an excellent time for affordable international travel, particularly because it falls within the shoulder season for many popular destinations.
Here are some enticing options:
- Tokyo, Japan: Flights from Las Vegas can be found for under $800, making it a great choice for those looking to experience a vibrant city with rich culture and delicious cuisine.
- Dublin, Ireland: With round-trip fares from Boston under $450, Dublin offers a charming atmosphere filled with history, music, and friendly locals.
- Milan, Italy: For under $600 from Dallas, Milan is perfect for fashion enthusiasts and food lovers alike, with its stunning architecture and renowned culinary scene.
- Taipei, Taiwan: Flights from Seattle for under $800 provide an opportunity to explore Taiwan’s unique blend of traditional and modern attractions.
- Berlin, Germany: With fares from Miami under $400, Berlin is a fantastic destination for history buffs and art lovers, especially with its rich cultural heritage.
- London, England: Round-trip tickets from Newark for under $400 make London an accessible option for those wanting to explore its iconic landmarks and diverse neighborhoods.
- Cancún, Mexico: For families or beach lovers, flights from Chicago for under $400 offer a chance to relax in the sun and enjoy beautiful beaches.
- Belize: With flights from Los Angeles for under $450, Belize is ideal for those seeking adventure, with opportunities for snorkeling, diving, and exploring ancient ruins.
- San Jose, Costa Rica: Affordable flights from Washington for under $350 allow travelers to immerse themselves in nature, with lush rainforests and stunning wildlife.
These destinations not only provide great travel deals but also allow you to avoid the crowds typically associated with peak travel seasons. So, if you’re looking for a memorable Thanksgiving getaway, consider venturing abroad!
Putting It All Together
With our three-step workflow in place—and the additional refinement to handle holiday-related queries—we finally revisited the original user query:
“Give me the US election related summary from the last week of October 2024?”
After applying our timeframe extraction workflow during the pre-retrieval phase, the chatbot was able to understand the temporal context and return a relevant answer. For example, it provided an article summary along with a link that, when clicked, revealed an article dated October 31, 2024.
This demonstrated that our system now accurately interpreted and handled dynamic timeframe prompts.
Conclusion
By breaking down our problem into isolated, manageable workflows, we created a simple AI agent capable of transforming vague, human-friendly time expressions into actionable, machine-readable date ranges. This process not only enhanced our chatbot’s response accuracy when it comes to temporal references but also showcased the power of combining prompt engineering into a AI agentic prompt chain workflow.
The result? A more reliable, context-aware system that can filter news articles based on precise date ranges—paving the way for more complex autonomous systems that can adapt and respond intelligently to diverse user queries. And as we refined our approach to handle even the nuances of holiday-related language, our journey in building AI agents grew even more robust.
Embracing this modular, agentic approach has opened up exciting possibilities in our AI journey. As we continue to refine our methods and integrate additional tools and memory capabilities, the potential for creating even more sophisticated autonomous systems is limitless.
Happy coding, and may your workflows always be creative!
