Formulating the prompt for SQL Question/Answering

[asemoon]

Hello,
I have just started using Langchain and I was following this tutorial to build SQL answering machine. My database sits on Postgres but it's a representation of a REVIT CAD model of a building architecture. it's quite a complicated structure with lots of tables and columns.

The problem is what is being sent as the database schema metadata to LLM to formulate the SQL query is way to big creating a massive number of tokens this has made calling chatgptAPIs super expensive and not viable thus far.

Here is my code snippet so far:

import os
from langchain.prompts import PromptTemplate
from langchain.chains import create_sql_query_chain
from langchain_community.utilities import SQLDatabase
from langchain_openai import ChatOpenAI
from dotenv import load_dotenv
from langchain_community.tools.sql_database.tool import QuerySQLDataBaseTool
from operator import itemgetter
from langchain_core.output_parsers import StrOutputParser
from langchain_core.prompts import PromptTemplate
from langchain_core.runnables import RunnablePassthrough
import logging

# Configure logging
logging.basicConfig(level=logging.DEBUG, format='%(asctime)s - %(levelname)s - %(message)s')

# Assuming `llm` is the instance of ChatOpenAI
def log_llm_call(prompt):
    logging.debug(f"LLM Input: {prompt}")
    response = llm(prompt)
    logging.debug(f"LLM Output: {response}")
    return response
#end of loggin

# Load environment variables
load_dotenv()

# Database connection
db_uri = f"postgresql://{os.getenv('DB_USER')}:{os.getenv('DB_PASS')}@{os.getenv('DB_HOST')}:{os.getenv('DB_PORT')}/{os.getenv('DB_NAME')}"
db = SQLDatabase.from_uri(db_uri)
llm = ChatOpenAI(api_key=os.getenv('OPENAI_API_KEY'), model_name='gpt-3.5-turbo')


execute_query = QuerySQLDataBaseTool(db=db)
write_query = create_sql_query_chain(llm, db)


answer_prompt = PromptTemplate.from_template(
    """Given the following user question, corresponding SQL query, and SQL result, answer the user question. You are querying a Revit model translated into SQL
    don't include the  sql ``` in your sql query output
    Question: {question}
    SQL Query: {query}
    SQL Result: {result}
    Answer: """
)

chain = (
    RunnablePassthrough.assign(query=write_query).assign(
        result=itemgetter("query") | execute_query
    )
    | answer_prompt
    | llm
    | log_llm_call  # Use the logging wrapper
    | StrOutputParser()
)

result = chain.invoke({"question": "what's the project name?"})
print(result)

It seems like I can pass the custom prompt to create_sql_query_chain, but I am not sure what's the best way to strip down the results from schema description results which is good enough to pass to LLM in order for the SQL to be formulated.

My goal is to be able to pass barebone metadata about the database schema to the llm while enabling the llm to be able to understand enough to produce the SQL statement to be executed. So that the cost and speed will be improved. Any ideas on how to go about doing this?

P.S: I understand that it would be better to provide the database schema but because it's absolutely so many tables and columns it would just clutter this conversation.

[HavenDV]

Please ask here: https://github.com/langchain-ai/langchain