When you listen to expert’s in AI talk about the future, most of them say AI is not going to replace everyone but it is for sure going to make those who don’t know how to use AI obsolete while those who have learned how to use it will thrive. What does that mean for you as someone who is a software engineer, DevOps engineer, SRE or other related roles? It means learning how to use AI beyond the basics is a survival mechanism. Saying that it is a survival mechanism might sound like I am blowing things out of proportion (I hope I am) but it just might be the case that I am not.
This article is about building a GenAI chat application on AWS which as you might already know is not the only way to use AI but as things stand is the most widely use form of AI. All that aside, let’s cut to the chase starting with an architecture diagram of the Application we will be building.
You can view this a three tier architecture with a frontend, middleware and backend layers. This three way split enforces the principle of separation of concerns offering you benefits such as reduced complexity, allowing you to build an application that is more maintainable and the ability to scale components independently as need arises.
Here is a list of service used for this project. For each service I will give a little bit of detail of how it is intended for us for this particular application architecture.
CloudFront: CloudFront sits just in front of the chat applications user interface or frontend. In case you are wondering what CloudFront does, it is a CDN service by AWS used to cache data closer to where your users are to reduce latency and improve application performance for faster access. This data (mostly static content like HTML, CSS, JS and image files) is cached in edge locations which are data centres that are globally positioned near end-users. We are aware that most of the content in a GenAI chatbot is dynamic content generated by the language model at inference time. CloudFront also optimizes this process because it routes requests from the user through the edge location to a backend via the AWS global network backbone rather than the public internet which avoids congestion that exists on the internet. To protect the application that the CloudFront distribution points to from common web vulnerabilities, we can use an AWS WAF protection pack (web ACL).
Amplify: For our use case, Amplify is used for hosting the Chatbot UI. I chose to use Amplify Hosting to host the ChatUI because it allows zero-config deployments for Next.js application. This allows you to get the ball rolling fast and also gives you more time to focus on other parts of your application. In a project such as this, you get things going fast by making use of a ready-made open source AI chatbot template. If I end up implementing this project, I won’t spend time building a ChatUI from scratch because I have ready-made ones like this one made built with Next.js by Vercel.
API Gateway: It sits between the Chat UI and Amazon Lex. You can view it as the bridge between the frontend and the backend. In our use case API gateway, it provides a secure endpoint for user requests flowing to the backend from the public internet and you can integrate with an authentication service like Amazon Cognito user pools to add an authentication feature to the application. As a summary, API gateway handled input validation, traffic management through throttling and rate limiting and also some CORS support.
Amazon Lex: Amazon Lex is the service that powers the conversational interface of our GenAI chatbot. It is a service for building conversational interfaces using voice and text. In this architecture, Lex is responsible for understanding user input and generating appropriate responses. It uses advanced natural language understanding (NLU) to process user input and can be integrated with other AWS services to enhance its capabilities. For instance, you can connect Lex to AWS Lambda functions to execute business logic or fetch data from other services.
Lambda: AWS Lambda is used to run backend code in response to events. In this architecture, Lambda functions serve multiple purposes. One key role is as the fulfillment Lambda function connected to Amazon Lex, which executes specific tasks based on user interactions, such as querying a database or processing data. Additionally, other Lambda functions are part of a Step Functions workflow designed for ingesting and refining data that comes into the knowledge S3 bucket. This workflow ensures that data is processed efficiently and is ready for use by the chatbot, enhancing its ability to provide accurate and relevant responses. As you probably know already, Lambda allows you to run code without provisioning or managing servers, making it a cost-effective and scalable solution for handling backend logic.
Redis Cache: In this architecture, Redis is used as an in-memory data store to cache frequent responses, significantly improving the application's performance and reducing costs. By caching responses that have been processed previously, Redis allows the application to quickly retrieve these responses without having to reprocess the same data. This reduces the computational load on the backend systems and minimizes the latency experienced by users.
When a user makes a request, the application first checks the Redis cache to see if a response for that request already exists. If it does, the cached response is returned immediately, providing a faster user experience. If not, the application processes the request, generates a response, and stores it in the Redis cache for future use. This caching mechanism is particularly beneficial for handling repetitive queries or frequently accessed data, ensuring that users receive quick responses while optimizing resource utilization.
RDS for PostgreSQL with PGVector: In this architecture, Amazon RDS is used to manage a PostgreSQL database with the PGVector extension. This setup allows for the storage and management of vector embeddings, which are crucial for performing similarity searches during inference time. Vector embeddings are representations of data in a continuous vector space, often used in AI applications to capture semantic meaning. By using PGVector, you can efficiently store and query these embeddings, enabling the chatbot to perform tasks such as finding similar phrases or concepts based on user input. RDS provides automated backups, patching, and scaling, ensuring that your database is reliable and performant. This choice of database is particularly beneficial for applications that require advanced search capabilities and the ability to handle complex queries involving vector data.
S3: In this architecture, Amazon S3 serves primarily as the knowledge bucket where files are ingested and further processed into chunks that are used to make text embeddings. Here, the S3 bucket plays a crucial role in enabling RAG by providing a scalable and reliable storage solution for large datasets. In a RAG architecture, S3 is used to store vast amounts of unstructured data, such as documents, articles, and other text sources. This data is then processed to create embeddings, which are stored in a database like RDS with PGVector for efficient similarity searches.
CloudWatch & Xray: AWS CloudWatch and X-Ray are used together to monitor and enhance the application's performance and health. CloudWatch provides insights into resource utilization, application metrics, and operational health by setting up alarms, visualizing logs, and creating dashboards to monitor the chatbot's performance. Meanwhile, X-Ray complements this by tracing requests as they travel through the application, offering a detailed view of the application's architecture and identifying bottlenecks or performance issues. Together, CloudWatch and X-Ray ensure that the application runs smoothly and efficiently, providing a comprehensive monitoring solution.
Secrets Manager: It is used to securely store and manage sensitive information such as API keys, access keys, and database credentials. It provides a centralized and secure way to manage secrets, ensuring that sensitive data is protected and only accessible to authorized services and applications. Secrets Manager automatically rotates secrets, eliminating the need for manual updates and reducing the risk of exposure. By integrating with IAM, it controls access to secrets, ensuring that only authorized users and applications can retrieve them. This enhances the security of the application by safeguarding critical information and simplifying secret management.
VPC Endpoints: Data exchanges between different services are routed through VPC endpoints, ensuring that the data does not traverse the public internet. VPC endpoints provide a secure and private connection between your VPC and AWS services, allowing data to flow directly within the AWS network. Using VPC endpoints offers several benefits. Firstly, it enhances security by keeping data within the AWS network, reducing exposure to potential threats on the public internet. Secondly, it improves performance by minimizing latency, as data does not have to travel through external networks. Lastly, it can reduce costs associated with data transfer over the internet, as traffic remains within the AWS infrastructure. This setup ensures that your application maintains high levels of security, reliability, and efficiency.
The architecture of this chatbot application is designed to incorporate both synchronous and asynchronous elements, optimizing for real-time interaction and efficient processing. On the synchronous side, the user-facing components, such as the chat UI, Amazon Lex, and API Gateway, handle requests and responses in real-time. This ensures that users experience immediate feedback during their interactions with the chatbot, maintaining a seamless conversational flow. Conversely, the backend processes leverage asynchronous operations to enhance scalability and resource efficiency. Services like AWS Lambda and Amazon S3 operate independently, processing events and storing data without requiring immediate responses. This asynchronous design allows the system to handle large volumes of data and complex workflows, such as data ingestion and refinement, without impacting the real-time user experience. By balancing synchronous and asynchronous operations, the architecture effectively supports both the immediacy of user interactions and the robustness of backend processing.
Conclusion
The architecture we've talked about in this article is just a starting point, kind of like a blueprint for building a GenAI chatbot in the real world. It's a solid foundation, but when you're ready to take it to the next level, you'll need to think about a few more things. Real-world applications often come with their own set of challenges, like specific business needs, how big you want to scale, keeping everything secure, and making sure it plays nice with other systems you might already have. By using this setup as your base, you can tweak and adjust it to fit exactly what you need, making sure your chatbot is not only up and running but also ready to handle whatever comes its way. This way, you're not just building something that works today, but something that's ready for the future too.
Comments
Nothing yet. Say the first thing.
Sign in to join the conversation.