Final Capstone: Build a Production NLP Intelligence System

• A multi-capability NLP Intelligence API that classifies intent, scores sentiment, and generates contextual responses — all served through a single REST endpoint.

• A modular PyTorch-based pipeline where each NLP component is independently testable and swappable without touching the rest of the system.

• A production-style deployment with Docker, a health-check endpoint, structured logging, and a minimal web demo interface.

You’ve spent 176 days learning how individual pieces work — tensors, layers, training loops, embeddings, sequence models. What separates a student from a practitioner is the ability to compose those pieces into a system that solves a real problem end-to-end. Every major NLP product at companies like OpenAI, Cohere, or Hugging Face is not one model — it’s a pipeline of coordinated components: input normalization, classification, generation, post-processing, and serving. This capstone is your first time building that pipeline yourself, which means by the end of Day 180, you’ll have produced something you can legitimately show in a job interview or deploy for real users.

A pipeline is a directed graph of transformations. Each node takes in a well-defined input and produces a well-defined output. The moment you start building pipelines this way, your code becomes composable. You can swap a rule-based intent classifier for a neural one without touching the sentiment module. You can add a new component — say, entity extraction — without modifying the response generator.

In production systems at Stripe or Shopify, the ML inference path looks exactly like this: an API gateway hands the raw text to a normalization service, which passes the cleaned text to a classification model, which produces a structured prediction object, which routes to a generation or retrieval system, which formats the final response. Each handoff is a typed contract. Today you’re building that exact pattern.

The dominant pattern for intent classification in production is fine-tuning a pre-trained transformer on a small labeled dataset. DistilBERT — a compressed version of BERT — gives you 97% of the accuracy at 40% of the size. This is the same trade-off that powers intent routing in voice assistants at Amazon and Google.

The key insight here is that the pre-trained model already understands language at a deep level. Your fine-tuning job is not to teach it language — it’s to teach it your categories. When you have only a few hundred labeled examples, this transfer learning approach outperforms training from scratch by a wide margin. Your final classifier is really just a linear layer sitting on top of a frozen or lightly-unfrozen transformer backbone.

The response generation in this system is a hybrid: a retrieval layer picks a candidate response template based on intent and sentiment score, and a neural reranker scores each candidate against the original query context. This avoids the hallucination risk of pure generation while still producing contextually relevant responses.

This pattern — retrieval-augmented generation with a reranker — is exactly what systems like Perplexity AI and Bing Chat use at scale. The reranker is a cross-encoder transformer that takes (query, candidate) pairs as input and outputs a relevance score. You’re not building the full RAG stack today, but the architecture you implement mirrors the production pattern precisely.

Serving an ML model is fundamentally different from serving a database. Model inference has unpredictable latency, GPU resource contention, and serialization overhead. Three patterns matter most:

Request batching: Group multiple incoming requests together before sending to the model. A batch of 32 requests takes only 1.5× the time of a single request on a GPU, so throughput multiplies dramatically. Your system implements a simple synchronous batch endpoint.

Response caching: Identical or near-identical queries should return cached predictions. A simple Redis-backed LRU cache cuts GPU load significantly in production. The architecture includes a cache layer even if you implement it in-memory for the capstone.

Structured error envelopes: Every response — success or failure — should follow the same JSON schema. Clients must be able to rely on the response structure regardless of what happened internally. Your API will always return {status, data, error, latency_ms}.

The preprocessor runs first and its output is shared by both the intent classifier and the sentiment scorer — they run in parallel, then their outputs converge at the response selector. This parallel fan-out pattern keeps latency bounded to max(t_intent, t_sentiment) rather than t_intent + t_sentiment.

                   NLP Intelligence API
   Input Text
       |
       v
   [Preprocessor]  -->  normalize, tokenize, truncate
       |
       +-----------------------------+
       v                             v
   [Intent Classifier]        [Sentiment Scorer]
   (DistilBERT + linear)      (LSTM / Day 176)
       |                             |
       +-------------+---------------+
                     v
            [Response Selector]
            (retrieve + rerank)
                     |
                     v
              [JSON Response]