Company Research Assistant Backend

This backend is part of the Company Research Assistant project. It automates the process of crawling company websites, extracting product and service information, classifying the company, and generating reports—including Google Docs—using AI and various integrations.

Features

Website Crawling: Scrapes company websites and subpages for content using Firecrawl.
Product/Service Extraction: Uses AI (OpenAI) to extract and summarize products/services from crawled data.
Category Classification: Classifies companies into software categories (e.g., CRM, ERP, HRM).
Google Docs Integration: Generates and uploads reports to Google Docs via Composio.
Supabase Integration: Stores and retrieves crawled data in a Supabase database.
Streamlit UI: Provides a chat-based interface for user interaction.

Project Structure

agent.py — Main workflow and Streamlit app.
schemas.py — Pydantic models for structured data.
utils.py — Utility functions (e.g., graph saving).
constants.py — Constants used throughout the backend.
prompts/prompts.py — AI prompt templates.
database/table_creator.py — Database table creation logic.

Setup

Clone the repository

Install dependencies

pip install -r requirements.txt
# or, if using pyproject.toml
pip install .

Set up environment variables
- Create a .env file in the backend/ directory with the following keys:
  - OPENAI_KEY
  - FIRECRAWL_KEY
  - SUPABASE_URL
  - SUPABASE_KEY
  - COMPOSIO_KEY
  - GOOGLEDOCS_CONNECTION_ID
Create the database table
- Run the table creator script to set up the CrawledWebpages table in Supabase:
```
python backend/database/table_creator.py
```
Run the Streamlit app
```
streamlit run backend/agent.py
```

Usage

Enter a company website URL in the chat interface.
The assistant will crawl the website, extract product/service information, classify the company, and offer to save the report to Google Docs.

Dependencies

See pyproject.toml for the full list. Key packages:

composio-core, composio-langgraph
firecrawl-py
streamlit
supabase
loguru
langchain, langgraph, openai

License

[Add your license here]