061 Architecture 34d53db9

# explanation/architecture.md
---
layout: libdoc/page
title: Architecture
order: 110
category: Explanation
---


This document explains the overall architecture, design principles, and key concepts behind Simple Chat. Understanding these foundations will help you make informed decisions about deployment, configuration, and usage.

## System Overview

Simple Chat is built as a modern, cloud-native application leveraging Azure's AI and data services to provide Retrieval-Augmented Generation (RAG) capabilities with enterprise-grade security and scalability.

### Core Principles

**Security-First Design**
- Azure Active Directory integration for authentication
- Role-based access control (RBAC) for authorization
- Azure Managed Identity for service-to-service communication
- Private networking support for enterprise deployments

**Scalable Architecture**
- Stateless application design with external session storage
- Horizontal scaling support across multiple App Service instances
- Configurable autoscaling for variable workloads
- Distributed caching with Azure Redis Cache

**Extensible Framework**
- Modular feature architecture with optional components
- Admin-configurable settings for all major features
- Plugin-style integration for additional AI services
- API-first design for custom integrations

## High-Level Architecture

```
┌─────────────┐    ┌──────────────┐    ┌─────────────────┐
│   Users     │───▶│ Azure AD     │───▶│ Simple Chat     │
│ (Browsers)  │    │ (Auth)       │    │ (App Service)   │
└─────────────┘    └──────────────┘    └─────────────────┘
                                                │
                   ┌─────────────────────────────┼─────────────────────────────┐
                   │                             ▼                             │
                   │        ┌─────────────────────────────────────┐           │
                   │        │          Data Layer                 │           │
                   │        │                                     │           │
                   │        │  ┌─────────────┐ ┌──────────────┐  │           │
                   │        │  │ Cosmos DB   │ │ AI Search    │  │           │
                   │        │  │(Metadata)   │ │(Documents)   │  │           │
                   │        │  └─────────────┘ └──────────────┘  │           │
                   │        └─────────────────────────────────────┘           │
                   │                                                          │
                   │        ┌─────────────────────────────────────┐           │
                   │        │           AI Services               │           │
                   │        │                                     │           │
                   │        │ ┌───────────┐ ┌─────────────────┐   │           │
                   │        │ │Azure      │ │ Document        │   │           │
                   │        │ │OpenAI     │ │ Intelligence    │   │           │
                   │        │ └───────────┘ └─────────────────┘   │           │
                   │        │                                     │           │
                   │        │ ┌───────────┐ ┌─────────────────┐   │           │
                   │        │ │Content    │ │ Other AI        │   │           │
                   │        │ │Safety     │ │ Services        │   │           │
                   │        │ └───────────┘ └─────────────────┘   │           │
                   │        └─────────────────────────────────────┘           │
                   └────────────────────────────────────────────────────────────┘
```

## Core Components

### Application Tier

**Azure App Service**
- **Purpose**: Hosts the Python web application
- **Technology**: Flask-based web framework
- **Scaling**: Horizontal scaling with session state externalization
- **Security**: Integrated with Azure AD, supports Managed Identity

**Key Responsibilities:**
- User interface rendering and interaction handling
- Business logic orchestration
- API endpoint management
- Authentication and authorization enforcement
- Integration with Azure AI services

### Data Layer

**Azure Cosmos DB**
- **Purpose**: Primary data store for application metadata
- **Data Model**: Document-based JSON storage
- **Containers**: Conversations, documents, users, groups, settings
- **Scaling**: Request Unit (RU) based autoscaling
- **Consistency**: Session consistency for user interactions

**Stored Data Types:**
- Conversation history and metadata
- Document metadata and processing status
- User preferences and group memberships
- Application configuration settings
- Feedback and audit logs

**Azure AI Search**
- **Purpose**: Document content indexing and retrieval
- **Technology**: Hybrid search (vector + keyword)
- **Indexes**: Separate indexes for personal and group documents
- **Scaling**: Search units (replicas + partitions)
- **Features**: Semantic search, custom ranking, faceted search

**Search Index Structure:**
- Document chunks with embeddings
- Metadata fields for filtering
- User and group access controls
- Classification and tagging information

**Azure Storage Account** (Enhanced Citations)
- **Purpose**: Stores processed document files for direct access
- **Organization**: User-scoped and document-scoped folders
- **Access**: Private with time-limited SAS tokens
- **Integration**: Links citations to original document pages/timestamps

### AI Services Layer

**Azure OpenAI**
- **Chat Models**: GPT-4, GPT-3.5-turbo for conversational AI
- **Embedding Models**: text-embedding-ada-002, text-embedding-3 variants
- **Image Generation**: DALL-E models for image creation
- **Integration**: Both direct endpoints and API Management support

**Azure AI Document Intelligence**
- **Purpose**: Extract text and structure from uploaded documents
- **Capabilities**: OCR, layout analysis, table extraction
- **File Types**: PDF, Office documents, images
- **Integration**: Async processing with status tracking

**Azure AI Content Safety**
- **Purpose**: Content moderation and safety filtering
- **Categories**: Hate, sexual, violence, self-harm detection
- **Custom Lists**: Organization-specific blocked terms
- **Integration**: Pre-processing filter for all user inputs

**Additional AI Services:**
- **Speech Service**: Audio file transcription
- **Video Indexer**: Video content analysis and transcription
- **Custom AI Models**: Integration points for specialized models

## Data Flow and Processing

### Document Ingestion Workflow

```
User Upload ─┐
             ├─▶ Document Intelligence ─┐
File Types   ┘                           ├─▶ Text Extraction
                                         │
Audio Files ─────▶ Speech Service ──────┘
                                         │
Video Files ─────▶ Video Indexer ───────┘
                                         │
                                         ▼
                              ┌─────────────────┐
                              │ Content Chunking │
                              │ & Vectorization │
                              └─────────────────┘
                                         │
                                         ▼
                              ┌─────────────────┐
                              │    Storage      │
                              │ ┌─────────────┐ │
                              │ │ Cosmos DB   │ │ ◄─── Metadata
                              │ │ (Metadata)  │ │
                              │ └─────────────┘ │
                              │ ┌─────────────┐ │
                              │ │ AI Search   │ │ ◄─── Content + Embeddings
                              │ │ (Content)   │ │
                              │ └─────────────┘ │
                              │ ┌─────────────┐ │
                              │ │ Storage     │ │ ◄─── Processed Files
                              │ │ (Files)     │ │      (Enhanced Citations)
                              │ └─────────────┘ │
                              └─────────────────┘
```

### Chat Processing Workflow

```
User Message ─┐
              │
              ▼
    ┌─────────────────┐
    │ Content Safety  │ ◄─── Optional pre-processing filter
    │   Filtering     │
    └─────────────────┘
              │
              ▼
    ┌─────────────────┐
    │   RAG Query     │
    │   Processing    │
    └─────────────────┘
              │
              ├─▶ AI Search ────┐
              │                 │
              ▼                 ▼
    ┌─────────────────┐  ┌─────────────────┐
    │  Document       │  │   Relevant      │
    │  Retrieval      │  │   Context       │
    └─────────────────┘  └─────────────────┘
              │                 │
              └─────────┬───────┘
                        │
                        ▼
              ┌─────────────────┐
              │  Azure OpenAI   │
              │  Generation     │
              └─────────────────┘
                        │
                        ▼
              ┌─────────────────┐
              │   Response      │
              │ + Citations     │
              └─────────────────┘
```

## Security Architecture

### Authentication & Authorization

**Azure Active Directory Integration**
- **Identity Provider**: Centralized identity management
- **Authentication Flow**: OAuth 2.0/OpenID Connect
- **Multi-tenancy**: Support for multiple Azure AD tenants
- **Device Security**: Conditional access policy support

**Role-Based Access Control (RBAC)**
```
Application Roles:
├── Admin
│   └── Full system configuration access
├── User  
│   └── Basic chat and document access
├── CreateGroups
│   └── Permission to create new groups
├── SafetyViolationAdmin
│   └── View and manage content safety violations
└── FeedbackAdmin
    └── Access user feedback and analytics
```

**Data Access Control**
- **Personal Workspaces**: User-scoped document access
- **Group Workspaces**: Role-based group membership
- **Document Permissions**: Fine-grained access controls
- **Search Isolation**: User/group-aware search results

### Network Security

**Private Networking Support**
- **Private Endpoints**: Secure service-to-service communication
- **VNet Integration**: Application subnet isolation  
- **NSG Rules**: Network traffic filtering and control
- **Private DNS**: Internal name resolution

**Service Security**
- **Managed Identity**: Eliminate stored secrets
- **Key Vault Integration**: Secure secret management
- **TLS Encryption**: End-to-end encryption in transit
- **At-Rest Encryption**: Azure service native encryption

## Scalability Architecture

### Horizontal Scaling Design

**Stateless Application**
- **Session Storage**: Externalized to Azure Redis Cache
- **No Local State**: All persistent data in external services
- **Load Balancer**: Azure App Service built-in load balancing
- **Health Checks**: Application health monitoring

**Auto-scaling Configuration**
```
App Service Scaling:
├── CPU-based scaling (70% threshold)
├── Memory-based scaling (80% threshold)
├── Request queue scaling
└── Custom metrics scaling

Database Scaling:
├── Cosmos DB autoscale (RU/s based)
├── AI Search replicas (query performance)
├── AI Search partitions (storage capacity)
└── Cache scaling (memory and connections)
```

### Performance Optimization

**Caching Strategy**
- **Application Cache**: Redis for session and temporary data
- **Search Cache**: AI Search query result caching
- **CDN Integration**: Static asset delivery optimization
- **Browser Caching**: Client-side caching headers

**Database Optimization**
- **Partition Strategy**: Efficient data distribution
- **Index Optimization**: Query-specific indexing
- **Connection Pooling**: Efficient connection management
- **Query Optimization**: Minimized RU consumption

## Integration Architecture

### External Service Integration

**API-First Design**
- **REST APIs**: Standard HTTP/JSON interfaces
- **Authentication**: Bearer token and Managed Identity
- **Rate Limiting**: Built-in throttling and retry logic
- **Error Handling**: Comprehensive error responses

**Extensibility Points**
```
Integration Capabilities:
├── Custom AI Models
│   └── Bring your own model endpoints
├── External Data Sources
│   └── Custom document connectors
├── Workflow Integrations
│   └── Business process automation
└── Reporting & Analytics
    └── Custom dashboard integration
```

### Monitoring and Observability

**Application Insights Integration**
- **Performance Monitoring**: Request/response tracking
- **Error Tracking**: Exception and failure analysis
- **User Analytics**: Usage patterns and behavior
- **Custom Telemetry**: Business-specific metrics

**Azure Monitor Integration**
- **Resource Health**: Service availability monitoring
- **Cost Monitoring**: Resource usage and cost tracking
- **Security Monitoring**: Audit log analysis
- **Alerting**: Proactive issue notification

## Deployment Architectures

### Single-Region Deployment

**Standard Configuration:**
- All services deployed in single Azure region
- VNet integration for private networking
- Backup and disaster recovery within region
- Suitable for most enterprise deployments

**Benefits:**
- Lower latency between components
- Simplified networking configuration
- Reduced cross-region data transfer costs
- Easier compliance with data residency requirements

### Multi-Region Deployment

**Global Distribution:**
- Primary and secondary region deployments
- Cross-region replication for data services
- Traffic manager for intelligent routing
- Disaster recovery and business continuity

**Considerations:**
- Increased complexity and cost
- Data synchronization challenges
- Network latency for cross-region calls
- Compliance with data sovereignty requirements

## Design Patterns and Best Practices

### Microservices Principles

**Service Separation**
- **Document Processing**: Independent processing pipeline
- **Search Service**: Dedicated search and retrieval
- **Chat Service**: Conversation management and AI integration
- **Admin Service**: Configuration and management APIs

**Communication Patterns**
- **Async Processing**: Message queues for long-running operations
- **Event-Driven**: Event-based service communication
- **Circuit Breakers**: Fault tolerance for external dependencies
- **Retry Logic**: Resilient service interactions

### Data Consistency Patterns

**Eventually Consistent**
- Document processing and search indexing
- Cross-service data synchronization
- User preference replication

**Strongly Consistent**
- User authentication and authorization
- Configuration changes
- Critical business operations

## Technology Choices and Rationale

### Azure Services Selection

**Why Azure OpenAI?**
- Enterprise-grade AI with Azure security controls
- Private deployment options for sensitive data
- Integration with Azure ecosystem
- Compliance with enterprise requirements

**Why Cosmos DB?**
- Global distribution capabilities
- Flexible schema for evolving data models
- Built-in scaling and performance
- Strong consistency options when needed

**Why AI Search?**
- Hybrid search capabilities (vector + keyword)
- Built-in semantic search features
- Integration with Azure AI services
- Scalable search infrastructure

### Framework and Language Choices

**Python/Flask**
- Rich AI and ML library ecosystem
- Rapid development and iteration
- Strong Azure SDK support
- Enterprise-ready deployment options

**React/TypeScript Frontend**
- Modern, responsive user interface
- Strong typing for maintainability
- Rich component ecosystem
- Mobile-responsive design capabilities

This architecture provides a solid foundation for understanding how Simple Chat components work together to deliver secure, scalable, and intelligent conversational AI capabilities.