freeleaps-ops/apps/gitea-webhook-ambassador-python/README.md

# Gitea Webhook Ambassador (Python)

A high-performance Python webhook service for connecting Gitea and Jenkins, supporting intelligent dispatch, high concurrency processing, and deduplication strategies.

## 🚀 Features

### 1. Intelligent Dispatch Strategy
- **dev branches** → Trigger alpha environment build
- **prod branches** → Trigger production environment build
- **other branches** → Configurable default strategy

### 2. High Concurrency Processing
- **Asynchronous Task Queue**: Use Celery + Redis for high concurrency
- **Task Queueing Mechanism**: Prevent build loss, ensure tasks are executed in order
- **Load Balancing**: Support multiple worker instances

### 3. Deduplication Strategy
- **Deduplication based on commit hash + branch**: Prevent repeated triggers
- **Time Window Deduplication**: Only trigger once for the same commit within a specified time window
- **Intelligent Deduplication**: Support configurable deduplication strategies

## 🏗️ Architecture Design

```
Gitea Webhook → FastAPI → Celery Queue → Jenkins Workers
     ↓              ↓           ↓            ↓
Signature Verification  Routing Dispatch  Task Queueing  Concurrent Execution
     ↓              ↓           ↓            ↓
Deduplication   Environment Judgment  Persistent Storage  Status Feedback
```

## 📁 Project Structure

```
gitea-webhook-ambassador-python/
├── app/
│   ├── __init__.py
│   ├── main.py                 # FastAPI application entry
│   ├── config.py               # Configuration management
│   ├── models/                 # Data models
│   │   ├── __init__.py
│   │   ├── gitea.py           # Gitea webhook model
│   │   └── jenkins.py         # Jenkins job model
│   ├── services/              # Business logic
│   │   ├── __init__.py
│   │   ├── webhook_service.py # Webhook processing service
│   │   ├── jenkins_service.py # Jenkins integration service
│   │   ├── queue_service.py   # Queue management service
│   │   └── dedup_service.py   # Deduplication service
│   ├── api/                   # API routes
│   │   ├── __init__.py
│   │   ├── webhook.py         # Webhook endpoint
│   │   ├── health.py          # Health check
│   │   └── admin.py           # Admin interface
│   ├── core/                  # Core components
│   │   ├── __init__.py
│   │   ├── security.py        # Security validation
│   │   ├── database.py        # Database connection
│   │   └── cache.py           # Cache management
│   └── tasks/                 # Celery tasks
│       ├── __init__.py
│       └── jenkins_tasks.py   # Jenkins task processing
├── tests/                     # Test files
├── docker/                    # Docker configuration
├── requirements.txt           # Python dependencies
├── docker-compose.yml         # Development environment
└── README.md
```

## 🛠️ Tech Stack

- **Web Framework**: FastAPI
- **Task Queue**: Celery + Redis
- **Database**: PostgreSQL (production) / SQLite (development)
- **Cache**: Redis
- **Monitoring**: Prometheus + Grafana
- **Logging**: Structured logging with JSON
- **Testing**: pytest + pytest-asyncio

## 🚀 Quick Start

### 1. Install Dependencies
```bash
pip install -r requirements.txt
```

### 2. Configure Environment
```bash
cp .env.example .env
# Edit the .env file to configure Jenkins and database connections
```

### 3. Start Service
```bash
# Start Redis
docker run -d -p 6379:6379 redis:alpine

# Start Celery worker
celery -A app.tasks worker --loglevel=info

# Start FastAPI application
uvicorn app.main:app --reload --host 0.0.0.0 --port 8000
```

## 📋 Configuration

### Environment Dispatch Configuration
```yaml
environments:
  dev:
    branches: ["dev", "develop", "development"]
    jenkins_job: "alpha-build"
    jenkins_url: "https://jenkins-alpha.example.com"
  prod:
    branches: ["prod", "production", "main", "master"]
    jenkins_job: "production-build"
    jenkins_url: "https://jenkins-prod.example.com"
  default:
    jenkins_job: "default-build"
    jenkins_url: "https://jenkins-default.example.com"
```

### Deduplication Configuration
```yaml
deduplication:
  enabled: true
  window_seconds: 300  # 5-minute deduplication window
  strategy: "commit_branch"  # commit_hash + branch
  cache_ttl: 3600  # Cache for 1 hour
```

### Queue Configuration
```yaml
queue:
  max_concurrent: 10
  max_retries: 3
  retry_delay: 60  # seconds
  priority_levels: 3
```

## 🔧 API Endpoints

### Webhook Endpoint
```
POST /webhook/gitea
```

### Health Check
```
GET /health
GET /health/queue
GET /health/jenkins
```

### Admin Endpoints
```
GET /admin/queue/status
GET /admin/queue/stats
POST /admin/queue/clear
```

## 🧪 Testing

```bash
# Run all tests
pytest

# Run specific test
pytest tests/test_webhook_service.py

# Run performance test
pytest tests/test_performance.py
```

## 📊 Monitoring Metrics

- Webhook receive rate
- Task queue length
- Jenkins build success rate
- Response time distribution
- Deduplication hit rate

## 🔒 Security Features

- Webhook signature verification
- API key authentication
- Request rate limiting
- Input validation and sanitization
- Secure logging