Tech Stack
Python
FastAPI
Redis
PostgreSQL
Docker
Linux
Git
Description
Async worker service for the multi-model AI chat platform. Consumes inference tasks from RabbitMQ, loads conversation context from Redis (with PostgreSQL fallback), calls Ollama, and writes results back to Redis and the database.
Flow: gateway publishes task → worker picks up → builds context → Ollama inference → assistant reply persisted → task status set to completed → frontend poll succeeds.
- Configurable retries (MAX_RETRIES=3) before dead-letter queue chat_tasks.dlq.
- Write-through caching: updates PostgreSQL and Redis (last 50 messages, 1-hour TTL).
- Environment-driven: RABBITMQ_URL, REDIS_URL, DATABASE_URL, OLLAMA_URL, prefetch and timeout tuning.
- Docker Compose stack for local dev with RabbitMQ, Redis, and Ollama (ollama pull llama3).
- Python 3.12+ with uv; structured logging and graceful shutdown.
Page Info
Worker pipeline
Consumes chat_tasks, builds context from Redis (PostgreSQL fallback), calls Ollama generate, write-through cache update.
