Video QA Platform
An AI-powered video processing platform that allows users to upload instructional videos and ask questions about them using natural language. Features multimodal search across video content, frame analysis, and intelligent chat interface.
Features
- Video Upload: Drag-and-drop interface with real-time processing status
- AI Chat Interface: Ask questions about uploaded videos with streaming responses
- Multimodal Search: Search across transcripts, frames, and uploaded images
- Frame Analysis: Automatic scene detection and frame extraction with AI vision
- Authentication: Simple demo authentication system
- Real-time Processing: Live status updates during video processing
System Architecture
+-------------+ +--------------+ +-------------+
| Browser |----->| Next.js App |----->| PostgreSQL |
| | | (video-qa) | | + pgvector |
+-------------+ +--------------+ +------+------+
| |
| writes | polls
| video + | jobs
| job |
V V
+-------------+ +--------------+
| data/ | | Worker |
| uploads/ |<-----| (video- |
| processed/ | | worker) |
| frames/ | +--------------+
+-------------+
Quick Start
Prerequisites
- Node.js 18+ and pnpm
- Docker and Docker Compose
- OpenAI API key
1. Setup Environment
# Clone both repositories
git clone <video-qa-repo>
git clone <video-worker-repo>
# Create environment file
cd video-qa
cat > .env.local << EOF
DATABASE_URL=postgresql://postgres:postgres@localhost:5432/videoqa
OPENAI_KEY=your_openai_api_key_here
EOF
git clone <video-qa-repo>
git clone <video-worker-repo>
# Create environment file
cd video-qa
cat > .env.local << EOF
DATABASE_URL=postgresql://postgres:postgres@localhost:5432/videoqa
OPENAI_KEY=your_openai_api_key_here
EOF
2. Build and Start Services
# Build worker image
cd ../video-qa-worker
docker build -t videoqa-worker:0.0.19 .
# Start database and worker
cd ../video-qa
docker-compose up -d
# Start Next.js app
pnpm install
pnpm dev
cd ../video-qa-worker
docker build -t videoqa-worker:0.0.19 .
# Start database and worker
cd ../video-qa
docker-compose up -d
# Start Next.js app
pnpm install
pnpm dev
3. Login and Upload
- Open http://localhost:3000
- Login with demo credentials:
demo/demo123 - Upload a video file (max 500MB)
- Monitor processing status
- Ask questions about your video
How It Works
Upload Flow
- File Upload: Video saved to
data/uploads/{id}_{name}.mp4 - Database: Metadata stored in
videostable withoriginal_path - Job Queue: Processing job created in
jobstable - Worker: Polls for jobs using
FOR UPDATE SKIP LOCKED
Processing Pipeline
The worker processes videos through 6 stages:
Input: uploads/{id}_{name}.mp4
|
+-> [1. NORMALIZE] - processed/{id}/normalized.mp4
| - processed/{id}/audio.wav
|
+-> [2. TRANSCRIBE] - transcript_segments table
| - subs/{id}.srt
|
+-> [3. SCENES] - scenes table (t_start, t_end)
|
+-> [4. FRAMES] - frames/{id}/scene_*.jpg
| - frames table (phash, path)
|
+-> [5. VISION] - frame_captions table (caption, entities)
|
+-> [6. EMBEDDINGS] - UPDATE embeddings (1536-dim vectors)
Output: video.status = 'ready'