Async Task Queue for GenAI Inference

Data & AI Pipelines diagram · made in Skemio

Architecture diagram showing a Web Client sending requests to FastAPI, enqueuing tasks in Redis, executed by Celery GPU workers, and pushed back via WebSockets.

Open it in Skemio Make your own

What it shows

API gateways return instant task IDs, freeing up client HTTP threads.
Redis queues buffer spikes to protect GPU workers from out-of-memory crashes.
Workers stream generated tokens into Redis and push updates via WebSockets.

The takeaway

Decoupling GenAI requests with async queues protects expensive GPUs from connection drops.

#SystemDesign
#FastAPI
#Redis
#GenerativeAI
#SoftwareArchitecture

More data & ai pipelines diagrams