An AI model is heavy and hungry. Served carelessly, it blows up response times and the server bill. On an audio-analysis SaaS that I designed and shipped solo, it had to stay fast, secure and economically viable.
The hosting decision. I served the model on a server I run, with NGINX and automatic file cleanup, rather than on usage-billed infrastructure. It means more deployment work, but the bill no longer follows the number of analyses. It can be forecast.
On the code side, two rules matter for a service like this.
1. Load the model once, at startup. Not on every request. With FastAPI, that's what the lifespan parameter is for. The application doesn't receive requests until startup is finished.
from contextlib import asynccontextmanager from fastapi import FastAPI
models = {}
@asynccontextmanager async def lifespan ( app : FastAPI ): models [ " main " ] = load_model () # slow: done once yield models .…