Choosing the Right Compute Model: Serverless Functions, Serverless Containers, and Managed Containers
The Rise of Serverless Function: A Silver Bullet or a Double-Edged Sword?
With the time to market for products and applications decreasing drastically in recent years, serverless computing has emerged as a popular way to build and deploy applications. It allows companies to build and maintain lean teams and focus on the core business processes rather than spending considerable time and resources on infrastructure when you're focus should be on looking for product-market fit.
Coupled with the ability to host applications for pennies, and frameworks like Next.js heavily pushing for serverless technologies like AWS Lambda and Cloudflare Workers, function-based technologies have now earned a place in a lot of these companies' tech stacks. However, I've also seen a lot of developers who bought into the promise of serverless caught off guard when they encounter 5-10 seconds cold starts right after launching their MVPs or when they see unexpectedly high costs or function timeouts when doing long running operations.
So which compute model is best? Does this mean serverless or Lambda is bad? Not necessarily. In fact, serverless functions, particularly Lambda, excel in handling sporadic, event-driven workloads. They allow applications to scale rapidly from zero to thousands of requests almost instantly. However, they come with trade-offs, and the real issue arises when developers blindly follow the hype without considering factors such as the performance characteristics or price characteristics of their architecture and application along with their specific workload or traffic patterns.
A Product Lifecycle Approach to Compute Models
There is no silver bullet. Each compute model has its own strengths and weaknesses. Generally, compute models fall into three main categories: serverless functions, serverless containers, and managed containers. These compute models each have trade-offs, and their suitability depends on where your product is in its lifecycle. A product typically progresses through multiple stages such as MVP, growth, and maturity. Making the right decision comes down to identifying the stage your product is in, understanding priorities, available resources, workloads, traffic patterns, and selecting the compute model that best aligns with your current needs.
Often, products transition across multiple compute models over time—and this is often a good approach, as we want to ensure that we're using the right tool for the right job, or in this case the right tool for the right time.
But does this mean we have to rewrite our entire application every time we switch compute models?
No. Ideally, we want to design our applications in a way that they are modular and loosely coupled. This allows us to swap out components or services without having to rewrite the entire application. There are approaches like Domain-Driven Design (DDD) that can help us write out business logic in a way that it's not tied to a specific interface (REST, GraphQL, CLI) or infrastructure.
Along with containerization now being supported by most services including lambda (though its cold starts for containerized deployments have been reported as problematic), it's now easier to move from one compute model to another without having to rewrite the entire application.
1. Serverless Functions (Function-as-a-Service, FaaS)
Serverless functions are best for early-stage applications, MVPs, and workloads with low or inconsistent traffic patterns. They are particularly useful when speed to market and minimizing infrastructure overhead are top priorities.
Key Characteristics:
- Best suited for sporadic, spiky, or unpredictable traffic
- AFAIK, nothing can scale as fast as lambda so if you need to go from 0 to 1000 in a few seconds, lambda is your best bet.
- Pay-per-use billing (per request and duration)
- Cold starts can impact performance
- In most cases nothing else matters except cold starts. Throughput is largely irrelevant.
- Each invocation is isolated
Isolation
I'd like to dive deeper into the isolation part as this is something that's a bit controversial in my opinion.
On one hand, many advocates highlight isolation as one of the strongest benefits of serverless functions. Each invocation is isolated from the others so this means that if one function invocation fails, it doesn't affect the others. This is great for fault tolerance (which is something that's generally harder to achieve unless you're using a language specifically designed for it like Elixir) and security. This also means that you can't have one function hogging all the resources.
However, this also means that you can't leverage language-level concurrency. For example, in Node.js, you can't have a function that's processing multiple requests at the same time. And this is a big deal because one of the biggest strengths of Node.js despite being single-threaded is it's high concurrency capabilities for I/O-bound workloads which if you haven't realized yet is what most web applications are these days (database queries and API calls combined with a bunch of business logic). Basically during these I/O operations, instead of Node.js processing other requests, it will just wait for the I/O operation to finish, where you are paying for the time the CPU is just there idle.
Use Cases:
- Proof of Concept (POC) and MVPs
- Event-driven architectures (e.g., API Gateway triggers, IoT events, file processing)
- Latency-sensitive workloads (e.g., real-time notifications, lightweight APIs)
- CPU intensive workloads: Think SSR with high Core Web Vitals requirements
Trade-offs:
| Pros | Cons |
|---|---|
| No infrastructure management | Execution time limits (e.g., max 15 min) |
| Pay only for execution time | Cold starts impact latency |
| Auto-scales instantly | Limited to stateless execution |
| Ideal for bursty traffic | Cannot utilize language concurrency effectively |
2. Serverless Containers
Note: When I initially mulled over this, I was mostly thinking of GCP's Cloud Run which has a very similar pricing model to lambda (per request and execution time). However, different services from different cloud providers have different pricing models so what I'm about to say might not be 100% accurate for all services. For example, fly.io, and app runner doesn't have a per request aspect in it's pricing model (which is a good thing since it gives us more options). That being said, the general idea should still hold.
Serverless containers are a more flexible alternative to serverless functions. They allow developers to use more traditional frameworks, reduce cold start impact, and support higher concurrency per instance.
Key Characteristics:
- Lower costs for moderate traffic due to concurrency
- Does not scale as fast as serverless functions
- Higher cold starts, but impact is reduced when handling concurrent requests
- Higher cold starts compared to serverless functions either because the overhead of spinning a container is just higher overall or firecracker is just that optimized for cold starts.
- Portable—can easily move to a different service if needed
Correlation Between Concurrency and Cost
Building on the topic of concurrency which was touched on in the previous section, I'd like to dive deeper into how concurrency can reduce costs when compared to serverless functions.
When you have a serverless function, you are billed based on the duration of the function execution and number of invocations among other things. This means that if you have a function that runs for 1 second, you are billed for 1 second. If you have a function that runs for 1 second but you have 10 requests coming in at the same time, you are billed for 10 seconds. This is because each request is handled by a separate instance of the function.
On the other hand, with serverless containers, assuming you are still billed based on both the duration of the container execution and the number of invocations, concurrency allows you to drastically reduce the cost of the duration aspect of the billing. If you have a container that runs for 1 second and can handle 10 concurrent requests during that 1 second, you are only billed for that 1 second duration instead of 10 seconds. This means we are able to squeeze more value out of the same vCPU and memory we're paying for at a given time, assuming you even have that many users in the first place.
This of course depends on the concurrency model of the language you're using, but as mentioned earlier in the Node.js example, most web applications these days have a lot or at least some I/O operation included in most requests which means that we should be able to take advantage of that to reduce costs. This is of course not limited to Node, as any highly concurrent language like Elixir should be able to benefit from this.
Finally, I should mention that concurrency is not a one-size-fits-all solution. Two applications using the same language and framework can have different concurrency characteristics, mostly depending on the workload it's running. For example, a Nodejs application that's mostly doing CPU-bound operations like image processing might not benefit as much from concurrency as a Nodejs application that's mostly doing I/O-bound operations like database queries and API calls. This is because Nodejs is a single-threaded language and CPU-bound operations can block the event loop.
Using Full-Featured Frameworks
One major advantage of serverless containers is that they make using batteries-included frameworks like Phoenix, NestJS, Spring Boot, Ruby on Rails, and Django a viable option compared to serverless functions.
Yes, these frameworks are heavier and have longer cold starts, but only during the first request. If a container can handle 100 concurrent requests, that means you only get 1 cold start for every 100 requests—which is not that bad.
In return, you gain powerful built-in features that can save significant development time and effort, helping you build more robust applications without needing to reimplement common functionality from scratch.
Portability
One of the biggest advantages of containers is that they are portable. You can run the same container on your local machine, on your server, on your friend's server, or even on your cloud provider’s server—without making any changes.
This portability also means that you’re not locked into a single cloud provider. If you ever outgrow your serverless container service, you can easily migrate to a managed container service like Google Kubernetes Engine (GKE) or Amazon Elastic Kubernetes Service (EKS) without having to rewrite your entire application.
Use Cases:
- Low-to-mid traffic web applications & APIs
- Microservices that require custom runtime environments
- Workloads where concurrency reduces cost (e.g., Elixir, Node.js for I/O-bound workloads)
Trade-offs:
| Pros | Cons |
|---|---|
| Concurrency reduces cost | Worse cold starts |
| Portability across cloud providers | Not as fast scaling as Lambda |
| Better suited for heavy-duty frameworks | Performance depends on workload type |
| Cold starts are amortized over many requests |
3. Managed Containers (Not Fully Serverless)
Managed containers provide predictable performance and are cost-effective for high, sustained traffic applications. Unlike serverless, which charges per request and execution time, managed containers typically charge for reserved compute instances. While costly for low traffic, this model becomes more cost-efficient for applications running at high, sustained loads.
When applications outgrow the limitations of serverless functions and containers, managed containers provide greater control over scaling, networking, and long-running processes.
Key Characteristics:
- Persistent resources—not scale-to-zero
- Auto-scaling requires manual configuration
- Best for predictable workloads or traffic patterns
- Fewer limitations compared to serverless
Use Cases:
- High-traffic applications with predictable workloads or traffic patterns
- Services requiring fine-tuned networking and runtime configurations
- Long-running operations and batch processing
- Websockets
Trade-offs:
| Pros | Cons |
|---|---|
| More control over networking & scaling | Cannot scale to zero |
| Suitable for sustained high traffic | Higher cost for low-traffic applications |
| No execution time limits | Requires manual scaling configurations |
While they offer greater control, managed containers often require more DevOps expertise to configure networking, storage, and scaling which can result in a higher total cost of ownership.
Comparison Summary: When to Choose What?
| Feature | Serverless Functions | Serverless Containers | Managed Containers |
|---|---|---|---|
| Scaling | Auto (instant) | Auto (scale-to-zero) | Auto (manual tuning) |
| Cost Efficiency | Best for sporadic workloads | Cost-efficient at moderate traffic | Best for high, predictable workloads |
| Startup Time | Low Cold Starts | Higher cold starts | No cold starts |
| Concurrency | Low (1 per function instance) | Moderate to high | High |
| Best for | MVPs, event-driven tasks | APIs, microservices, moderate workloads | Predictable workloads at scale, websockets, long-running operations |
Final Thoughts: Choosing the Right Compute Model
A product often moves through different compute models over time, and this is completely normal. In fact, you'll find many teams use a hybrid approach—using different compute models at the same time depending on the workload. It's always good to evaluate compute models based on your product’s stage, workloads, and cost considerations:
- Start with serverless functions for MVPs, quick iterations, and sporadic workloads.
- Transition to serverless containers when you need better concurrency management or reduce cold start issues.
- Move to managed containers when you require high, predictable traffic scaling and better cost optimization at scale.
By understanding these principles, you can design scalable, cost-effective architectures that align with your business needs.