Saarthi
An end-to-end intelligent serverless computing platform integrating an ML-based request scheduler, custom gateway, load balancer, and autoscaler. Achieves 1.84× cost reduction and 97.2% SLA compliance benchmarked against AWS Power Tuning and standard Kubernetes HPA.
Performance
Benchmarked against AWS Power Tuning, standard HPA, and baseline OpenFaaS.
Architecture
Prediction Layer
Advanced ML models forecast workload patterns and resource requirements ahead of arrival, enabling proactive warm-up and eliminating cold starts under predictable traffic.
Optimisation Layer
Integer Linear Programming (ILP) solver computes mathematically optimal resource allocation, maximising utility subject to SLA and capacity constraints.
Execution Layer
Custom OpenFaaS gateway with idle-first pod selection, real-time request routing, and Kubernetes Custom Resource Definitions (CRDs) for function lifecycle management.
Monitoring Layer
Prometheus-backed multi-dimensional telemetry with cost analytics, per-function SLA tracking, and real-time rebalancing triggers.
Technical Implementation
Core Technologies
- Kubernetes-native deployment with Custom Resource Definitions
- OpenFaaS with custom gateway and provider extensions
- Integer Linear Programming (ILP) for resource allocation
- Prometheus & Grafana for telemetry collection and dashboards
- Multi-node Kubernetes clusters (5–20 nodes) on Melbourne Research Cloud (NeCTAR) and AWS
Advanced Capabilities
- Idle-first pod selection for minimal cold start overhead
- Predictive autoscaling independent of standard HPA
- Sub-millisecond scheduling inference at production load
- Validated against experimental benchmarks
Publication
Saarthi: An End-to-End Intelligent Platform for Optimising Distributed Serverless Workloads
Agarwal, S. et al. — arXiv:2511.06599
View on arXiv