DEVOPSdigest

Fastly announced the launch of Fastly AI Accelerator, the company’s first AI solution designed to create a better experience for developers by helping improve performance and reduce costs across the use of similar prompts for large language models (LLM) apps.

Fastly AI Accelerator is designed to reduce API calls and costs with intelligent, semantic caching. Built on Fastly’s Edge Cloud Platform and leveraging industry leading caching technology, AI Accelerator uses a specialized API gateway to dramatically improve performance for apps using popular LLMs, beginning with ChatGPT and expanding support to include additional models.

Fastly AI Accelerator’s semantic caching provides a cached response for repeated queries directly from Fastly’s high performance edge platform, instead of going back to the AI provider, helping to deliver a better experience by improving performance while reducing costs.

“At Fastly, we’re always listening to developers, to understand both what they’re excited about and what their biggest pain points are,” said Anil Dash, Vice President of Developer Experience with Fastly. “Fastly AI Accelerator gives developers exactly what they want, by making the experience of their favorite LLMs a lot faster and more efficient, so they can focus on what makes their app or site unique, and what keeps their users happy.”

When using Fastly AI Accelerator, developers only need to update their app to use a new API endpoint, which typically only requires changing a single line of code. Fastly AI Accelerator will then transparently implement semantic caching for OpenAI compatible APIs. This approach goes beyond traditional caching as Fastly AI Accelerator is able to understand the context of the requests and queries, and will send a similar response if two or more requests are alike.

To help developers build faster, more secure and more engaging experiences, Fastly is also now making it even easier for developers to try Fastly with an expanded free account tier that helps coders set up a new site, create a new app, or launch a new service in just a few minutes. Free tier accounts also include access to Fastly’s Content Delivery Network (CDN), generous memory and storage allotments, uncapped redirects, page rules and regular expressions. Plus, the free Fastly tier includes security features such as TLS and always-on DDoS mitigation, observability tools, and much more.