Trailblazing AI answer engine Perplexity.AI doubles throughput, cuts cost with Azure AI Studio

Far more than they bargained for

Aravind Srinivas and his partners didn’t realize at first that they were creating a groundbreaking, AI-based conversational answer engine. They thought they were creating an internal tool to assist their own company in its growth. Then the team saw what it could do—and realized what it could do for others.

“We were a small startup looking to do cool things with large language models,” says Srinivas, Chief Executive Officer and Cofounder of Perplexity.AI. “And we created a service for ourselves, a personal assistant always ready to answer the questions that any business faces and have intelligent conversations about those answers. Not a search engine, because that just gives you links that you have to explore on your own. We wanted a conversational answer engine that combines traditional search with generative AI and large language models to deliver fully up-to-date answers highly customized for each user, complete with a citation to back up every fact.”

Once Srinivas and his colleagues had the vision to “democratize AI” by commercializing their tool, Perplexity Ask was born. Now they had a new range of issues to address. They needed a technology platform that wouldn’t just scale seamlessly to support millions of users but would do so cost-effectively. It was a keen concern for a startup that was in no position to build, maintain, and continually expand a hardware infrastructure. 

If their answer engine was to survive and thrive in the highly competitive marketplace, the Perplexity.AI team needed to deliver the utmost in reliability, security, and trust. They envisioned extended experimentation with language models to continually refine their engine. So, the team also needed their platform and tools to be a force multiplier for the lean, three-person staff they started with. Their goal was to boost productivity, both to enable Perplexity.AI to get to market quickly and to enable rapid continual development.

“We wanted a sophisticated platform that would be simple to use,” explains Denis Yarats, Chief Technology Officer and Cofounder of Perplexity.AI. “We didn’t want to waste time and money reinventing the wheel.”

A mature platform meets all needs

The emergence of large language models and OpenAI made it possible for Perplexity.AI to begin work on Perplexity Ask. However, the company found that OpenAI didn’t fully meet its requirements. The Perplexity.AI team considered its options and decided to also use Azure AI Studio and Azure OpenAI Service. 

Azure AI Studio aligned with Perplexity.AI’s requirements, and the company saw Azure as a mature, reliable, well-known platform with enterprise-grade compliance and a reputation that would help sell Perplexity Ask to potential customers. The Perplexity.AI team also liked that Azure AI Studio provides extensive support for fine-tuning APIs and provides easy access to ChatGPT 3.5 and 4.0 and other large language models. In addition, Microsoft provided the company with early access to Azure AI models.

“Azure AI Studio improved the experience for creating AI products,” says Yarats. “We found it mapped perfectly to our needs for faster development and time to market, and greater throughput, scalability, security, and trust.”

Perplexity Ask supports research, solves problems, debugs code, and provides both straightforward and complex information. It also guides users through interactive follow-up questions for deeper, more accurate results. About 70 percent of Perplexity Ask users are in the business sector. The rest are largely researchers and students.

Fast time to market with a lean staff and startup funding

Perplexity.AI brought its answer engine to market in just six months—even with a lean staff and tight, startup funding—with help from Azure AI Studio. 

“Trying out large language models available with Azure OpenAI Service was easy, with just a few clicks to get going,” recalls Yarats. “That’s an important differentiator of Azure AI Studio: we had our first prototype in hours. We had more time to try more things, even with our minimal headcount.”

As a result, Perplexity.AI ran twice as many experiments, in parallel, as it could before adopting Azure, and with less human intervention. Retraining new versions of the model was also twice as fast.

Massive scalability at a less-than-massive price

The answer engine currently serves 7 million users per month. That number requires a lot of compute capacity, but Perplexity.AI had no issues scaling to that volume on Azure AI Studio, according to Yarats. Adding ChatGPT 4.0 via Azure OpenAI Service doubled throughput from 300,000 tokens per minute to 600,000, with latency declining by 30 percent. That increase delivers a faster, more satisfying user experience while enabling Perplexity Ask to serve more users. And with the favorable economics of Azure, the cost per token actually declined, making the higher throughput highly cost-effective. 

Perplexity.AI also got greater reliability and lower latency by choosing the subscription model that best met its needs. The company uses Provision Throughput Units (PTUs), which provide dedicated capacity and more effective caching to foster lower latency even during periods of high use. 

“The ability to cache user questions makes a day-and-night difference in scalability,” says Lauren Yang, AI Product Manager at Perplexity.AI. “It also makes our answer engine more cost-effective.”

An instant reputation for security, privacy, and trust

Security, privacy, and trust were always top of mind for Perplexity.AI. “Given the information that users provide through their queries, they need to know that Perplexity Ask is highly secure,” says Yarats. “We know—and our customers know—that Azure has that level of security. We don’t have to say more after we say we run on Azure.”

Srinivas and his colleagues anticipate further enhancing and improving Perplexity Ask while supporting a large and growing body of users. Srinivas also takes a moment to reflect: “We were able to build Perplexity Ask because we made the decision to move to Azure,” he says. “Our mission is to empower everyone through the democratization of AI. Azure AI Studio makes that possible.”

Find out more about Perplexity.AI on Twitter and LinkedIn.