How to take advantage of great language models without breaking the bank
Access our on-demand library to view VB Transform 2023 sessions. Sign up here
Generative AI continues to make headlines. At its beginnings, we were all taken by the novelty. But now we're way beyond fun and games - we're seeing its real impact on business. And everyone dives head first.
MSFT, AWS, and Google have embarked on an "AI arms race" in a bid to dominate. Companies are rushing to pivot for fear of being left behind or missing out on a huge opportunity. New LLM-powered companies are emerging by the minute, fueled by VCs chasing their next bet.
But with every new technology comes challenges. The truth and bias of the model and the cost of training are among the topics of the day. Identity and security, though related to the misuse of patterns rather than inherent technology issues, are also starting to make headlines.
The cost of operating models, a major threat to innovationGenerative AI is also bringing back the good old debate between open source and closed source. While both have their place in the enterprise, open source offers lower deployment and release costs. They also offer great accessibility and a large selection. However, we now see an abundance of open source models, but not enough technological advancement to deploy them in a viable way.
EventVB Transform 2023 on demand
Did you miss a session of VB Transform 2023? Sign up to access the on-demand library for all of our featured sessions.
Register nowAll that aside, there is one issue that still needs much more attention: the cost of running these large models in production (inference costs) poses a major threat to innovation. Generative models are exceptionally large, complex, and computationally intensive, making them much more expensive to run than other types of machine learning models.
Imagine you're creating a home decor app that helps customers imagine their room in different design styles. With a few tweaks, the Stable Diffusion model can do this relatively easily. You go with a service that charges $1.50 for 1,000 images, which might not seem like much, but what if the app goes viral? Let's say you get 1 million daily active users who create ten images each. Your inference costs are now $5.4 million per year.
LLM Cost: Inference is EternalNow, if you are a company deploying a generative model or an LLM as the backbone of your application, your...
Access our on-demand library to view VB Transform 2023 sessions. Sign up here
Generative AI continues to make headlines. At its beginnings, we were all taken by the novelty. But now we're way beyond fun and games - we're seeing its real impact on business. And everyone dives head first.
MSFT, AWS, and Google have embarked on an "AI arms race" in a bid to dominate. Companies are rushing to pivot for fear of being left behind or missing out on a huge opportunity. New LLM-powered companies are emerging by the minute, fueled by VCs chasing their next bet.
But with every new technology comes challenges. The truth and bias of the model and the cost of training are among the topics of the day. Identity and security, though related to the misuse of patterns rather than inherent technology issues, are also starting to make headlines.
The cost of operating models, a major threat to innovationGenerative AI is also bringing back the good old debate between open source and closed source. While both have their place in the enterprise, open source offers lower deployment and release costs. They also offer great accessibility and a large selection. However, we now see an abundance of open source models, but not enough technological advancement to deploy them in a viable way.
EventVB Transform 2023 on demand
Did you miss a session of VB Transform 2023? Sign up to access the on-demand library for all of our featured sessions.
Register nowAll that aside, there is one issue that still needs much more attention: the cost of running these large models in production (inference costs) poses a major threat to innovation. Generative models are exceptionally large, complex, and computationally intensive, making them much more expensive to run than other types of machine learning models.
Imagine you're creating a home decor app that helps customers imagine their room in different design styles. With a few tweaks, the Stable Diffusion model can do this relatively easily. You go with a service that charges $1.50 for 1,000 images, which might not seem like much, but what if the app goes viral? Let's say you get 1 million daily active users who create ten images each. Your inference costs are now $5.4 million per year.
LLM Cost: Inference is EternalNow, if you are a company deploying a generative model or an LLM as the backbone of your application, your...
What's Your Reaction?
