GenAI Apps: Not a walk in the park: The balance India needs to maintain between open-source and closed-source GenAI models

Silicon Valley has recently been embroiled in a lobbying battle between… large technology companies including GoalOn the one hand, Mistral and IBM advocate an “open science” approach to AI development, putting them at odds with rivals Anthropic, Microsoft and OpenAI, the maker of ChatGPT, who support closed software. Indeed, OpenAI is often the target of criticism on social media for not living up to its name.

But as India begins to emerge as the capital of use cases, GenAI ApplicationsTechnology leaders and startup founders believe that both technologies have a unique role to play in optimizing costs, ensuring data sovereignty, and delivering the best performance.

ET spoke to a representative group of startups and enterprises to understand the benefits and challenges of using open language versus natural language. Closed source GenAI models.

Implementation cost

By definition, open source software Source code is code that is available for everyone to use, modify, and distribute in the public domain. On the other hand, closed source code means that the source code is restricted to private use and users cannot modify it or create anything from it.

In the context of GenAI, while open models are free and flexible, their implementation cost can exceed that of closed ones, industry experts say.

“If you want to host an open model on-premises, tune it and customize it for the organization’s specific needs, there is a substantial infrastructure cost, LLMOps, lifecycle management and inference costs involved,” said Arun Chandrasekaran, distinguished analyst at Gartner. “Open source models, while often freely accessible, can incur high deployment and maintenance costs due to technical requirements,” said Rashid Khan, co-founder of conversational AI startup Yellow.ai.

In contrast, closed API models are typically optimized, well-maintained, and continually updated, which can save time and resources. “They offer a more complete solution with dedicated vendor support, ease of integration, and regular updates,” he said.

In fact, over the past 12 months, prices for closed assets LLM API has dropped by 65-90%, which is helping startups expand their margins, invest in research and development, improve performance, hire more talent and price their products competitively.

“Implementing and fine-tuning an open source model is a cost for the enterprise and for low-volume early stages or with some use cases that do not require 100% LLM availability, the preferred approach is to use paid APIs,” said Ankur Dhawan, President of Product and Technology at edtech company upGrad.

However, if an organization has the experience and resources to optimally manage the infrastructure and the technical skills to fine-tune it, it can eliminate the recurring costs associated with closed-model API access.

For example, Gnani.ai, which develops chatbots and voice bots for customer support, says the tokens consumed in a closed model often determine costs. “In contrast, if you have an open model fine-tuned with your own data, you may have less to worry about in terms of costs except for implementation,” said Ganesh Gopalan, co-founder and CEO of Gnani.ai.

Performance

AI researchers and scientists are divided on benchmarking performance. Stanford studies showed that closed-source models still outperform their open-source counterparts. But the general consensus suggests that open models are quickly catching up. The best closed models are improving, and the best open models are improving faster than they are.

“Previously, we had seen certain closed-source models, such as GPT-4o, leading the charge in terms of quality, but open-source models are quickly catching up. For example, the Llama 3.1 405B model recently announced by Meta is on par with other top models such as GPT-4o and Claude Sonnet 3.5,” said Baris Gultekin, AI Director at Snowflake, a US-based data cloud company that hosts top-of-the-line GenAI models.

Meta, which is leading the open source revolution with its Llama models, believes that organizations need to be more transparent in their assessments.

“At Meta we publish not only the benchmarks but also the methodology which goes beyond what many other proprietary vendors do,” said Ragavan Srinivasan, vice president of product management at Meta. “…people are not as open and transparent. There is only one scorecard. How do we evaluate this? Did you use 10 metrics? What are those 10 metrics? Are they all similar?”

When it comes to critical applications, especially with agent AI, controlling hallucinations and inaccuracies is a key priority for companies to preserve brand value.

“It’s a trade-off between cost and quality,” says Jonathan Frankle, chief AI scientist at Databricks, one of the most valuable AI companies. He added that cost takes many different forms. It can be speed or latency. “For coding assistants and real-time chatbots, speed is very important.”

“What we’ve seen over the last few months is that this trade-off has improved for everyone. For whatever amount of money you spend, you’re going to get a better model,” he said.

Data sovereignty and privacy

“When I talk to organizations, they often prefer open source models,” said Snowflake’s Gultekin. “Companies want to bring AI closer to their data, ensuring that the security and privacy of the data is respected. Where the model is run is really important.”

For example, banks are uncomfortable with their data being moved to a public cloud where these closed models are accessible, says Gartner’s Chandrasekaran.

“But tomorrow, if the same thing is installed on-premises through some licensing agreement, it could solve security or data privacy issues,” he added.

Gnani.ai, which works with BFSI and healthcare organizations, said that for mission-critical applications, customers deploy on private clouds or even on-premise servers for clients in regulated industries in both the US and India, Gopalan said.

In this case, open models are winning. There is an inherent limitation in the amount of transparency that can be provided with closed models, said Databricks’ Frankle.

“So, as Databricks, I love the idea of ​​being able to sit down with a customer and explain what’s going on in our systems and how everything works, and have the model weights available to them.”

“LLMs are black boxes, you cannot understand exactly why and how they do what they do,” said Dhawan of UpGrad. “Hence, the decision on which one to use will depend on other factors like maintenance, availability and running cost of the model. The trend will be similar to using the cloud to store data instead of on-premise.”

Orchestration layer

There is no one-size-fits-all solution. Therefore, organizations are experimenting with a multi-vendor strategy that combines cost, performance, and security to deliver the required results.

“Today, many CIOs and organizations are considering a multi-model strategy because they want to reduce dependency on a single vendor. But the fundamental question is how to align the right model for the right use cases,” said Gartner’s Chandrasekaran.

“So, if you have a model garden, you need an orchestration tool on the backend, which not only optimizes my costs but also delivers the best outcome through automated selection,” he said.

Microsoft Azure, the leading cloud services provider currently hosting over 1,600 open and closed models, states that “…factors such as cost, efficiency, latency, and accuracy make choosing the right model a critical step for enterprise adoption.”

“That’s why Microsoft is committed to giving our customers flexibility in how they can develop and deploy custom AI solutions, whether ready-to-use or fine-tuned, through Azure AI,” a company spokesperson said.

Google, which led the open software revolution with the Android operating system on millions of mobile devices, today finds itself at a crucial intersection between open and closed systems when it comes to GenAI. It has a family of open (Gemma) and closed (Gemini) models. In fact, in 2019, Google scientists co-authored the seminal paper introducing the Transformer architecture that today forms the basis of all GenAI models. It has also created key developer tools and infrastructure products such as Transformer, TensorFlow, and Jax required for engineering these models.

“Openly contributing to the research ecosystem is deeply embedded in our DNA… Decisions about whether to open source or release models to the public should be made on a case-by-case basis,” a Google spokesperson said. “It is important to carefully assess the balance of potential benefits and harms before releasing any AI system to the general public.”