AI Language Models: Commodity Tools or Innovation Drivers?
The advent of open-source AI language models like DeepSeek and S1 has ignited a debate about the commoditization of these powerful tools.
S1: A Challenger to Chat-GPT
Developed by researchers at Stanford and the University of Washington, S1 is a "reasoning" model that rivals OpenAI’s o1. Unlike traditional language models, S1 approaches questions by breaking them down into smaller, solvable steps. For instance, to estimate the cost of replacing Uber vehicles with Waymos, S1 would determine the number of Ubers on the road and the manufacturing cost of Waymos.
Mimicking Gemini for Cost-Effective Training
S1’s creation is a testament to the low-hanging fruit in AI research. The researchers harnessed an existing language model and trained it to reason by studying Google’s Gemini 2.0 model. Gemini reveals the thought process behind its answers, enabling S1 to mimic this process with minimal training data.
Ingenious Simplicity Improves Reasoning
Remarkably, the researchers improved S1’s reasoning ability through an ingeniously simple method: they added a special token to the model’s input, prompting it to "think more like a human." This tweak demonstrates the crude nature of chatbots and language models, which lack natural human reasoning capabilities.
Ethical Concerns and Open Source Access
OpenAI has expressed concern about the DeepSeek team training their model on ChatGPT outputs, highlighting the ethical conundrum of training models on copyrighted material. Similarly, Google prohibits competitors from training on Gemini’s outputs.
The open-source nature of AI models also raises safety concerns. Allowing unrestricted access and customization could facilitate the creation of harmful content like spam and deepfakes.
Inference Costs and the Future of AI
Despite the cost-effectiveness of model training, inference remains computationally expensive. As AI models become more pervasive, the demand for computing resources will soar. OpenAI’s massive server farm project underscores the ongoing need for infrastructure investment.
The Role of User Interfaces and Applications
OpenAI and other companies emphasize the importance of building useful applications on top of language models. The user interface, such as OpenAI’s Operator or the unique data sets accessed by xAI, will ultimately differentiate products.
Commodity or Innovation Catalyst?
The rise of open-source models has sparked debate about whether language models will become commoditized. Some believe that OpenAI will face challenges if its models can be easily replicated. However, others argue that the true value lies in building applications on top of these models.
Hype or Breakthrough?
The hype surrounding AI has raised questions about whether it represents a bubble. Nevertheless, the impressive performance of models like S1 suggests that there is still much potential in this field. The future will determine whether AI transforms our lives as predicted or ultimately fizzles out.
