Artificial intelligence (AI) researchers at Stanford University were able to create a chatbot called Alpaca using their own version of ChatGPT in just under two months. However, they decided to terminate the project, citing the high cost of hosting and issues with content filters that affected the behavior of the large language model (LLM).
The announcement of termination was made just a week after the chatbot’s release. The source code for the ChatGPT model developed by the researchers for less than $600 is publicly available. According to the researchers, Alpaca had a similar performance to OpenAI’s ChatGPT 3.5. The researchers stated that Alpaca is not intended for general use in the near future but is only for academic research purposes. Tatsunori Hashimoto, one of the Alpaca researchers, said that they were more interested in developing methods on top of Alpaca. The chatbot was developed on Meta AI’s LLaMA 7B model, and the researchers generated training data using the self-instruct method.
Douwe Kiela, an adjunct professor and AI researcher at Facebook, stated that “As soon as the LLaMA model came out, the race was on.” He added that the Alpaca team was the first to instruction-finetune the model, which is one of the reasons why it went viral. The source code of Alpaca is available on GitHub and has been viewed 17,500 times. More than 2,400 people have used the code for their own models. Hashimoto also stated that much of the observed performance of Alpaca came from LLaMA and that the base language model was still a key bottleneck.
As the use of AI systems continues to grow, experts have been debating the transparency of the technology and the publishing of source code and data used by companies to train their AI models. Kiela believes that Stanford is one of the big players in the LLM space and that universities should continue to do cutting-edge research in the open.
