We introduce Vicuna-13B, an open-source chatbot trained by fine-tuning LLaMA on user-shared conversations collected from ShareGPT. Preliminary evaluation using GPT-4 as a judge shows Vicuna-13B achieves more than 90%* quality of OpenAI ChatGPT and Google Bard while outperforming other models like LLaMA and Stanford Alpaca in more than 90%* of cases. The cost of training Vicuna-13B is around $300. The code and weights, along with an online demo, are publicly available for non-commercial use.
The ultimate guide to chatbot analytics. Find out what bot metrics and KPIs you should measure and discover easy ways to optimize your chatbot performance.
These measurements are indispensable for tracking the results of your chatbot, identifying any stumbling blocks and continuously improving its performance. But which metrics should you choose?
R. Amant, T. Horton, и F. Ritter. CHI '04: Proceedings of the SIGCHI conference on Human factors in computing systems, стр. 343-350. New York, NY, USA, ACM Press, (2004)
D. Swallow, M. Blythe, и P. Wright. EACE '05: Proceedings of the 2005 annual conference on European association of cognitive ergonomics, стр. 91-98. University of Athens, (2005)
V. Basili. ICSE '96: Proceedings of the 18th international conference on Software engineering, стр. 442--449. Washington, DC, USA, IEEE Computer Society, (1996)
K. Baker, S. Greenberg, и C. Gutwin. CSCW '02: Proceedings of the 2002 ACM conference on Computer supported cooperative work, стр. 96-105. New York, NY, USA, ACM Press, (2002)
D. Neale, J. Carroll, и M. Rosson. CSCW '04: Proceedings of the 2004 ACM conference on Computer supported cooperative work, стр. 112-121. New York, NY, USA, ACM Press, (2004)
C. Nodder, G. Williams, и D. Dubrow. GROUP '99: Proceedings of the international ACM SIGGROUP conference on Supporting group work, стр. 150-159. New York, NY, USA, ACM Press, (1999)