LangSmith Helps Fine-Tune LLaMA2 to Beat GPT-3.5

LangChain has published a workflow guide demonstrating how to use its LangSmith platform to manage datasets and evaluate fine-tuned models. To test the system, the company fine-tuned Meta's LLaMA2-7b-chat and OpenAI's gpt-3.5-turbo for a specialized knowledge graph triple extraction task. Using the BenchIE and CarbIE public datasets, they compiled a training set of approximately 1,500 labeled sentences and a test set of 100 sentences, converting them into a standardized JSON format.

For the open-source LLaMA2-7b-chat model, LangChain addressed hardware limitations by quantizing the model parameters to 4-bit precision. This reduced the memory footprint from 28GB to 3.5GB, allowing it to fit on a single 16GB T4 GPU. They utilized parameter-efficient fine-tuning via qLoRA to freeze the quantized weights and train only the LoRA adapters. The supervised fine-tuning process took less than 15 minutes on an NVIDIA A100 GPU. In contrast, fine-tuning OpenAI's gpt-3.5-turbo required only 50 formatted chat examples and completed in a few minutes using the OpenAI SDK.

The models were evaluated on a 0-to-100 scale using a custom GPT-4 evaluator within LangSmith to detect factual discrepancies. The results showed that the fine-tuned LLaMA2-7b-chat model improved by approximately 29 percent over its baseline version and actually outperformed few-shot prompting on the standard gpt-3.5-turbo. However, few-shot prompting on GPT-4 still delivered the highest overall performance. The fine-tuned gpt-3.5-turbo successfully output the correct triplet format in every test case.

This development highlights how targeted fine-tuning of smaller, open-source models can yield highly competitive results for specific tasks while reducing operational costs. However, LangChain cautions that fine-tuning is best suited for formatting and specialized structures rather than teaching models new factual knowledge. Developers should still evaluate retrieval-augmented generation or few-shot prompting before committing to the more complex fine-tuning pipeline.