alpaca-lora

🦙🌲🤏 Alpaca (Low-Rank Edition)

This repository contains code for reproducing the Stanford Alpaca results. Users will need to be ready to fork transformers.

Setup

1. Install dependencies (install zphang's transformers fork)

   pip install -q datasets accelerate loralib sentencepiece
   
   pip install -q git+https://github.com/zphang/transformers@llama_push
   pip install -q git+https://github.com/huggingface/peft.git
   

2. Install bitsandbytes from source

Inference

See generate.py. This file reads the decapoda-research/llama-7b-hf model from the Huggingface model hub and the LoRA weights from tloen/alpaca-lora-7b, and runs inference on a specified input. Users should treat this as example code for the use of the model, and modify it as needed.

Training

Under construction.

To do

• Hyperparameter tuning
• Documentation for notebook
• Support for 13b, 30b, 65b
• Inference CLI and evaluation
• Better disclaimers about why using LLaMA without permission is very bad!