暫無描述

Eric Wang 63121244c8 Licenses and whatnot		3 年之前
.gitignore	26f64780ad initial commit	3 年之前
DATA_LICENSE	63121244c8 Licenses and whatnot	3 年之前
LICENSE	63121244c8 Licenses and whatnot	3 年之前
README.md	63121244c8 Licenses and whatnot	3 年之前
alpaca_data.json	26f64780ad initial commit	3 年之前
conversion.py	26f64780ad initial commit	3 年之前
finetune.py	63121244c8 Licenses and whatnot	3 年之前
generate.py	63121244c8 Licenses and whatnot	3 年之前
iteration.ipynb	26f64780ad initial commit	3 年之前
lengths.ipynb	26f64780ad initial commit	3 年之前

alpaca-lora

This repository contains code for reproducing the Stanford Alpaca results. Users will need to have LLaMA weights on hand and be ready to fork transformers.

Install dependencies

pip install -q bitsandbytes datasets accelerate loralib

pip install -q git+https://github.com/zphang/transformers@llama_push
pip install -q git+https://github.com/huggingface/peft.git\

Convert weights

python conversion.py --input_dir [LLAMA_DIR]/LLaMA --model_size 7B --output_dir ./7B

Modify hyperparams in finetune.py

MICRO_BATCH_SIZE = 12
BATCH_SIZE = 36
EPOCHS = 3
LEARNING_RATE = 2e-5

Run experiments
```
python finetune.py
```

README.md

alpaca-lora