Нема описа

Eric Wang 63121244c8 Licenses and whatnot		пре 3 година
.gitignore	26f64780ad initial commit	пре 3 година
DATA_LICENSE	63121244c8 Licenses and whatnot	пре 3 година
LICENSE	63121244c8 Licenses and whatnot	пре 3 година
README.md	63121244c8 Licenses and whatnot	пре 3 година
alpaca_data.json	26f64780ad initial commit	пре 3 година
conversion.py	26f64780ad initial commit	пре 3 година
finetune.py	63121244c8 Licenses and whatnot	пре 3 година
generate.py	63121244c8 Licenses and whatnot	пре 3 година
iteration.ipynb	26f64780ad initial commit	пре 3 година
lengths.ipynb	26f64780ad initial commit	пре 3 година

alpaca-lora

This repository contains code for reproducing the Stanford Alpaca results. Users will need to have LLaMA weights on hand and be ready to fork transformers.

Install dependencies

pip install -q bitsandbytes datasets accelerate loralib

pip install -q git+https://github.com/zphang/transformers@llama_push
pip install -q git+https://github.com/huggingface/peft.git\

Convert weights

python conversion.py --input_dir [LLAMA_DIR]/LLaMA --model_size 7B --output_dir ./7B

Modify hyperparams in finetune.py

MICRO_BATCH_SIZE = 12
BATCH_SIZE = 36
EPOCHS = 3
LEARNING_RATE = 2e-5

Run experiments
```
python finetune.py
```

README.md

alpaca-lora