Nincs leírás

1 Ágak

Eric Wang 63121244c8 Licenses and whatnot		3 éve
.gitignore	26f64780ad initial commit	3 éve
DATA_LICENSE	63121244c8 Licenses and whatnot	3 éve
LICENSE	63121244c8 Licenses and whatnot	3 éve
README.md	63121244c8 Licenses and whatnot	3 éve
alpaca_data.json	26f64780ad initial commit	3 éve
conversion.py	26f64780ad initial commit	3 éve
finetune.py	63121244c8 Licenses and whatnot	3 éve
generate.py	63121244c8 Licenses and whatnot	3 éve
iteration.ipynb	26f64780ad initial commit	3 éve
lengths.ipynb	26f64780ad initial commit	3 éve

alpaca-lora

This repository contains code for reproducing the Stanford Alpaca results. Users will need to have LLaMA weights on hand and be ready to fork transformers.

Install dependencies

pip install -q bitsandbytes datasets accelerate loralib

pip install -q git+https://github.com/zphang/transformers@llama_push
pip install -q git+https://github.com/huggingface/peft.git\

Convert weights

python conversion.py --input_dir [LLAMA_DIR]/LLaMA --model_size 7B --output_dir ./7B

Modify hyperparams in finetune.py

MICRO_BATCH_SIZE = 12
BATCH_SIZE = 36
EPOCHS = 3
LEARNING_RATE = 2e-5

Run experiments
```
python finetune.py
```

README.md

alpaca-lora