Няма описание

Eric Wang 63121244c8 Licenses and whatnot		преди 3 години
.gitignore	26f64780ad initial commit	преди 3 години
DATA_LICENSE	63121244c8 Licenses and whatnot	преди 3 години
LICENSE	63121244c8 Licenses and whatnot	преди 3 години
README.md	63121244c8 Licenses and whatnot	преди 3 години
alpaca_data.json	26f64780ad initial commit	преди 3 години
conversion.py	26f64780ad initial commit	преди 3 години
finetune.py	63121244c8 Licenses and whatnot	преди 3 години
generate.py	63121244c8 Licenses and whatnot	преди 3 години
iteration.ipynb	26f64780ad initial commit	преди 3 години
lengths.ipynb	26f64780ad initial commit	преди 3 години

alpaca-lora

This repository contains code for reproducing the Stanford Alpaca results. Users will need to have LLaMA weights on hand and be ready to fork transformers.

Install dependencies

pip install -q bitsandbytes datasets accelerate loralib

pip install -q git+https://github.com/zphang/transformers@llama_push
pip install -q git+https://github.com/huggingface/peft.git\

Convert weights

python conversion.py --input_dir [LLAMA_DIR]/LLaMA --model_size 7B --output_dir ./7B

Modify hyperparams in finetune.py

MICRO_BATCH_SIZE = 12
BATCH_SIZE = 36
EPOCHS = 3
LEARNING_RATE = 2e-5

Run experiments
```
python finetune.py
```

README.md

alpaca-lora