Startseite Erkunden Hilfe
Anmelden
dongzenan
/
alpaca-lora
1
0
Fork 0
Dateien Issues 0 Pull-Requests 0 Wiki

Keine Beschreibung

Struktur: 63121244c8
Branches Tags
alpaca-lora
Eric Wang 63121244c8 Licenses and whatnot vor 3 Jahren
.gitignore 26f64780ad initial commit vor 3 Jahren
DATA_LICENSE 63121244c8 Licenses and whatnot vor 3 Jahren
LICENSE 63121244c8 Licenses and whatnot vor 3 Jahren
README.md 63121244c8 Licenses and whatnot vor 3 Jahren
alpaca_data.json 26f64780ad initial commit vor 3 Jahren
conversion.py 26f64780ad initial commit vor 3 Jahren
finetune.py 63121244c8 Licenses and whatnot vor 3 Jahren
generate.py 63121244c8 Licenses and whatnot vor 3 Jahren
iteration.ipynb 26f64780ad initial commit vor 3 Jahren
lengths.ipynb 26f64780ad initial commit vor 3 Jahren

README.md

alpaca-lora

This repository contains code for reproducing the Stanford Alpaca results. Users will need to have LLaMA weights on hand and be ready to fork transformers.

  1. Install dependencies

    pip install -q bitsandbytes datasets accelerate loralib

pip install -q git+https://github.com/zphang/transformers@llama_push
pip install -q git+https://github.com/huggingface/peft.git\

  2. Convert weights

    python conversion.py --input_dir [LLAMA_DIR]/LLaMA --model_size 7B --output_dir ./7B

  3. Modify hyperparams in finetune.py

    MICRO_BATCH_SIZE = 12
BATCH_SIZE = 36
EPOCHS = 3
LEARNING_RATE = 2e-5

  4. Run experiments

    python finetune.py