ACP → CODE Fine-tuning Pipeline

Model: meta-llama/Meta-Llama-3.1-8B-Instruct + QLoRA

---

Files

File	Purpose
prepare_dataset.py	Convert raw JSON dataset → train/val JSONL
finetune.py	QLoRA fine-tuning with trl SFTTrainer
inference.py	Interactive + batch inference on the trained model

---

Setup

pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121
pip install transformers datasets trl peft bitsandbytes accelerate

GPU requirement: 16 GB+ VRAM (A100, A10G, RTX 3090/4090).
For <16 GB, reduce --max_seq_length 1024 and --per_device_train_batch_size 1.

HuggingFace access: You need to accept the Llama-3.1 license at:
https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct

---

Step 1 — Prepare Data

python prepare_dataset.py \
    --input  email_agent_dataset_format.json \
    --outdir ./data \
    --val_split 0.2

Outputs: ./data/train.jsonl, ./data/val.jsonl

---

Step 2 — Fine-tune

export HF_TOKEN=hf_your_token_here

python finetune.py \
    --train_file ./data/train.jsonl \
    --val_file   ./data/val.jsonl \
    --output_dir ./output/acp_llama \
    --num_epochs 5 \
    --per_device_train_batch_size 2 \
    --gradient_accumulation_steps 4 \
    --learning_rate 2e-4

The LoRA adapter is saved to ./output/acp_llama/final/.
The base model (~16 GB) is NOT duplicated — only the adapter weights (~80 MB) are saved.

---

Step 3 — Inference

Interactive:

python inference.py \
    --adapter_dir ./output/acp_llama/final \
    --base_model  meta-llama/Meta-Llama-3.1-8B-Instruct

Batch (evaluate on val set):

python inference.py \
    --adapter_dir  ./output/acp_llama/final \
    --input_file   ./data/val.jsonl \
    --output_file  ./results/predictions.jsonl

---

Dataset Size Guidelines

Examples	Expected Quality
< 50	Model will memorize, not generalize
50–150	Reasonable for narrow task scope
150–500	Good generalization across your defined classes
500+	Strong; handles edge cases well

---

Key Design Decisions

Why QLoRA?

• Fine-tunes only ~0.5% of parameters (LoRA adapters)
• 4-bit quantization reduces memory from ~60 GB → ~10 GB
• Inference is fast; adapter can be merged into base model after training

Why completion-only loss?

DataCollatorForCompletionOnlyLM masks the system/user tokens so the model only learns to predict the assistant's CODE output. Without this, the model wastes capacity memorizing the prompt.

Why greedy decoding at inference?

The task requires valid JSON. Sampling (temperature > 0) introduces randomness that can break JSON structure. Use greedy (do_sample=False) for production.

---

Hyperparameter Tuning

Parameter	Default	Notes
lora_r	16	Increase to 32/64 if underfitting
lora_alpha	32	Usually 2x lora_r
learning_rate	2e-4	Lower to 1e-4 if loss is unstable
num_epochs	5	With small data, may overfit after 3
max_seq_length	2048	Your examples are ~800 tokens; 1024 is fine

Monitor eval_loss — if it starts rising while train_loss falls, you're overfitting. Reduce epochs or add more data.