This guide covers how to set up and configure fine-tuning on a Linux VPS. Whether you're running a production environment or a development setup, these instructions will help you get started quickly and securely.
Prerequisites
- Root or sudo access to the server
- A VPS running Ubuntu 22.04 or later (2GB+ RAM recommended)
- Basic familiarity with the Linux command line
- At least 4GB RAM (8GB+ recommended for model loading)
- Python 3.10+ installed
Installing Dependencies
Performance benchmarks show that properly tuned fine-tuning can handle significantly more concurrent connections than the default configuration. The key improvements come from adjusting worker processes and connection pooling.
# Install Python dependencies
pip install torch transformers accelerate
pip install fine-tuning fastapi uvicorn
This configuration provides a good balance between performance and resource usage. For high-traffic scenarios, you may need to increase the limits further.
Model Configuration
After applying these changes, monitor the server's resource usage for at least 24 hours to ensure stability. Tools like htop, iostat, and vmstat can provide real-time insights into system performance.
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
model_name = "fine-tuning/llm"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
model_name,
torch_dtype=torch.float16,
device_map="auto",
low_cpu_mem_usage=True
)
The configuration above sets the recommended values for a VPS with 2-4GB of RAM. Adjust the memory-related settings proportionally if your server has different specifications.
Wrapping Up
Following this guide, your fine-tuning setup should be production-ready. Keep an eye on resource usage as your traffic grows and don't forget to test your backup and recovery procedures periodically.