ALIBABA · objective · 16% of the exam
LLM Fine-Tuning
Exam objectives are the official skill domains LLM Engineer (Professional) tests. Below are the official ALIBABA docs our questions on this objective cite — review them, then practise.
Sign in to track your progress on this objective.
Official references for this objective
-
Alibabacloud — Create a tuning job - Alibaba Cloud Model Studio - Alibaba Cloud Documentation Center
During warmup, the learning rate linearly increases from a small initial value to the configured rate. Limits the extent of parameter changes during early training,
-
Alibabacloud — model fine-tuning, API, model tuning, model fine-tuning, Dashscope, fine tuning, model training - - Alibaba Cloud Documentation Center
but only the final assistant output is trained. The \n characters before and after the think tags must be retained.
-
Alibabacloud — Introduction to model fine-tuning - Alibaba Cloud Model Studio - Alibaba Cloud Documentation Center
Supplements knowledge ( Injects domain knowledge ) Learns to perform tasks ( Follows instructions ) Performs tasks better ( Aligns with human preferences )
-
Alibabacloud — Deploy and fine-tune Llama 3 models - Platform For AI - Alibaba Cloud Documentation Center
lora_dim int 32 No LoRA dimension. When lora_dim > 0, LoRA/QLoRA training is enabled.
-
Alibabacloud — Fine-tuning operations - Alibaba Cloud Model Studio - Alibaba Cloud Documentation Center
When you perform a second round of efficient fine-tuning on a model that has already been efficiently fine-tuned, the lora_rank , lora_alpha , and lora_dropout
-
Alibabacloud — gpu memory, large language model, llm, fine-tuning, inference, calculator - Platform For AI - Alibaba Cloud Documentation Center
model parameters, activation values, and gradients are typically stored in 16-bit precision. The Adam/AdamW optimizer is used, and its state is stored in 32-bit precision.