Implement generative AI quality assurance and observability — Microsoft Certified: Machine Learning Operations Engineer Associate

Official references for this objective

• Microsoft — Monitoring Generative AI applications | Microsoft Learn

  A collection of spans forms a trace that represents the complete path and lifecycle of a request.

• Microsoft — Evaluation metrics | Microsoft Learn

  Reference-based metrics are used to compare generated text to a reference, the human annotated ground truth text.

• Microsoft — Evaluating the performance of LLM summarization prompts with G-Eval | Microsoft Learn

  A technique known as G-Eval has been developed, which uses GPT-4 to evaluate the quality of summaries without a ground truth.