ML Solution Monitoring, Maintenance, and Security

Official references for this objective

• Amazon Web Services — Amazon SageMaker AI metrics in Amazon CloudWatch - Amazon SageMaker AI

  Invocation5XXErrors The number of InvokeEndpoint requests where the model returned a 5xx HTTP response code.

• Amazon Web Services — Model Monitor FAQs - Amazon SageMaker AI

  Data Capture happens asynchronously without impacting production traffic.

• Amazon Web Services — Security Hub CSPM controls for SageMaker AI - AWS Security Hub

  You can't change the internet access setting after creating a notebook instance. Instead, you can stop, delete, and recreate the instance with blocked internet access.

• Amazon Web Services — AWS Identity and Access Management for Amazon SageMaker AI - Amazon SageMaker AI

  Identity-based policies can be inline policies (embedded directly into a single identity) or managed policies (standalone policies attached to multiple identities)

• Amazon Web Services — Baseline calculation, drift detection and lifecycle with ClarifyCheck and QualityCheck steps in Amazon SageMaker Pipelines - Amazon SageMaker AI

  you can enable the checks you want by setting the skip_check property of the corresponding check step set to False

• Amazon Web Services — Best practices for endpoint security and health with Amazon SageMaker AI - Amazon SageMaker AI

  if you incorrectly modify your endpoint dependencies, Amazon SageMaker AI can't automatically patch your endpoints or replace your unhealthy instances

• Amazon Web Services — Bias drift for models in production - Amazon SageMaker AI

  it uses the Normal Bootstrap Interval method to construct an interval C=(c min ​,c max ​) such that SageMaker Clarify is confident that the true

• Amazon Web Services — Data and model quality monitoring with Amazon SageMaker Model Monitor - Amazon SageMaker AI

  We recommended that you keep your disk utilization below 75% to ensure data capture continues capturing requests.

• Amazon Web Services — Feature attribution drift for models in production - Amazon SageMaker AI

  In SageMaker Clarify, if the NDCG value is below 0.90, we automatically raise an alert.