Sunday, September 20, 2020

M365 Service Health Notification

Microsoft

Microsoft Teams service alert

Incident information

Title: Some admins may have been unable to access the Microsoft Teams admin center
ID: TM222357

Status

Post-Incident Report Published

Details

A post-incident report has been published.

Thank you,
The Microsoft team

To customize what's included in this email, who gets it, or to unsubscribe, set your Service health preferences. If you are receiving this email because your Admin added you as a recipient, please contact your Admin to unsubscribe.
Microsoft respects your privacy. Review our online Privacy Statement.
Microsoft Corporation
One Microsoft Way
Redmond, WA USA 98052

Thursday, September 17, 2020

M365 Service Health Notification

Microsoft

Exchange Online service alert

Incident information

Title: Users with Android 8 devices may be unable to send email in the Outlook mobile app
ID: EX222397

Status

Service Degradation

Details

Title: Users with Android 8 devices may be unable to send email in the Outlook mobile app

User Impact: Users may be unable to send email in the Outlook mobile app when using an Android 8 device.

More info: Users may see the 'send' button grayed out on their screen, preventing them from sending email.

Current status: We determined that a recent service update is unexpectedly causing issues with Android 8 Outlook mobile. We've developed and deployed a fix for the issue and are monitoring the deployment progress.

Scope of impact: This could potentially affect any of your users attempting to send email through the Outlook mobile app on their Android 8 devices.

Start time: Saturday, September 12, 2020, at 7:00 AM UTC

Root cause: A recent standard service update is causing unexpected impact to the Outlook mobile app for Android 8 devices.

Next update by: Friday, September 18, 2020, at 8:30 PM UTC

Thank you,
The Microsoft team

To customize what's included in this email, who gets it, or to unsubscribe, set your Service health preferences. If you are receiving this email because your Admin added you as a recipient, please contact your Admin to unsubscribe.
Microsoft respects your privacy. Review our online Privacy Statement.
Microsoft Corporation
One Microsoft Way
Redmond, WA USA 98052

M365 Service Health Notification

Microsoft

Microsoft Teams service alert

Incident information

Title: Users experiencing failures joining meetings using a Conferencing Virtual Assistant (CVA) in Microsoft Teams
ID: TM222384

Status

Service Restored

Details

Title: Users experiencing failures joining meetings using a Conferencing Virtual Assistant (CVA) in Microsoft Teams

User Impact: Users may intermittently experience dropped calls when connecting through a CVA in Microsoft Teams.

More info: This was a continuation of the issue communicated in TM222372.

Due to the intermittent nature of impact, retries may have been successful.

Final status: The fix has been deployed and we've monitored through several peak cycles to confirm that the issue has not reoccurred and the issue is mitigated.

Scope of impact: Around 30 percent of calls served through the affected infrastructure and using CVA were impacted.

Start time: Thursday, September 17, 2020, at 3:00 PM UTC

End time: Thursday, September 17, 2020, at 6:20 PM UTC

Root cause: An unexpected increase in traffic resulted in impact.

Next steps:

- We're continuing to monitor the affected environments while looking into further efficiency improvements to prevent future occurrences.

This is the final update for the event.

Thank you,
The Microsoft team

To customize what's included in this email, who gets it, or to unsubscribe, set your Service health preferences. If you are receiving this email because your Admin added you as a recipient, please contact your Admin to unsubscribe.
Microsoft respects your privacy. Review our online Privacy Statement.
Microsoft Corporation
One Microsoft Way
Redmond, WA USA 98052

M365 Service Health Notification

Microsoft

Microsoft Teams service alert

Incident information

Title: Users experiencing failures joining meetings using a Conferencing Virtual Assistant (CVA) in Microsoft Teams
ID: TM222384

Status

Service Degradation

Details

Title: Users experiencing failures joining meetings using a Conferencing Virtual Assistant (CVA) in Microsoft Teams

User Impact: Users may intermittently experience dropped calls when connecting through a CVA in Microsoft Teams.

More info: This is is a continuation of the issue communicated in TM222372.

Due to the intermittent nature of impact, retries may be successful.

Current status: Our telemetry has indicated the issue has reoccured and we're working to expand our load balancing efforts to mitigate impact while continuing our code review to isolate the underlying cause.

Scope of impact: Around 30 percent of calls served through the affected infrastructure and using CVA are impacted.

Start time: Thursday, September 17, 2020, at 3:00 PM UTC

Root cause: An unexpected increase in traffic is causing impact.

Next update by: Thursday, September 17, 2020, at 9:00 PM UTC

Thank you,
The Microsoft team

To customize what's included in this email, who gets it, or to unsubscribe, set your Service health preferences. If you are receiving this email because your Admin added you as a recipient, please contact your Admin to unsubscribe.
Microsoft respects your privacy. Review our online Privacy Statement.
Microsoft Corporation
One Microsoft Way
Redmond, WA USA 98052

M365 Service Health Notification

Microsoft

Exchange Online service alert

Incident information

Title: On-premises users experiencing email delivery delays of over an hour in the Exchange Online service
ID: EX221688

Status

Service Restored

Details

Title: On-premises users experiencing email delivery delays of over an hour in the Exchange Online service

User Impact: Users could have seen delays of over an hour before receiving email sent from on-premises to users hosted the cloud.

More info: The problem affected all connection methods including the Outlook client and Outlook on the web.

Admins may have seen the following error corresponding to this issue when reviewing client logs: 452 4.3.1 Insufficient system resources (TSTE).

Final status: We've confirmed via telemetry that the configuration changes we made have successfully mitigated this issue.

Scope of impact: Impact was specific to users sending email from on-premises accounts to users hosted in Exchange Online.

Start time: Tuesday, September 1, 2020, at 2:00 PM UTC

End time: Friday, September 11, 2020, at 11:30 PM UTC

Root cause: A specific portion of infrastructure facilitating mail flow between the on-premises and Exchange Online environment wasn't processing traffic as expected during peak times, resulting in impact.

Next steps:

-We're continuing our investigation into the cause of this issue to develop a permanent solution.

This is the final update for the event.

Thank you,
The Microsoft team

To customize what's included in this email, who gets it, or to unsubscribe, set your Service health preferences. If you are receiving this email because your Admin added you as a recipient, please contact your Admin to unsubscribe.
Microsoft respects your privacy. Review our online Privacy Statement.
Microsoft Corporation
One Microsoft Way
Redmond, WA USA 98052

M365 Service Health Notification

Microsoft

Microsoft Teams service alert

Incident information

Title: Users experiencing failures joining meetings using a Conferencing Virtual Assistant (CVA) in Microsoft Teams
ID: TM222372

Status

Service Restored

Details

Title: Users experiencing failures joining meetings using a Conferencing Virtual Assistant (CVA) in Microsoft Teams

User Impact: Users may have intermittently experienced dropped calls when connecting through a CVA in Microsoft Teams.

More info: Due to the intermittent nature of impact, retries may have been successful.

Final status: We've verified via telemetry that rebalancing load has mitigated impact.

Scope of impact: Around 30 percent of calls served through the affected infrastructure and using CVA were impacted.

Start time: Thursday, September 17, 2020, at 3:00 PM UTC

End time: Thursday, September 17, 2020, at 5:14 PM UTC

Root cause: An unexpected increase in traffic resulted in impact.

Next steps:

- We're furthering reviewing the source of the spike in traffic and why it resulted in impact to prevent similar issues in the future.

This is the final update for the event.

Thank you,
The Microsoft team

To customize what's included in this email, who gets it, or to unsubscribe, set your Service health preferences. If you are receiving this email because your Admin added you as a recipient, please contact your Admin to unsubscribe.
Microsoft respects your privacy. Review our online Privacy Statement.
Microsoft Corporation
One Microsoft Way
Redmond, WA USA 98052

M365 Service Health Notification

Microsoft

Microsoft Teams service alert

Incident information

Title: Users experiencing failures joining meetings using a Conferencing Virtual Assistant (CVA) in Microsoft Teams
ID: TM222372

Status

Service Degradation

Details

Title: Users experiencing failures joining meetings using a Conferencing Virtual Assistant (CVA) in Microsoft Teams

User Impact: Users may intermittently experience dropped calls when connecting through a CVA in Microsoft Teams.

More info: Due to the intermittent nature of impact, retries may be successful.

Current status: We've identified an unexpected increase in CVA traffic and have rebalanced load to mitigate impact. We're continuing to monitor telemetry to confirm system health.

Scope of impact: Around 30 percent of calls served through the affected infrastructure and using CVA are impacted.

Start time: Thursday, September 17, 2020, at 3:00 PM UTC

Next update by: Thursday, September 17, 2020, at 6:30 PM UTC

Thank you,
The Microsoft team

To customize what's included in this email, who gets it, or to unsubscribe, set your Service health preferences. If you are receiving this email because your Admin added you as a recipient, please contact your Admin to unsubscribe.
Microsoft respects your privacy. Review our online Privacy Statement.
Microsoft Corporation
One Microsoft Way
Redmond, WA USA 98052