✨ Logging - Enable App Service and Application Insights logs for production troubleshooting

[ssw-yakshaver]

Requested by: @Calinator444 via YakShaver.ai 🦬
cc: @Calinator444, @RobJT-wq, @tiagov8, @adamcogan

Hi Team!

  🟥  Watch the video (2 min 0 sec)

Pain

Following a production outage on the Rules (Content) landing page, the team was unable to access sufficient App Service or Application Insights logs to diagnose the issue. This PBI aims to enable and validate appropriate logging for the Rules (Content) App Service and Application Insights so future production incidents can be investigated quickly with reliable diagnostic data.

Acceptance Criteria

1. Application Insights logging is enabled and correctly configured for the Rules (Content) App Service
2. App Service logs (application and platform logs) are enabled for the Rules (Content) App Service
3. Logs are retained for an agreed minimum retention period suitable for production troubleshooting
4. Logs can be accessed by the team during a production incident without additional configuration
5. Health check failures and app startup/runtime errors are visible in logs
6. Documentation or notes are added to the project describing where and how to access these logs

Screenshot

Figure: Metrics showing outage timing and lack of diagnostic logs

[chrisschultzssw]

Please also:

1. Check what is being logged here: https://portal.azure.com/#@sswcom.onmicrosoft.com/resource/subscriptions/b8b18dcf-d83b-47e2-9886-00c2e983629e/resourceGroups/SSW.Rules/providers/Microsoft.OperationalInsights/workspaces/log-sswrules-prod/Overview
2. Make sure all logging is cost optimised - i.e. we're only logging what we need
3. Check in the staging resource group as well