Add ASGI support to Django

[Kyle-Verhoog]

Motivation

In Django 3.1 the internal implementation was modified to follow a completely different codepath for async requests. Our integration only patches get_response and not get_response_async which means that the top level django.request span doesn't get generated.

Solution

Using the ASGI middleware we can relatively cleanly add async/ASGI
support for Django.

The previous method of instrumentation was first attempted but it was
decided against for a few reasons:

  - Tracing the ASGI application would require more than just wrapping
    the get_response_async method (although this would provide most of
    the visibility desired) as we'd need to trace the invocation of the
    response method.
  - We already have an ASGI middleware implementation.
  - The ASGI middleware provides more accurate timings as it is able to run 
    before the Django application.

Builds off of #1812

[majorgreys]

I was wondering whether we could use asgiref.sync.sync_to_async in cases like this to reduce the repeated code here:

https://docs.djangoproject.com/en/3.2/topics/async/#asgiref.sync.sync_to_async

[mergify]

@Kyle-Verhoog this pull request is now in conflict 😩

[Kyle-Verhoog]

@majorgreys looking at the implementation of sync_to_async it seems as though it'll introduce quite a bit of overhead unfortunately

[mabdinur]

Aside from Gab's comments LGTM

[mergify]

@Kyle-Verhoog this pull request is now in conflict 😩

[codecov-commenter]

Codecov Report

❌ Patch coverage is 50.00000% with 16 lines in your changes missing coverage. Please review.
✅ Project coverage is 84.93%. Comparing base (c695018) to head (495588a).

Files with missing lines	Patch %	Lines
ddtrace/contrib/django/_asgi.py	46.66%	8 Missing ⚠️
ddtrace/contrib/django/patch.py	56.25%	7 Missing ⚠️
tests/contrib/django/asgi.py	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #2656      +/-   ##
==========================================
- Coverage   84.95%   84.93%   -0.03%     
==========================================
  Files         590      591       +1     
  Lines       43083    43113      +30     
==========================================
+ Hits        36601    36617      +16     
- Misses       6482     6496      +14     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[majorgreys]

If I'm understanding correctly, enabling the ASGI middleware will generate a trace with two django.request spans with some variation on the tags:

https://github.com/DataDog/dd-trace-py/blob/e85e959fdd966692b93f4fb8384672524471ce8c/tests/snapshots/tests.contrib.django.test_django_snapshots.test_asgi_200_30.snap#L21-L38

vs.

https://github.com/DataDog/dd-trace-py/blob/e85e959fdd966692b93f4fb8384672524471ce8c/tests/snapshots/tests.contrib.django.test_django_snapshots.test_asgi_200_30.snap#L1-L20

I'm concerned that this could create confusion to have two django-request span.

Can we have just one?

[Kyle-Verhoog]

Oh good catch @majorgreys, I missed that! Yeah we should add checks to only do the middleware if get_response_async is available. What's happening here is that we're essentially double patching because get_response is being used for async

[majorgreys]

@Mergifyio backport 0.51

[mergify]

Command backport 0.51: success

Backports have been created

• #2752 Add ASGI support to Django (backport #2656) has been created for branch 0.51