First, thanks for pointing out "--privileged is essential for DRI support." - this saved me when trying to use Nvidia's CUDA containers to also get OpenGL working.
I'm not exactly sure of the implications of using --privileged, though it sounds like you are basically just giving the container the same privileges your user would have for device access (i.e., not necessarily root access level). That sounds reasonable to me, though some of what I read casually indicated it could be used for exploits.
Are there possibly more granular options that can be used on the horizon?