I haven't seen such issue yet so I'm wondering how is this possible.
What values is "GPU Core Load" and "D3D Usage" showing in idle and under load?
And what kind of workload are you running - gaming or computing?
That looks more like the game you're using is not based on Direct3D, rather than D3D usage reporting a reverse number. Do you know which interface that game uses?
Note that D3D Usage reflects only load via the Direct3D interface, so if you're running other type of workload (computing or graphics via a different interface), it won't be reflected here.
Hence GPU Core Load is the best indicator.