Thanks to Lawrence Hortizuela on YouTube for the report on this issue. This affects most versions of the skin, including 1.7, on osu!stable client. osu!lazer is currently unaffected.
Solution
- Delete
hitcircleoverlay@2x.png from the skin folder.
- Rename
hitcircleoverlay-0@2x.png to hitcircleoverlay@2x.png.
- Reload the skin in osu! (Ctrl+Shift+Alt+S), and the issue should be resolved.
Why did this happen?
Usually the skin preview on osu!stable client shows the osu!standard set:

Since this skin is primarily an osu!mania set, I wanted to show the arrows instead:

A skin preview is generated from the following unanimated files:
hitcircleoverlay@2x.png
hit50@2x.png
hit100@2x.png
hit300@2x.png
In this skin, I set those files to each one of the osu!mania arrows. Now here's the interesting part of the trick - if an animated version of an element exists, it is used in gameplay instead of the unanimated version. An osu!standard player would have seen the following files instead (-0 means the first frame of an animation), which contain regular ol' osu!standard elements as you would expect:
hitcircleoverlay-0@2x.png
hit50-0@2x.png
hit100-0@2x.png
hit300-0@2x.png
Unfortunately this trick doesn't seem to work for the hitcircleoverlay any more. In a recent update to the osu!wiki, hitcircleoverlay is now shown as non-animatable. I don't know when this was changed in the osu!stable client, but I'm guessing fairly recently (as of July 2022).