The reason you are struggling is that the size of symbols in plot views isn't really meant to match the size of symbols in a symbol key.
Symbols are drawn to physical size and the scale of a plot view will affect how large a symbol appears. You could have multiple plot views in a plot, all using different scales. You can also override the scale of a symbol per fixture. In effect the same symbol could appear in many different sizes on the same plot.
Symbols keys on the other hand merely list symbols. They have no relation to other physical objects in your design and are unaware of the scale(s) of any plot view(s) on the plot. In a symbol key it is the text size of the symbol key that controls the overall size of all symbols in the key, not the individual symbol's physical size.