It's a fact that a huge part of the extensions require X. Listing the common components in each of those dep files would increase the load time unnecessarily for each such extension.
Certainly it's a bit more hassle if you usually use only a cli and then load some GUI app. But if you're loading a GUI app on purpose, surely you're also loading an X server (we don't force any choice, you can pick Xvesa, Xfbdev, Xorg, etc) and so can load the two other related extensions.