1 - Get Recalbox on a GPi Case 2 and you'll have access to just about every system from before 2000 (including support for commodore and other similar systems). It can handle PSP games as well, but not PS2 or NDS. There are other cases available for a raspberry pi system, but I recommend the GPi Case 2 because you can play it "docked" and handheld. I recommend Recalbox since it already has a lot of support for the GPi case built into it, but if you're tech-savvy you may prefer Lakka for its flexibility. You may be able to get more modern emulators to run on the lakka as well.

2 - Gaming PC with Lakka, Citra, or whatever other emulators you'd like. And unless you're playing a lot of super new games, you don't need anything fancy- you could probably just throw windows 7 on a $100 refurbished business PC and run just about any game from 2010 or earlier, TBH.

3 - Wii or Wii U. I personally find emulation of these (specifically with a wii-mote) to be a bit finicky. If you don't use a Wii, you can substitute your personal console of choice for this one.

4 - Oculus Quest- though I'm not sure if it counts since you aren't connecting to a TV. This isn't the best VR headset but it is the cheapest. It has a good library of standalone games, and for anything else you can use airlink or the virtual desktop to run games off of a VR-ready PC (If you went with one that was beefy for #2). The quest has a lot of modding support through the sidequest. The main concern with this is that you need a phone to set up a Quest when you buy it/after a factory reset. So if Facebook goes under or a meteor hits silicon valley, this could conceivably turn into a fancy paperweight. To my knowledge, nobody has cracked the Quest to skip over this step. If historical preservation is more important to you than money, I would recommend choosing literally any other VR headset because of the setup thing.