I’m currently part way through a fantastic course on hardware design. It’s called Nand2Tetris and walks through building a simulated computer from the very simplest components (individual transistors) right through to a fully functioning computer.
Everything runs in a simulator so there’s no actual hardware to buy or break or futz about with (which you may or may not find attractive) and this means you can focus on the underlying theory and understand what’s going on rather than getting bogged down in implementation details.
Still early days (I’m busy building some of the more complex components from basic logic gates at the moment) but it’s very good so far. No previous experience is required, but if you have none, be prepared to do some fairly serious thinking and maybe a bot of background reading along the way.