I don't want to waste your time, here's the TLDR.
1. An Apple Podcasts subscribe link
2. An RSS subscribe link
3. An easily accessible (non-autoplaying) auto-updating audio player. Some call this a "playlist" player.
4. Subscribe in Spotify link
5. Subscribe on Google Podcasts link
Among the 1001 things the beginning podcaster is concerned with, a fully-featured and user-friendly website is a high priority for most, particularly those that plan on building a business around or supported by their podcast.
The greatest advantage of having your own site is the central location of information on your show. When someone drops your title into a web search, it's likely to return a myriad of links. Your Apple Podcasts listing, perhaps your Stitcher listing, maybe even a directory your media host provides that includes your episodes will turn up. But, if you've done your SEO homework, your own website should always top the list in any search, making it the most likely destination for someone looking to learn more, subscribe, buy merchandise etc.
What should they find when they find your website?
First and foremost, they should find an easy way to play it right away. Something that autoupdates for you so the latest episode is always at the top. This is the lowest friction point for ANYONE to interact with your content, no matter their knowledge of podcast or technology, everyone can follow a link and click "play". Make sure they can if they follow your link.
Next, you need to promote your listing in Apple Podcasts. Personally, I suggest doing through this two means. First, and most obviously, a button. You can create your own, or use the branding and images provided by Apple through the Podcasts Connect website, but either way have something that is obviously for Apple users to click and subscribe on their devices.
The less obvious way you should integrate Apple Podcasts into your website is through the Smart App banners introduced several years ago. While primarily aimed at App Store developers highlighting their apps on their sites, it works admirably for Podcasts and even books.
For full documentation on implementing Smart App banners on YOUR site, see Apple's guidelines here.
To ensure you're show is listed in the Google Podcasts directory, and to grab the direct link to use on your site, use Google's developer tools here.
The RSS link should be easy to grab from your media host or website plugin that creates that feed. And finally, for Spotify, the best bet is to share the show from the app itself and copy the link. You might be asking, why Spotify? The other links above are either OS's themselves (Apple and Google) with their own proprietary apps, or the standard RSS that podcasting is built on, and which can be used in any podcast app, across platforms completely. So why Spotify and not Overcast, or Stitcher or iHeartRadio etc.?
Rob Walch with Libsyn and Todd Cochrane with Blubrry (the two largest podcast media hosts) have both said they believe Spotify will soon be the number two destination for downloads after Apple Podcasts (and depending on exactly how hard Google pushes their new initiative). Spotify is platform agnostic, it works on Mac, it works on PC, it works on Android it works on iPhone etc., AND Spotify is actively pushing the podcasting medium among avid music listeners who aren't CURRENTLY listening to podcasts through other apps. Why does this matter for Spotify? Well, we're infinitely cheaper than streaming music, since they don't pay us royalties (yet anyway), so there's every reason to think this current interest in podcasting from Spotify is not just a passing craze.
For an example of how all this can work together well on one site see the image of my site above.
The primary thing to remember is that your website isn't where most of your fans and listeners will hang out. They'll be interacting with your content from their favorite podcast app for the most part. But your website should be the easiest, most direct route to understand more about your show, to find all the appropriate social media links, and enable new listeners to easily subscribe to your show whether they really understand what a "podcast" is or not.