It's pretty simple! You just need the specific food for the particular pet you want to tame (listed below). Once you have that, go to your inventory and click on it.
Your character should now be holding the item out in front of them. You can now walk up to the animal and they should attempt to eat the food and have a heart over their head once they are tame.

You'll need to find Reins. These are region-locked items, so you'll have to find them each time you enter a new region. Talk to NPCs all over and they will hopefully tell you where they are eventually (you can also just get lucky and run into them while exploring).
Once you learn their location, head over there and collect them. You can now use the T key on a rideable pet to hop on.

When you are about to tame a pet, make sure that it is away from its flock or some other similar creature, since once the first one finishes eating and the others do not, they will become aggressive towards you and attack you, independently If the creature is neutral, hostile or friendly.

The food can be found by slaying various creatures or out of chests you can find around the lands! You can also oddly enough find them from deposits. For example, I found a Lollipop from an Emerald Deposit! This is pretty random, so hopefully there ends up a way to more reliably find some of these items. You'll know the food is for a pet because it will be listed under the pets tab in your inventory.

You can also buy in the village shop that sell empty flasks for one coin, in every village you will not have the same foods offer.
The list of purchaseable food is: candy, carrot, bubble gum, chocolate donut, waffle, cotton candy.

I will modify this guide according to updates and captures that adapt to pets and items along with some other useful information.