Those side quests are actually really worth your time especially when it comes to permanent upgrades. The across the realms favor, for instance, gives you this thing called the meal of comfort which is pretty great since it adds +5 to all your stats permanently. Something interesting about freeing the hafgufa - while it mainly helps with clearing up the weather in the forbidden sands, the exploration possibilities it opens up are super valuable

You'll find some really good stuff through these quests like when you take on the mysterious orb, which gets you lunda's lost armor set, or freya's missing peace that hooks you up with some powerful runic summons. The nice thing is, everything you earn from these side activities, whether its xp, gear, or crafting materials, sticks with you for your whole playthrough

If you're looking for the best rewards, maybe focus on the story-connected favors first, like defend your valor - those tend to give you some seriously good equipment that'll stay useful right through to the endgame. Pretty much every favor adds something permanent to your character's progression, even if it's just materials for upgrading your gear later on