After stumbling upon Gravatar, a site serving your avatar globally, I looked for an easy way to add this functionality to Joomla!. After ditching Gravatar/Avatar for needing some other plugin to work in the first place, I installed gAvatar.
This plugin worked without dependencies and gAvatar is dead simple to set up. Just enable the plugin, change the rendering mode and you are done.
The only thing that can go wrong (and did for me) was the fact that you can choose for a floating avatar (which ended up floating in the text), a table avatar (using the default Joomla header style - which this site doesn't use) or advanced templating.
That last option seemed like a good solution except that nowhere was described what it did or how it worked. A quick dive in the source showed me everything I needed to know: gAvatar adds a 'avatar' parameter to the article object. To use it, find the classes generating the article layouts and modify those.
To find those class files, I used the following in the template folder:
fgrep "article->text" * -R
And add the following (for example) to insert the avatar:
<?php if (isset($this->article->avatar)) : ?> <div style="float: right"><?php echo $this->article->avatar; ?></div> <?php endif; ?>