I can confirm you need a last.fm account for rhythmbox's built-in cover art plugin - "coverart" to work with last.fm
(There are however musicbrainz and podcast coverart python scripts in /usr/local/lib/rhythmbox/plugins/artdisplay in addition to the last.fm one.)
The plugin communicates with last.fm via a browser using gvfs - this implies:
the gvfs extension is loaded
/home/tc/.local/share/applications/mimeapps.list has an entry pointing to a browser
the browser desktop file exec line contains "%u"
..if all goes well, rhythmbox will open your browser/a tab in your browser to allow you to login to last.fm and request that you allow access to rhythmbox. Once this is done, all last.fm functionality seems to work except the coverart download
If you'd like to manually download coverart from last.fm (or anywhere else), rhythmbox will display it in the following cases:
the file is copied to /home/tc/.cache/rhythmbox/covers/artist - album.{jpg, png}
the file is copied to the same local folder as the tracks from an album and renamed cover.{jpg, png}