Well, it also depends on what you're a growing. I think 5 gal. is a little small for a baobab tree.
But it's overkill for a dwarf tom.
The thing is, most plants will limit their size to the available root ball. So 5 gal. will do. But if you want BIG plants then you have to give them a LOT of dirt.