Mix dry ingredients (flours, baking soda, cinnamon, nutmeg, salt) in a medium bowl. In another bowl, mash the banana with the sweet potato, then mix in oil, milk*, maple, and vanilla.
*Start by adding ¾ cup and mix everything to combine. If the mixture seems a little too dry (it should be thick and slightly sticky, but still easy to stir), add another 2-4 tablespoons of the milk.