Hi Gideon
<<1. creates a new menu (or ribbon, I'm not sure which)>>
Depends on which version of Excel you're targeting. 2007/2010 = Ribbon; earlier versions = toolbar/menu. But the controls available for the Ribbon will let you create menu controls within a tab+group. This interface can either be part of a COM Add-in (such as you'd create using VSTO) or be in an Office document that's loaded as a document add-in.
The major difference between VBA and VSTO is that VBA runs in-process with the application and is that appication's "native" language. This can make coding "simpler" (you rarely need to fully qualify things) and the code that automates the application itself will execute (much) more quickly. Compared to a c-Language XLL you'll notice a significant performance hit with a .NET Add-in. So...
<<Rather than pursue the XLL path, I thought maybe I could go to VBA, and write all my code there, and for the parts that require a lot of number crunching, I would call C dlls.>>
...this is a definite possibility. If you go this route you will not require VSTO; VSTO is designed to run in-process in order to enhance the UI. For "crunching", a straight-forward (COM-Visible) DLL will do the job just fine, and an Express edition can create that. For VSTO you'd need to have the "Professional" version of Visual Studio (or higher).
Another possibility would be a so-called "Shared Add-in" (Visual Studio template also available in VS Express). VSTO Add-ins base on the same IDTExtensibility2 COM interface, but VSTO takes care of some of the "plumbing" for you and extends certain parts of the object model.
Cindy Meister, VSTO/Word MVP