After clicking around the administration screens in Team Foundation Service looking for a way to delete a Team Project, I looked more broadly for an answer.

The answer is to use the tfsdeleteproject.exe from the command line.

On a computer with Team Explorer 2012 installed, navigate to the following directory.

C:Program Files (x86)Microsoft Visual Studio 11.0Common7IDE>

Now you’ll use the tfsdeleteproject command and specify both the URL to your collection on Team Foundation Service and the name of the Team project you wish to delete.

tfsdeleteproject /collection:https://yourname.visualstudio.com/defaultcollection projname

Remember to substitute your account name in the URL and replace projname with the name of the team project you wish to delete. If your project name includes spaces, remember to put quotation marks around the name.