Good question! Applications that are showing as in-progress in your dashboard can be deleted. Applications that have already been submitted/paid for are not able to be deleted.
To delete an application, locate the application you would like to delete in your Applications tab. When you hover your mouse over that application, you will see a Delete button next to the Continue button. Select the Delete to delete that application. But be careful! There is no way to undo this change.