The easiest way is to start with a proper folder structure and work for there.
The folder\File structure I use looks like this
and a file might look like this
..\ACDC\Back In Black\01\01-ACDC-Hells Bells.flac.
I was annoyed for awhile until I figured out how to zero file tracks/discs numbers.
The key to this approach is to have proper ID3tag's
If you use a Tag editor you should be able to rename your existing files and move them to a new folder (back up the source files/folder first).
What should happen is that when you try to update a file that already exists it should warn you about duplicates or give you an error message.
The software I use is called MP3TAG and it will display a dialogue box with a list of files it couldn't move.
I recently did something similar to what you are after where there were thousands of files in a single folder.
I used this software to read the tags based on the filenames and then I was able to move them into subfolders.
Once you have moved your files then what's left should be anomalies/mismatched/duplicate files.
You will spend a fair amount of time organising your collections and making sure the tags are correct but I am sure you will be happy with the results.
You might even find some tracks that you forgot you had.
Maybe we should start a separate discussion on what tags people use.