Find unused methods and variables in code

Tooling API can be used to retrieve the SymbolTable for each apex class. This Symbol Table will tell you the methods that are defined in the apex class, and the references to methods that are consumed in the externalReferences.

By scanning the methods and corresponding references in each apex classes symbol table you can find methods that aren't called by other apex classes.

There is already such a tool available as a canvas app. See Spring Cleaning Apex Code with the Tooling API.

Note, as this is only scanning Apex classes it won't detect declarative references from Visualforce markup.