Typically, record access is determined by Ownership, Role Hierarchy, Territory Hierarchy, Sharing Rules, Profile settings (View/Manage All Data), and Team membership (ex. Account Teams).
However, a user might find that he or she is able to view a record that would not normally be accessible per the normal security settings.
To understand why a user can view a particular Account and its related records, the System Administrator may click on the Sharing button on the Account in question. This will list all the users with access to that Account, as well as the reason why.
If the sharing reason states "Associated record owner or sharing", this Account is visible to that user because he or she has read access to a related record (Contact, Opportunity, etc.). Additionally, having Read access to the Account will grant Read access to the Contacts associated to that Account.
Examples of this behavior:
Per the org's security settings, User A is typically unable to see User B's Account. However, User A owns an Opportunity related to User B's Account. User A will then have Read-Only access to User B's Account and the related Contacts.
Further, User C does not typically have access to User A and User B's Accounts and Opportunities. However, User A grants Read-Only sharing access for the Opportunity to User C. User C now has Read-Only access to the Opportunity and *also* Read-Only access to the Account and to the Contacts associated with that Account.