Creating mass merge envelope / label templates is basically the same as creating a normal mail merge document, for Envelopes and Label you will need to create them as per normal using Microsoft Word. (http://support.microsoft..com/kb/294685/)
Then similar to a normal merge document instead of adding in recipients, you can insert the merge fields into these document using the office edition integration or enter them in manually .(Creating New Mail Merge Templates)
Once all the merge fields has been entered and your document saved, you can upload it using the following steps, you will be prompted to pick what type of template you are uploading from a choice of Document, Envelope and Label. (Uploading Mail Merge Templates to Salesforce)
Additionally, you may refer to this Documentation (Managing Mail Merge Templates) to find a downloadable zip with sample templates.