A cover letter should not be confused with a query letter. And, while most short story markets DO NOT require a query letter, they do like to have a brief cover letter. Fortunately, cover letters are one of the easiest parts of the submission “extras” to write.
Here are the basics.
Your letter should begin with Dear [Editor]. Find the name of the editor of the publication you're submitting to. It is usually available in the “About Us” info on the magazine's website. If it's a large publication there may be separate editors for different departments. i.e. Fantasy stories may go to one editor while sci-fi goes to someone else. If this is the case, make sure you get the correct name. Even if a publication doesn't have different editors for different departments, chances are they do have more than one editor. The general rule of thumb is to address the letter to the Grand High Poohbah A.K.A: the senior editor.
Next should be a sentence that looks something like this: Attached is my short story (The Weather is Always Fine in Paradise) of approximately 7000 words. It is important to note that most publications DO NOT want a logline, summary or synopsis (however brief) of your short story. In fact, many submission guidelines specify the story should be able to stand on it's own and they will find out what it's about when they read it. The only thing they want to know is the title of the story and how long it is. If you're dealing with publications that do take simultaneous submissions, this is also the place to say if a particular story is being subbed elsewhere.
Thirdly, unless specified in the submission guidelines you DO NOT need to provide an author's bio in your cover letter. You can provide a few details about your writing history such as: I have a B.A. in Creative Writing/English/Speculative Fiction/Whatever. My stories have been published online/in print with Magazine X, Anthology Y and Publication Z. My rule of thumb is to mention whatever publication credits you have, up to three or four. Once you have more than that, mention the most recent/relevant.
You DON'T need to state you've been writing since the age of seven or that storytelling is your passion. The first is irrelevant, the second is usually a given.
If, however, you've met the editor before and been encouraged to submit you might put in a line reminding them of that meeting. Or if you've submitted to the publication before and been rejected but encouraged to submit something else, you can say that too. But keep it brief and business-like.
This is also the place to put down any “special circumstances” that apply to the story you are submitting. This could be: My high school paper published this story 14 years ago. Or: I spent two years working as a crew member on a sailing ship/zeppelin/velocipede team. DO NOT say something like: I saw an episode of Nova on PBS and was inspired to write about black holes.
If you haven't been published before and/or don't have any background writing fiction THAT IS OKAY. Everyone starts somewhere. But (BUT!) don't put that in the cover letter. Just skip this paragraph and head straight for the last section.
Fourthly, a brief line thanking the editor is always nice. I usually say: Thank you for your time and consideration.
Lastly, put your name.
The only remaining question is: is a cover letter necessary?
Some publications not only don't require one, but even say NOT to include one unless there is something specific (not covered in the other parts of the submission form) that needs to be brought to the editors attention. (Daily Science Fiction and Strange Horizons both politely discourage cover letters unless absolutely necessary.) And, while there are some publications that DO require a cover letter, many fall in the middle ground, neither requiring nor discouraging them.
Here's my two cents. If you have ANY publication credits (whether pro- semi- or otherwise) it never hurts to mention them. (This is assuming they are relevant credits. Non-fiction sales will mean little to fantasy magazine.) And, in my mind, it never hurts to show a little extra effort in the submission process by actually thanking the editor (or slush-reader) for their time. This is the handshake at the end of the job interview. By itself it won't make a difference between “Yes, we want it” and “No, thanks”. But it can show you are serious about what you are doing.
So, what have you submitted today?