Two factors differentiate a comet from an asteroid: orbit and chemical composition. Comets have eccentric orbits so their distance from the Sun varies considerably. The nucleus of a comet is composed of volatile material. When a comet is far from the sun, this material usually stays pristine but when the comet comes closer to the sun, solar radiation and solar winds cause it to lose some volatile compounds from its surface. This gives it a coma i.e. a nebulous appearance and a thin, transient atmosphere, which differentiates it from asteroids.
In 2006 a unified term was coined that encompasses both comets and asteroids: "Small Solar System Body".
Asteroids are named by their discoverers and comets are named for their discoverer. For example, Italian astronomer Giuseppe Piazzi discovered the first asteroid and named it Ceres. The comet discovered by Edmund Halley, who demonstrated that the comets of 1531, 1607, and 1682 were the same body and successfully predicted its return in 1759, is called Halley's Comet (or Comet Halley).
Asteroids are assigned both names and numbers because there are so many of them. They are numbered sequentially. The first asteroid, discovered by astronomer Giuseppe Piazzi in Italy, was named Ceres and is numbered 1. By 2009, about 450,000 asteroids had been discovered and 200,000 of them had been numbered. When a new asteroid is discovered, its orbital elements are calculated and then it is numbered. The discoverer then has a right to name the asteroid.
When comets are discovered by several persons simultaneously, they are given an impersonal designation. When they are discovered by an instrument and not a human, the name of the instrument is used as if it were a person's name. Official names of non-periodic comets begin with a "C"; comets that have been lost or disappeared have names that begin with a "D". Periodic comets' names begin with a "P" and "X" denotes a comet whose orbit could not be reliably calculated.
edit Dual listings
A few objects have ended up being dual-listed as both asteroid and comet because they were first classified as minor planets (asteroids) but later showed evidence of cometary activity. Conversely, when comets are depleted of their surface volatile ices, they become asteroids. Most asteroids with eccentric orbits are probably dormant or extinct comets.
edit Video explaining the difference
This video from the History channel explains the differences and similarities between asteroids and comets: