Jail: A place of detention; a place where a person convicted or suspected of a crime is detained.
Prison: A place of long-term confinement for those convicted of serious crimes.
The most notable difference is that prison inmates have been tried and convicted of crimes, while those in jail may be awaiting trial. A prison is under the jurisdiction of either federal or state, while the jail holds people accused under federal, state, county and/or city laws. A jail holds inmates from two days up to one year.
Note that this definition only applies in the United States of America.
edit In the United States
Jails are most often run by sheriffs and/or local governments and are designed to hold individuals awaiting dispostion of their case, waiting for transport to the state prison system after they have been convicted, or they are serving time on a misdemeanor sentence.
State Prisons are operated by the state where the person was convicted of a felony.
Federal Prisons are operated by the Federal Bureau of Prisons (BOP) and are designed to hold individuals convicted of federal crimes such as racketeering (RICO), bank robbery, and others.
Jails operate work release programs, boot camps, and other specialized services. They try to address educational needs, substance abuse needs, and vocational needs while managing inmate behavior. Inmate idleness contributes to management problems.
State prison systems operate halfway houses, work release centers and community restitution centers - all considered medium or minimum custody. Inmates assigned to such facilities are usually reaching the end of their sentences.
edit Video explaining the differences