It's a tradition that dates back to when the 1st Kentucky Derby first originated in 1875.
Back in the day : A woman's crowning glory was the hat. Especially popular in the south where the southern belle's would adorn themselves with this centerpieces.
The culture is actually a hybrid between southern and British tradition.
According to the Kentucky Derby's website, this tradition was started by founder Col. Meriwether Lewis Clark, Jr. , who shaped the horse race after other high-class European racing events.
The hat-wearing tradition doesn't stop in the USA. It actually can be found as a popular adornment across the great seas and in many other countries around the world.