Joel was born and raised in the Dunean and Judson textile communities on the Mill Village in Greenville, South Carolina. He graduated from Parker High School in 1954. After working a few years and serving in the Army, he went on to receive his Bachelor of Science degree in Education from Clemson University. Just prior to graduating from Clemson, Joel met the love of his life, Patricia. They have been married for over 44 years.
Beginning in 1955, Joel began working at Judson Mill and continued working there and at other plants with Milliken and Company (the parent company of Judson), until he retired from Milliken and Company in 2002. So, for 47 years his entire Textile career was with Milliken and Company.
Joel was raised in a Christian family who attended Tremont Avenue Church of God. He still is a member of this church. Joel attributes his Christian upbringing, his belief in Jesus Christ, and his family - then and now - to making him the person he is today.