"Weak labor market conditions, generally soft business conditions and waning public confidence in questionable business practices have helped erode consumer confidence," said Lynn Franco, director of the Conference Board's Consumer Research Center.
Franco added that the latest readings still point to continued consumer spending and moderate economic growth.
The group said the index fell to 106.4 from an upwardly revised 110.3 in May, roughly in line with analysts' expectations. But the index remained above the 7-1/2 year low of 84.9 hit last November.
The Present Situation Index, which measures Americans' views of the economy right now, fell to 105.7 in June from 111.2 in May. The Expectations Index, a gauge of consumers' six-month outlook, fell to 106.9 in June from 109.7 in May.
The Conference Board surveys 5,000 households to compile its monthly report.