The apparel industry has a big problem. At a time when the economy is growing, unemployment is low, wages are rebounding and consumers are eager to buy, Americans are spending less and less on clothing. The woes of retailers are often blamed on Amazon.com Inc. and its vise grip on e-commerce shoppers. Consumers glued to their phones would rather browse online instead of venturing out to their local malls, and that’s crushed sales and hastened the bankruptcies of brick-and-mortar stalwarts from American Apparel to Wet Seal. But that’s not the whole story. Apparel is being displaced by travel, eating out and activities—what’s routinely lumped together as “experiences”—which have grown to 18 percent of purchases.