If Apple could be considered niche back then, not so bad being niche, I guess. Estimates for total number of Amiga's sold over the years is 4.8 million - Apple sold nearly 400,000 Mac's the first year they were out, and 1 million a few years later (1 millionth Mac was made March 1987).
A common argument I know I saw first hand over here in Canada was that Commodore was perceived as a low cost, "toy" type computer company, left over from the C64 days. Of course, that's a totally wrong perception when it comes to the Amiga. Over in Europe, the Amiga was a far bigger seller and player. North America was always a weird market for the Miggy.
Often curious how sales would have went for C= had they only focused on the lower end, considering the vast amount of Amiga sales were not on the "big box" Amiga's, but the wedge ones like the A500. That being said, I likely wouldn't have ever owned an Amiga at all if the big box systems were never made

Never owned an A500, and didn't own an A1200 until 2009, I just didn't care for the wedge miggy's.