The answer isn’t really that complicated.
If you eat an abundance of sugar, then yes, you will gain weight. The same can be said if you ate an abundance of carrots, or an abundance of whole-grain bread.
The miscommunication here is that many of us are under the mistaken belief that a calorie represents a unit of nutrition. Technically, that’s not true. Calories are a unit of energy, and are a component (a subset, if you will) of a nutrient. The definition is nuanced here, certainly, but the difference is important. Calling a calorie a nutrient is akin to calling a spark plug a car.
For the record, nutrients are classified as carbohydrates, dietary fiber, fats, minerals, protein, vitamins, and water.
Sugar has calories, and in the long term, all calories are equal. The difference between 100 calories of sugar and 100 calories of carrots is that the carrots have far more nutritional value when consumed, as they contain more fiber, vitamins, minerals, etc. The carrots, therefore, bring more to the table. Sugar, on the other hand, does not have such added value. As your body works better with Vitamin A and other nutrients than without, two options are available to the person who chooses to eat 100 calories of sugar – either eat the carrots and get the necessary vitamin A and deal with the additional calories consumed, or go without the carrots , and deal with the ramifications of not getting Vitamin A. As we, as a country, have ready access to a full bounty of food, we often opt for the former.
One Side note here – only tangentially related. Did you know that there’s technically no such thing as baby carrots? The nubs you find in the grocery store are actually fully grown carrots that have been whittled down. Oh the things I learn.