There is sugar, absolutely. And that's probably where most of the calories come from. But there is also water, cellulose (fiber), and vitamins/minerals - doesn't have much non-sugar caloric value to change that balance, but it's still important. And nobody serious is suggesting you eat only fruit, so you can get non-sugar calories from other sources and it can be balanced in the big picture.
It's kind of like an appropriate amount of dressing on a salad, the good outweighs the bad and makes you more likely to actually eat that nutrition-positive food.
Source: I'm some guy on the Internet. You can trust me.