That's essentially the crux of the matter Takk.
People have placed a huge amount of importance in the presence of alternative weapons such as swords (let's ignore the fact that the main weapon in The Spirits Within, a gun, also features heavily in other Final Fantasy titles - Barret, Vincent, Balthier, Advent Children, X-2 and so on

).
Couple that with the lack of a magic spell or two and suddenly you have a whole bunch of people who dislike the movie as a whole.
For that matter, how much magic use do we actually see in the likes of Advent Children? Kadaj summons Bahamut, Loz (arguably) equips Haste and we see a case of materia lying in the Sector 7 Church. There is perhaps more examples that I have forgotten but the point is, magic doesn't play a major role in the movie. The majority of combat is swordplay.
By having such a narrow definition of what Final Fantasy should be, that is, it must have magic, summons and perhaps a sword featuring, we really limit any potential that could be achieved.
It also means that if Square had to stick to the formula: Final Fantasy = Swords + Magic then any new productions, be it games or movies, will just be more of the same old thing.
I really thought The Spirits Within threw a good mix of new ideas in and really bringing something refreshingly different to the series.
I think most of the criticism this film faces is from a few die-hards that like the comfort of familiar territory.
