I've read he ran into some scriptural disputes with a fair few of those Jews (seeing as how Islam comes from the Judeo-Christian tradition, but interpretations differing) which duly led to a falling out and sorting out the theological differences the old-fashioned way well liked by everyone with an army.

Which is about everyone on the Arabian peninsula those days, since around every tribe and community almost by definition also constituted a military power.

There's also the raw Realpolitik angle - Mohammed and his burgeoning Muslim community were fighting for their survival and jockeying for power in a rather complicated and fluid political scene where communitarian warfare and raiding was regarded as perfectly normal, and not a few of his allies were such for purely opportunistic reasons. Those obviously didn't actually care a jack about the sectarian issues but rather more about good old standbys like power, loot and glory, or siding with the apparent winning side (a few later champions of the Muslim conquests were originally some of the Prophet's most troublesome and bitter foes...). Plus sorting out old grudges if the opportunity presented itself.

So he orderd and islamic law :"Two religions may not dwell together on the arabian peninsula".
This sounds rather more like that one decree one of the first Caliphs made, IMO. And it has been pointed out in some sources that may well have been but an excuse to relocate certain non-Muslim communities of skilled armourers closer to the front line in the Levant and Mesopotamia.