Plainly, that is not the meaning of 'division of labour'.
The necessity for a 'division of labour' arises in the first instance, from the plurality of human needs, and the plurality of natural resources. At any one point in time, it is not possible for all people to do the same thing, or indeed for them as a totality to do things that do not correspond to the realisation of human need. But there is no need for each of them individually to do the same thing in perpetuity.
As the quotation that you cite suggests, even the capitalist system fragments that narrow specialisation typical of craft production, by constantly revolutionising (albeit in a haphazard and destructive way) the means of production (hence, for example, Kent miners retrain as nightclub bouncers, or sociologists, as their first skill is no longer needed).
There is no contradiction between the German Ideology piece and the later one, except that Marx is clearer in the later, using the phrase 'division of labour' with greater precision, distinguishing between its transhistorical meaning, and its epoch-specific meaning (most explicitly in the letter to Kugleman, just quoted).
It seems to me that your difficulty with Marx is your difficulty, and no-one else's (well, maybe Horkheimer's or Marcuse's, but that is not company you should want to keep, at least not in this respect). The *technical* division of labour carries no horrors, it is just what it is - like the rain. The particular social form of the division of labour, that is, as attained through the exchange of commodities, has become a barrier to human self-realisation.