We completely describe the algebraic part of the rational cohomology of the Torelli groups of the manifolds $\#gSn\times Sn$ relative to a disc in a stable range, for $2n\ge 6$. Our calculation is also valid for $2n=2$ assuming that the rational cohomology groups of these Torelli groups are finite dimensional in a stable range.