I don't understand the tu quoque issue to have been so clear cut nor that the Dönitz ruling signaled acceptance of such a defense. In fact, I think that the principle of denying the tu quoque was, if a bit tortuously, maintained in the Dönitz judgment.
First, legal commentators seem generally to concur that tu quoque was not a permitted defense at the IMT.
For example, just two examples: (1) Robert Cryer, Håkan Friman, Darryl Robinson, & Elizabeth Wilmshurst, “the defense was not permitted to raise the issue of crimes committed by the Allies.” (An Introduction to International Criminal Law and Procedure, 2007) (2) Susan Mary Twist, for example, wrote, “the IMT ruled the defence of tu quoque inadmissible.” (“Retrospectivity at Nuremberg: The Nature and Limits of a Schmittian Analysis” (PhD dissertation, University of Central Lancashire, 2012)
However, the commentators also recognize the issue of submarine warfare and Dönitz’s defense in that matter.
Different defenses were raised in this matter at the IMT. Defense attorney Servatius, for example, argued that the orders were not illegal because the Hague Convention was obsolete. (Conot, Justice at Nuremberg, 1983, 2009, p 325)
The strongest case, however, was argued by defense attorney Kranzbühler who put it like this (IMT, Blue Series, v VIII, p 549):
In other words, Kranzbühler’s argument was that no one broke international law, not that everyone violated the law. The gray area is that Kranzbühler got to this proposition via the reality that everyone acted the same way. Persico describes Kranzbühler’s interrogatory as ingenious (Nuremberg: Infamy on Trial, 1994, p 338).
I understand Kranzbühler’s plea to be different to the case in which everyone is driving over the speed limit and the driver who gets pulled over pleads “they did it too" meaning that other drivers were also violating the speed limit. It is more like the driver who was singled out arguing that there was no speed limit because the limit didn't apply in the circumstances and thus everyone was within his or her rights to drive at fast speeds through the zone.
Conot makes this explicit in discussing Nimitz’s testimony: “Kranzbühler thus won his point. . . .[H]e had obtained the affirmation that American submarine practices had paralleled the Germans . . . and that, therefore, in practice, if not strictly in theory, German naval warfare had been ‘legal.’” 0p 417) Telford Taylor was of the same view as Conot, writing in The Anatomy of the Nuremberg Trials (1993, 2013) that Kranzbühler had “turned from the tu quoque to an issue of the interpretation of the London Agreement of 1936,” namely the technical definition of merchant vessels under that agreement (p 400).
The tribunal’s acceptance of Kranzbühler’s interrogatory spoke of its being “appropriate to construe the international law of submarine warfare by determining what actions were taken by the powers during the war.” (p 411) Taylor also wrote that the questions later put to Nimitz didn’t go to either the London Agreement or the definition of merchant ship, the very points on which Kranzbühler’s argument relied. (p 409)
All this, as Taylor noted, was not so clear as Kranzbühler’s argument but was not really the tribunal's admitting a tu quoque pleading either. Taylor’s opinion on the matter? “[T]he line between tu quoque and the meaning of ‘merchant ship’ in the London Charter was a very thin one” (p 409), but Nimitz’s testimony “had supported . . . Kranzbühler’s argument that armed merchant ships were not ‘merchant vessels’ within the meaning and protection of the 1939 Agreement . . .” (p 483)