This graph shows how the Canadian federal parties built and retired debt from 1968 to 2016. You can see that the vast bulk of the debt was generated by Tory governments (in blue). The figures come from Statistics Canada.
Note that after a change of government, it takes time to reverse fiscal policies. Thus, the third Liberal government took a few years to start reversing the earlier debt and the third Conservative government took a few years to start rebuilding it.