1. Problem Statement - Migration to stricter config could be tough

Like it or not, TypeScript is the de facto language choice for front-end projects nowadays. The benefit of stricter typings are obvious, it adds a layer of typings on our code to both help us code faster and also with less bugs.

There is no doubt that stricter is better since it is the reason to adopt TypeScript in the first place. Yet though TypeScript was launched 10+ years ago, I still see projects with loose config. This could be resulted from a lot of reasons but could simply summarized as migration to stricter config requires too many code changes.

The concern is reasonable.

1. Not enough bandwidth to make the changes.
2. Code base is too huge and owned by different people, one might not be confident enough to touch other people’s code.
3. It is hard to verify the changes.
4. Not able to prioritize the migration.
5. …

Actually it doesn’t need to be so painful as here I’m going to introduce you the way of doing it right and with joy.

2. Solution Proposal - Migrate to stricter config progressively

The core idea is Be Progressive, don’t aim at solving all issues at one pull request. The implementation could vary based on your requirements.

You can break down the migration into smaller tasks by files, by user flows or by config rules, and then use a longer time span to tackle them.

Or you can just fix the issues of files touched in each of your pull requests, and just ignore the files you never touch. This results in files being fixed in the order of their hotness, which is quite fair because if you never touch one piece of code, there is no need to fix them.

I’d like to give an example on the second approach because it means we don’t need prioritize special tasks for the migration thus addresses the issue of not enough bandwidth.

3. Code Demo - progressively migrate noImplicitAny to true.

This is a common issue when migrating from existing JavaScript project. noImplicitAny is recommended to add better type safety.

In order to turn it on progressively, I’d like to

1. create a script that filter out errors reported by tsc
   1. newly introduced errors
   2. existing errors of implicit any from touched files
2. set up CI with above script to block pull requests
3. create a script to track errors of implicit any
4. set up CI with above script to track the progress and attribute impact to individuals.

The requirement is simple:

1. input: two commit hashes
2. output: tsc errors that are newly introduced or in the changed files.

I’ve created a demo repo that has implicit any(commit), it is very simple and has multiple errors across files.

ts
// a.ts
export function a(t) {
  return String(t);
}
// b.ts
export function b(t) {
  return String(t);
}

ts
// a.ts
export function a(t) {
  return String(t);
}
// b.ts
export function b(t) {
  return String(t);
}

The tsconfig is not strict so above code doesn’t get complained.

json
{
  "compilerOptions": {
    "module": "esnext",
    "outDir": "dist"
  },
  "include": ["src/**/*"]
}

json
{
  "compilerOptions": {
    "module": "esnext",
    "outDir": "dist"
  },
  "include": ["src/**/*"]
}

If we turn on noImplicitAny, all errors are reported which is not scalable.

ts
// a.ts
export function a(t) {
Parameter 't' implicitly has an 'any' type.7006Parameter 't' implicitly has an 'any' type.
  return String(t);
}
 
// b.ts
export function b(t) {
Parameter 't' implicitly has an 'any' type.7006Parameter 't' implicitly has an 'any' type.
  return String(t);
}

ts
// a.ts
export function a(t) {
Parameter 't' implicitly has an 'any' type.7006Parameter 't' implicitly has an 'any' type.
  return String(t);
}
 
// b.ts
export function b(t) {
Parameter 't' implicitly has an 'any' type.7006Parameter 't' implicitly has an 'any' type.
  return String(t);
}

To migrate it progressively, first of all, let’s create a function to collect errors reported by tsc that forces noImplicitAny to be true.

import ts from "typescript";
function collectTscDiagnostics(
  basePath: string,
  pathToConfig: string,
  extraCompilerOptions?: Record<string, any>
) {
  const { config } = ts.readConfigFile(pathToConfig, ts.sys.readFile);
  if (config == null) {
    throw new Error("unable to read config file:" + pathToConfig);
  }
  config.compilerOptions = {
    ...config.compilerOptions,
    ...extraCompilerOptions,
  };
  const { options, fileNames, errors } = ts.parseJsonConfigFileContent(
    config,
    ts.sys,
    basePath
  );
  const program = ts.createProgram({
    options,
    rootNames: fileNames,
  });
  const diagnostics = ts.getPreEmitDiagnostics(program).map((diagnostic) => ({
    path: diagnostic.file.fileName,
    start: diagnostic.start,
    length: diagnostic.length,
    code: diagnostic.code,
    message: diagnostic.messageText,
  }));
  return diagnostics;
}
collectTscDiagnostics(__dirname + "/../../", "./tsconfig.json", {
  noImplicitAny: true,
});

import ts from "typescript";
function collectTscDiagnostics(
  basePath: string,
  pathToConfig: string,
  extraCompilerOptions?: Record<string, any>
) {
  const { config } = ts.readConfigFile(pathToConfig, ts.sys.readFile);
  if (config == null) {
    throw new Error("unable to read config file:" + pathToConfig);
  }
  config.compilerOptions = {
    ...config.compilerOptions,
    ...extraCompilerOptions,
  };
  const { options, fileNames, errors } = ts.parseJsonConfigFileContent(
    config,
    ts.sys,
    basePath
  );
  const program = ts.createProgram({
    options,
    rootNames: fileNames,
  });
  const diagnostics = ts.getPreEmitDiagnostics(program).map((diagnostic) => ({
    path: diagnostic.file.fileName,
    start: diagnostic.start,
    length: diagnostic.length,
    code: diagnostic.code,
    message: diagnostic.messageText,
  }));
  return diagnostics;
}
collectTscDiagnostics(__dirname + "/../../", "./tsconfig.json", {
  noImplicitAny: true,
});

With above code, even we don’t set noImplicitAny in tsconfig, we are still able to get errors reported.

[
  {
    path: '/Users/Documents/src/demo-ts-progressive-migration/src/a.ts',
    start: 18,
    length: 1,
    code: 7006,
    message: "Parameter 't' implicitly has an 'any' type."
------------------------------------------------------
  },
  {
    path: '/Users/Documents/src/demo-ts-progressive-migration/src/b.ts',
    start: 18,
    length: 1,
    code: 7006,
    message: "Parameter 't' implicitly has an 'any' type."
------------------------------------------------------
  }
]

[
  {
    path: '/Users/Documents/src/demo-ts-progressive-migration/src/a.ts',
    start: 18,
    length: 1,
    code: 7006,
    message: "Parameter 't' implicitly has an 'any' type."
------------------------------------------------------
  },
  {
    path: '/Users/Documents/src/demo-ts-progressive-migration/src/b.ts',
    start: 18,
    length: 1,
    code: 7006,
    message: "Parameter 't' implicitly has an 'any' type."
------------------------------------------------------
  }
]

Obviously what we need to do next, is to collect errors from both commits and then try to diff and keep the necessary ones to fix.

function getAllDiagnostics(basePath: string) {
  return collectTscDiagnostics(basePath, "./tsconfig.json", {
    noImplicitAny: true,
  });
}
async function diffTscDiagnostics(from: string, to: string, basePath: string) {
  const currentBranch = (await git.branch()).current;
  git.checkout(from);
  const diagnosticsFrom = getAllDiagnostics(basePath);
  git.checkout(to);
  const diagnosticsTo = getAllDiagnostics(basePath);
  git.checkout(currentBranch);
  // we need to filter out new issues and existing issues in diff list
  const existingDiagnostics = new Map<string, Diagnostic>();
  diagnosticsFrom.forEach((diagnostic) => {
    const hash = JSON.stringify(diagnostic);
    existingDiagnostics.set(hash, diagnostic);
  });
  const diff = await git.diffSummary([from, to]);
  const changedFiles = new Set(
    diff.files.map((file) => path.resolve(basePath + file.file))
  );
  const diagnosticsToFix = diagnosticsTo.filter((diagnostic) => {
    if (diagnostic.path == null) return false;
    const hash = JSON.stringify(diagnostic);
    return !existingDiagnostics.has(hash) || changedFiles.has(diagnostic.path);
-------------------------------------------------------------------
Only report new errors and errors from touched files
  });
  return diagnosticsToFix;
}

function getAllDiagnostics(basePath: string) {
  return collectTscDiagnostics(basePath, "./tsconfig.json", {
    noImplicitAny: true,
  });
}
async function diffTscDiagnostics(from: string, to: string, basePath: string) {
  const currentBranch = (await git.branch()).current;
  git.checkout(from);
  const diagnosticsFrom = getAllDiagnostics(basePath);
  git.checkout(to);
  const diagnosticsTo = getAllDiagnostics(basePath);
  git.checkout(currentBranch);
  // we need to filter out new issues and existing issues in diff list
  const existingDiagnostics = new Map<string, Diagnostic>();
  diagnosticsFrom.forEach((diagnostic) => {
    const hash = JSON.stringify(diagnostic);
    existingDiagnostics.set(hash, diagnostic);
  });
  const diff = await git.diffSummary([from, to]);
  const changedFiles = new Set(
    diff.files.map((file) => path.resolve(basePath + file.file))
  );
  const diagnosticsToFix = diagnosticsTo.filter((diagnostic) => {
    if (diagnostic.path == null) return false;
    const hash = JSON.stringify(diagnostic);
    return !existingDiagnostics.has(hash) || changedFiles.has(diagnostic.path);
-------------------------------------------------------------------
Only report new errors and errors from touched files
  });
  return diagnosticsToFix;
}

Now edit the b.ts by adding a comment which should not affect diagnostics.

diff
+// a comment
export function b(t) {
  return String(t);
}

diff
+// a comment
export function b(t) {
  return String(t);
}

Then diff the issues from previous commit.

typescript
diffTscDiagnostics(
  "1d67625d3991525cda22d42f077dfd54c364a899",
  "225a888521ea8ddb48d2c514488897d8bbfd8f84",
  __dirname + "/../../"
).then(console.log);

typescript
diffTscDiagnostics(
  "1d67625d3991525cda22d42f077dfd54c364a899",
  "225a888521ea8ddb48d2c514488897d8bbfd8f84",
  __dirname + "/../../"
).then(console.log);

We can see that existing errors from b.ts is reported while still ignoring the one from a.ts.

$ npm run migrate
> [email protected] migrate
> ts-node ./src/scripts/migrate.ts
[
  {
    path: '/Users/Documents/src/demo-ts-progressive-migration/src/b.ts',
    start: 31,
    length: 1,
    code: 7006,
    message: "Parameter 't' implicitly has an 'any' type."
  }
Now we only get errors from b.ts since only it got touched!
]

$ npm run migrate
> [email protected] migrate
> ts-node ./src/scripts/migrate.ts
[
  {
    path: '/Users/Documents/src/demo-ts-progressive-migration/src/b.ts',
    start: 31,
    length: 1,
    code: 7006,
    message: "Parameter 't' implicitly has an 'any' type."
  }
Now we only get errors from b.ts since only it got touched!
]

Obviously there needs to be more of this code to work on CI, like sending comments on the pull request, but it could be easily done by calling github API, you can figure it yourself.

The full code is on github.

4. Summary

In this post, I’ve demonstrated the power of being progressive in code migration, it could be useful in migrating JavaScript project to TypeScript, or to introducing stricter rules in tsconfig.

The approach, or the idea itself, could actually be applied to anything that needs to be broke down. How do you like it?