Advanced DocumentRegistry: what's expected of SourceFile

[mihailik]

I want to speed up the startup of language service. The biggest hit by far is parsing of lib.d.ts, which may easily take many seconds on underpowered devices.

Wiki page Using the Language Service API suggests I can implement a custom document service, which would cache SourceFiles.

Sounds good: lib.d.ts virtually never changes and can be cached safely. Hopefully, eliminating parsing and boosting startup speeds.

But what is the nature of the data that should be cached? I can see SourceFile interface is not quite clear on what it is holding. Note the /* @internal */ members, are those expected to be cached/restored too?

    export interface SourceFile {
        /* @internal */ version: string;
        /* @internal */ scriptSnapshot: IScriptSnapshot;
        /* @internal */ nameTable: Map<string>;

        /* @internal */ getNamedDeclarations(): Map<Declaration[]>;

        getLineAndCharacterOfPosition(pos: number): LineAndCharacter;
        getLineStarts(): number[];
        getPositionOfLineAndCharacter(line: number, character: number): number;
        update(newText: string, textChangeRange: TextChangeRange): SourceFile;
    }

Comments on DocumentService.acquireDocument give a slight hint that the main piece of information SourceFile holds is IScriptSnapshot. Is that right? Or is it more to it?

Many thanks!

[mhegazy]

So just to be clear do you want to serialize a SourceFile instance and re-hydrate it later on? or do you want to persist a single instance of 'lib.d.ts' source file instance in memory across different compilations?

[mihailik]

The former: serialize SourceFile for lib.d.ts and re-hydrate it for faster startup in subsequent sessions (assuming re-hydration is signifcantly faster than creating SourceFile from text).

[mhegazy]

then that is an uncharted territory:) the interface you want to look at is the one types.ts, this is the actual object that the compiler creates. the LS code augments that with some additional helpers by setting the prototype at: https://github.com/Microsoft/TypeScript/blob/master/src/services/services.ts#L7348

I think it should just work if you do not set the parent pointers when you parse the file (i.e. ts.createSoruceFile). you will need to filter out diagnotics, as they can have circular references, but it might be safe to say do not serialize if there are any parse errors. I do not think you need to care about the table either.

let us know how this goes, and the perf characteristics you get at the end. this sounds like an interesting investigation.

[mihailik]

Not entirely clear still, but hopefully it gets easier as I go along :-)

Thanks, if I get any results will report back.

[mhegazy]

there are two interfaces for SoruceFile. 1. in types.ts: https://github.com/Microsoft/TypeScript/blob/master/src/compiler/types.ts#L1214
and 2. in services.ts: https://github.com/Microsoft/TypeScript/blob/master/src/services/services.ts#L61

The one in types.ts is the truth. it is all data properties, no methods. this is what you get back when you call ts.createSoruceFile, the methods defined in the second interface declaration (services.ts) is added by patching the prototype, so that should be doable again once you have re-hydrated the object.

SourceFile is just a Node, and Nodes should be serializable, modulo parent pointers (as they introduce cycles). diagnostics also introduce cycles as a diagnostic object has a reference to a SoruceFile itself. so assuming you ignore properties that are of Diagnostics[], you should be able to serialize and deserialize the sourcefFile object.

[mihailik]

Thanks a lot 👍