React App - Worker & Markdown Render

背景

最近需要开发一个内部的文档编辑器, 解决旧编辑器的卡顿/支持功能较少的问题, 经过一段时间的调研和开发, 解决了各种问题, 这里记录一下几个重要的方案

需求

• markdown实时渲染, 所见即所得
• 大文件编辑/渲染不卡顿
• 支持mermaid

其实还有部分后台的需求如多人编辑的锁/多语言切换和某些自定义渲染页面等, 但是这些需求实现起来也不复杂, 所以这里不再记录

调研

markdown实时渲染

常见的库有两个: marked 和 remark

研究之后发现从可控性和易用性来说, 都是marked较好一点, 但是它的渲染速度比remark稍慢, 而在大文件渲染时它们都比较慢, 所以我们选择marked后只需要解决渲染较慢的问题

而且由于会有比较多的自定义功能, 也没有使用常见的react封装过的marked组件, 而是选择使用div来自己实现渲染数据的填充

编辑组件

也有比较多的现成markdown编辑组件, 但是仔细研究一遍过后, 发现居然还是回到了原始的codemirror和新兴的monaco-editor(vscode内置使用的编辑器), 他们都支持虚拟滚动和定位, 在超过2w行以上的数据量的时候也不会卡顿, 由于我们不绑定编辑器, 所以选择都支持, 并且提供根据喜好切换编辑器的功能

渲染

这块是需要最重点解决的问题, 旧的编辑器就是渲染卡顿导致页面切换时假死, 这里使用web-worker进行后台渲染解决页面卡顿的问题

mermaid

尝试了一下mermaid的相关api, 发现它的渲染需要依赖document, 不能放在web-worker中渲染, 所以最后选择了使用通过class标注, 然后在页面内实时渲染

实现

rpc-worker

ts对worker的支持如类型提示和定义等其实比较少的, 都是直接使用原始的postMessage和onmessage来接受和传输数据, 所以封装了一个通用的rpc-worker来对worker-server和worker-client实现类型支持

export type RpcWorkerOptions = {
  rpcTimeout?: number;
};
export type RpcWorkerAllOptions = {
  version: string;
  rpcTimeout?: number;
};

const defaultOptions: RpcWorkerAllOptions = {
  version: '0.0.1',
  rpcTimeout: 30 * 1000,
};

export type RpcWorkerServerHandler = Record<string, (value: any) => any>;

export type RpcWorkerRequest = {
  rpcId: string;
  method: string;
  data: any;
};

export type RpcWorkerResponse = {
  rpcId: string;
  data: any;
  error: any;
};

class RpcWorkerServer<T extends RpcWorkerServerHandler> {
  private ctx: Worker;
  private _rpcHandler: T;
  constructor(ctx: Worker) {
    this.ctx = ctx;
    this._rpcHandler = {} as T;
    this.ctx.onmessage = async (event) => {
      const data: RpcWorkerRequest = event.data;
      try {
        const result = await this._rpcHandler[data.method](data.data);
        this.sendMessage(data, result);
      } catch (e) {
        this.sendMessage(data, null, e);
      }
    };
  }

  addHandler<K extends keyof T>(key: K, fn: T[K]) {
    this._rpcHandler[key] = fn;
  }

  private postMessage(response: RpcWorkerResponse) {
    this.ctx.postMessage(response);
  }

  private sendMessage(request: RpcWorkerRequest, result: any, error: any = null) {
    this.postMessage({ rpcId: request.rpcId, data: result, error: error });
  }
}

class RpcWorkerClient<T extends RpcWorkerServerHandler> {
  private worker: Worker;
  private options: RpcWorkerAllOptions;
  private _rpcId: number;
  private _rpcHandler: {
    [key: string]: { resolve: (value: any) => void; reject: (error: any) => void };
  };

  public addEventListener: Worker['addEventListener'];

  constructor(worker: Worker, options?: RpcWorkerOptions) {
    this.worker = worker;
    this.options = { ...defaultOptions, ...options };
    this._rpcId = 1;
    this._rpcHandler = {};
    this.addEventListener = this.worker.addEventListener.bind(this.worker);
    this.worker.onmessage = (event) => {
      const response: RpcWorkerResponse = event.data;
      const _rpcId = response.rpcId;
      if (!this._rpcHandler[_rpcId]) {
        return;
      }
      if (response.error) {
        this._rpcHandler[_rpcId].reject(response.error);
      } else {
        this._rpcHandler[_rpcId].resolve(response.data);
      }
    };
  }

  terminate() {
    this.worker.terminate();
  }

  private getRpcId(): string {
    return (this._rpcId++).toString();
  }

  call<K extends keyof T>(method: K, data: Parameters<T[K]>[0], options?: RpcWorkerOptions): Promise<ReturnType<T[K]>> {
    return new Promise<ReturnType<T[K]>>((resolve, reject) => {
      const rpcId = this.getRpcId();
      let isTimeout = false;
      const timeout = options?.rpcTimeout || this.options.rpcTimeout;
      let timer: null | number = null;
      if (timeout) {
        timer = window.setTimeout(() => {
          isTimeout = true;
          reject(new Error(`Request Timeout: ${method}`));
        }, timeout);
      }

      this._rpcHandler[rpcId] = {
        resolve: (value: any) => {
          if (timer !== null && !isTimeout) {
            clearTimeout(timer);
          }
          if (!timer || !isTimeout) {
            resolve(value);
          }
        },
        reject: (error: any) => {
          if (timer !== null && !isTimeout) {
            clearTimeout(timer);
          }
          if (!timer || !isTimeout) {
            reject(error);
          }
        },
      };
      this.postMessage({
        rpcId: rpcId,
        method: method as string,
        data: data,
      });
    });
  }

  private postMessage(request: RpcWorkerRequest) {
    this.worker.postMessage(request);
  }
}

export { RpcWorkerServer, RpcWorkerClient };

webpack配置

这一块其实比较简单, 直接参考已有配置和worker-loader说明即可

const rules = [
  // for web worker
  {
    test: /\.worker\.ts$/, // ts结尾,这也很重要
    include: paths.appSrc,
    use: [
      {
        loader: 'worker-loader',
        options: {
          name: 'static/js/[name].[hash:8].js', // 打包后chunk的名称
          // inline: true, // 开启内联模式,免得爆缺少标签或者跨域的错误
        },
      },
      {
        loader: require.resolve('babel-loader'),
        options: {
          customize: require.resolve('babel-preset-react-app/webpack-overrides'),

          plugins: [
            [
              require.resolve('babel-plugin-named-asset-import'),
              {
                loaderMap: {
                  svg: {
                    ReactComponent: '@svgr/webpack?-svgo,+titleProp,+ref![path]',
                  },
                },
              },
            ],
          ],
          // This is a feature of `babel-loader` for webpack (not Babel itself).
          // It enables caching results in ./node_modules/.cache/babel-loader/
          // directory for faster rebuilds.
          cacheDirectory: true,
          // See #6846 for context on why cacheCompression is disabled
          cacheCompression: false,
          compact: isEnvProduction,
        },
      },
    ],
  },
];

render-worker

这里实现了RpcWorkerServer, 支持toc, sanitize防注入, prettier代码格式化以及大数据量时的优化, 图片鉴权, tab-code等功能

import { RpcWorkerServer } from './rpc-worker';
import marked, { Tokens } from 'marked';
import { TAB_CONTENT } from '../multi-tab';
import prettier from 'prettier/standalone';
import prettierParserMarkdown from 'prettier/parser-markdown';
import highlight from 'highlight.js';
import sanitizeHtml from 'sanitize-html';
import md5 from 'md5';

export type MarkdownWorkerToc = { id: string; anchor: string; level: number; text: string };

export type MarkdownWorkerMethod = {
  markdownRender: ({ text }: { text: string }) => { content: string; toc: MarkdownWorkerToc[] };
  prettierFormat: ({ text, bigTextLength, formatMd5List }: { text: string; bigTextLength: number; formatMd5List: string[] }) => {
    text: string;
    formatMd5List: string[];
  };
};

const longPrettierHeaderCount = 200;

// eslint-disable-next-line no-restricted-globals
const ctx: Worker = self as any;

const server = new RpcWorkerServer<MarkdownWorkerMethod>(ctx);

marked.setOptions({
  highlight: function (code, lang) {
    return highlight.highlight(highlight.getLanguage(lang) ? lang : 'plaintext', code, true).value;
  },
});

const UNIT_KEY = `//tab=unit_key_for_tab`;

server.addHandler('markdownRender', ({ text }) => {
  const renderer = new marked.Renderer();
  const _oldCode = renderer.code.bind(renderer);
  renderer.code = (code, language, ...args) => {
    if (code.indexOf(UNIT_KEY) > -1) {
      const tabStr = _.nth(_.split(code, '-'), 1);
      const finalCode = _.replace(code, `${UNIT_KEY}-${tabStr}-`, '');
      return `<div class="${TAB_CONTENT}" data-language="${tabStr}"><div class='code-container'><div class='copy-code-btn'></div>${_oldCode(
        finalCode,
        language,
        ...args,
      )}</div></div>`;
    }

    if (language === 'mermaid') {
      return '<div class="mermaid">' + code + '</div>';
    }

    return `<div class='code-container'><div class='copy-code-btn'></div>${_oldCode(code, language, ...args)}</div>`;
  };
  const toc: MarkdownWorkerToc[] = [];
  const levelNIndex: Record<number, number> = {};
  renderer.heading = function (text, level, raw, slugger) {
    if (!levelNIndex[level]) {
      levelNIndex[level] = 1;
    } else {
      levelNIndex[level] = levelNIndex[level] + 1;
    }

    const headerIdMatch = /\s+\{#([a-zA-Z0-9_-]+)\}$/gi.exec(text);
    const anchor = encodeURIComponent(headerIdMatch ? headerIdMatch[1] : slugger.slug(raw));
    text = text.replace(/\s+\{#([a-zA-Z0-9_-]+)\}$/gi, '');
    toc.push({
      id: anchor,
      anchor: `#${anchor}`,
      level: level,
      text: text,
    });
    if (level > 3) {
      return `<h${level} id='${anchor}' class='toc-id'>${text}</h${level}>\n`;
    } else {
      return `<h${level} id='${anchor}' class='toc-id'><a id='user-content-${anchor}' class='anchor' href='#${anchor}'><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>${text}</h${level}>\n`;
    }
  };

  const _oldImage = renderer.image.bind(renderer);
  renderer.image = function (href: string, title: string, text: string) {
    if (href.indexOf('.....') > -1) {
      href = `${'...'}?target=${encodeURIComponent(href)}`;
    }
    return _oldImage(href, title, text);
  };
  const _oldLink = renderer.link.bind(renderer);
  renderer.link = function (href: string, title: string, text: string) {
    if (href.indexOf('.....') > -1) {
      href = `${'...'}?target=${encodeURIComponent(href)}`;
    }
    return _oldLink(href, title, text);
  };

  let preLineTab = '';
  const html = marked(text, {
    renderer: renderer,
    walkTokens: (tokens) => {
      const token = _.first(_.castArray(tokens)) as Tokens.HTML | Tokens.Code;
      if (token.type === 'html') {
        const tabMatch = token.raw.match(/<!-- tab=(\S+) -->/);
        if (tabMatch) {
          preLineTab = tabMatch[1];
        }
      } else if (preLineTab && token.type === 'code') {
        token.text = `${UNIT_KEY}-${preLineTab}-${token.text}`;
        preLineTab = '';
      } else if (preLineTab && token.type !== 'code') {
        preLineTab = '';
      }
    },
  });

  const endHtml = `\n<div class='markdown-preview-padding-bottom'></div>\n`;
  return {
    content: sanitizeHtml(html + endHtml, {
      allowedTags: sanitizeHtml.defaults.allowedTags.concat(['input', 'img', 'svg', 'path', 'details', 'summary', 'del']),
      allowedAttributes: {
        '*': ['class', 'id', 'href', 'style', 'width', 'height', 'data-*'],
        input: ['checked', 'type', 'disabled'],
        img: ['src', 'alt'],
        svg: ['xmlns', 'viewBox', 'version', 'aria-hidden'],
        path: ['d', 'fill-rule'],
      },
      parser: {
        lowerCaseAttributeNames: false,
      },
    }),
    toc: toc,
  };
});

const longTextPritter = (text: string, formatMd5List: Set<string>): { text: string; formatMd5List: string[] } => {
  console.info(`in long text pritter`);
  const textLines = text.split('\n');
  let inCodeBlock = false;
  const headerBlocks: string[] = [];
  let currentHeaderLines: string[] = [];
  const newMd5List = Array.from(formatMd5List);
  for (let i = textLines.length - 1; i >= 0; i--) {
    const line = textLines[i];
    currentHeaderLines.unshift(line);
    if (!inCodeBlock && currentHeaderLines.length > longPrettierHeaderCount && /^#{1,6} /.test(line)) {
      headerBlocks.push(currentHeaderLines.join('\n'));
      currentHeaderLines = [];
    }
    if (line.startsWith('```')) {
      inCodeBlock = !inCodeBlock;
    }
  }
  if (currentHeaderLines.length) {
    headerBlocks.push(currentHeaderLines.join('\n'));
  }
  headerBlocks.reverse();
  _.map(headerBlocks, (headerText, index) => {
    const headerMd5 = md5(headerText);
    if (formatMd5List.has(headerMd5)) {
      return;
    } else {
      headerBlocks[index] = prettier.format(headerText, { parser: 'markdown', plugins: [prettierParserMarkdown], tabWidth: 2 });
      newMd5List.push(md5(headerBlocks[index]));
    }
  });
  return { text: headerBlocks.join('\n'), formatMd5List: newMd5List };
};

server.addHandler('prettierFormat', ({ text, bigTextLength, formatMd5List }) => {
  console.time('prettier');
  text = text.replace(/\t/gi, '    ');
  const result: { text: string; formatMd5List: string[] } =
    text.length > bigTextLength
      ? longTextPritter(text, new Set(formatMd5List))
      : {
          text: prettier.format(text, { parser: 'markdown', plugins: [prettierParserMarkdown], tabWidth: 2 }),
          formatMd5List: Array.from(formatMd5List),
        };
  console.timeEnd('prettier');
  return result;
});

export default null as any;

使用worker

这里还是可以加上一层本地缓存, 如果有缓存就不用再次渲染

import { MarkdownWorkerMethod } from './markdown.worker';
// @ts-ignore
import MarkdownWorker from './markdown.worker.ts';

const markdownWorkerClient = new RpcWorkerClient<MarkdownWorkerMethod>(new MarkdownWorker());

export const callMarkdownRender = async (allPath: string, content: string): Promise<ReturnType<MarkdownWorkerMethod['markdownRender']>> => {
  if (allPath && content && content.length > CONSTANT.BIG_CONTENT_LENGTH) {
    try {
      const renderCache = store.get('...') || {};
      const cache: { md5: string; renderResult: { content: string; toc: MarkdownWorkerToc[] } } = renderCache[allPath];
      const contentMd5 = md5(content);
      if (cache && cache.md5 === contentMd5) {
        return cache.renderResult;
      } else {
        const result = await markdownWorkerClient.call('markdownRender', {
          text: content,
        });
        renderCache[allPath] = {
          md5: contentMd5,
          renderResult: result,
        };
        store.set('...', renderCache);
        return result;
      }
    } catch (e) {
      console.warn(e);
    }
  }
  return await markdownWorkerClient.call('markdownRender', {
    text: content,
  });
};

mermaid加载

useLayoutEffect(() => {
  if (document.querySelector('.mermaid')) {
    mermaid.init('.mermaid');
  }
  initMultiTab('.markdown-body');
  initCopyCodeBtn();
}, [contentData]);

dangerouslyHTML

const DangerouslyHTMLDiv: React.FC<{ content: string }> = React.memo(({ content }) => <div dangerouslySetInnerHTML={{ __html: content }}></div>);

小结

这次的页面开发周期比较长, 中间也踩了很多坑, 最后这里也只记录了一小部分可复用的核心内容, 其他更多的问题往往是遇见后再去解决就可以了