preface

Reprinted from Sohu - front end Engineering - building enterprise general scaffold

With the concept of front-end engineering getting deeper and deeper into FEer's heart, the standardization and standardization of technology selection, code specification, construction and release processes in the front-end development process need tools to escort, rather than manually copying and pasting repeated work every time. Scaffold can be used as an engineering auxiliary tool to improve the efficiency of front-end R & D to a great extent.

What is a scaffold?

What is the scaffold?

In previous work, we may need to do the following before we can start writing business code:

• Technology selection
• Initialize the project, select package management tools, and install dependencies
• Write basic configuration item
• Configure the local service and start the project
• Start coding

With the rise of Vue/React, we can use the official scaffold Vue cli or create react app to quickly generate projects according to our requirements and preferences through selection or input on the command line. They allow us to focus on code rather than building tools.

Scaffold capacity

However, these scaffolds are specific to specific languages (Vue/React), and in our actual work, different BU is specific to different ends (PC, Wap, applet...) The technology stacks adopted may also be different. Often, the technology stacks adopted at a specific end can be reused in other similar projects to a certain extent. We expect to build projects with different technology stacks at different ends through several commands, selection and input on the command line.

The above is just an example of a new project. The front-end development process is more than that. Generally, there are the following scenarios:

• Create project + integration common code. The project template contains a large number of general codes, such as general tools and methods, general styles, general request library, HTTP request processing, internal component library, buried point monitoring
• Git operation. Generally, you need to manually create a warehouse, solve code conflicts, remote code synchronization, create a version, publish and Tag in Gitlab And so on.
• CICD. After the business code is written, you also need to build and package it, upload the server, bind the domain name, distinguish the formal environment for testing, and support rollback Such as continuous integration and continuous deployment.

Why not use automated build tools

In general, we will use Jenkins, Gitlab CI, Webhooks, etc. for automatic construction. Why do we need scaffolds?

Because these automated construction tools are executed on the server, the local functions of R & D students cannot be covered in the cloud, such as the above creation projects, local Git operations, etc; In addition, the customization process of these automated tools requires the development of plug-ins. The front-end students need to learn the language and implementation at a certain time cost. The front-end students also expect to realize these functions only by using JavaScript.

Scaffold core value

In conclusion, the existence of front-end scaffold is of great significance. The core goal of scaffold is to improve the efficiency of the whole front-end R & D process.

• Automation. Avoid duplicate code copy and deletion of the project; Automate Git operations throughout the project cycle.
• Standardization. Quickly create projects based on templates; Provide CICD capability.
• Data. Through the statistics of the embedded points of the scaffold itself, the time-consuming is quantified to form an intuitive comparison.

Often, companies implement a complete code release management system similar to Git operation and CICD for some automated and standardized functions to help us manage projects on Gitlab and provide the ability of continuous integration and deployment. What's more, projects for small programs will also conduct code release management and standardize them.

We may just need to consider

• Create project + integrate common code
• Solutions to common pain points (quickly generate pages and configure routes...)
• Configuration (eslint, tsconfig, prettier...)
• Efficiency improvement tool (copy various files)
• Plug in (solve a problem in the webpack build process...)
• ...

The following describes our attempts based on these scenarios within the company.

Use scaffolding

First, create a new project on the terminal through the command focus create projectName. Where focus represents the main command, create represents the command, and projectName represents the param of the command. Then select and input the final generated project according to the terminal interaction.

We provide different template projects for each BU, each end and each technology stack. At the same time, each student can precipitate and refine the projects in the group into a template project, and integrate them into the scaffold according to certain specifications to feed the whole BU.

@focus/cli architecture

The following architecture diagram is adopted Lerna As a project management tool, at present babel,vue-cli,create-react-app Large projects are managed by Lerna. Its advantages are:

• Significantly reduce repetitive operations. The local link, unit test, code submission and code release of multiple packages can be operated by Lerna one click.
• Improve the standardization of operation. The release versions and interdependencies of multiple packages can be consistent through Lerna.

In @ focus/cli scaffold, split according to function:

• @Main functions of focus/cli storage scaffold

  • focus create projectName pull template project
  • focus add material creates a new material, which can be a package, page, component The particle size can be large or small
  • focus cache clears the cache, configuration file information, and temporarily stored templates
  • focus domain copy configuration file
  • focus upgrade updates the scaffold version and also has an automatic query update mechanism
• @Focus / eslint config focus Fe unified eslint rules in the storage group
• You can also create a new sub Package through focus add material to realize specific functions

Dependency overview

The core function of a scaffold needs to be supported by the following basic libraries.

• chalk : console character style
• commander : node. Complete solution of JS command line interface
• fs-extra : enhanced base file operation Library
• inquirer : enables interaction between command lines
• ora : elegant terminal Spinner waiting animation
• axios : combine Gitlab API to obtain warehouse list, Tags
• download-git-repo : pull warehouse code from Github/Gitlab
• consolidate : template engine consolidation library. It mainly uses ejs to realize template character replacement
• ncp : copy directories and files like cp -r
• metalsmith : pluggable static website generator; For example, after obtaining the final content after user-defined input or selecting the rendering variable with ejs, insert and modify it.
• semver : get the valid version number of the library
• ini : an ini format parser and serializer for nodes. It mainly encodes and decodes the configuration.
• jscodeshift : the file can be parsed to convert the code from AST to ast. For example, after creating a new page, you need to create a new page in routes Create a new route in TS.

Typescript coding and babel compilation.

In addition to tsc, babel7 can also compile typescript code, which is the result of one year's cooperation between the two teams.
However, because of the characteristics of single file compilation, babel cannot achieve the same effect as tsc's multi file type compilation. Several features are not supported (mainly namespace cross file merging and export of non const values), but the impact is small and the whole is available.
For babel code compilation, you still need to use tsc for type checking, and execute tsc --noEmit alone. Quoted from Why is it better to compile typescript with babel

{
  "scripts": {
    "dev": "npx babel src -d lib -w -x \".ts, .tsx\"",
    "build": "npx babel src -d lib -x \".ts, .tsx\"",
    "lint": "eslint src/**/*.ts --ignore-pattern src/types/*",
    "typeCheck": "tsc --noEmit"
  },  
}

In pre commit, you need to first NPM run lint & & NPM run typecheck and then build before submitting the code.

focus create projectName core process

After having a preliminary understanding of the dependencies and making preparations, let's understand the process of the core function focus create xxx.

• 1. Run focus create xxx on the terminal and print the logo with figlet first
  2. After obtaining the valid version number with semver, set it to automatically detect the latest version after N days and prompt whether to update it
• 1. Combined with Gitlab API capabilities, pull all template items through axios and list them for selection
• 1. After selecting a specific template, pull all Tags of the template
• 1. After selecting a specific Tag, you need to install the package management tool npm/yarn required for dependency
• 1. Use download git repo to pull the specific template Tag in Gitlab and cache it to In focusTemplate
• 1. If ask for cli is not provided in the template project JS file, use ncp to copy the code directly to the local
  2. If it exists, use the inquirer to enter and select the render (consolidate.ejs) variable according to the user, and finally traverse all files through metalsmith for insertion and modification
• 1. Install the dependency and execute git init to initialize the repository
• 1. complete

Core code implementation

One of the noteworthy is in step 6

In Src / create / index Copy in TS

// Copy operation
if (!fs.existsSync(path.join(result, CONFIG.ASK_FOR_CLI as string))) {
  // There is no direct copy to local
  await ncp(result, path.resolve(projectName));
  successTip();
} else {
  const args = require(path.join(result, CONFIG.ASK_FOR_CLI as string));
  await new Promise<void>((resolve, reject) => {
    MetalSmith(__dirname)
      .source(result)
      .destination(path.resolve(projectName))
      .use(async (files, metal, done) => {
        // If requiredPrompts does not exist, it will be exported by default
        const obj = await Inquirer.prompt(args.requiredPrompts || args);
        const meta = metal.metadata();
        Object.assign(meta, obj);
        delete files[CONFIG.ASK_FOR_CLI];
        done(null, files, metal);
      })
      .use((files, metal, done) => {
        const obj = metal.metadata();
        const effectFiles = args.effectFiles || [];
        Reflect.ownKeys(files).forEach(async (file) => {
          // When effectFiles is empty, it needs to be traversed
          if (effectFiles.length === 0 || effectFiles.includes(file)) {
            let content = files[file as string].contents.toString();
            if (/<%=([\s\S]+?)%>/g.test(content)) {
              content = await ejs.render(content, obj);
              files[file as string].contents = Buffer.from(content);
            }
          }
        });
        successTip();
        done(null, files, metal);
      })
      .build((err) => {
        if (err) {
          reject();
        } else {
          resolve();
        }
      });
  });
}

In ask for cli Configuration variables in JS

// Fields to be filled in and modified according to users
const requiredPrompts = [
  {
    type: 'input',
    name: 'repoNameEn',
    message: 'please input repo English Name ? (e.g. `smart-case`.focus.cn)',
  },
  {
    type: 'input',
    name: 'repoNameZh',
    message: 'please input repo Chinese Name ?(e.g. `Smart case field`)',
  },
];
// You need to modify the file where the field is located
const effectFiles = [
  `README.md`,
  `code/package.json`,
  `code/client/package.json`,
  `code/client/README.md`,
  // ...
]
module.exports = {
  requiredPrompts,
  effectFiles,
};

In readme Using ejs variable syntax placeholder in MD

## <% = reponamezh% > Project

Access address <%=repoNameEn%>.focus.cn

For example, the value of repoNameEn entered by the user is smart case, and the value of repoNameZh is smart case field

Eventually readme MD is rendered as follows

## Smart case field project

Access address smart-case.focus.cn

Summary

We can also use variables to other configurations of the project, such as publicPath, base, baseURL

Through the above steps, the project initialization is realized, and the new students in the group can happily enter the business code without paying attention to various cumbersome configurations.

focus add material core process

In the process of developing a page, you may need the following steps

• 1. Create a new NewPage directory in src/pages / and index tsx/index. less/index. d.ts
• 1. Create a new newpage in src/models / TS file, to do state management
• 1. Create a new newpage in src/servers / TS file to manage interface calls
• 1. In config / routes Insert a NewPage route into the TS file

Each new page requires such cumbersome operations. In fact, we can integrate the above steps into the scaffold and get the effect through one line of command and selection.

The general idea is as follows

• 1. Prepare the index in advance tsx/index. less/index. d.ts/models. ts/servers. TS template, which can be subdivided according to functions, such as common List pages, Drawer components
• 1. Copy the template to the specified directory
• 1. Use jscodeshift to read the routing configuration file of the project, and then insert a route
• 1. complete

Core code implementation

1. In Src / add / UMI page/template. Prepare the jsContent/cssContent/modelsContent/servicesContent template in TS

   export const jsContent = `
   import React from 'react';
   import './index.less';
   interface IProps {}
   const Page: React.FC<IProps> = (props) => {
     console.log(props);
     return <div>Page</div>;
   };
   `;
   
   export const cssContent = `
   // TODO: write here ...
   `;
   
   export const modelsContent = (upperPageName: string, lowerPageName: string) => (`
   import type { Effect, Reducer } from 'umi';
   import {
     get${upperPageName}List,
   } from '@/services/${lowerPageName}';
   
   export type ${upperPageName}ModelState = {
     ${lowerPageName}List: {
    list: any[];
     };
   };
   
   export type ${upperPageName}ModelType = {
     namespace: string;
     state: ${upperPageName}ModelState;
     effects: {
    get${upperPageName}List: Effect;
     };
     reducers: {
    updateState: Reducer;
     };
   };
   
   const ${upperPageName}Model: ${upperPageName}ModelType = {
     namespace: '${lowerPageName}',
   
     state: {
    ${lowerPageName}List: {
      list: [],
    },
     },
   
     effects: {
    *get${upperPageName}List({ payload }, { call, put }) {
      const res = yield call(get${upperPageName}List, payload);
      yield put({
        type: 'updateState',
        payload: {
          ${lowerPageName}List: {
            list: res ? res.map((l: any) => ({
              ...l, 
              id: l.${lowerPageName}Id,
              key: l.${lowerPageName}Id,
            })) : []
          },
        },
      });
    },
     },
   
     reducers: {
    updateState(state, action) {
      return {
        ...state,
        ...action.payload,
      };
    },
     },
   };
   export default ${upperPageName}Model;
   `);
   
   export const servicesContent = (upperPageName: string, lowerPageName: string) => (`
   import { MainDomain } from '@/utils/env';
   import request from './decorator';
   export async function get${upperPageName}List(
     params: any,
   ): Promise<any> {
     return request(\`\${MainDomain}/${lowerPageName}\`, {
    params,
     });
   }
   `);

2. In Src / add / UMI page/index. TS maps the destination address of the copy to the template

   import fs from 'fs';
   import path from 'path';
   import jf from 'jscodeshift';
   import {
     cssContent,
     jsContent,
     modelsContent,
     servicesContent,
   } from './template';
   import { firstToUpper, getUmiPrefix } from '../../../utils/util';
   import { IGenerateRule } from '../../../index.d';
   
   module.exports = (cwdDir: string, pageName: string): IGenerateRule => {
     const lowerPageName = pageName.toLocaleLowerCase();
     const upperPageName = firstToUpper(pageName);
     const pagesPrefix = getUmiPrefix(cwdDir, 'src/pages');
     const modelsPrefix = getUmiPrefix(cwdDir, 'src/models');
     const servicesPrefix = getUmiPrefix(cwdDir, 'src/services');
     const routesPrefix = getUmiPrefix(cwdDir, 'config');
     const routesPath = path.resolve(cwdDir, `${routesPrefix}/routes.ts`);
     const routeContent = fs.readFileSync(routesPath, 'utf-8');
     const routeContentRoot = jf(routeContent);
     routeContentRoot.find(jf.ArrayExpression)
    .forEach((p, pIndex) => {
      if (pIndex === 1) {
        p.get('elements').unshift(`{
     path: '/${pageName}', // TODO: do you need to adjust the position of the menu?
     name: '${pageName}',
     component: './${upperPageName}',
   }`);
      }
    });
     return {
    [`${pagesPrefix}/${upperPageName}/index.tsx`]: jsContent,
    [`${pagesPrefix}/${upperPageName}/index.less`]: cssContent,
    [`${modelsPrefix}/${lowerPageName}.ts`]: modelsContent(upperPageName, lowerPageName),
    [`${servicesPrefix}/${lowerPageName}.ts`]: servicesContent(upperPageName, lowerPageName),
    [`${routesPrefix}/routes.ts`]: routeContentRoot.toSource(),
     };
   };

Using jscodeshift, first read the route configuration in the project, find the first item of the route, and then insert an unshift route.

1. In Src / add / index Read all material templates and mapping relationships in TS, and finally make copies.

   import chalk from 'chalk';
   import inquirer from 'inquirer';
   import path from 'path';
   import { getDirName } from '../../utils/util';
   import writeFileTree from '../../utils/writeFileTree';
   import { UMI_DIR_ARR } from '../../utils/constants';
   
   module.exports = async (pageName: string) => {
     const cwdDirArr = process.cwd().split('/');
     const cwdDirTail = cwdDirArr[cwdDirArr.length - 1];
     if (!UMI_DIR_ARR.includes(cwdDirTail)) {
    console.log(`${chalk.red('please make sure in the "src" directory when executing the "focus add material" command !')}`);
    return;
     }
     const pages = getDirName(__dirname);
     if (!pages.length) {
    console.log(`${chalk.red('please support page !')}`);
    return;
     }
     const { pageType } = await inquirer.prompt({
    name: 'pageType',
    type: 'list',
    message: 'please choose a type to add page',
    choices: pages,
     });
     const generateRule = require(path.resolve(__dirname, `${pageType}`));
     const fileTree = await generateRule(process.cwd(), pageName);
     writeFileTree(process.cwd(), fileTree);
   };

2. In Src / utils / writefiletree The logic of copying in TS

   import chalk from 'chalk';
   import fs from 'fs-extra';
   import path from 'path';
   const writeFileTree = async (dir: string, files: any) => {
     Object.keys(files).forEach((name) => {
    const filePath = path.join(dir, name);
    fs.ensureDirSync(path.dirname(filePath));
    fs.writeFileSync(filePath, files[name]);
    console.log(`${chalk.green(name)} write done .`);
     });
   };
   export default writeFileTree;

Summary

The above code realizes the scene of quickly creating a new page. Not only that, we can refine the template for the repeated operations associated with multiple files and frequently copy and paste in the work, and place them in the src/add / directory of the scaffold according to certain specifications to realize one click new materials.

General capability

From the use and core implementation of focus create projectName and focus add material, this paper expounds the effect of scaffold @ focus/cli in the front-end R & D process. We implemented a solution to create projects + integrate common code and common pain points (quickly generate pages and configure routes...).

• [x] Create project + integrate common code
• [x] Solutions to common pain points (quickly generate pages and configure routes...)
• [] configuration (eslint, tsconfig, prettier...)
• [] efficiency improvement tools (copy various files)
• [] plug-in (solve a problem in the webpack build process...)

We also partially support the above three items based on specific business scenarios, which makes us focus on tools and light engineering in the development process, greatly improves the delivery speed, and allows the R & D students in the group to participate in the joint construction. For example, how to build a new scaffold through scaffold? Create all materials through scaffolding?

summary

The above codes are stored in the warehouse @careteen/cli.

Original address: Sohu - front end Engineering - building enterprise general scaffold

The core goal of scaffold is to improve the efficiency of the whole front-end R & D process. Although the scaffold has no fixed form and has different realization in different companies, it has the necessary elements.

• From the perspective of function realization, we should consider the high matching with the business.
• From the perspective of the underlying framework, it should have a high degree of scalability and execution environment diversity support.