Sonar scan plug-in custom extension

Ali published a popular Java development manual. pdf a long time ago, which defines many development specifications, of which the most basic specifications must be observed. Ali has issued a set of Sonar plug-ins to help scan

The essence of the development plug-in is to use the entry left by the source designer to get the source event and then implement it. Finally, your implementation will be scheduled and executed at a time and place you don't know. This is the charm of the design mode, which ensures the function expansion under the opening and closing principle

So what entrances did the designers of Sonar leave for us?

At this time, we often meet Baidu and Google. After an hour, you will find that you still don't understand anything. Because there are too few and scattered materials, you need this blog~~~

To get back to business, there are quite a lot of extensions left by Sonar. Please don't worry. Let's first understand what Sonar generates after parsing our java files

AST syntax tree

Wikipedia

AST (Abstract Syntax Tree) is also called Abstract Syntax Tree. Sonar will parse our java files into AST Syntax Tree. So what is AST Syntax Tree? In fact, back-end developers may not know much about it, but they will be familiar with the front-end

	public class TimeCase2 {
        public void test(){
            String s = "";
            Long start = System.currentTimeMillis();
        }
}

Such a piece of Java code will generate the following AST tree structure

The whole AST syntax tree is too huge to draw one by one. The main core parts are drawn. In fact, the role of each branch node in the corresponding implementation class in Sonar can be roughly understood through the diagram. In fact, each node of the AST tree has no restricted type, and different tree nodes are mainly made according to the expression and behavior of the code: description type, behavior type Tree nodes such as expression type, return value type, etc

class	effect
Tree	The root node of the entire Class and the metadata information of the Class
ModifierKeywordTreeImpl	Class description, annotation, name, etc
InternalSyntaxToken	Record the start symbol and end symbol of the class, and adapt to various grammars
MethodTreeImpl	Method implementation Record the name, description, annotation, input parameter, output parameter, etc. of the method
BlockTreeImpl	Method internal code block implementation Match internal line code records

In fact, there are many implementation types. Their top-level interfaces are Tree interfaces. This top interface defines the following interface methods:

	//Judge the type of the current concrete implementation class
	boolean is(Tree.Kind... var1);
	//Function execution
    void accept(TreeVisitor var1);
	//Parent node of the current node
    @Nullable
    Tree parent();
	//Start symbol of current node
    @Nullable
    SyntaxToken firstToken();
	//End symbol of current node
    @Nullable
    SyntaxToken lastToken();
	//Current node type
    Tree.Kind kind();

How many subclasses of the top interface of Tree inherit? Very many, each subclass interface extends new functions again:

		COMPILATION_UNIT(CompilationUnitTree.class),
        CLASS(ClassTree.class),
        ENUM(ClassTree.class),
        INTERFACE(ClassTree.class),
        ANNOTATION_TYPE(ClassTree.class),
        ENUM_CONSTANT(EnumConstantTree.class),
        INITIALIZER(BlockTree.class),
        STATIC_INITIALIZER(StaticInitializerTree.class),
        CONSTRUCTOR(MethodTree.class),
        METHOD(MethodTree.class),
        BLOCK(BlockTree.class),
        EMPTY_STATEMENT(EmptyStatementTree.class),
        LABELED_STATEMENT(LabeledStatementTree.class),
        EXPRESSION_STATEMENT(ExpressionStatementTree.class),
        IF_STATEMENT(IfStatementTree.class),
        ASSERT_STATEMENT(AssertStatementTree.class),
        SWITCH_STATEMENT(SwitchStatementTree.class),
        CASE_GROUP(CaseGroupTree.class),
        CASE_LABEL(CaseLabelTree.class),
        WHILE_STATEMENT(WhileStatementTree.class),
        DO_STATEMENT(DoWhileStatementTree.class),
        FOR_STATEMENT(ForStatementTree.class),
        FOR_EACH_STATEMENT(ForEachStatement.class),
        BREAK_STATEMENT(BreakStatementTree.class),
        CONTINUE_STATEMENT(ContinueStatementTree.class),
        RETURN_STATEMENT(ReturnStatementTree.class),
        THROW_STATEMENT(ThrowStatementTree.class),
        SYNCHRONIZED_STATEMENT(SynchronizedStatementTree.class),
        TRY_STATEMENT(TryStatementTree.class),
        CATCH(CatchTree.class),
        POSTFIX_INCREMENT(UnaryExpressionTree.class),
        POSTFIX_DECREMENT(UnaryExpressionTree.class),
        PREFIX_INCREMENT(UnaryExpressionTree.class),
        PREFIX_DECREMENT(UnaryExpressionTree.class),
        UNARY_PLUS(UnaryExpressionTree.class),
        UNARY_MINUS(UnaryExpressionTree.class),
        BITWISE_COMPLEMENT(UnaryExpressionTree.class),
        LOGICAL_COMPLEMENT(UnaryExpressionTree.class),
        MULTIPLY(BinaryExpressionTree.class),
        DIVIDE(BinaryExpressionTree.class),
        REMAINDER(BinaryExpressionTree.class),
        PLUS(BinaryExpressionTree.class),
        MINUS(BinaryExpressionTree.class),
        LEFT_SHIFT(BinaryExpressionTree.class),
        RIGHT_SHIFT(BinaryExpressionTree.class),
        UNSIGNED_RIGHT_SHIFT(BinaryExpressionTree.class),
        LESS_THAN(BinaryExpressionTree.class),
        GREATER_THAN(BinaryExpressionTree.class),
        LESS_THAN_OR_EQUAL_TO(BinaryExpressionTree.class),
        GREATER_THAN_OR_EQUAL_TO(BinaryExpressionTree.class),
        EQUAL_TO(BinaryExpressionTree.class),
        NOT_EQUAL_TO(BinaryExpressionTree.class),
        AND(BinaryExpressionTree.class),
        XOR(BinaryExpressionTree.class),
        OR(BinaryExpressionTree.class),
        CONDITIONAL_AND(BinaryExpressionTree.class),
        CONDITIONAL_OR(BinaryExpressionTree.class),
        CONDITIONAL_EXPRESSION(ConditionalExpressionTree.class),
        ARRAY_ACCESS_EXPRESSION(ArrayAccessExpressionTree.class),
        MEMBER_SELECT(MemberSelectExpressionTree.class),
        NEW_CLASS(NewClassTree.class),
        NEW_ARRAY(NewArrayTree.class),
        METHOD_INVOCATION(MethodInvocationTree.class),
        TYPE_CAST(TypeCastTree.class),
        INSTANCE_OF(InstanceOfTree.class),
        PARENTHESIZED_EXPRESSION(ParenthesizedTree.class),
        ASSIGNMENT(AssignmentExpressionTree.class),
        MULTIPLY_ASSIGNMENT(AssignmentExpressionTree.class),
        DIVIDE_ASSIGNMENT(AssignmentExpressionTree.class),
        REMAINDER_ASSIGNMENT(AssignmentExpressionTree.class),
        PLUS_ASSIGNMENT(AssignmentExpressionTree.class),
        MINUS_ASSIGNMENT(AssignmentExpressionTree.class),
        LEFT_SHIFT_ASSIGNMENT(AssignmentExpressionTree.class),
        RIGHT_SHIFT_ASSIGNMENT(AssignmentExpressionTree.class),
        UNSIGNED_RIGHT_SHIFT_ASSIGNMENT(AssignmentExpressionTree.class),
        AND_ASSIGNMENT(AssignmentExpressionTree.class),
        XOR_ASSIGNMENT(AssignmentExpressionTree.class),
        OR_ASSIGNMENT(AssignmentExpressionTree.class),
        INT_LITERAL(LiteralTree.class),
        LONG_LITERAL(LiteralTree.class),
        FLOAT_LITERAL(LiteralTree.class),
        DOUBLE_LITERAL(LiteralTree.class),
        BOOLEAN_LITERAL(LiteralTree.class),
        CHAR_LITERAL(LiteralTree.class),
        STRING_LITERAL(LiteralTree.class),
        NULL_LITERAL(LiteralTree.class),
        IDENTIFIER(IdentifierTree.class),
        VARIABLE(VariableTree.class),
        ARRAY_TYPE(ArrayTypeTree.class),
        PARAMETERIZED_TYPE(ParameterizedTypeTree.class),
        UNION_TYPE(UnionTypeTree.class),
        UNBOUNDED_WILDCARD(WildcardTree.class),
        EXTENDS_WILDCARD(WildcardTree.class),
        SUPER_WILDCARD(WildcardTree.class),
        ANNOTATION(AnnotationTree.class),
        MODIFIERS(ModifiersTree.class),
        LAMBDA_EXPRESSION(LambdaExpressionTree.class),
        PRIMITIVE_TYPE(PrimitiveTypeTree.class),
        TYPE_PARAMETER(TypeParameterTree.class),
        IMPORT(ImportTree.class),
        PACKAGE(PackageDeclarationTree.class),
        ARRAY_DIMENSION(ArrayDimensionTree.class),
        OTHER(Tree.class),
        TOKEN(SyntaxToken.class),
        TRIVIA(SyntaxTrivia.class),
        INFERED_TYPE(InferedTypeTree.class),
        TYPE_ARGUMENTS(TypeArguments.class),
        METHOD_REFERENCE(MethodReferenceTree.class),
        TYPE_PARAMETERS(TypeParameters.class),
        ARGUMENTS(Arguments.class),
        LIST(ListTree.class);

This enumeration class lists the subclass interface definitions of all Tree interfaces. Different subclass interfaces correspond to different expressions, descriptions and other implementations of the code. Therefore, when we extend the sonar plug-in later, we can directly obtain the information we want from the super many subclass implementations above by scanning the code of the rule types we need to extend, We don't need to get it from the top Tree one by one. Please continue to see the analysis below~~

Sonar storage core table structure

Sonar uses PGSQL by default

Table name	describe
projects	The table holds the basic information of all items analyzed by sonar
metrics	This table stores test indicators, such as test coverage, code complexity, and so on
rules_profiles	All test indicators are stored in the metrics table. rules_profiles is a subset of these metrics, which can be regarded as a set of customized test standards. Each project will have corresponding rules_ Corresponding to project
snapshots	With projects and rules_profiles. According to some rules_profile a sonar analysis of a project will generate some snapshots. But there is no real index value stored here. Instead, it is stored in the project_measures in this table. Snapshots and projects_ Measures are associated by foreign keys

Develop Sonar plug-ins

Create a new maven project and modify POM

<?xml version="1.0" encoding="UTF-8"?>
<project>
  ...
  <build>
    <plugins>
      <plugin>
        <groupId>org.sonarsource.sonar-packaging-maven-plugin</groupId>
        <artifactId>sonar-packaging-maven-plugin</artifactId>
        <extensions>true</extensions>
        <configuration>
          <pluginClass>org.sonarqube.plugins.example.ExamplePlugin</pluginClass>
        </configuration>
      </plugin>
    </plugins>
  </build>
</project>

Implement the Plugin interface of Sonar

public class MyJavaRulesPlugin implements Plugin {

  @Override
  public void define(Context context) {
    // server extensions -> objects are instantiated during server startup
    context.addExtension(MyJavaRulesDefinition.class);
    // batch extensions -> objects are instantiated during code analysis
    context.addExtension(MyJavaFileCheckRegistrar.class);
  }
}

context.addExtension is to add an extension execution class. MyJavaRulesDefinition implements RulesDefinition to describe some rule metadata information, mainly the description of your own custom rules

public class MyJavaRulesDefinition implements RulesDefinition {
    public static final String REPOSITORY_KEY = "finger-java-custom-rules";
    private final Gson gson = new Gson();

    @Override
    public void define(Context context) {
        NewRepository repository = context
                .createRepository( REPOSITORY_KEY, Java.KEY )
                .setName( "Finger Java Custom Rules" );

        new AnnotationBasedRulesDefinition( repository, Java.KEY )
                .addRuleClasses(/* don't fail if no SQALE annotations */ false, RulesList.getChecks() );

        for (NewRule rule : repository.rules()) {
            String metadataKey = rule.key();
            System.out.println( metadataKey );
            // Setting internal key is essential for rule templates (see SONAR-6162), and it is not done by AnnotationBasedRulesDefinition from
            // sslr-squid-bridge version 2.5.1:
            rule.setInternalKey( metadataKey );
            rule.setHtmlDescription( readRuleDefinitionResource( metadataKey + ".html" ) );
            addMetadata( rule, metadataKey );
        }

        repository.done();
    }

    @Nullable
    private static String readRuleDefinitionResource(String fileName) {
        URL resource = MyJavaRulesDefinition.class.getResource( "/org/sonar/l10n/java/rules/squid/" + fileName );
        if (resource == null) {
            return null;
        }
        try {
            return Resources.toString( resource, Charsets.UTF_8 );
        } catch (IOException e) {
            throw new IllegalStateException( "Failed to read: " + resource, e );
        }
    }

    private void addMetadata(NewRule rule, String metadataKey) {
        String json = readRuleDefinitionResource( metadataKey + ".json" );
        if (json != null) {
            RuleMetadata metadata = gson.fromJson( json, RuleMetadata.class );
            rule.setSeverity( metadata.defaultSeverity.toUpperCase( Locale.US ) );
            rule.setName( metadata.title );
            rule.setTags( metadata.tags );
            rule.setStatus( RuleStatus.valueOf( metadata.status.toUpperCase( Locale.US ) ) );

            if (metadata.remediation != null) {
                // metadata.remediation is null for template rules
                rule.setDebtRemediationFunction( metadata.remediation.remediationFunction( rule.debtRemediationFunctions() ) );
                rule.setGapDescription( metadata.remediation.linearDesc );
            }
        }
    }

    private static class RuleMetadata {
        String title;
        String status;
        @Nullable
        Remediation remediation;

        String[] tags;
        String defaultSeverity;
    }

    private static class Remediation {
        String func;
        String constantCost;
        String linearDesc;
        String linearOffset;
        String linearFactor;

        private DebtRemediationFunction remediationFunction(DebtRemediationFunctions drf) {
            if (func.startsWith( "Constant" )) {
                return drf.constantPerIssue( constantCost.replace( "mn", "min" ) );
            }
            if ("Linear".equals( func )) {
                return drf.linear( linearFactor.replace( "mn", "min" ) );
            }
            return drf.linearWithOffset( linearFactor.replace( "mn", "min" ), linearOffset.replace( "mn", "min" ) );
        }
    }
}

MyJavaFileCheckRegistrar is the main extension rule class, which is similar to the responsibility class mode. You can add the extension class to be executed

@SonarLintSide
public class MyJavaFileCheckRegistrar implements CheckRegistrar {

  /**
   * Register the classes that will be used to instantiate checks during analysis.
   */
  @Override
  public void register(RegistrarContext registrarContext) {
    // Call to registerClassesForRepository to associate the classes with the correct repository key
    registrarContext.registerClassesForRepository(MyJavaRulesDefinition.REPOSITORY_KEY, Arrays.asList(checkClasses()), Arrays.asList(testCheckClasses()));
  }

  /**
   * Lists all the checks provided by the plugin
   */
  public static Class<? extends JavaCheck>[] checkClasses() {
    return new Class[] {
            xx.class, //Just fill in your extended rule class
            xx1.class
    };
  }

  /**
   * Lists all the test checks provided by the plugin
   */
  public static Class<? extends JavaCheck>[] testCheckClasses() {
    return new Class[] {};
  }
}

Specific rule extension class development

The above has correctly accessed the Sonar extension, and the next step is to realize the specific extension

/**
 * <p>Title:Thread pool specification verification</p>
 * <p>Description:</p>
 *
 * @author QIQI
 * @params
 * @return
 * @throws
 * @date 2021/06/10 17:26
 */
@Rule(key = "ThreadPoolCheck")
public class ThreadPoolCheck extends BaseTreeVisitor implements JavaFileScanner {
    private static final Logger LOGGER = LoggerFactory.getLogger( ThreadPoolCheck.class );


    private JavaFileScannerContext context;

    @Override
    public void scanFile(JavaFileScannerContext context) {
        this.context = context;
        scan( context.getTree() );
    }

    @Override
    public void visitBlock(BlockTree tree) {
        List<StatementTree> statementTrees = tree.body();
        if (statementTrees != null && statementTrees.size() > 0) {
            LOGGER.info( "ThreadPoolCheck start ...." );
            for (StatementTree tree1 : statementTrees) {
                if (tree1 instanceof VariableTreeImpl) {
                    CompletableFuture.runAsync( () -> System.out.println(1) );
                    executorServiceCheck( (VariableTreeImpl) tree1 );
                } else if (tree1 instanceof ExpressionStatementTreeImpl) {
                    completableFutureCheck( (ExpressionStatementTreeImpl) tree1 );
                }
            }
        }
        super.visitBlock( tree );
    }

    @Override
    public void visitAnnotation(AnnotationTree annotationTree) {
        if(annotationTree.annotationType().toString().equals( "Async" )){
            if(annotationTree.arguments().size() == 0){
                context.reportIssue( this, annotationTree, "Unified thread pool is used except for special requirement thread pool" );
            }
        }
        super.visitAnnotation( annotationTree );
    }

    private void executorServiceCheck(VariableTreeImpl variableTree) {
        if (variableTree.type().toString().equals( "ExecutorService" )) {
            List<Tree> treeList = variableTree.getChildren();
            if (treeList != null && treeList.size() > 0) {
                for (Tree tree2 : treeList) {
                    if (tree2 instanceof MethodInvocationTreeImpl) {
                        MethodInvocationTreeImpl methodInvocationTree = (MethodInvocationTreeImpl) tree2;
                        if (methodInvocationTree.symbol().name().equals( "newFixedThreadPool" )) {
                            context.reportIssue( this, methodInvocationTree, "Unified thread pool is used except for special requirement thread pool" );
                        }
                    }
                }
            }
        }
    }

    private void completableFutureCheck(ExpressionStatementTreeImpl expressionStatementTree) {
        MethodInvocationTreeImpl methodInvocationTree = (MethodInvocationTreeImpl) expressionStatementTree.expression();
        if(methodInvocationTree != null && methodInvocationTree.arguments().size() < 2){
            context.reportIssue( this, expressionStatementTree, "Unified thread pool is used except for special requirement thread pool" );
        }
    }
}

Fixed inheritance of BaseTreeVisitor and implementation of JavaFileScanner
Next, it depends on the @ Override method you need. The entire parent class has a lot of @ Override. Different @ Override corresponds to different rule verification scenarios. Sonar will throw different Tree subclass implementations into different @ Override implementation classes and pass in the parameters

I need to verify the @ block subclass in the tree interface, so I need to verify the @ block subclass
Now I need to verify the method annotation, so I @ Override visitAnnotation again, and Sonar passes in the subclass interface of AnnotationTree

There can be @ Override multiple in the same extension class, which can be developed according to the needs of the scenario

Dear students, it's not over yet~~~~

Now we have just developed the rules and have not done the case display ~. After we are scanned for bugs, there are cases to tell you how to modify, so we also need to define cases

Careful students have found that the above code has an annotation @ Rule(key = "ThreadPoolCheck"), which must correspond to the name of the following case file

ThreadPoolCheck.html

<p>MachineTimeCheck Check</p>
<h2>Noncompliant Code Example</h2>
<pre>
    Executors.newFixedThreadPool();
    CompletableFuture.runAsync( () -> System.out.println(1) );
    @Async;
</pre>
<h2>Compliant Solution</h2>
<pre>
    ThreadFactory.getThreadExecutor( APOLLO-CONFIG );
    CompletableFuture.runAsync( () -> System.out.println(1), ThreadFactory.getThreadExecutor( APOLLO-CONFIG ) );
    (https://duapp.yuque.com/team_tech/confluence-data-iwskfg/ul9gg1)
</pre>

ThreadPoolCheck.json (this file represents the location of the problem after scanning: odor? BUG? Vulnerability?)

{
  "title": "Unified thread pool is used except for special requirement thread pool",
  "status": "ready",
  "remediation": {
    "func": "Constant\/Issue",
    "constantCost": "5min"
  },
  "tags": [
    "bug",
    "pitfall"
  ],
  "defaultSeverity": "CRITICAL"
}

ThreadPoolCheck_java.json

{
  "title": "Unified thread pool is used except for special requirement thread pool",
  "status": "ready",
  "remediation": {
    "func": "Constant\/Issue",
    "constantCost": "5min"
  },
  "tags": [
    "bug",
    "pitfall"
  ],
  "defaultSeverity": "CRITICAL"
}

Finally, how to write the test case of the plug-in?

Create a new files folder under test and a threadpoolcase Java file, which writes a counter example that does not conform to the rules

/**
* <p>Title:</p>
* <p>Description:</p>
* @author QIQI
* @params
* @return
* @throws
* @date 2021/06/10 18:11
*/
public class ThreadPoolCase {
    public void test(){
        ExecutorService executorService = Executors.newFixedThreadPool( 1 ); // Noncompliant
    }
}

Then create a new test case

	@Test
    public void test() {
        JavaCheckVerifier.verify("src/test/files/ThreadPoolCase.java", new ThreadPoolCheck());
    }

Keywords: Programming Sonar

Added by darknessmdk on Tue, 01 Feb 2022 17:13:00 +0200

Programming VIP

Sonar scan plug-in custom extension

AST syntax tree

Sonar storage core table structure

Develop Sonar plug-ins

Finally, how to write the test case of the plug-in?

Popular Keywords