This paper First appeared in My blog , if you think it's useful, please like the collection and let more friends see it.

Recently, I developed a very simple tool, the total code is less than 200 lines. Today, briefly introduce it. What tool is this? It is a tool for visualizing Go Module dependencies.

Why development

Why do you want to develop this tool? There are two main reasons:

One is that we often see you discussing Go Module in the community recently. So I spent some time studying it. During this process, we met a requirement to clearly identify the relationship between the dependencies in the module. After some understanding, I found the go mod graph.

The results are as follows:

$ go mod graph
github.com/poloxue/testmod golang.org/x/text@v0.3.2
github.com/poloxue/testmod rsc.io/quote/v3@v3.1.0
github.com/poloxue/testmod rsc.io/sampler@v1.3.1
golang.org/x/text@v0.3.2 golang.org/x/tools@v0.0.0-20180917221912-90fa682c2a6e
rsc.io/quote/v3@v3.1.0 rsc.io/sampler@v1.3.0
rsc.io/sampler@v1.3.1 golang.org/x/text@v0.0.0-20170915032832-14c0d48ead0c
rsc.io/sampler@v1.3.0 golang.org/x/text@v0.0.0-20170915032832-14c0d48ead0c

The format of each line is module dependent, which can basically meet the requirements, but it is still not so intuitive.

Second, I have a project in my hand before, and the package management has always used dep. So I got to know it and read the official documents carefully. Among them A chapter This paper introduces the method of dependency visualization.

Package diagram given in the document:

When you see this picture, your eyes will be bright in an instant. Graphing is excellence. The relationship between different dependencies is clear at a glance. Isn't that what I want? 666, like it.

But... The following problem is that go mod does not have this ability. What should I do?

How to achieve

Let's see if someone has already done it. I searched the Internet, but I couldn't find it. Can it be realized by itself? Should we learn from the idea of dep?

The following is how dep relies on Visualization:

# linux
$ sudo apt-get install graphviz
$ dep status -dot | dot -T png | display

# macOS
$ brew install graphviz
$ dep status -dot | dot -T png | open -f -a /Applications/Preview.app

# Windows
> choco install graphviz.portable
> dep status -dot | dot -T png -o status.png; start status.png

Here we show the usage of the three systems. They all have a software package, graphviz. In terms of name, this should be a software for visualization, that is, for drawing. It's the same thing. You can see it. Official website.

Looking at its use, we find that it is all combined by pipeline commands, and the previous parts are basically the same. They are dep status -dot | dot -T png. The latter part is different in different systems. Linux is display, Mac OS is open-f-a / applications / preview.app, and Window is start status.png.

After a little analysis, it will be clear that the front is to generate pictures, and the back is to display pictures. Because the image display commands of different systems are different, the following parts are also different.

Now the focus is on what dep status -dot | dot -T png has done, and how does it realize drawing? Presumably, dot-t PNG is the data provided by DEP status-dot to generate pictures. Continue to see the execution effect of dep status -dot.

$ dep status -dot
digraph {
    node [shape=box];
    2609291568 [label="github.com/poloxue/hellodep"];
    953278068 [label="rsc.io/quote\nv3.1.0"];
    3852693168 [label="rsc.io/sampler\nv1.0.0"];
    2609291568 -> 953278068;
    953278068 -> 3852693168;
}

At first glance, the output is a piece of code that doesn't seem to know what it is. This should be the language graphviz uses to draw charts. Is that still learning? Of course not. It's very simple here. Just apply it directly.

Try to analyze it. The first two lines don't need to be concerned. This should be graphviz specific writing method, indicating what kind of picture to draw. Our main concern is how to provide data in the right form.

2609291568 [label="github.com/poloxue/hellodep"];
953278068 [label="rsc.io/quote\nv3.1.0"];
3852693168 [label="rsc.io/sampler\nv1.0.0"];
2609291568 -> 953278068;
953278068 -> 3852693168;

At first glance, we know that there are two kinds of structures, one is to associate ID for dependency, and the other is to represent the relationship between dependencies through ID and - >.

According to the above conjecture, we can try to draw a simple diagram to show that a module depends on b module. Execute the command as follows to send the drawing code to the dot command through each pipeline.

$ echo 'digraph {
node [shape=box];
1 [label="a"];
2 [label="b"];
1 -> 2;
}' | dot -T png | open -f -a /Applications/Preview.app 

The results are as follows:

It's so easy to draw a dependency graph.

It's very easy to find out the problem here. We only need to transform the output of go mod graph into a similar structure to realize visualization.

Introduction to development process

Next, let's develop this small program. I will name this small program modv, which means module visible. Project source code is located in poloxue/modv.

Receive input from pipeline

First, check whether the data input pipeline is normal.

Our goal is to use a drawing method similar to that in dep, go mod graph will pipe data to modv. Therefore, first check os.Stdin, that is, check whether the standard input status is normal and whether it is pipeline transmission.

Here is the code for the main function, located at main.go Medium.

func main() {
    info, err := os.Stdin.Stat()
    if err != nil {
        fmt.Println("os.Stdin.Stat:", err)
        PrintUsage()
        os.Exit(1)
    }

    // Is it pipeline transmission?
    if info.Mode()&os.ModeNamedPipe == 0 {
        fmt.Println("command err: command is intended to work with pipes.")
        PrintUsage()
        os.Exit(1)
    }

Once the input device is confirmed to be normal, we can enter the process of data reading, parsing and rendering.

    mg := NewModuleGraph(os.Stdin)
    mg.Parse()
    mg.Render(os.Stdout)
}

Next, let's see how to implement the data processing flow.

Abstract implementation structure

First, define a structure, and roughly define the whole process.

type ModGraph struct {
    Reader io.Reader  // Read data stream
}

func NewModGraph(r io.Reader) *ModGraph {
    return &ModGraph{Reader: r}
}

// Perform data processing and transformation
func (m *ModGraph) Parse() error {}

// Result rendering and output
func (m *ModGraph) Render(w io.Writer) error {}

Take a look at the output of go mod graph, as follows:

github.com/poloxue/testmod golang.org/x/text@v0.3.2
github.com/poloxue/testmod rsc.io/quote/v3@v3.1.0
...

The structure of each row is a module dependency. The goal now is to parse it into the following structure:

digraph {
    node [shape=box];
    1 github.com/poloxue/testmod;
    2 golang.org/x/text@v0.3.2;
    3 rsc.io/quote/v3@v3.1.0;
    1 -> 2;
    1 -> 3;
}

As mentioned earlier, there are two different structures, namely module and ID Association, and module ID represents the dependency association between modules. Add two members to the ModGraph structure to represent them.

type ModGraph struct {
    r io.Reader  // Data stream reading instance, which is os.Stdin
 
    // Mapping of each item name and ID
    Mods         map[string]int
    // ID and dependency ID relationship mapping, an ID may depend on multiple items
    Dependencies map[int][]int
}

Note that after adding two map members, remember to initialize them in NewModGraph.

Analysis of mod graph output

How to parse?

Introduced here, the goal has been very clear. To Parse the input data into Mods and Dependencies, the implementation code is in the Parse method.

To facilitate data reading, first of all, we use bufio to create a new buf reader based on the reader.

func (m *ModGraph) Parse() error {
    bufReader := bufio.NewReader(m.Reader)
    ...

To facilitate parsing data by row, we read the data in os.Stdin row by row through the ReadBytes() method of bufReader. Then, each row of data is divided into two spaces to get two items of dependency. The code is as follows:

for {
    relationBytes, err := bufReader.ReadBytes('\n')
    if err != nil {
        if err == io.EOF {
            return nil
        }
        return err
    }

    relation := bytes.Split(relationBytes, []byte(" "))
    // module and dependency
    mod, depMod := strings.TrimSpace(string(relation[0])), strings.TrimSpace(string(relation[1]))

    ...
}

Next, organize the resolved Dependencies into Mods and Dependencies. The module ID is the simplest way to generate rules, which is self increasing from 1. The implementation code is as follows:

modId, ok := m.Mods[mod]
if !ok {
    modId = serialID
    m.Mods[mod] = modId
    serialID += 1
}

depModId, ok := m.Mods[depMod]
if !ok {
    depModId = serialID
    m.Mods[depMod] = depModId
    serialID += 1
}

if _, ok := m.Dependencies[modId]; ok {
    m.Dependencies[modId] = append(m.Dependencies[modId], depModId)
} else {
    m.Dependencies[modId] = []int{depModId}
}

This is the end of parsing.

Render the result of parsing

The last step of the tool is to Render the parsed data to meet the drawing requirements of graphviz tool. The implementation code is the Render part:

First, define a template to generate an output format that meets the requirements.

var graphTemplate = `digraph {
node [shape=box];
{{ range $mod, $modId := .mods -}}
{{ $modId }} [label="{{ $mod }}"];
{{ end -}}
{{- range $modId, $depModIds := .dependencies -}}
{{- range $_, $depModId := $depModIds -}}
{{ $modId }} -> {{ $depModId }};
{{  end -}}
{{- end -}}
}
`

There is nothing to introduce in this section, mainly to be familiar with the syntax specification of the text/template template template in Go. In order to show friendliness, we use - to achieve line breaking removal, which does not affect reading as a whole.

Next, look at the implementation of the Render method. Put the previously parsed Mods and Dependencies into the template for rendering.

func (m *ModuleGraph) Render(w io.Writer) error {
    templ, err := template.New("graph").Parse(graphTemplate)
    if err != nil {
        return fmt.Errorf("templ.Parse: %v", err)
    }

    if err := templ.Execute(w, map[string]interface{}{
        "mods":         m.Mods,
        "dependencies": m.Dependencies,
    }); err != nil {
        return fmt.Errorf("templ.Execute: %v", err)
    }

    return nil
}

Now, all the work is done. Finally, the process is integrated into the main function. The next step is to use it.

User's experience

Let's start. In addition, I only test the use of this tool under Mac now. If you have any questions, please come up with them.

First, you need to install graphviz. The installation method is introduced at the beginning of this article. Choose your system installation method.

Next, install modv. The command is as follows:

$ go get github.com/poloxue/modv

Installation complete! Simply test its use.

Take MacOS for example. Download the test library, github.com/pollox/testmod. Enter testmod directory to execute the command:

$ go mod graph | modv | dot -T png | open -f -a /Applications/Preview.app

If the execution is successful, you will see the following effects:

It perfectly shows the dependencies among the modules.

Some thoughts

This is a practical article, from a simple idea to a successful presentation of a usable tool. Although, it is not difficult to develop. It only takes an hour or two from development to completion. But my feeling is that it's really a valuable tool.

There are still some ideas that have not been implemented and verified, such as whether it is convenient to display the dependency tree of a specific node rather than the whole project once the project is large. What's more, whether the gadget can generate some value when other projects migrate to the Go Module.

Welcome to my wechat account.