preface

Coupon website m.cps3 cn

I'm writing the little sparrow project FastGithub, which mainly involves Pipeline mode and Factory+Provider mode. These two design modes make the project feel like a duck to water in the two core functions of "ip scanning" and "ip search". I'll share it with you here.

Pipeline

Pipeline mode is also called pipeline mode or pipeline mode. The input data is processed through a series of preset stages. The output of each stage is the input of the next stage. Each stage can choose whether to continue the next stage.

Context object

In the implementation, we encapsulate all the required data in the context object, and each stage can share the same context object.

middleware

In terms of implementation, we encapsulate the processing of each stage into middleware. One middleware can access the context object and the processing object of the next stage, and can access or modify the data of the context object during execution.

Implementation details

For complete Pipeline construction code, see https://github.com/xljiulang/FastGithub/tree/master/FastGithub.Core

Stage processing object

/// <summary>
///Represents all middleware execution delegates
/// </summary>
///< typeparam name = "tcontext" > middleware context type < / typeparam >
///< param name = "context" > middleware context < / param >
/// <returns></returns>
public delegate Task InvokeDelegate<TContext>(TContext context);

Delegate Middleware

Func < invokedelegate < tcontext >, invokedelegate < tcontext > > is a delegated middleware. The first invokedelegate < tcontext > represents the next processing stage of incoming data, and the second invokedelegate < tcontext > represents the current processing stage.

/// <summary>
///Define the interface of middleware pipeline Creator
/// </summary>
///< typeparam name = "tcontext" > middleware context < / typeparam >
public interface IPipelineBuilder<TContext>
{
    /// <summary>
    ///Using middleware
    /// </summary>
    ///< param name = "middleware" > middleware < / param >
    /// <returns></returns>
    IPipelineBuilder<TContext> Use(Func<InvokeDelegate<TContext>, InvokeDelegate<TContext>> middleware);

    /// <summary>
    ///Create all middleware execution processors
    /// </summary>
    /// <returns></returns>
    InvokeDelegate<TContext> Build();
}

Strongly typed Middleware

We can convert the delegated middleware into the following strong type middleware. The InvokeAsync method is the processing stage and the next parameter is the packaging of the next stage of the delegated middleware.

/// <summary>
///Define the interface of Middleware
/// </summary>
/// <typeparam name="TContext"></typeparam>
public interface IMiddleware<TContext>
{
    /// <summary>
    ///Execution Middleware
    /// </summary>
    ///< param name = "context" > context < / param >
    ///< param name = "next" > next middleware < / param >
    /// <returns></returns>
    Task InvokeAsync(TContext context, Func<Task> next);
}

Detailed explanation of use

The scanning task is divided into complete scanning and historical result scanning. The middleware used is a little different, but it is only necessary to string the required middleware.

/// <summary>
///github scanning service
/// </summary>
/// <param name="domainAddressFactory"></param>
/// <param name="scanResults"></param>
/// <param name="appService"></param>
/// <param name="logger"></param>
public GithubScanService(
    DomainAddressFacotry domainAddressFactory,
    GithubContextCollection scanResults,
    IServiceProvider appService,
    ILogger<GithubScanService> logger)
{
    this.domainAddressFactory = domainAddressFactory;
    this.scanResults = scanResults;
    this.logger = logger;

    this.fullScanDelegate = new PipelineBuilder<GithubContext>(appService, ctx => Task.CompletedTask)
        .Use<ConcurrentMiddleware>()
        .Use<StatisticsMiddleware>()
        .Use<TcpScanMiddleware>()
        .Use<HttpsScanMiddleware>()
        .Build();

    this.resultScanDelegate = new PipelineBuilder<GithubContext>(appService, ctx => Task.CompletedTask)
        .Use<StatisticsMiddleware>()
        .Use<HttpsScanMiddleware>()
        .Build();
}

Factory+Provider

Multiple providers can use different means to obtain the ip of github, and then the Factory integrates the ip obtained by each Provider. They all get the same function: to obtain the ip of github, only each Provider works specifically, and there is no relationship between providers.

IDomainAddressProvider

/// <summary>
///Define the ip valuer of the domain name
/// </summary>
interface IDomainAddressProvider
{
    /// <summary>
    ///Create the relationship between domain name and ip
    /// </summary>
    /// <returns></returns>
    Task<IEnumerable<DomainAddress>> CreateDomainAddressesAsync();
}

DomainAddressFacotry

/// <summary>
///Domain name and ip relationship factory
/// </summary>
[Service(ServiceLifetime.Singleton)]
sealed class DomainAddressFacotry
{
    private readonly IEnumerable<IDomainAddressProvider> providers;

    /// <summary>
    ///Domain name and ip relationship factory
    /// </summary>
    /// <param name="providers"></param>
    public DomainAddressFacotry(IEnumerable<IDomainAddressProvider> providers)
    {
        this.providers = providers;
    }

    /// <summary>
    ///Create the relationship between domain name and ip
    /// </summary>
    /// <returns></returns>
    public async Task<IEnumerable<DomainAddress>> CreateDomainAddressesAsync()
    {
        var hashSet = new HashSet<DomainAddress>();
        foreach (var provider in this.providers)
        {
            var domainAddresses = await provider.CreateDomainAddressesAsync();
            foreach (var item in domainAddresses)
            {
                hashSet.Add(item);
            }
        }
        return hashSet;
    }
}

Mode advantage analysis

FastGithub uses the above two modes at the same time, and its workflow is very simple: use domainaddressfactory to create the ip to be scanned, and then use the scanning delegate created by pipeline to scan. If you want to get more ip addresses, you can add a DomainAddressProvider without affecting any other code flow. If you want to do other scanning logic in the scanning process, you can add a scanning middleware and install it in a suitable location.