Since 2021, the self driving track has entered an explosive period, and the industry has become a battleground for large manufacturers and start-ups. Among them, many companies have adopted computer vision as the technical base of automatic driving. Through image segmentation technology, cars can effectively understand the road scene and distinguish where is the road and where is the person. In addition to the field of automatic driving, image segmentation technology often appears in other important scenes, such as:

• Medical image segmentation: help doctors conduct diagnostic tests

• Satellite image analysis: suitable for in-depth study of a large number of image data

• Video entertainment apps: human image matting to avoid video barrage covering the face

Therefore, the application of image segmentation technology is very important and extensive. HMS Core machine learning service image segmentation service adopts an innovative semantic segmentation framework. This framework labels every pixel in the image, and even the hair details can be clearly and completely retained. In addition, the image segmentation service also improves the processing ability of images with different image quality and different sizes. For the white edges that often appear in the segmentation algorithm, a more structured learning training method is adopted to make the edges more natural.

So how to realize such a stable and refined segmentation ability?

Development practice

1, Development preparation

For the configuration steps of Maven warehouse and SDK, please refer to the application development introduction in the developer website: https://developer.huawei.com/consumer/cn/doc/development/hiai-Guides/image-segmentation-0000001050040109#section1658976113112?ha_source=hms1

1. Configure Huawei Maven warehouse address.

buildscript {
repositories {
google()
jcenter()
maven {url 'https://developer.huawei.com/repo/'}
}
dependencies {
...
classpath 'com.huawei.agconnect:agcp:1.4.1.300'
}
}
 
allprojects {
repositories {
google()
jcenter()
maven {url 'https://developer.huawei.com/repo/'}
}
}

2 add and compile SDK dependencies

dependencies {
    // Import basic SDK
implementation 'com.huawei.hms:ml-computer-vision-segmentation:2.1.0.301'
    // Introducing portrait segmentation model package
implementation 'com.huawei.hms:ml-computer-vision-image-segmentation-body-model:2.1.0.303'
}

3 on Android manifest Add permissions to XML

// Use storage permissions
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />

2, Development steps

@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
    setContentView(R.layout.activity_main);
    if (!allPermissionsGranted()) {
        getRuntimePermissions();
    }
}
private boolean allPermissionsGranted() {
    for (String permission : getRequiredPermissions()) {
        if (!isPermissionGranted(this, permission)) {
            return false;
        }
    }
    return true;
}
private void getRuntimePermissions() {
    List<String> allNeededPermissions = new ArrayList<>();
    for (String permission : getRequiredPermissions()) {
        if (!isPermissionGranted(this, permission)) {
            allNeededPermissions.add(permission);
        }
    }
    if (!allNeededPermissions.isEmpty()) {
        ActivityCompat.requestPermissions(
                this, allNeededPermissions.toArray(new String[0]), PERMISSION_REQUESTS);
    }
}
private static boolean isPermissionGranted(Context context, String permission) {
    if (ContextCompat.checkSelfPermission(context, permission) == PackageManager.PERMISSION_GRANTED) {
        return true; 
    }
    return false; 
}
private String[] getRequiredPermissions() { 
    try {
        PackageInfo info =
                this.getPackageManager()
                        .getPackageInfo(this.getPackageName(), PackageManager.GET_PERMISSIONS); 
        String[] ps = info.requestedPermissions;
        if (ps != null && ps.length > 0) { 
            return ps;
        } else { 
            return new String[0];
        } 
    } catch (RuntimeException e) {
        throw e; 
    } catch (Exception e) { 
        return new String[0];
    }
}

2 create image segmentation detector

MLImageSegmentationSetting setting = new MLImageSegmentationSetting.Factory()
// Set as portrait segmentation
        .setAnalyzerType(MLImageSegmentationSetting.BODY_SEG)
        .create();
this.analyzer = MLAnalyzerFactory.getInstance().getImageSegmentationAnalyzer(setting);

3 through Android graphics. Bitmap creates an "MLFrame" object for the analyzer to detect pictures

MLFrame mlFrame = new MLFrame.Creator().setBitmap(this.originBitmap).create();

4 call the "asyncAnalyseFrame" method for image segmentation

// Create a task to process the results returned by the image segmentation detector.
Task<MLImageSegmentation> task = this.analyzer.asyncAnalyseFrame(mlFrame);
// Asynchronously process the results returned by the image segmentation detector.
task.addOnSuccessListener(new OnSuccessListener<MLImageSegmentation>() {
    @Override
    public void onSuccess(MLImageSegmentation mlImageSegmentationResults) {.
        if (mlImageSegmentationResults != null) {
//Obtain the foreground image of the portrait segmented from the picture
            foreground = mlImageSegmentationResults.getForeground();
            preview.setImageBitmap(MainActivity.this.foreground);
        }
    }
}).addOnFailureListener(new OnFailureListener() {
    @Override
    public void onFailure(Exception e) {
        return;
    }
});

5 replace the picture background

// Get pictures from album
backgroundBitmap = Utils.loadFromPath(this, id, targetedSize.first, targetedSize.second);
BitmapDrawable drawable = new BitmapDrawable(backgroundBitmap);
preview.setBackground(drawable);
preview.setImageBitmap(this.foreground);

6 replace the picture background

MLFrame mlFrame = new MLFrame.Creator().setBitmap(this.originBitmap).create();

(demo demo video is shown below)

Learn more > >

visit Official website of Huawei developer Alliance
obtain Development guidance document
Huawei mobile service open source warehouse address: GitHub,Gitee

Follow us and learn the latest technical information of HMS Core at the first time~