Automatic plug-in

In your first application, you learned to build a simple media player for Ogg/Vorbis files. By using a lt ernative elements, you can build media players for other media types, such as Ogg/Speex, MP3, and even video formats. However, you prefer to build an application that can automatically detect the media type of the stream and automatically generate the best possible pipeline by viewing all available elements in the system. This process is called automatic plug-in, and GStreamer contains high-quality automatic plug-ins. If you are looking for an automatic plug-in, please do not continue reading and go to < playback components >. This chapter explains the concepts of automatic plug-ins and type lookup. It will explain which systems GStreamer contains to dynamically detect the type of media stream and how to generate the pipeline of decoder element to play this media. The same principle can also be used for transcoding. Due to the fully dynamic nature of this concept, GStreamer can be automatically extended to support new media types without any adjustment to its automatic plug-in.

We will first introduce the concept of media type as a dynamic and scalable way to identify media streams. After that, we will introduce the concept of type lookup to find the type of media stream. Finally, we will explain how to use the auto plug-in and GStreamer registry to set up a pipeline that converts media from one media type to another, such as for media decoding.

Media type as a way to identify streams

We have previously introduced the concept of capabilities. When data flows from one element to another, capabilities is a way for elements (or, more precisely, pad) to agree on media types (see capabilities of PAD). As we have explained, capabilities} is a combination of media types and a set of attributes. For most container formats (those are files you find on your hard disk; for example, Ogg is a container format), no attributes are required to describe the stream. Only one media type is required. A complete list of media types and accompanying properties can be found in the < plug in writing guide >.

When an element is loaded into the system, it must associate a media type with its source pad and sink pad. GStreamer understands different elements and their expected data types and issued data types through the registry of GStreamer. As we will see, this allows the creation of very dynamic and extensible elements.

In your first application, we learned to build a music player for Ogg/Vorbis files. Let's look at the media types associated with each pad in this pipeline. The Hello world pipeline with media type shows the media type of each pad in the pipeline.

 

Now that we know how GStreamer identifies known media streams, we can look at the method GStreamer uses to set the pipeline for media processing and media type detection.

Media stream type detection

Typically, when loading a media stream, the type of stream is unknown. This means that before we can select a pipeline to decode the stream, we first need to detect the stream type. GStreamer uses the concept of type lookup for this purpose. Type lookup is a normal part of pipeline. As long as the type of stream is unknown, it will read data. During this time, it will provide data to all plug-ins that implement the type finder. When one of the typefinder s recognizes the stream, the typefind element will send a signal and act as a passthrough module from then on. If no type is found, it issues an error and stops further media processing.

Once the typefind element finds a type, the application can use it to insert a pipeline to decode the media stream. This will be discussed in the next section.

As mentioned earlier, the plug-in in GStreamer can implement the type finder function. The plug-in implementing this function will submit a media type, an optional set of file extensions commonly used for the media type, and a typefind function. Once the typefind function inside the plug-in is called, the plug-in will check whether the data in the media stream matches the specific pattern of the media type marked by the media type. If so, it will notify the typefind element of the fact, telling us which media type has been identified and how sure we are that the stream is indeed that media type. Once this run has been completed for all plug-ins that implement the typefind feature, the typefind element will tell the application the type of media stream it considers recognized.

The following code should explain how to use typefind element. It prints the detected media type or tells you that the media type was not found. The next section describes more useful behaviors, such as concatenating decoded pipeline s together.

#include <gst/gst.h>

[.. my_bus_callback goes here ..]

static gboolean
idle_exit_loop (gpointer data)
{
  g_main_loop_quit ((GMainLoop *) data);

  /* once */
  return FALSE;
}

static void
cb_typefound (GstElement *typefind,
          guint       probability,
          GstCaps    *caps,
          gpointer    data)
{
  GMainLoop *loop = data;
  gchar *type;

  type = gst_caps_to_string (caps);
  g_print ("Media type %s found, probability %d%%\n", type, probability);
  g_free (type);

  /* since we connect to a signal in the pipeline thread context, we need
   * to set an idle handler to exit the main loop in the mainloop context.
   * Normally, your app should not need to worry about such things. */
  g_idle_add (idle_exit_loop, loop);
}

gint
main (gint   argc,
      gchar *argv[])
{
  GMainLoop *loop;
  GstElement *pipeline, *filesrc, *typefind, *fakesink;
  GstBus *bus;

  /* init GStreamer */
  gst_init (&argc, &argv);
  loop = g_main_loop_new (NULL, FALSE);

  /* check args */
  if (argc != 2) {
    g_print ("Usage: %s <filename>\n", argv[0]);
    return -1;
  }

  /* create a new pipeline to hold the elements */
  pipeline = gst_pipeline_new ("pipe");

  bus = gst_pipeline_get_bus (GST_PIPELINE (pipeline));
  gst_bus_add_watch (bus, my_bus_callback, NULL);
  gst_object_unref (bus);

  /* create file source and typefind element */
  filesrc = gst_element_factory_make ("filesrc", "source");
  g_object_set (G_OBJECT (filesrc), "location", argv[1], NULL);
  typefind = gst_element_factory_make ("typefind", "typefinder");
  g_signal_connect (typefind, "have-type", G_CALLBACK (cb_typefound), loop);
  fakesink = gst_element_factory_make ("fakesink", "sink");

  /* setup */
  gst_bin_add_many (GST_BIN (pipeline), filesrc, typefind, fakesink, NULL);
  gst_element_link_many (filesrc, typefind, fakesink, NULL);
  gst_element_set_state (GST_ELEMENT (pipeline), GST_STATE_PLAYING);
  g_main_loop_run (loop);

  /* unset */
  gst_element_set_state (GST_ELEMENT (pipeline), GST_STATE_NULL);
  gst_object_unref (GST_OBJECT (pipeline));

  return 0;
}

After the media type is detected, you can insert an element (such as a diverter or decoder) into the source pad of the typefind element, and then the decoding of the media stream will begin immediately.

Dynamic auto plug-in pipeline

See playback components to use advanced objects that can be used to build pipeline s dynamically.