Dreamscene 2 version 1.3 has been released and now supports mouse and desktop interaction. This function will not affect performance and basically does not occupy CPU. This function gives me a deeper understanding of Windows message mechanism. In this blog, I will introduce the implementation in detail.

Welcome to Star and Fork https://github.com/he55/DreamScene2

Implementation principle

Using WIN32 API SetWindowsHookEx Function Hook mouse and keyboard messages, in the hook processing function to capture mouse and keyboard messages and then call PostMessage Function to send a forward message to the dynamic desktop window.

Set mouse and keyboard hooks

The first parameter of the function is the Hook type. The Hook mouse message can be passed to WH_MOUSE_LL, Hook keyboard messages can be transmitted to WH_KEYBOARD_LL. The second parameter is the address of the custom Hook message processing function. The third parameter of the function is the module handle where the Hook function is located. When the Hook type is WH_MOUSE_LL or WH_KEYBOARD_LL, you can directly transfer the current module handle. The fourth parameter of the function is the thread Id. pass NULL to capture all messages.

Set the Hook code. Save the return value of SetWindowsHookEx function, which is required when unloading Hook

HHOOK g_hLowLevelMouseHook = NULL;
HHOOK g_hLowLevelKeyboardHook = NULL;

HMODULE hModule = GetModuleHandle(NULL);
g_hLowLevelMouseHook = SetWindowsHookEx(WH_MOUSE_LL, LowLevelMouseProc, hModule, NULL);
g_hLowLevelKeyboardHook = SetWindowsHookEx(WH_KEYBOARD_LL, LowLevelKeyboardProc, hModule, NULL);

Uninstall Hook code

UnhookWindowsHookEx(g_hLowLevelMouseHook);
UnhookWindowsHookEx(g_hLowLevelKeyboardHook);

Write hook handling functions

WH_MOUSE_LL and wh_ KEYBOARD_ The signature of the hook processing function of ll is the same. The wParam parameter is a message type, and the lParam parameter is a pointer, which is related to the type of the hook function. When the hook type is wh_ MOUSE_ LParam parameter when ll is MSLLHOOKSTRUCT Structure pointer. When the hook type is wh_ KEYBOARD_ lParam parameter when ll is KBDLLHOOKSTRUCT Structure pointer.

Hook handler signature

LRESULT CALLBACK xxxProc(
  _In_ int    nCode,
  _In_ WPARAM wParam,
  _In_ LPARAM lParam
);

Mouse hook handler

LRESULT CALLBACK LowLevelMouseProc(int    nCode, WPARAM wParam, LPARAM lParam) {
    return CallNextHookEx(NULL, nCode, wParam, lParam);
}

Process WM_LBUTTONDOWN mouse down message

The wParam parameter of the mouse hook processing function is the mouse message type. The lParam parameter needs to be converted into the pointer of msllhooktrue structure, the pt field of msllhooktrue structure and the coordinates of the mouse relative to the screen. To forward the mouse press message, you need to see WM_LBUTTONDOWN Definition of message: WM_ The wParam parameter of lbuttondown message is the status of the key, and the low byte of lParam parameter is the x coordinate of the cursor and the high byte is the y coordinate of the cursor. Note that the meanings of the wParam and lParam parameters of the mouse hook processing function and the PostMessage function are different, and they need to be converted into the parameters required by the PostMessage function.

WM_LBUTTONDOWN processing method

LRESULT CALLBACK LowLevelMouseProc(int    nCode, WPARAM wParam, LPARAM lParam) {
    MSLLHOOKSTRUCT* p = (MSLLHOOKSTRUCT*)lParam;
    LONG lp = MAKELONG(p->pt.x, p->pt.y); // Low byte x coordinate, high byte y coordinate

    if (wParam == WM_LBUTTONDOWN) {
        PostMessage(g_hWnd, (UINT)wParam, MK_LBUTTON, lp); // Send mouse down message to dynamic desktop window
    }
    return CallNextHookEx(NULL, nCode, wParam, lParam);
}

WM_LBUTTONUP and WM_MOUSEMOVE is handled in the same way

LRESULT CALLBACK LowLevelMouseProc(int    nCode, WPARAM wParam, LPARAM lParam) {
    MSLLHOOKSTRUCT* p = (MSLLHOOKSTRUCT*)lParam;
    LONG lp = MAKELONG(p->pt.x, p->pt.y);

    if (wParam == WM_MOUSEMOVE) {
        PostMessage(g_hWnd, (UINT)wParam, MK_XBUTTON1, lp);
    }
    else  if (wParam == WM_LBUTTONDOWN || wParam == WM_LBUTTONUP) {
        PostMessage(g_hWnd, (UINT)wParam, MK_LBUTTON, lp);
    }
    return CallNextHookEx(NULL, nCode, wParam, lParam);
}

Optimize mouse message forwarding

The above code will forward all mouse messages. In fact, it does not want to forward all mouse messages. For the messages of mouse press and release, only the mouse messages focusing on the desktop are forwarded.

Determine whether the foreground window is a desktop

BOOL DS2_IsDesktop(void) {
    HWND hProgman = FindWindow("Progman", "Program Manager");
    HWND hWorkerW = NULL;

    HWND   hShellViewWin = FindWindowEx(hProgman, NULL, "SHELLDLL_DefView", NULL);
    if (!hShellViewWin)
    {
        HWND hDesktopWnd = GetDesktopWindow();
        do
        {
            hWorkerW = FindWindowEx(hDesktopWnd, hWorkerW, "WorkerW", NULL);
            hShellViewWin = FindWindowEx(hWorkerW, NULL, "SHELLDLL_DefView", NULL);
        } while (!hShellViewWin && hWorkerW);
    }

    HWND hForegroundWindow = GetForegroundWindow();
    return hForegroundWindow == hWorkerW || hForegroundWindow == hProgman;
}

For the message of mouse movement, forward the message of mouse movement on the desktop.

LRESULT CALLBACK LowLevelMouseProc(int    nCode, WPARAM wParam, LPARAM lParam) {
    MSLLHOOKSTRUCT* p = (MSLLHOOKSTRUCT*)lParam;
    LONG lp = MAKELONG(p->pt.x, p->pt.y);

    if (wParam == WM_MOUSEMOVE) {
        RECT rect;
        GetWindowRect(GetForegroundWindow(), &rect);

        if (!PtInRect(&rect, p->pt)) {
            PostMessage(g_hWnd, (UINT)wParam, MK_XBUTTON1, lp);
        }
    }
    return CallNextHookEx(NULL, nCode, wParam, lParam);
}

Complete mouse hook processing function code

LRESULT CALLBACK LowLevelMouseProc(int    nCode, WPARAM wParam, LPARAM lParam) {
    MSLLHOOKSTRUCT* p = (MSLLHOOKSTRUCT*)lParam;
    LONG lp = MAKELONG(p->pt.x, p->pt.y);

    if (DS2_IsDesktop()) {
        if (wParam == WM_MOUSEMOVE) {
            PostMessage(g_hWnd, (UINT)wParam, MK_XBUTTON1, lp);
        }
        else  if (wParam == WM_LBUTTONDOWN || wParam == WM_LBUTTONUP) {
            PostMessage(g_hWnd, (UINT)wParam, MK_LBUTTON, lp);
        }
        else  if (wParam == WM_MOUSEWHEEL) {
            // TODO:
        }
    }
    else  if (wParam == WM_MOUSEMOVE) {
        RECT rect;
        GetWindowRect(GetForegroundWindow(), &rect);

        if (!PtInRect(&rect, p->pt)) {
            PostMessage(g_hWnd, (UINT)wParam, MK_XBUTTON1, lp);
        }
    }

    return CallNextHookEx(NULL, nCode, wParam, lParam);
}

Keyboard hook handler

The wParam parameter of the keyboard hook processing function is the keyboard message type, and the lParam parameter needs to be converted into a KBDLLHOOKSTRUCT structure pointer. scanCode field and vkCode field are used in KBDLLHOOKSTRUCT structure. Keyboard message WM_KEYDOWN and WM_KEYUP The wParam parameter of the message is vkCode, and the meaning of the lParam parameter is complex.

WM_ lParam parameter bit description of Keydown message

Bits	explain
0-15	The duplicate count of the current message.
16-23	Scan code
24	Indicates that the key is an extension key. If it is an extension key, the value is 1, otherwise it is 0.
25-28	Reserved, not used.
29	Context code. For WM_KEYDOWN message this value is always 0.
30	Previous key status. The value is 1 if the key is off before sending the message, and 0 if the key is started.
31	Transition state. For WM_KEYDOWN message this value is always 0.

WM_ lParam parameter bit description of Keyup message

Bits	explain
0-15	The duplicate count of the current message. For WM_ For Keyup messages, the repeat count is always 1.
16-23	Scan code
24	Indicates that the key is an extension key. If it is an extension key, the value is 1, otherwise it is 0.
25-28	Reserved, not used.
29	Context code. For WM_ The value of the Keyup message is always 0.
30	Previous key status. For WM_ The value of the Keyup message is always 1.
31	Transition state. For WM_ The value of the Keyup message is always 1.

Complete keyboard hook processing function code

LRESULT CALLBACK LowLevelKeyboardProc(int    nCode, WPARAM wParam, LPARAM lParam) {
    if (DS2_IsDesktop()) {
        KBDLLHOOKSTRUCT* p = (KBDLLHOOKSTRUCT*)lParam;

        if (wParam == WM_KEYDOWN) {
            int lp = 1 | (p->scanCode << 16) | (1 << 24) | (0 << 29) | (0 << 30) | (0 << 31);
            PostMessage(g_hWnd, (UINT)wParam, p->vkCode, lp);
        }
        else if (wParam == WM_KEYUP) {
            int lp = 1 | (p->scanCode << 16) | (1 << 24) | (0 << 29) | (1 << 30) | (1 << 31);
            PostMessage(g_hWnd, (UINT)wParam, p->vkCode, lp);
        }
    }

    return CallNextHookEx(NULL, nCode, wParam, lParam);
}

All codes

https://github.com/he55/DreamScene2

How to use Kanban https://www.cnblogs.com/he55/p/15705047.html

Write at the end

The next step is to add ffmpeg video playback engine