Microsoft Longhorn

DWM & Graphics Fixing DCE in Longhorn build 4008

In Longhorn build 4008, DCE was broken and attempting to start DCE would trigger a crash within UXDeskSB.exe. This would also break theming, which was also provided by themesrv.dll in the same process. In this article, we look into the root cause of the failure, and how a patch to winlogon.exe can fix it by changing the launch context of UxDeskSB.exe to be more robust and explicit.

Download the patched winlogon.

Summary

In Longhorn build 4008, winlogon.exe launches UxDeskSB.exe too early and with an underspecified desktop context. UxDeskSB.exe is started by winlogon!sub_1034BAD using CreateProcessW with a STARTUPINFO structure that does not set lpDesktop.

This matters because the launch happens after winlogon has created the WinSta0, Winlogon, and Default desktop objects, but before the startup thread completes its normal desktop selection and switching logic. As a result, UxDeskSB.exe inherits an ambiguous or transitional desktop context instead of being pinned to WinSta0\\Default.

Inside UxDeskSB.exe, dce.dll initializes Direct3D early. On Longhorn build 4008, that hosted launch context is sufficient to make IDirect3D9::GetDeviceCaps fail with D3DERR_NOTAVAILABLE (0x8876086A), even though the same machine can return valid HAL caps in a later standalone interactive test.

The patch forces winlogon to launch UxDeskSB.exe on WinSta0\\Default, matching the stronger launch behavior used by the later 4011 line.

Observed Failure

The visible error in dce.dll is:

Failed HrInit hr=8876086a

Tracing showed the failure occurs in the hosted DCE path during:

IDirect3D9::GetDeviceCaps(adapterOrdinal, D3DDEVTYPE_HAL, &caps)

and not in CreateDevice.

Excluding other causes

To rule out wider DirectX or driver issues, a test application was created that performs a similar D3D9 initialization sequence, including the same HAL caps query.

#include <windows.h>
#include <d3d9.h>
#include <stdio.h>

static void PrintHr(const char *label, HRESULT hr)
{
    printf("%s: 0x%08lX\n", label, (unsigned long)hr);
}

int main(void)
{
    IDirect3D9 *pD3D;
    UINT adapterCount;
    UINT i;

    pD3D = Direct3DCreate9(D3D_SDK_VERSION);
    if (pD3D == NULL)
    {
        printf("Direct3DCreate9 failed\n");
        return 1;
    }

    adapterCount = IDirect3D9_GetAdapterCount(pD3D);
    printf("AdapterCount: %u\n", adapterCount);

    for (i = 0; i < adapterCount; ++i)
    {
        D3DADAPTER_IDENTIFIER9 id;
        D3DDISPLAYMODE mode;
        D3DCAPS9 caps;
        HRESULT hr;

        ZeroMemory(&id, sizeof(id));
        ZeroMemory(&mode, sizeof(mode));
        ZeroMemory(&caps, sizeof(caps));

        hr = IDirect3D9_GetAdapterIdentifier(pD3D, i, 0, &id);
        PrintHr("GetAdapterIdentifier", hr);
        if (SUCCEEDED(hr))
        {
            printf("  Adapter %u\n", i);
            printf("  Description: %s\n", id.Description);
            printf("  Driver: %s\n", id.Driver);
            printf("  DeviceName: %s\n", id.DeviceName);
            printf("  VendorId: 0x%04lX\n", (unsigned long)id.VendorId);
            printf("  DeviceId: 0x%04lX\n", (unsigned long)id.DeviceId);
            printf("  SubSysId: 0x%08lX\n", (unsigned long)id.SubSysId);
            printf("  Revision: 0x%08lX\n", (unsigned long)id.Revision);
            printf("  WHQLLevel: 0x%08lX\n", (unsigned long)id.WHQLLevel);
        }

        hr = IDirect3D9_GetAdapterDisplayMode(pD3D, i, &mode);
        PrintHr("GetAdapterDisplayMode", hr);
        if (SUCCEEDED(hr))
        {
            printf("  Mode: %lux%lu fmt=%lu refresh=%lu\n",
                (unsigned long)mode.Width,
                (unsigned long)mode.Height,
                (unsigned long)mode.Format,
                (unsigned long)mode.RefreshRate);
        }

        hr = IDirect3D9_GetDeviceCaps(pD3D, i, D3DDEVTYPE_HAL, &caps);
        PrintHr("GetDeviceCaps(HAL)", hr);
        if (SUCCEEDED(hr))
        {
            printf("  DevCaps: 0x%08lX\n", (unsigned long)caps.DevCaps);
            printf("  Caps2: 0x%08lX\n", (unsigned long)caps.Caps2);
            printf("  TextureCaps: 0x%08lX\n", (unsigned long)caps.TextureCaps);
            printf("  SrcBlendCaps: 0x%08lX\n", (unsigned long)caps.SrcBlendCaps);
            printf("  DestBlendCaps: 0x%08lX\n", (unsigned long)caps.DestBlendCaps);
            printf("  AdapterOrdinal: %lu\n", (unsigned long)caps.AdapterOrdinal);
            printf("  DeviceType: %lu\n", (unsigned long)caps.DeviceType);
        }

        hr = IDirect3D9_GetDeviceCaps(pD3D, i, D3DDEVTYPE_REF, &caps);
        PrintHr("GetDeviceCaps(REF)", hr);

        printf("\n");
    }

    IDirect3D9_Release(pD3D);
    return 0;
}

When run in a normal interactive context on Longhorn build 4008, the test app successfully retrieves valid HAL caps:

AdapterCount: 1
GetAdapterIdentifier: 0x00000000
  Adapter 0
  Description: VMware SVGA II
  Driver: vmx_fb.dll
  DeviceName: \\.\DISPLAY1
  VendorId: 0x15AD
  DeviceId: 0x0405
  SubSysId: 0x040515AD
  Revision: 0x00000000
  WHQLLevel: 0x00000000
GetAdapterDisplayMode: 0x00000000
  Mode: 1280x720 fmt=22 refresh=60
GetDeviceCaps(HAL): 0x00000000
  DevCaps: 0x0019AEF0
  Caps2: 0xA0000000
  TextureCaps: 0x0007ED47
  SrcBlendCaps: 0x00003FFF
  DestBlendCaps: 0x000023FF
  AdapterOrdinal: 0
  DeviceType: 1
GetDeviceCaps(REF): 0x8876086A

Note that GetDeviceCaps(HAL) succeeds - it returns 0x0, which is S_OK. The failure in UxDeskSB.exe is not due to a machine-wide lack of D3D9 HAL support, but rather due to the specific environmental conditions of the hosted launch context.

So, how does UxDeskSB get launched?

4008 Launch Sequence

Longhorn build 4008 is unique amongst the builds that we have available in that winlogon.exe is responsible for launching UxDeskSB.exe. In earlier M4 builds, winsrv.dll is responsible for launching UxDeskSB.exe and in later M4 builds, uxtheme.dll is responsible for launching UxDeskSB.exe.

Relevant winlogon.exe behavior

winlogon prepares the session UI objects:

  • creates the window station WinSta0
  • sets the process window station to WinSta0
  • creates desktop Winlogon
  • creates desktop Default

In simplified pseudocode:

void SetupSessionUi(void)
{
    HWINSTA hwinsta = CreateWindowStationW(L"WinSta0", ...);
    SetProcessWindowStation(hwinsta);

    HDESK hWinlogon = CreateDesktopW(L"Winlogon", ...);
    HDESK hDefault  = CreateDesktopW(L"Default", ...);

    gSession->hWinSta = hwinsta;
    gSession->hWinlogon = hWinlogon;
    gSession->hDefault = hDefault;
}

After that, winlogon directly starts UxDeskSB.exe:

void LaunchUxDeskSb(void)
{
    STARTUPINFOW si;
    PROCESS_INFORMATION pi;
    WCHAR cmdLine[MAX_PATH];

    ZeroMemory(&si, sizeof(si));
    ZeroMemory(&pi, sizeof(pi));

    lstrcpyW(cmdLine, L"UxDeskSB.exe");
    si.cb = sizeof(si);

    CreateProcessW(
        NULL,
        cmdLine,
        NULL,
        NULL,
        FALSE,
        NORMAL_PRIORITY_CLASS,
        NULL,
        NULL,
        &si,
        &pi);
}

It is important to note that the STARTUPINFO structure does not set lpDesktop, which means the launched process inherits the desktop association of the parent thread at that moment. The parent thread is still in the process of setting up the session UI, and has not yet completed its normal desktop transition logic.

Why this is unsafe

The startup thread does not complete its desktop transition before launching UxDeskSB.exe. The sequence is effectively:

SetupSessionUi();     // creates WinSta0, Winlogon, Default
LaunchUxDeskSb_4008();
SetThreadDesktop(gSession->hWinlogon);
// ... later desktop/input-desktop query and switch logic ...
SwitchDesktop(...);

That means UxDeskSB.exe starts with:

  • no explicit desktop string
  • no user token
  • no user environment block
  • launch timing that precedes the normal interactive desktop transition

The child process therefore inherits a context from winlogon that is not guaranteed to be the final interactive desktop state. This is a problem because in this era of Windows, hardware acceleration and Direct3D support are only available on the interactive desktop (WinSta0\\Default).

How Longhorn build 4011 differs

At this point, it seemed instructive to compare this to how Longhorn build 4011 works, to see if there was any insight that could be gained from how Microsoft responded to this problem themselves. Longhorn build 4011 moved UxDeskSB.exe startup into uxtheme.dll, with a number of changes aimed at hardening the launch process. That implementation is materially stronger:

void LaunchUxDeskSb(HANDLE hTokenOrNull)
{
    STARTUPINFOW si;
    PROCESS_INFORMATION pi;
    WCHAR cmdLine[MAX_PATH];

    ZeroMemory(&si, sizeof(si));
    ZeroMemory(&pi, sizeof(pi));

    si.cb = sizeof(si);
    si.lpDesktop = L"WinSta0\\Default";

    ResolveConfiguredSbExecutable(cmdLine, ARRAYSIZE(cmdLine));

    if (hTokenOrNull != NULL)
    {
        CreateProcessAsUserW(hTokenOrNull, NULL, cmdLine, NULL, NULL,
            FALSE, NORMAL_PRIORITY_CLASS, NULL, NULL, &si, &pi);
    }
    else
    {
        CreateProcessW(NULL, cmdLine, NULL, NULL,
            FALSE, NORMAL_PRIORITY_CLASS, NULL, NULL, &si, &pi);
    }

    RegisterWaitForSingleObject(...); // restart / lifecycle management
}

It attempts to start UxDeskSB.exe with CreateProcessAsUserW if a user token is available, which is a more appropriate API for launching an interactive user process from a system service. However, even within the fallback path that uses CreateProcessW, the launch still explicitly forces the desktop context to the interactive desktop:

si.lpDesktop = L"WinSta0\\Default";

That strongly suggests the older null-desktop launch in 4008 was not considered robust enough. I would not be surprised if these fixes may have even been originally implemented within winlogon.exe before they got moved to uxtheme.dll later.

Patch Design

The patch modifies winlogon to set the desktop before calling CreateProcessW.

Desired logic:

void LaunchUxDeskSb_Patched(void)
{
    STARTUPINFOW si;
    PROCESS_INFORMATION pi;
    WCHAR cmdLine[MAX_PATH];

    ZeroMemory(&si, sizeof(si));
    ZeroMemory(&pi, sizeof(pi));

    lstrcpyW(cmdLine, L"UxDeskSB.exe");
    si.cb = sizeof(si);
    si.lpDesktop = L"WinSta0\\Default";

    CreateProcessW(
        NULL,
        cmdLine,
        NULL,
        NULL,
        FALSE,
        NORMAL_PRIORITY_CLASS,
        NULL,
        NULL,
        &si,
        &pi);
}

Because there is not enough inline space in the sub-routine for the extra instructions, the binary patch uses a trampoline:

  1. replace the start of the original launch setup with a jump to a code cave
  2. replay the overwritten original instructions in the cave
  3. add: si.lpDesktop = L"WinSta0\\Default";
  4. jump back to the original function just before the remaining CreateProcessW argument setup

Why the Patch Helps

The patch does not change token handling or launch timing in line with how 4011 works. It only changes the desktop binding.

That still matters because it removes the most obvious ambiguity in the original launch path:

  • before patch: child inherits whatever desktop association the parent thread happens to have at that moment
  • after patch: child is explicitly created on WinSta0\\Default

In practical terms, the patch aligns 4008 with the key launch behavior already present in 4011. This alone proved sufficient to fix the D3D9 initialization failure in dce.dll without any other changes.

Conclusion

The patch worked successfully for enabling DCE in Longhorn build 4008, as you can see in the video below. The D3D9 initialization sequence in dce.dll now completes successfully, and the expected DCE effects are rendered without error.

I’d like to thank Lukáš for our long back-and-forth on this issue (and others), and Mainnn for testing this and confirming the patch worked as intended. As further disclosure, I also used DeepSeek 4 Pro and GPT 5.4 to assist with the reverse engineering and patch development process, as I did with the 3706 DCE patch, with GPT 5.4 in particular being used to create the patch itself.

Download the patched winlogon.