Pou's IT Life

• Multi-View Application

利用 ProjectionManager APIs 在 App 裏面建立多個 window 呈現在不同的 screen 裏面 (利用 cable 或是 Miracast)，統一由一個 Windows 10 device 管理。

幾個重要的元件：

• DevicePicker

負責列出一個清單可讓用戶選擇設備的 flyout。CastingDevicePicker, DialDevicePicker 跟 DevicePicker 是很相近的類型，往下説明幾個重點：

Type	Name	Description
Events	DevicePickerDismissed	Indicates that the device picker was light dismissed by the user.
	DeviceSelected	Indicates that the user selected a device in the picker.
	DisconnectButtonClicked	Indicates that the user clicked or tapped the disconnect button for a device in the picker.
Methods	Hide	Hides the picker.
	PickSingleDeviceAsync	Shows the picker UI and returns the selected device; does not require you to register for an event.
	SetDisplayStatus	Updates the picker UI to reflect the provided status and display options for a specified device.
	Show	Shows the picker UI.
Properties	Filter	Gets the filter used to choose what devices to show in the picker. DevicePickerFilter 用于過濾與決定那些設備可以被放在 picker 上。使用 OR-ed 的機制做判斷。重要的屬性： SupportedDeviceClasses Gets a list of supported device classes to show in the picker. This defaults to an empty list (no filter). You can add device classes to this vector and filter the devices list to those that are in one or more of the provided classes. SupportedDeviceSelectors Gets a list of AQS filter strings. This defaults to empty list (no filter). You can add one or more AQS filter strings to this vector and filter the devices list to those that meet one or more of the provided filters.
	RequestProperties	Gets a collection of properties for the returned device information object.

• ProjectionManager 

定義很多 methods 支援管理 windows(app views) 傳送到第二以上的屏幕。

Type	Name	Description
Event	ProjectionDisplayAvailableChanged	Occurs when a projector or other secondary display becomes available or unavailable.
Method	GetDeviceSelector	Returns a string that is used to enumerate device services.
	RequestStartProjectingAsync(int32, int32, rect)	Makes a request to asynchronously sends a window (app view) to the projector or other secondary display.
	RequestStartProjectingAsync(int32, int32, rect, placement)	Makes a request to asynchronously sends a window (app view) to the projector or other secondary display with the specified preferred placement.
	StartProjectingAsync(int32, int32)	Asynchronously sends a window (app view) to the projector or other secondary display.
	StartProjectingAsync(int32, int32, DeviceInformation)	Asynchronously sends a window (app view) to the projector or other secondary display, and provides info about the display.
	StopProjectingASync	Asynchronously hides a window (app view) displayed by a projector or other secondary display.
	SwapDisplaysForViewsAsync	Asynchronously swaps the calling window (app view) with the window displayed on the projector or other secondary display.
Property	ProjectionDisplayAvailable	Gets whether or not a projection display is available to use.

 

• CoreApplication

允許 App 處理 state 改變，管理 winows (app views) 與 整合多種 UI frameworks。針對管理 window 的部分加以説明：

Type	Name	Description
Method	CreateNewView	Creates a new view for the app.
	GetCurrentView	Gets the active view for the app.
Property	MainView	Gets the main CoreApplicationView instance for all running apps that use this CoreApplication instance.
	Views	Gets all views (CoreApplicationView) for the app.

 

• ApplicationView

代表啓動的 application view 與相關狀態與行爲。

Method	Description
GetForCurrentView	Gets the view state and behavior settings of the active application.
GetApplicationViewIdFromWindow	Gets the window ID that corresponds to a specific CoreWindow managed by the app.

參考<Show multiple views for an app>的説明：

• 一個 app view 是一組 1:1 的 thread 與 window，app 利用 view 來顯示内容，而 view = CoreApplicationView object
• views 被 CoreApplication 管理，所以可以呼叫 CoreApplication.CreateNewView 來建立新的 view (CoreApplicationView)
• CoreApplicationView 具有 CoreWindow 與 CoreDispatcher ，可以視爲 Windows Runtime 利用它與 Windows System 互動
• 一般不會直接操作 CoreApplicationView , 而是改用 Windows Runtime 提供的 ApplicationView 來操作
• ApplicationView 包裝了很多的 properties, methods, events ，讓 app 可以跟 System 互動
• 因此，操作都會利用 ApplicationView.GetForCurrentView 取得 ApplicationView 來綁定當前的 CoerApplicationView 的 thread
• 同樣的 CoreWindw 不會直接操作而是利用 Window 物件來操作

可以總結，操作有關 view 的部分要使用 ApplicationView 與 Window，而 CoreWindow, CoreApplicationView 則是比較跟 System 互動的。

 

理解了 view 的概念之後，藉由下列的範例説明怎麽做到 Projection。

1. 建立 DevicePicker 并設定找出可以使用 Projection 的設備

private void OnProjectionClick(object sender, RoutedEventArgs e)
{
    if (devicePicker == null)
    {
    	devicePicker = new DevicePicker();
        // 設定來源是支援 Projection 的設備
    	devicePicker.Filter.SupportedDeviceSelectors.Add(ProjectionManager.GetDeviceSelector());
	devicePicker.DevicePickerDismissed += DevicePicker_DevicePickerDismissed;
	devicePicker.DeviceSelected += DevicePicker_DeviceSelected;
	devicePicker.DisconnectButtonClicked += DevicePicker_DisconnectButtonClicked;
    }
    // 從按下的 button 出現 picker 内容
    Button btn = sender as Button;
    GeneralTransform transform = btn.TransformToVisual(Window.Current.Content as UIElement);
    Point pt = transform.TransformPoint(new Point(0, 0));
    devicePicker.Show(new Rect(pt.X, pt.Y, btn.ActualWidth, btn.ActualHeight), Windows.UI.Popups.Placement.Above);
}

 

2. 處理 DeviceSelected 事件，建立一個 view 放到被選擇的設備

2-1. 建立一個 ProjectionBroker 記錄目前的 main view id 跟 控制新建立 view 的 ViewLifeTime：

這裏參考 ViewLifetimeControl.cs  用來管理新建立 view 的生命周期，藉由處理當 view 被關閉的時候釋放相關的狀態，加上新建立的 view 跟原本 app 啓動時使用的 view 是不同的，所以建立新 view 時才可以 new 這個類別，不然還是拿到原本的 main view。

public class ProjectionBroker
{
   public int MainViewId { get; set; }

   public ViewLifetimeControl ProjectionViewPageControl { get; set; }

   public ProjectionPage Content { get; set; }
}

2-2. 建立一個 ProjectionPage.xaml 放置要 Projection 的内容，與處理當設備被 StopProjectionAsync 的時候要關掉 view：

private ProjectionBroker projectionBroker;

protected override void OnNavigatedTo(NavigationEventArgs e)   
{
   base.OnNavigatedTo(e);
   if (e.Parameter != null)
   {
	Player.Source = new Uri("ms-appx:///Assets/Videos/kkbox_ja_adv.mp4");
	Player.Position = TimeSpan.FromSeconds(0);
	// 承接 新的 view 的 Life time control
	if (projectionBroker == null)
	{
	   projectionBroker = e.Parameter as ProjectionBroker;
	   projectionBroker.ProjectionViewPageControl.Released += LifetimeControl_Released;
	   projectionBroker.Content = this;
	}
   }
}

public async void StopProjection()
{

   // 停止 projection
   projectionBroker?.ProjectionViewPageControl?.StopViewInUse();
   await ProjectionManager.StopProjectingAsync(projectionBroker.ProjectionViewPageControl.Id, projectionBroker.MainViewId);
}

private void LifetimeControl_Released(object sender, EventArgs e)
{
   // 記得要釋放這些内容
   Player.Stop();
   Player.Source = null;
   projectionBroker.Content = null;
   projectionBroker.ProjectionViewPageControl = null;
   Window.Current.Close();
}

2-3. 處理用戶選擇設備之後要建立新的 view 跟通知 ProjectionManager 要開始 projection 新的 view 到設備上：

private async void DevicePicker_DeviceSelected(DevicePicker sender, DeviceSelectedEventArgs args)
{
    await Dispatcher.RunAsync(Windows.UI.Core.CoreDispatcherPriority.Normal, async () =>
    {
        // 更新 picker 上設備的 status
        sender.SetDisplayStatus(args.SelectedDevice, "connecting", DevicePickerDisplayStatusOptions.ShowProgress);

        // 取得目前選到設備的資訊
	activeDevice = args.SelectedDevice;

	// 現在 view 的 Id 與 CoreDispatcher
	int currentViewId = ApplicationView.GetForCurrentView().Id;
	CoreDispatcher currentDispatcher = Window.Current.Dispatcher;

	// 建立新的 view,
	if (projectionInstance.ProjectionViewPageControl == null)
	{
	    await CoreApplication.CreateNewView().Dispatcher.RunAsync(Windows.UI.Core.CoreDispatcherPriority.Normal, () =>
	    {
		// 建立新 viewe 的生命管理器
		projectionInstance.ProjectionViewPageControl = ViewLifetimeControl.CreateForCurrentView();
		projectionInstance.MainViewId = currentViewId;

		var rootFrame = new Frame();
		rootFrame.Navigate(typeof(ProjectionPage), projectionInstance);

		// 這裏的 Window 代表是新建立這個 view 的 Window
		// 但是要等到呼叫 ProjectionManager.StartProjectingAsync 才會顯示
		Window.Current.Content = rootFrame;
		Window.Current.Activate();
	    });
	}

	// 直接切換到指定的 view id
	//bool viewShown = await ApplicationViewSwitcher.TryShowAsStandaloneAsync(projectionInstance.ProjectionViewPageControl.Id);

	// 通知要使用新的 view
	projectionInstance.ProjectionViewPageControl.StartViewInUse();

	try
	{
	    await ProjectionManager.StartProjectingAsync(projectionInstance.ProjectionViewPageControl.Id, currentViewId, activeDevice);

	    player.Pause();

            sender.SetDisplayStatus(args.SelectedDevice, "connected", DevicePickerDisplayStatusOptions.ShowDisconnectButton);
	}
	catch (Exception ex)
	{
            sender.SetDisplayStatus(args.SelectedDevice, ex.Message, DevicePickerDisplayStatusOptions.ShowRetryButton);
            if (ProjectionManager.ProjectionDisplayAvailable == false)
            {
		throw;
            }
	}
    });
}

 

3. 處理 DisconnectionButtonClicked 事件，通知 StopProjectingAsync

private async void DevicePicker_DisconnectButtonClicked(DevicePicker sender, DeviceDisconnectButtonClickedEventArgs args) 
{
    await Dispatcher.RunAsync(Windows.UI.Core.CoreDispatcherPriority.Normal, () =>   
    {
         // 關閉新建立的 view         
         projectionInstance.Content.StopProjection();

         //Update the display status for the selected device.
         sender.SetDisplayStatus(args.Device, "Disconnecting", DevicePickerDisplayStatusOptions.ShowProgress);

         //Update the display status for the selected device.
         sender.SetDisplayStatus(args.Device, "Disconnected", DevicePickerDisplayStatusOptions.None);

         // Set the active device variables to null
         activeDevice = null;
    });
}

[注意]

• Projection 的 View 使用的 CoreDispatcher 是來自 CoreApplication 的，所以要小心它使用時可能會是 null。
• Projection 的 View 要確實的關閉，以免發生 OOM
• 傳輸過程也有可能遇到 exception，要記得多判斷 ProjectionManager.ProjectionDisplayAvailable
• 如果沒有要 Projection 衹是要建立新的 view，可以搭配  ApplicationViewSwitcher.TryShowAsStandaloneAsync 直接顯示

 

 

• Combine Application Level methods

根據上圖三種情境介紹了 CastingDevicePicker, DialDevicePicker, DevicePicker。這些功能可以獨立加入也可以混搭，統一使用 DevicePicker 來搜尋，重點在加入多個 Selector，如下範例：

private void OnCombinePickerClick(object sender, RoutedEventArgs e)
{
   if (devicePicker== null)
   {    
        devicePicker = new DevicePicker();

        // add casting
        devicePicker.Filter.SupportedDeviceSelectors.Add(CastingDevice.GetDeviceSelector(CastingPlaybackTypes.Video));

        // add dial
        devicePicker.Filter.SupportedDeviceSelectors.Add(DialDevice.GetDeviceSelector("castingsample"));

        // add projection
        devicePicker.Filter.SupportedDeviceSelectors.Add(ProjectionManager.GetDeviceSelector());

        devicePicker.DevicePickerDismissed += DevicePicker_DevicePickerDismissed;
        devicePicker.DeviceSelected += DevicePicker_DeviceSelected;
        devicePicker.DisconnectButtonClicked += DevicePicker_DisconnectButtonClicked;
    }

    player.Pause();

    // 從按下的 button 出現 picker 内容
    Button btn = sender as Button;
    GeneralTransform transform = btn.TransformToVisual(Window.Current.Content as UIElement);
    Point pt = transform.TransformPoint(new Point(0, 0));
    devicePicker.Show(new Rect(pt.X, pt.Y, btn.ActualWidth, btn.ActualHeight), Windows.UI.Popups.Placement.Above);
}

加入可以被 Projection, casting Video 内容，與支援 Dial 且是 castingsample 的 app name。

接著處理 DeviceSelected 事件，判斷這次選擇的設備支援那個類型，如果下範例：

private async void DevicePicker_DeviceSelected(DevicePicker sender, DeviceSelectedEventArgs args)
{
    await Dispatcher.RunAsync(Windows.UI.Core.CoreDispatcherPriority.Normal, async () =>
    {
        // 判斷是那種類型處理對應的轉換
	DeviceInformation selectedDevice = args.SelectedDevice;
	if (await DialDevice.DeviceInfoSupportsDialAsync(selectedDevice))
	{
            await SendDialParameter(sender, args);
	}
	else if (await CastingDevice.DeviceInfoSupportsCastingAsync(selectedDevice))
	{
	    await CastingVideoToScreen(sender, args);
	}
	else if (ProjectionManager.ProjectionDisplayAvailable)
	{
	    await ProjectioinViewToScreen(sender, args);
	}
    });
}

更多詳細的内容可以參考下面的範例程式，不過 combine 的機制還是有 bug，所以如果有比較新的用法可以參考：<AdvancedCasting>裏面的範例。

[補充]

• DeviceInformation

代表一個設備，可以取得或維護該設備部分(well-know) 的屬性，或是建立監控有哪些設備加入或刪除...等。

Type	Name	Description
Method	CreateFromIdAsync	Creates a DeviceInformation object from a DeviceInformation ID.
	CreateWatcher	Creates a DeviceWatcher for all devices.
	FindAllAsync	Enumerates all DeviceInformation objects.
	GetAqsFilterFromDeviceClass	Creates a filter to use to enumerate through a subset of device types.
	GetGlyphThumbnailAsync	Gets a glyph for the device.
	GetThumbnaiAsync	Returns a thumbnail image for the device.
	Update	Updates the properties of an existing DeviceInformation object.
Properties	Id	A string representing the identity of the device.
	EnclosureLocation	The physical location of the device in its enclosure.
	IsDefault	Indicates whether this device is the default device for the class.
	IsEnabled	Indicates whether this device is enabled.
	Kind	Gets the type of DeviceInformation represented by this object.
	Name	The name of the device.
	Pairing	Gets the information about the capabilities for this device to pair.
	Properties	Property store containing well-known values as well as additional properties that can be specified during device enumeration.

• ApplicationViewSwitcher

快速切換 app view 的功能。主要方法有：

Method	Description
DisableShowingMainViewOnActivation	Disables the primary window (app view) when the app is activated, showing the most recently displayed window instead.
DisableSystemViewActivationPolicy	Disables Windows shell control of the view selection on activation, and lets the app handle it instead.
PrepareForCustomAnimatedSwitchAsync	Prepares your app to visually transition between two windows with a custom animation.
SwitchAsync(Int32)	Visually replaces the calling window (app view) with a specified window.
TryShowAsStandaloneAsync(Int32)	Displays another window (app view) for the app on the screen, adjacent to the original window

 

[範例程式]

• CastingSample

======

利用 ProjectionManager 建立多個 View 非常適合在 Desktop 環境下使用，不一定需要投影到其他屏幕上面。因爲 Desktop 的用戶習慣多工的操作方式。

在 Mobile 的話比較就比較適合映射 Media 的内容。

 

Reference：

• Screen Casting: Develop Multi-Screen Universal Windows Apps Using Casting Technologies
• Media casting
• Windows API reference for Windows Runtime apps
• Media playback with MediaSource
• Create custom transport controls
• Media Transport Controls sample
• Show multiple views for an app
• Guidelines for projection manager