Notatki o Windows 8 Consumer Preview - odc. 13

W 13-tym odcinku poruszam multimedia - nagrywanie i odtwarzanie audio oraz video, strumieniowanie PlayTo, konwersja i edycja.

W wersji Consumer Preview można dostrzec pewne drobne zmiany (np. inna nazwa metody lub dwie metody zamiast jednej albo odwrotnie, nieco zretuszowane przykłady, nieco więcej informacji np. ograniczenie na liczbę efektów dla video), ale ogólnie znaczących różnic nie zauważyłem. Moją uwagę jedynie zwróciły  możliwość wyłączenia PlayTo dla danego audio/video na stronie oraz wskazania, od którego audio/video na stronie PlayTo ma zacząć, a także pewne rozszerzenia MediaElement w stosunku np. do Silverlight będące odpowiednikami funkcjonalności Windows 8 wspieranych w HTML5 i JS.

JS

Multimedia

Quickstart: capturing video using the MediaCapture API

var oMediaCapture;
var profile;
var captureInitSettings;
var deviceList = new Array();
var recordState = false;
var storageFile;

function errorHandler(e) {
    sdkSample.displayStatus(e.message + ", Error Code: " + e.code.toString(16));
}

// Begin initialization.
function initialization() {
    document.getElementById("message").innerHTML = "Initialization started.";
    enumerateCameras();  
}

// Identify available cameras.
function enumerateCameras() {
    var deviceInfo = Windows.Devices.Enumeration.DeviceInformation;
    deviceInfo.findAllAsync(Windows.Devices.Enumeration.DeviceClass.videoCapture).then(function (devices) {
        // Add the devices to deviceList
        if (devices.length > 0) {
          
            for (var i = 0; i < devices.length; i++) {
                deviceList.push(devices[i]);             
            }
    
            initCaptureSettings();
            initMediaCapture();
            document.getElementById("message").innerHTML = "Initialization complete.";

        } else {
            sdkSample.displayError("No camera device is found ");           
        }
    }, errorHandler);
}

// Initialize the MediaCaptureInitialzationSettings.
function initCaptureSettings() {
    captureInitSettings = null;
    captureInitSettings = new Windows.Media.Capture.MediaCaptureInitializationSettings();
    captureInitSettings.audioDeviceId = "";
    captureInitSettings.videoDeviceId = "";
    captureInitSettings.streamingCaptureMode = Windows.Media.Capture.StreamingCaptureMode.audioAndVideo;
    captureInitSettings.photoCaptureSource = Windows.Media.Capture.PhotoCaptureSource.videoPreview;
    captureInitSettings.realTimeModeEnabled = true;
    if (deviceList.length > 0)
        captureInitSettings.videoDeviceId = deviceList[0].id;
}

// Create a profile.
function createProfile() {
    profile = Windows.Media.MediaProperties.MediaEncodingProfile.createMp4(
        Windows.Media.MediaProperties.VideoEncodingQuality.qvga);
}

// Create and initialze the MediaCapture object.
function initMediaCapture() {
    oMediaCapture = null;
    oMediaCapture = new Windows.Media.Capture.MediaCapture();
    oMediaCapture.initializeAsync(captureInitSettings).then (function (result) {
       createProfile();
    }, errorHandler);   
}

// Start the video capture.
function startMediaCaptureSession() {
   Windows.Storage.KnownFolders.videosLibrary.createFileAsync("cameraCapture.mp4", Windows.Storage.CreationCollisionOption.generateUniqueName).then(function (newFile) {
        storageFile = newFile;
        oMediaCapture.startRecordToStorageFileAsync(profile, storageFile).then(function (result) {          
          
        }, errorHandler);
    }  );  
}

// Stop the video capture.
function stopMediaCaptureSession() {
    oMediaCapture.stopRecordAsync().then(function (result) {
       
    }, errorHandler);        
}

How to rotate captured video

You can rotate video captured from a camera by using the SetPreviewRotation and SetRecordRotation methods on the MediaCapture class.

The MediaCapture class contains methods and properties for managing captured video. To capture and rotate video from a camera, you will need to use InitializeAsync, StartPreviewAsync, and SetPreviewRotation. Use SetRecordRotation to set the rotation for video that is being recorded.

Similarly, the rotation that is used for previewing a video is set by calling the GetPreviewRotation and SetPreviewRotation methods. These methods also use the VideoRotation enumeration values to specify the amount by which to rotate the video.

Quickstart: adding audio in a Metro style app

<audio>

Stream category	Description	Background Capable?
Alert	Looping or longer running alert sounds: Alarms Ring tones Ringing notification Sounds that need to attenuate existing audio	No
BackgroundCapableMedia	For audio that needs to continue playing in the background. Examples include the following local media playback scenarios: Local playlist Streaming radio Streaming playlist Music videos Streaming audio/radio, YouTube, Netflix, etc.	Yes
Communications	For streaming communication audio such as the following: VOIP Real time chat or other type of phone call	Yes
ForeGroundOnlyMedia	Games or other sounds designed only to work in the foreground, but will mute existing background media sounds. Game audio needed for the game experience (dancing games, music games) Feature films (designed to pause when they go to the background)	No
GameEffects	Game sound effects designed to mix with existing audio Characters talking All non-music sounds	No
GameMedia	Background music played by a game	No
SoundEffects	Game or other sound effects designed to mix with existing audio: Characters talking Beeps, dings, brief sounds	No
Other	Default audio type, and recommended for all audio media that does not need to continue playing in the background.	No

How to configure keys for media controls

// Assign the button object to MediaControls
MediaControls = Windows.Media.MediaControl;

// Add event listeners for the buttons
MediaControls.addEventListener(“playselected”, play, false);
MediaControls.addEventListener(“pauseselected”, pause, false);
MediaControls.addEventListener(“playpausetoggleselected”, playpausetoggle, false);

// Add event listeners for the audio element
document.GetElementById(“audiotag”).addEventListener(“playing”, playing, false);
document.GetElementById(“audiotag”).addEventListener(“paused”, paused, false);
document.GetElementById(“audiotag”).addEventListener(“ended”, ended, false);

// Define functions that will be the event handlers
function play() {
    document.getElementById(“audiotag”).play();
}
function pause() {
    document.getElementById(“audiotag”).pause();
}

function playpausetoggle() {
    if(MediaControls.isPlaying === true) {
        document.getElementById(“audiotag”).pause();
    } else {
        document.getElementById(“audiotag”).play();
    }
}

function playing() {
    MediaControls.isPlaying = true;
}

function paused() {
    MediaControls.isPlaying = false;
}

function ended() {
    MediaControls.isPlaying = false;
}

At the beginning of the playlist there is no previous track. Therefore, the Previous Track button is disabled for the first track of audio in the sample. Similarly, at the end of the playlist, there is no next track, so the Next Track button is disabled.

// Assign the button object to MediaControls
MediaControls = Windows.Media.MediaControl;
…

// Disable the previous track button
MediaControls.removeEventListener(“previoustrackselected”, previoustrack);

// Disable the next track button
MediaControls.removeEventListener(“nexttrackselected”, nexttrack);

After the first track, but before the last track, the Previous and Next Track buttons are enabled. If your playlist has three or more tracks in it, then when your audio app is playing a track other than the first or last track, it is good practice to enable the Previous and Next Track buttons.

// Assign the button object to MediaControls
MediaControls = Windows.Media.MediaControl;
…

// Enable the previous track button
MediaControls.addEventListener(“previoustrackselected”, previoustrack, false);

// Enable the next track button
MediaControls.addEventListener(“nexttrackselected”, nexttrack, false);

Playing and previewing audio and video

How to enable effects in a <video> tag

function effects() {
    var myVideo = document.getElementById("videoTag1");
    myVideo.msInsertVideoEffect(
        "Windows.Media.VideoEffects.videoStabilization", // ClassID
        true,   // effectRequired
        null);  // config
    myVideo.play();
}

The number of effects on each active stream is limited to a maximum of 2. For example, a video tag can have two video effects enabled and two audio effects enabled if there is an audio stream being played out of the same video file.

How to optimize video rendering

For msIsLayoutOptimalForPlayback to be true, avoid the following:

• Video elements with CSS outlines set.
• Rendering a video element through a Canvas.
• Embedding video elements in a SVG.

How to play a local media file using HTML5

function playVideo() {
    var openPicker = new Windows.Storage.Pickers.FileOpenPicker();
    openPicker.pickSingleFileAsync().then(function (fileItem) {
        if (fileItem) {
            var video = document.getElementById("myVideo");
            video.src = URL.createObjectURL(fileItem);
            video.play();
        }
    });
}

How to select audio tracks in different languages

You can use the audioTracks property to switch between multiple audio tracks on a media file. For example, media files could contain multiple audio tracks in different languages. The audioTracks attribute is available on the Video and Audio elements.

function selectLang() {
    var myVideo = document.getElementById("videoTag1");
   
    if (myVideo.audioTracks.length > 1) {
        for (var i = 0; i < myVideo.audioTracks.length ; i++) {
            if (myVideo.audioTracks[i].language == "en-gb") {
                myVideo.audioTracks[i].enabled = true;
            }
            else {
                myVideo.audioTracks[i].enabled = false;
            }
        }
    }

    myVideo.play();
}

How to zoom in and remove letterbox from a video

The msZoom property can be used to zoom into a video to remove the letterbox or pillarbox from the rendered video.

If the native aspect ratio of a video frame, which is defined by the videoWidth and videoHeight attributes, does not match the aspect ratio of the Video tag, which is defined by the width and height attributes, the video is rendered with letterbox or pillarbox format. Letterbox and pillarbox are the black bars on either the left and right of the video or the top and bottom of the video.

When the msZoom attribute is set to True, the rendered video is trimmed to fit the dimensions of the Video object. Either the top and bottom of the video is trimmed or the left and right of the video is trimmed.

Using media extensions

To use a built-in audio or video effect, you simply need to use the msInsertVideoEffect and msInsertAudioEffect methods to add the effect to the media pipeline, and msClearEffects to remove the effect. For transcoding, use the MediaTranscoder.AddVideoEffect and MediaTranscoder.AddAudioEffect.

Custom extensions

A Media Extension consists of a hybrid object that implements both Component Object Model (COM) and Windows Runtime interfaces. The COM interfaces interact with the Microsoft Media Foundation pipeline. The Windows Runtime interfaces activate the component and interact with the Metro style app.

In most situations, it is recommended that you use Visual C++ with Component Extensions (C++/CX ) to interact with the Windows Runtime. But in the case of hybrid components that implement both COM and Windows Runtime interfaces, such as Media Extensions, this is not possible. C++/CX can only create Windows Runtime objects. So, for hybrid objects it is recommended that you use Windows Runtime C++ Template Library to interact with the Windows Runtime. Be aware that Windows Runtime C++ Template Library has limited support for implementing COM interfaces.

A custom plugin cannot override a Microsoft provided component that supports the same format. For example, since Microsoft provides support for MPEG-Layer 3 (MP3), an application will not be able to use a plugin that is registered for MP3.

To create a custom effect, you must perform the following steps.

1. Create a native Media Foundation extension and implement IMediaExtension.
2. Register and activate the plugin.
3. Add the plugin in your application to the media pipeline.

Media Plugin Sample

Streaming media to devices using Play To

By default, Play To is enabled for all audio and video elements on an application page. You can disable this default behavior in your app, or you can identify which audio or video elements are not available to the default Play To behavior.

To disable the default Play To behavior in your application, set the defaultSourceSelection property of the PlayToManager object to false.

var ptm = Windows.Media.PlayTo.PlayToManager.getForCurrentView();
ptm.defaultSourceSelection = false;

To disable the default Play To behavior for a specific audio or video element, include the x-ms-playToDisabled property in the element markup.

<video src="http://sample.microsoft.com/video.mp4" x-ms-playToDisabled />

You can also disable the default Play To behavior for an element in JavaScript by setting the msPlayToDisabled property to true.

If you have more than one audio or video element on a page and you want to identify an element as the first element for Play To to stream you can include the x-ms-playToPrimary property in the element markup.

<video src="http://sample.microsoft.com/showvideo.mp4" x-ms-playToPrimary />

You can also identify the first element for Play To to stream in JavaScript by setting the msPlayToPrimary property to true.

var ptm = Windows.Media.PlayTo.PlayToManager.getForCurrentView();
ptm.addEventListener("sourcerequested", sourceRequestHandler, false);

function sourceRequestHandler(e) {
    try {
        var sr = e.sourceRequest;
        var controller;

        try {
            controller = mediaElement.msPlayToSource;
        } catch (ex) {
            id("messageDiv").innerHTML +=
                "msPlaytoSource failed: " + ex.message + "<br/>";
        }

        sr.setSource(controller);
    } catch (ex) {

        id("messageDiv").innerHTML +=
            "Exception encountered: " + ex.message + "<br/>";
    }
}

In order to use Windows Media Player as a target device your computer must be connected to a private network.

Quickstart: streaming a slide show using Play To

This sample displays images as a slide show using the images in the root folder of the pictures library. This is accomplished by first obtaining the list of images from the pictures library, and then using two <image> tags to cycle through them.

The reason that two <image> tags are used is for when the slide show is streaming to a target device. In this case, we make use of the ability to buffer the next media item using Play To. This is optional, but is useful for scenarios where it takes additional time to obtain the next media item to stream.

To buffer the next media item, we set the Play To source of the next item to the next property of the current item.We then set the source of the next item while the current item is displayed. When the current item completes, we call the playNext method of the current item to stream the next media source to the target device.

In the sourcerequested event handler, you pass the msPlayToSource property of the <image> element to the setSource method of the PlayToManager and set the next properties of each media element.

Transcoding

Transcoding video files

/// <param name="srcFile" type="IStorageFile"/>
/// <param name="destFile" type="IStorageFile"/>
function TranscodeFile(srcFile, destFile) {

    var profile = Windows.Media.MediaProperties.MediaEncodingProfile.createMp4(
            Windows.Media.MediaProperties.VideoEncodingQuality.hd720p);

    var transcoder = new Windows.Media.Transcoding.MediaTranscoder();

    transcoder.prepareFileTranscodeAsync(srcFile, destFile, profile).then(function (result) {
        if (result.canTranscode) {
            result.transcodeAsync().then(transcodeComplete, transcoderErrorHandler, transcodeProgress);
        } else {
            // Handle error condition. result.failureReason
        }
    });
};

How to defer a transcode session

   function(file){   
        destination = file;
               
        var profile = Windows.Media.MediaProperties.MediaEncodingProfile.createMp4(Windows.Media.MediaProperties.VideoEncodingQuality.hd720p);

        var transcoder = new Windows.Media.Transcoding.MediaTranscoder();
        return transcoder.prepareFileTranscodeAsync(source, destination, profile);
    }).then(

    // Begin phase 2.

    function(deferral){
        return prepareTranscodeResult.transcodeAsync();
    }).then(
    function(){
        // Update the UI for completion.
    },
    function(error){
        // Handle errors.
    },
    function(percent)
    {
        // Update the UI for progress.
    });

C#

Capturing or rendering audio, video, and images

How to preview video from a webcam

<CaptureElement Name="capturePreview" Height="400" />

MediaCapture captureMgr = new MediaCapture();
await captureMgr.InitializeAsync();

capturePreview.Source = captureMgr;

await captureMgr.StartPreviewAsync();

How to rotate captured video

captureMgr.SetPreviewRotation(VideoRotation.Clockwise90Degrees);

captureMgr.SetRecordRotation(VideoRotation.Clockwise90Degrees);

How to add video stabilization effect to captured video

Add the video stabilization effect during live capture

MediaCapture captureMgr = new MediaCapture();
await captureMgr.InitializeAsync();

await captureMgr.AddEffectAsync(
    MediaStreamType.VideoRecord,
    Windows.Media.VideoEffects.VideoStabilization,
    null);

// captureMgr is of type MediaCapture.
await captureMgr.ClearEffectsAsync(MediaStreamType.VideoRecord);

Add the video stabilization effect during transcoding

MediaTranscoder transcoder = new MediaTranscoder();

transcoder.AddVideoEffect(
    "Windows.Media.VideoEffects.VideoStabilization");

MediaTranscoder transcoder = new MediaTranscoder();

transcoder.AddVideoEffect(
    "Windows.Media.VideoEffects.VideoStabilization");

Playing and previewing audio and video

How to play a local media file using a MediaElement

mediaControl.SetSource(stream, file.ContentType); mediaControl.Play();

How to select audio tracks in different languages

You can use the AudioStreamIndex and GetAudioStreamLanguage properties of a MediaElement to switch between multiple language audio tracks on a media file. You can also use multiple audio tracks to include additional tracks, such as director commentaries on films.

// media is a MediaElement defined in XAML
for (int index = 0; index < media.AudioStreamCount; index++)
{
    if (media.GetAudioStreamLanguage(index) == "ja")
    {
        media.AudioStreamIndex = index;        
    }
}

How to enable low-latency playback

Setting the RealTimePlayback property on a MediaElement enables low-latency playback.

media.RealTimePlayback = true;

Streaming media to devices using Play To

private Windows.Media.PlayTo.PlayToManager ptm;

protected override void OnNavigatedTo(NavigationEventArgs e)
{
    ptm = Windows.Media.PlayTo.PlayToManager.GetForCurrentView();
    ptm.SourceRequested += SourceRequested;

    ShowMedia("VideoPlayer");
    LoadVideo();
}

private void SourceRequested(Windows.Media.PlayTo.PlayToManager sender,
                             Windows.Media.PlayTo.PlayToSourceRequestedEventArgs e)
{
    try
    {
        Windows.Media.PlayTo.PlayToSourceRequest sr = e.SourceRequest;
        Windows.Media.PlayTo.PlayToSource controller = null;

        try
        {
            if (mediaElement is Image)
            {
                controller = ((Image)mediaElement).PlayToSource;
            }
            else
            {
                controller = ((MediaElement)mediaElement).PlayToSource;
            }

        }
        catch (Exception ex)
        {
            MessageBlock.Text += "Exception encountered: " + ex.Message + "\n";
        }

        sr.SetSource(controller);
    }
    catch (Exception ex)
    {
        MessageBlock.Text += "Exception encountered: " + ex.Message + "\n";
    }
}

Streaming a slide show using Play To

Transcoding

Transcoding video files

async void TranscodeFile(StorageFile srcFile, StorageFile destFile)
{
    MediaEncodingProfile profile =
        MediaEncodingProfile.CreateMp4(VideoEncodingQuality.HD720p);

    MediaTranscoder transcoder = new MediaTranscoder();

    PrepareTranscodeResult prepareOp = await
        transcoder.PrepareFileTranscodeAsync(srcFile, destFile, profile);

    if (prepareOp.CanTranscode)
    {
        var transcodeOp = prepareOp.TranscodeAsync();
        transcodeOp.Progress +=
            new AsyncActionProgressHandler<double>(TranscodeProgress);
        transcodeOp.Completed +=
            new AsyncActionWithProgressCompletedHandler<double>(TranscodeComplete);
    }
    else
    {
        switch (prepareOp.FailureReason)
        {
            case TranscodeFailureReason.CodecNotFound:
                OutputText("Codec not found.");
                break;
            case TranscodeFailureReason.InvalidProfile:
                OutputText("Invalid profile.");
                break;
            default:
                OutputText("Unknown failure.");
                break;
        }
    }
}