Hello everyone.

We have decided to phase out the Mixed Reality Forums over the next few months in favor of other ways to connect with us.

The first way we want to connect with you is our mixed reality developer program, which you can sign up for at https://aka.ms/IWantMR.

The plan between now and the beginning of May is to clean up old, unanswered questions that are no longer relevant. The forums will remain open and usable.

On May 1st we will be locking the forums to new posts and replies. They will remain available for another three months for the purposes of searching them, and then they will be closed altogether on August 1st.

So, where does that leave our awesome community to ask questions? Well, there are a few places we want to engage with you. For technical questions, please use Stack Overflow, and tag your questions using either hololens or windows-mixed-reality. If you want to join in discussions, please do so in the HoloDevelopers Slack, which you can join by going to https://aka.ms/holodevelopers. And always feel free to hit us up on Twitter @MxdRealityDev.

MediaCapture/FrameReader only reads 10 frames on first run

I have a problem where the MediaCapture and FrameReader don't record more than 10 frames on the first run. I'm doing some image processing in a second thread and for some reason it just stops. I've checked the FrameArrived event that it is only being fired 10 times. This is a project built with Unity with an image recording/processing script written with the Windows API. On the first run it stops, but if I stop the recording and restart it everything works perfectly.

So first, I'm initializing the camera like this:

private async Task InitializeCameraAsync()
    {
        if (mediaCapture != null) return;
        var allGroups = await MediaFrameSourceGroup.FindAllAsync();

        if (allGroups.Count <= 0) return;

        mediaCapture = new MediaCapture();
        var settings = new MediaCaptureInitializationSettings
        {
            SourceGroup = allGroups[0],
            SharingMode = MediaCaptureSharingMode.SharedReadOnly,
            StreamingCaptureMode = StreamingCaptureMode.Video,
            MemoryPreference = MediaCaptureMemoryPreference.Cpu
        };

        await mediaCapture.InitializeAsync(settings);

        try
        {
            var mediaFrameSourceVideoPreview = mediaCapture.FrameSources.Values.Single(x => x.Info.MediaStreamType == MediaStreamType.VideoPreview);
            var minFormat = mediaFrameSourceVideoPreview.SupportedFormats.OrderBy(x => x.VideoFormat.Width * x.VideoFormat.Height).FirstOrDefault();
            await mediaFrameSourceVideoPreview.SetFormatAsync(minFormat);
            frameReader = await mediaCapture.CreateFrameReaderAsync(mediaFrameSourceVideoPreview, minFormat.Subtype);
            frameReader.FrameArrived += FrameReader_FrameArrived;
        }
        catch (Exception e)
        {
            System.Diagnostics.Debug.WriteLine("Exception: " + e.Message);
            return;
        }

        await frameReader.StartAsync();
    }

And the image processing runs in an IAsyncAction, where recording is set to true in another function before calling this one and it is set to false from the main thread with a voice command.

public void Record()
    {
        IAsyncAction asyncAction = Windows.System.Threading.ThreadPool.RunAsync(
            async (workItem) =>
            {
                while (true)
                {
                    if (!recording)
                    {
                        await CleanupCameraAsync();
                        return;
                    }
                    else
                    {
                        var frame = frameReader.TryAcquireLatestFrame();
                        if (frame == null) continue;

                        var softwareBitmap = SoftwareBitmap.Convert(frame.VideoMediaFrame.SoftwareBitmap, BitmapPixelFormat.Rgba8, BitmapAlphaMode.Ignore);
                        Process(softwareBitmap);
                        frame.Dispose();
                        await Task.Delay((int)(1.0f / fps) * 1000);
                    }
                }
            });
    }

CleaupCameraAsync() looks like this:

    private async Task CleanupCameraAsync()
    {
        if (frameReader != null)
        {
            await frameReader.StopAsync();
            frameReader.Dispose();
        }
        if (mediaCapture != null)
        {
            mediaCapture.Dispose();
            mediaCapture = null;
        }
    }

For some reason this isn't working and I'm not sure why. I read somewhere that it might have to do with frames not being disposed, but I am disposing the frames I get and everything works after running it once.

So any ideas what the problem might be?

Comments

  • Hi jnthn,
    I am facing the exact same issue (except that I'm developping for Hololens). Did you finally find a way to fix this?

    Thanks

Sign In or Register to comment.