Computer screen stream with C#

Asked

Viewed 794 times

0

I’m having a problem with streaming the computer screen with C#, I’ve done the software where the server would create a TCP socket and the client would send a print of it in Bitmap form. But the problem of this method was that with a lot of client connected (5 or more) the software was reading a lot, besides, there was a lot of delay. I tried with UDP too with the same failure. Is there any method to perform this project in some way with C#, or only with lower level language like C and C++?
EDIT 01:
Following the idea of a colleague there Gypsy Morrison Mendez I decided to take a test where I take several screenshots with Directx because it is much faster than a method I used before. I modified a code that I found on the net to only take what changes in the second print, because what I found it united the differences and was a very strange image. The problem I have now is how I do using only what was changed in the image to update the server image. 'Cause if I had Picturebox display only what changed the screen it would be mostly white.

using SlimDX.Direct3D9;
using System;
using System.Collections.Generic; 
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Drawing.Imaging;
using System.Linq;
using System.Runtime.InteropServices;
using System.Text;
using System.Threading;
using System.Threading.Tasks;
using System.Windows.Forms;

namespace StreamScreen
{
public partial class Form1 : Form
{

    DxScreenCapture sc;
    Thread _t1;

    Bitmap older;

    public Form1(){
        InitializeComponent();

        sc = new DxScreenCapture();

        /*Surface s = sc.CaptureScreen();
        Bitmap _b1 = new Bitmap(SlimDX.Direct3D9.Surface.ToStream(s, SlimDX.Direct3D9.ImageFileFormat.Bmp));
        Thread.Sleep(1000);
        s = sc.CaptureScreen();
        Bitmap _b2 = new Bitmap(SlimDX.Direct3D9.Surface.ToStream(s, SlimDX.Direct3D9.ImageFileFormat.Bmp));

        Bitmap diff = PixelDiff(_b1, _b2);
        setImage(diff, _b1, _b2);*/
        Surface s = sc.CaptureScreen();
        older =  new Bitmap(SlimDX.Direct3D9.Surface.ToStream(s, SlimDX.Direct3D9.ImageFileFormat.Bmp));

        _t1 = new Thread(new ThreadStart(capture));
        _t1.Start();
    }



    private void setImage(Bitmap bitmap, Bitmap _b1, Bitmap _b2)
    { 
        if(pictureBox1.InvokeRequired){
            this.Invoke((MethodInvoker)delegate{
                pictureBox1.Image = bitmap;
                pictureBox2.Image = _b1;
                pictureBox3.Image = _b2;
            });
        }else{
            pictureBox1.Image = bitmap;
            pictureBox2.Image = _b1;
            pictureBox3.Image = _b2;
        }
    }

    private void capture() {
        while(true){
            Surface s = sc.CaptureScreen();
            Bitmap _b1 = new Bitmap(SlimDX.Direct3D9.Surface.ToStream(s, SlimDX.Direct3D9.ImageFileFormat.Bmp));
            Bitmap diff = PixelDiff(older, _b1);
            setImage(diff, _b1, older);
            older = _b1;
            //setImage(new Bitmap(SlimDX.Direct3D9.Surface.ToStream(s, SlimDX.Direct3D9.ImageFileFormat.Bmp)));
        }
    }



    unsafe Bitmap PixelDiff(Bitmap a, Bitmap b){
        Bitmap output = new Bitmap(a.Width, a.Height, PixelFormat.Format32bppArgb);
        Rectangle rect = new Rectangle(Point.Empty, a.Size);
        using (var aData = a.LockBitsDisposable(rect, ImageLockMode.ReadOnly, PixelFormat.Format32bppArgb))
        using (var bData = b.LockBitsDisposable(rect, ImageLockMode.ReadOnly, PixelFormat.Format32bppArgb))
        using (var outputData = output.LockBitsDisposable(rect, ImageLockMode.ReadWrite, PixelFormat.Format32bppArgb))
        {
            byte* aPtr = (byte*)aData.Scan0;
            byte* bPtr = (byte*)bData.Scan0;
            byte* outputPtr = (byte*)outputData.Scan0;
            int len = aData.Stride * aData.Height;
            for (int i = 0; i < len; i++){
                // For alpha use the average of both images (otherwise pixels with the same alpha won't be visible)
                /*if ((i + 1) % 4 == 0){
                    //*outputPtr = (byte)((*aPtr + *aPtr) / 2);
                    *outputPtr = (byte)~(*aPtr ^ *bPtr);
                }else{
                    *outputPtr = (byte)(*aPtr ^ *bPtr);
                }*/
                /* inicio da adaptacao tecnica */
                byte bb = (byte)~(*aPtr ^ *bPtr);
                if (bb != ((byte)~(*aPtr ^ *aPtr))){
                    *outputPtr = *bPtr;
                }else
                    *outputPtr = (byte)~(*aPtr ^ *bPtr);

                /*fim da adaptacao tecnica */

                outputPtr++;
                aPtr++;
                bPtr++;
            }
        }
        return output;
    }
}
}

static class Extensions
{
public static DisposableImageData LockBitsDisposable(this Bitmap bitmap, Rectangle rect, ImageLockMode flags, PixelFormat format)
{
    return new DisposableImageData(bitmap, rect, flags, format);
}

public class DisposableImageData : IDisposable
{
    private readonly Bitmap _bitmap;
    private readonly BitmapData _data;

    internal DisposableImageData(Bitmap bitmap, Rectangle rect, ImageLockMode flags, PixelFormat format)
    {
        //bitmap.("bitmap");
        _bitmap = bitmap;
        _data = bitmap.LockBits(rect, flags, format);
    }

    public void Dispose()
    {
        _bitmap.UnlockBits(_data);
    }

    public IntPtr Scan0
    {
        get { return _data.Scan0; }
    }

    public int Stride
    {
        get { return _data.Stride; }
    }

    public int Width
    {
        get { return _data.Width; }
    }

    public int Height
    {
        get { return _data.Height; }
    }

    public PixelFormat PixelFormat
    {
        get { return _data.PixelFormat; }
    }

    public int Reserved
    {
        get { return _data.Reserved; }
    }
}
}

EDIT 02:
I was able to join two images using the following command.

Bitmap diff = PixelDiff(_b1, _b2);
using (Graphics grfx = Graphics.FromImage(_b1)){
     grfx.DrawImage(diff, new System.Drawing.Rectangle(0, 0, _b1.Width, _b1.Height));
}

But I fit finding another problem, the image that brings only what is different stays with the white background, and when drawing over the white replaces what has not changed yet. Does anyone know how to turn this white into transparent?

  • The software that does this does not command bitmaps integers. They send only a few portions of the screen as it changes. That’s why the velocity fluidity.

  • Can you tell me how to detect and capture only the part that changes on the screen?

  • 1

    Usually applications like this keep a copy of the last image sent (the key frame) in an area called frame buffer. With each new image captured, it is compared to the old one, and only the difference between the two is compressed and sent. The target client unpacks and applies the differences, and the new image replaces the old one in the frame buffer.

  • And how would I do that? I’m kind of a layman with image manipulation.

No answers

Browser other questions tagged

You are not signed in. Login or sign up in order to post.