Identify the text of the specified window

Identify the text of the specified window

1. Simple requirements

Read the text of a specified window through image and text recognition. Get the window handle, save the screenshot as bitmap, and call the image and text recognition library.

The test result is that the recognition of Chinese is not particularly good.

It should be noted that tessdata must be downloaded from the specified directory page.

2. Reference package

a. Reference tesseract4.1 b. Emgu.CV component

3. Upload the code

using System;
 using System.Drawing;
 using System.Drawing.Imaging;
 using System.Runtime.InteropServices;
 using Emgu.CV;
 using Emgu.CV.OCR;
 using Emgu.CV.Structure;
 using Tesseract;
 using System.Windows.Forms;
 using Pix = Tesseract.Pix;
 using PageSegMode = Tesseract.PageSegMode;

 public class Program
 {
     [DllImport("user32.dll", SetLastError = true)]
     public static extern IntPtr FindWindow(string lpClassName, string lpWindowName);

     [DllImport("user32.dll")]
     public static extern IntPtr GetWindowDC(IntPtr hWnd);

     [DllImport("user32.dll")]
     public static extern bool ReleaseDC(IntPtr hWnd, IntPtr hDC);

     [DllImport("gdi32.dll")]
     public static extern uint GetPixel(IntPtr hdc, int nXPos, int nYPos);

     [DllImport("user32.dll", SetLastError = true)]
     public static extern bool GetWindowRect(IntPtr hwnd, out RECT lpRect);


     [DllImport("user32.dll")]
     public static extern bool SetForegroundWindow(IntPtr hWnd);

     [StructLayout(LayoutKind.Sequential)]
     public struct RECT
     {
         public int Left;
         public int Top;
         public int Right;
         public int Bottom;
     }


     [DllImport("user32.dll", CharSet = CharSet.Auto, ExactSpelling = true, CallingConvention = CallingConvention.Winapi)]
     public static extern void keybd_event(byte bVk, byte bScan, uint dwFlags, int dwExtraInfo);

     private const int KEYEVENTF_EXTENDEDKEY = 0x0001;
     private const int KEYEVENTF_KEYUP = 0x0002;

     private const int VK_MENU = 0x12; // Alt
     private const int VK_S = 0x53; // S key


     public static Bitmap CaptureWindow(IntPtr hWnd)
     {
         RECT rect;
         GetWindowRect(hWnd, out rect);

         Bitmap bmp = new Bitmap(rect.Right - rect.Left, rect.Bottom - rect.Top, PixelFormat.Format32bppArgb);
         Graphics g = Graphics.FromImage(bmp);
         g.CopyFromScreen(rect.Left, rect.Top, 0, 0, bmp.Size, CopyPixelOperation.SourceCopy);

         return bmp;
     }

     public static void Main()
     {
         string windowTitle = "*Untitled - Notepad";

         //*Untitled - Notepad
         IntPtr hWnd = FindWindow(null, windowTitle);

         if (hWnd == IntPtr.Zero)
         {
             Console.WriteLine($"Could not find window with title '{windowTitle}'");
             return;
         }

         SetForegroundWindow(hWnd);
         //SendKeys.SendWait("%(s)"); //Send Alt+S event


         // Press Alt + S
         //keybd_event(VK_MENU, 0x45, KEYEVENTF_EXTENDEDKEY | 0, 0);
         //keybd_event(VK_S, 0x45, KEYEVENTF_EXTENDEDKEY | 0, 0);

         //// Release Alt + S
         //keybd_event(VK_S, 0x45, KEYEVENTF_EXTENDEDKEY | KEYEVENTF_KEYUP, 0);
         //keybd_event(VK_MENU, 0x45, KEYEVENTF_EXTENDEDKEY | KEYEVENTF_KEYUP, 0);




         //Image and text recognition
         Bitmap bmp = CaptureWindow(hWnd);

         //tessdata/tessdata-4.1.0
         TesseractEngine engine = new TesseractEngine(@"./tessdata", "eng", EngineMode.TesseractAndLstm);

         using (Bitmap bmpImage = new Bitmap(bmp))
         {
             byte[] imageData = ImageToByte(bmpImage);
             using (var image = Pix.LoadFromMemory(imageData))
             {
                 using (var page = engine.Process(image))
                 {
                     var text = page.GetText();
                     Console.WriteLine("Text recognized from image: {0}", text);
                 }
             }
         }

     }

     private static byte[] ImageToByte(Bitmap img)
     {
         ImageConverter converter = new ImageConverter();
         return (byte[])converter.ConvertTo(img, typeof(byte[]));
     }
 }

The above code is developed using net6.0. There is a problem with console programs simulating windows events, especially keyboard shortcuts. Not processed yet.

That’s it, just for fun. It’s too cold, so I don’t want to write too much. Just making up the word count.

This article comes from Blog Park, author: Zhidao Zhonghe, please indicate the original link when reprinting: https://www.cnblogs.com/voidobject/p/17912444.html