Area extraction on scanned paper is performed with C # and OpenCV (OpenCVSharp).

As shown in FIG.
Form a quadrangle from two points and extract the area inside.
Before forming a rectangle, rotate it based on the ruled lines (red line in the figure) on the paper.

There is some skew when scanning, so I rotate it,
It may not be leveled cleanly and may be slightly slanted.

In that case, a quadrangle that expands and contracts vertically and horizontally is detected as shown in Fig. 2.
* Fig. 2 is extremely tilted for explanation. Actually, the inclination is less.

Detects inside the rectangle compared to the original rectangle,
Since a slight deviation remains, the detection position is additionally adjusted up, down, left and right depending on the degree of vertical and horizontal deviation (difference in length).

The problem i am having

The current method is to adjust the additional detection position up and down, so
During development and testing, we cannot deny the feeling of responding when a gap occurs.
Also, as a response, it will not be behind.

Is there a way to deal with similar or similar problems?

I'm sorry that the question was fluffy because it was a conceptual problem.

Supplementary information

Since this is my first time using it, I am not accustomed to asking questions. .. ..
There may be some points for improvement/confirmation.

I think that the source code can be presented as needed.

  • Answer # 1

    * Fig. 2 is extremely tilted for explanation. Actually, the slope is less


    There are many ruled lines on the paper

    So, for example, if you want to find the amount of rotation for correction by the following simple processing, why not see if it improves from the current situation?

    Find the edge strength and gradient direction of each pixel per Sobel filter

    For a group of pixels whose edge strength is above a certain level and whose gradient direction is somewhat close to the ideal angle of the "ruled line" used to estimate the amount of rotation (probably one or both of {vertical lines, horizontal lines}, kana). , Make a histogram (frequency distribution) in the gradient direction.

    As a result of 2., the direction corresponding to the edge of the "ruled line" will be significantly larger, so the amount of rotation is determined from that.

    The difference from the current method using the Hough transform is
    The "tilt" is obtained from each edge point, and the amount of rotation is directly determined by the majority vote.

Related articles