Ytukyg

I have a web site that allows users to upload images of cars and I would like to put a privacy filter in place to detect registration plates on the vehicle and blur them.

The blurring is not a problem but is there a library or component (open source preferred) that will help with finding a licence within a photo?

Caveats;

1. I know nothing is perfect and image recognition of this type will provide false positive and negatives.


2. I appreciate that we could ask the user to select the area to blur and we will do this as well, but the question is specifically about finding that data programmatically; so answers such as 'get a person to check every image' is not helpful.


3. This software method is called 'Automatic Number Plate Recognition' in the UK but I cannot see any implementations of it as libraries.


4. Any language is great although .Net is preferred.

I coded a C# version based on JAVA ANPR, but I changed the awt library functions with OpenCV.
You can check it at http://anprmx.codeplex.com

EDIT: I wrote a Python script for this.

As your objective is blurring (for privacy protection), you basically need a high recall detector as a first step. Here's how to go about doing this. The included code hints use OpenCV with Python.

1. Convert to Grayscale.


2. 
   

   Apply Gaussian Blur.

   
   
   

   img = cv2.imread('input.jpg',1)
   
   img_gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
   
   img_gray = cv2.GaussianBlur(img_gray, (5,5), 0)  
   
   

   
   

Let the input image be the following.

3. Apply Sobel Filter to detect vertical edges.


4. 
   

   Threshold the resultant image using strict threshold or OTSU's binarization.

   
   
   

   cv2.Sobel(image, -1, 1, 0)
   
   cv2.threshold() 
   
   

   
   


5. 
   

   Apply a Morphological Closing operation using suitable structuring element. (I used 16x4 as structuring element)

   
   
   

   se = cv2.getStructuringElement(cv2.MORPH_RECT,(16,4))
   
   cv2.morphologyEx(image, cv2.MORPH_CLOSE, se)  
   
   

   
   

Resultant Image after Step 5.

6. 
   

   Find external contours of this image.

   
   
   

   cv2.findContours(image, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE) 
   
   

   
   



7. For each contour, find the minAreaRect() bounding it.



8. Select rectangles based on aspect ratio, minimum and maximum area, and angle with the horizontal. (I used 2.2 <= Aspect Ratio <= 8, 500 <= Area <=15000, and angle <= 45 degrees)

All minAreaRect()s are shown in orange and the one which satisfies our criteria is in green.

9. There may be false positives after this step, to filter it, use edge density. Edge Density is defined as the number of white pixels/total number of pixels in a rectangle. Set a threshold for edge density. (I used 0.5)

10. Blur the detected regions.

You can apply other filters you deem suitable to increase recall and precision. The detection can also be trained using HOG+SVM to increase precision.

There is a new, open source library on GitHub that does ANPR for US and European plates. It looks pretty accurate and it should do exactly what you need (recognize the plate regions). Here is the GitHub project:
https://github.com/openalpr/openalpr

I came across this one that is written in java javaANPR, I am looking for a c# library as well.

I would like a system where I can point a video camera at some sailing boats, all of which have large, identifiable numbers on them, and have it identify the boats and send a tweet when they sail past a video camera.

I have done some googling about this a couple of months ago. There are quite a few papers about this topic, but I never found any concrete open-source implementation. There are a lot of commercial implementations though, but none of them with a price quote, so they're probably pretty expensive.