Fixing the cropping issue

I've spent some time now fixing up my cropping function. The reason I've been spending my time on this is because I think that it's really important to have quality training data. If my training data is crap, then nothing will work. I want to be able to cleanly cut out letters. So, I changed my cropping algorithm to the following:
cut the original image into 4 pieces, as follows:


Here is the left piece:


Right piece:


Top piece:


Bottom piece:


I run the hough transform on all four pieces. In each, I know where a line would be, if there exists one. I have some way of thresholding whether or not there is a line there (if the maximum number of votes is higher than a certain percentage of the length of the image). If so, I cut the appropriate piece out.

The performance isn't as great as expected. I think that I need to fix up my hough transform function because sometimes, when there appears to be a dark line, it won't find it (the number of votes is low).

Here are the overall results anyway:


The letters straight out of the training sheet look like this, with no postprocessing:


The next step for me is to fix my hough transform function, get the training data looking good, retrying nearest neighbor, and then moving on to opencv (boosting).