Coffee bean sorting machine

Coffee growing and harvesting are common in Vietnam and Thailand, where sorting beans after harvest is an important step to improve product quality and value. Manual sorting is time-consuming, difficult to scale for large volumes, and can miss defective beans that look similar to good ones. At the time of this project, commercial sorting machines were also relatively expensive and difficult to maintain or modify. To address these limitations, an organization from Thailand collaborated with our industrial laboratory to develop a more affordable coffee bean sorting machine with in-house control of the core vision technology.

The project therefore involved more than developing a classification algorithm. It required the integration of several engineering components into a complete machine:

• A controlled imaging system with camera and illumination
• An embedded computing platform for real-time image processing
• Computer-vision algorithms for bean segmentation and quality assessment
• Communication between the vision computer and the STM32 controller
• A pneumatic ejector mechanism for removing defective beans

The project was developed by a two-member team, and I was responsible for the computer vision and embedded processing side of the system.

The table below shows the members' responsibilities

Camera selection

I considered three camera options for capturing beans in motion:

• Rolling-shutter camera: inexpensive, but fast-moving beans can appear distorted because image rows are captured at slightly different times.
• Global-shutter camera: captures the full frame at once, avoiding motion distortion and providing a good balance between performance and cost.
• Line-scan camera: well suited to continuous industrial inspection, but its cost was too high for the project budget.

Based on these trade-offs, I selected a global-shutter camera as the most practical option for the machine.

Computing platform

A Raspberry Pi 3 was first used to validate camera acquisition and the basic vision pipeline. For the later prototype, I moved to the NVIDIA Jetson Nano to improve processing performance and system stability, while also providing GPU capability for possible future AI-based extensions.

From conveyor imaging to vertical free-fall

The first prototype used a conveyor belt to move beans underneath the camera. Although this setup was useful for collecting data and testing the vision algorithm, it introduced two practical problems: beans could be thrown from the belt at higher speeds, and the camera could observe only one side of each bean.

I therefore redesigned the machine around a vertical free-fall mechanism. Beans fall through a controlled imaging area surrounded by lighting and are captured by the global-shutter camera before reaching the ejector. This design removed the conveyor-related problem and also made future two-sided imaging easier to implement.

The core vision problem was to determine whether each coffee bean should be kept or rejected based on its visual appearance. From discussions around practical sorting requirements, I treated bean quality using two main criteria: shape and color. Broken or malformed beans could be identified from their geometry and surface structure, while defects such as dark, black, or sour beans were mainly reflected in their color.

During development, the algorithm evolved through several stages. The first approach combined shape analysis with the CIE L*a*b* color space. Although it produced promising recognition results under controlled conditions, testing on the physical machine exposed sensitivity to non-uniform illumination and additional computational cost. I therefore redesigned the preprocessing stage and later replaced the Lab-based color analysis with a simpler RGB histogram approach that was more suitable for embedded deployment.

Stage 01 · Design choice Why Classical Computer Vision?

At the beginning of the research, I considered two main directions: a deep-learning-based classifier and a classical computer-vision pipeline.

I chose classical image processing mainly because it better matched the constraints of the project at that time. The collected dataset was relatively small, while the target system was intended to operate on affordable embedded hardware. Using a large neural network would therefore have introduced additional requirements for training data and computational resources.

At the same time, this problem had several characteristics that made classical vision a reasonable engineering choice. The camera position was fixed, the imaging background could be controlled, the illumination system was designed as part of the machine, and the defects of interest had visible characteristics related to color, contour, geometry, and surface texture. These properties could be explicitly extracted and evaluated without relying on a large learned model.

Stage 02 · Initial method Initial Approach — Shape + CIELAB

The first complete algorithm evaluated each bean using two independent criteria: shape and color.

A bean was accepted as good only when it satisfied both criteria. If either its geometry or its color indicated a defect, its position was recorded so that the bean could later be removed by the ejector.

Image segmentation and bean extraction

The first step was to separate the coffee beans from the background. After comparing the individual RGB channels, I converted the image to gray and Otsu automatic thresholding was then used to divide the image into foreground and background, while Gaussian filtering reduced noise in the segmented image.

From the binary image, I extracted the contours of individual beans. The boundaries were thinned to approximately one-pixel width, and additional processing was used to remove isolated pixels, broken contour fragments, and branching noise. Connected-component labeling was then used to isolate individual bean regions.

The resulting information served two purposes:

• contour coordinates were used to evaluate the bean geometry;
• the isolated RGB regions were used for color analysis.

Shape-based defect detection

The initial shape method was based on a simple observation: a normal coffee bean generally has a relatively smooth external contour, while a broken or malformed bean often contains abnormal concave regions.

The contour of each bean was divided into several sections. Within each section, I evaluated the position of the boundary relative to a line connecting the section endpoints. This made it possible to detect local concavities or irregularities that could indicate a broken or malformed bean. If one or more sections contained sufficiently strong geometric defects, the bean was classified as defective by shape.

Using CIELAB to Analyze Color

For color-based defects, I converted each segmented bean from RGB to CIE L*a*b*, which separates:

• L*: lightness
• a*: green-red
• b*: blue-yellow

My initial idea was to remove L* because it is related to illumination. However, after testing, I found that a* was less useful for separating good and defective beans. I therefore used the mean L*, b*, and hue-related h* values as a 3D feature vector.

The two classes formed partially separated clusters, so I used Pocket Learning to find a separating plane while handling the non-linearly separable samples.

Stage 03 · Deployment findings Limitation of the Initial Approach

This whole pipeline of this approach produced promising results, but practical deployment revealed two limitations:

• Sensitivity to non-uniform illumination

The laboratory images used during early algorithm development were relatively controlled. However, when I started processing images captured from the actual machine, the illumination was not perfectly uniform across the entire camera frame. And with the previous pipeline, I can't segment the image to foreground and background completely, even after trying several modifications to the hardware lighting setup.

• Extra processing cost from the CIELAB conversion

The color-analysis branch required RGB → XYZ → CIE L*a*b* conversion before the color features could be calculated. While this was acceptable during algorithm research, these additional calculations became less attractive once the algorithm had to run continuously on the embedded machine.

Stage 04 · Improvement 1 Improvement 1 — Robust Preprocessing

The next problem was:

How can I make the bean appearance more consistent under non-uniform illumination?

I first tested homomorphic filtering, which improved illumination uniformity but required costly transformations between the spatial and frequency domains. For embedded deployment, I therefore used a simpler background-subtraction approach. A reference image without coffee beans was captured to represent the background and lighting distribution:

Captured image = bean information + background/environmental component

By subtracting this reference from the current image, much of the static background and illumination variation could be removed.

Instead of converting the RGB image to grayscale using the conventional formula, I used the blue channel, which provided better contrast between the beans and the background in my experiments. I also applied an intensity-enhancement factor before subtraction to further improve this contrast.

The resulting image preserved the bean information while reducing the effects of non-uniform illumination and background noise.

After background correction, the illumination across the image became more uniform and the contrast between beans and background increased. This made Otsu segmentation considerably more reliable in the practical machine environment.

Stage 05 · Improvement 2 Improvement 2 — Faster and More Robust Classification

Once the segmentation became more reliable, I revisited the classification stage itself.

The final approach kept the basic idea that bean quality should not depend on a single feature. A bean could be defective because it was broken, malformed, internally damaged, abnormally dark, or affected by another color defect.

I therefore expanded the final algorithm to combine several complementary characteristics:

This was a more complete representation of bean quality than relying only on color or only on external contour.

Improving Shape Recognition

The original contour-concavity method was retained, but it was not sufficient to detect all abnormal beans. I therefore added roundness to capture overall shape irregularities and GLCM features to describe gray-level variations within the bean surface.

Together, these features allowed the algorithm to evaluate both external geometry and internal appearance.

Replacing CIELAB with Direct RGB Histogram Analysis for faster speed

The largest change to the color branch was to remove the CIE L*a*b* conversion entirely.

After background correction, I analyzed the red-channel intensity histogram directly.

During experiments, I observed that color-defective beans tended to contain a larger proportion of pixels in the lower-intensity region of the red-channel histogram than normal beans. In other words, instead of representing each bean in a transformed three-dimensional color space and running a learned separating function, I could characterize the relevant color difference directly from the pixel distribution.

For each isolated bean, I therefore:

1. calculated its red-channel histogram;
2. normalized the histogram into an intensity distribution;
3. measured the proportion of pixels within a selected low-intensity range;
4. compared this value with an experimentally determined threshold.

This produced a much simpler color decision.

This redesign eliminated the repeated RGB-to-CIELAB conversion and removed the Pocket Learning stage from the deployed color pipeline. More importantly, it worked directly on the background-corrected image, connecting the illumination improvement and the color classifier into one simpler processing chain.

---

Stage 06 · Final design Final Vision Pipeline

Final deployed vision pipeline Shared preprocessing followed by complementary shape/structure and color branches

During deployment on the Jetson Nano, I optimized the pipeline at three levels: algorithm complexity, memory usage, and execution speed. The objective was to reduce unnecessary computation and memory overhead while keeping the classification performance suitable for the sorting task.

Algorithm Optimization

Prefer lightweight algorithms with comparable effectiveness

When several methods produced similar results, I prioritized the one with lower computational cost.

For example, I tested homomorphic filtering to compensate for non-uniform illumination. Although it produced good image correction, it required transformations between the spatial and frequency domains, making it relatively expensive for continuous embedded processing. I therefore replaced it with background subtraction using an additive illumination model, which provided effective illumination correction with simpler computation.

The same principle was later applied when replacing the CIELAB-based color analysis with direct RGB histogram analysis.

Process only the region of interest

After segmentation and connected-component extraction, I avoided applying later classification operations to the entire 320 x 240 image.

Instead, each detected coffee bean was extracted into its own region of interest (ROI), and shape, texture, and color processing was performed only on that area. This reduced computation on background pixels that did not contribute to classification.

---

Memory Optimization

Use appropriate data types

I used data types according to the actual range and precision required by each operation. For example, the input images were 8-bit, so image buffers did not need larger integer representations when the processing stage only required the original pixel range.

For operations requiring numerical precision, larger data types could be used only where necessary rather than throughout the entire pipeline.

Reuse image buffers and avoid unnecessary copies

Image-processing pipelines can easily create multiple copies of the same frame at different stages. Where the previous result was no longer required, the same memory buffer could be reused or processed in place.

A new image buffer was therefore only necessary when both the previous image and the new result had to remain available for later processing.

Manage large image buffers on the heap

Large image matrices were allocated dynamically on the heap rather than on the limited stack. Smaller local variables could remain on the stack, while larger image buffers were managed separately to reduce the risk of stack overflow.

Dynamic memory also needed to be managed carefully so that buffers were reused where possible and released when they were no longer needed, avoiding unnecessary memory growth during continuous operation.

---

Speed Optimization

Combine operations when possible

When several calculations used the same pixel data, I combined them into the same for loop where possible instead of scanning the image repeatedly. This reduced memory accesses and loop overhead.

Replace expensive computations with simpler alternatives

For frequently executed mathematical operations, an exact high-cost function is not always necessary. Depending on the required precision, repeated calculations can sometimes be replaced by:

• precomputed lookup tables;
• simpler equivalent expressions; or
• approximation formulas.

Reduce repeated dynamic allocation

In my implementation, I measured slower access when working with dynamically allocated heap memory compared with stack memory. Because large image buffers could not practically be stored on the stack, I minimized the overhead by allocating the required heap memory once and reusing it instead of repeatedly allocating and releasing buffers during processing.

Store 2D images as contiguous 1D memory

Instead of dynamically allocating an image as multiple separate rows, I allocated the image as a single contiguous 1D block on the heap and accessed it using calculated 2D indices:

pixel(row, col) → buffer[row x width + col]

This still allowed the program to treat the data logically as a 2D image while keeping the physical memory contiguous. This reduced the overhead associated with dynamically allocated 2D arrays, which was important because pixel access occurred repeatedly throughout the vision pipeline.

Contiguous image-buffer layout A logical 2D pixel location is mapped into one heap-allocated 1D block

The final pipeline was evaluated on the NVIDIA Jetson Nano using 2,280 coffee beans, including both good and defective samples.

Accuracy and Classification Performance

The final algorithm was evaluated on good and defective coffee beans using both shape and color criteria. It achieved strong and balanced classification performance:

The high recall was especially important for the sorting task because the system should detect and remove as many defective beans as possible, while the F1-score showed that this was achieved without excessively sacrificing precision.

Processing Time

Replacing the CIELAB-based color analysis with the improved RGB approach also reduced the average processing time on the Jetson Nano:

The RGB-based approach reduced processing time by simplifying the color-recognition stage and avoiding the additional CIELAB conversion.

• The final processing time still left room for further optimization. For example, SIMD vectorization could accelerate convolution-based filters, which still consumed a significant portion of the processing time, while OpenMP could parallelize independent loops and processing tasks across CPU cores.
• The current imaging setup captured only one side of each bean; a dual-camera system could improve inspection coverage.
• Due to COVID-19, project funding was interrupted, which limited further development. Additional integration and refinement of the ejector system would therefore be required before the machine could become a fully production-ready system.