Figure 1 Petri dishes with mixed growth of multiple strains

This multi-species hybrid, especially in the presence of mold and actinomycetes, makes it difficult to count automated colonies. This is because the current automated colony count is based on image segmentation technology, and image segmentation is mainly based on the difference between the target and the background, such as grayscale differences, color differences, or edge contours. Mutation and so on.
Figure 2 shows the segmentation effect of a multi-species mixed dish containing mold using conventional image segmentation techniques. Among them, Fig. 2-a is the original drawing of the culture dish, Fig. 2-b is the dividing result by the conventional image segmentation technique, and Fig. 2-c is the color filling of the divided particles to facilitate the observation. It is not difficult to see: (1) Because the surface of the mold is granulated, the image is divided into small pieces. (2) The color change of the surface of the mold, the edge of which is milky white cloud, which is converted into gray scale and is close to other white colonies; and the middle becomes dark green, which is converted into gray scale and is close to the gray level of the background. This leads to the traditional gray-scale segmentation technology that treats the edge portion as a colony and the center portion as a background. (3) The cloudiness or ambiguity of the edge of the mold leads to no obvious edge gradient change, so that the edge gradient segmentation technique cannot be used.

Figure 2 Processing results of traditional image segmentation techniques for mixed cases of multiple strains

The image segmentation technology based on the horizontal set active contour model is currently the most advanced image processing technology in the world. This technology combines the level set and the active contour model. In the process of minimizing the energy functional, the active contour is continuously approached to the target to achieve the segmentation of the target. This feature is especially suitable for the detection of mold. In fact, after several years of research, Hangzhou Xunwei Technology Co., Ltd. has successfully applied the level set activity contour model to solve the detection of mold actinomycetes, and achieved ideal results (see our website for details). "Colony Count _ Innovative Technology (1~3)").
But when the mold actinomycetes are mixed with other strains, the problem becomes more complicated. At this time, the horizontal set active contour model is often effective for one colony and not good for the other. After adjusting the model parameters or constraints, it is effective against another colony, and the effect on the colony is not good. In fact, the above-mentioned culture dish shows a complex problem of multiple species, multiple targets, and multiple features. With a single level set active contour model, the desired effect has not been achieved. For the multi-objective and multi-feature segmentation problems, there are two main methods in the world: (1) multi-phase CV model and (2) multi-level level set framework model.
1. Multi-phase CV model A multi-phase level set segmentation algorithm proposed by Chan and Vese, namely multi-phase CV model ^[1] . After determining the number of target categories, the algorithm uses m level set functions to divide each other constrainedly, and can represent 2 ^m non-overlapping regions with m level set functions, thereby avoiding vacuum inside the regions or overlapping between regions. .
The energy functional of the multiphase CV model is:

Where u ₀ is the original image data, c _I is the average gray value of the I region, x _I is the level set represents the I-th region, [Phi] _i is the i th level set function. Taking the four-phase image as an example, two horizontal set functions Ф ₁ and Ф _{2 are introduced} . Corresponding to this, two initial contours must be set. The image u ₀ is divided into four non-overlapping regions:

Then the energy functional evolves into:

Where Ф = ( φ ₁ , φ ₂ ), c = ( c ₁₁ , c ₁₀ , c ₀₁ , c ₀₀ ) represent the average of the image u ₀ in the Ω ₁₁ , Ω ₁₂ , Ω ₂₁ and Ω ₂₂ regions, respectively. Grayscale, as shown in Figure 3.

Figure 3 two-phase level set schematic

The two contour short lines in the model evolve simultaneously, and the corresponding level set functions are also iteratively operated. The gradient descent flow formula can be obtained by solving the Euler-Lagrange equation corresponding to the energy functional of the upper formula:

The gray level averages c ₁₁ , c ₁₀ , c ₀₁ and c ₀₀ can be updated in each iteration as follows:

2 , multi-level level set framework ^[2]
Unlike multi-phase CV models, which use multiple level set functions, the multi-level level set framework is based on the concept of a layer and uses only a level set function based on the CV model to evolve in a hierarchical way. Multi-target segmentation is a serial multi-phase segmentation algorithm.
Define the original layer as

among them, L represents the class of the i-th target region. l _m l denotes the number of regions of the target class, and sets L represents a collection of object classes, For the background area.
Firstly, the original layer L _{0 is} segmented by a single level set, the first type target area is extracted, and then the first type target area is masked by the gray level average of the area other than the first type target area, and a new background area class is generated. , the layer status changes to:

The conversion process of the original layer L ₀ to the new layer L ₁ is as shown in FIG. 4 . The original layer L ₀ contains two types of targets with When the first type of target area When segmented and replaced with the gray-scale average of the second-type target region and the background region, the layer L ₁ is formed such that only the second-type target region remains in L ₁ , and the remaining region is regarded as the background. When there are multiple types of target areas in the image to be segmented, this idea can be used to separate a certain type of target area one by one until there is no target, and the mathematics is summarized as follows:

Figure 4 Conversion of the original image layer L ₀ to the new image layer L ₁

Finally get the layer L _n without any target, that is, the background layer L _background :

Level Set frame is still dependent on the model CV, the traditional CV model is different, a level set function is only applied to a layer, a new layer and generated by iterative evolution, the level set function, the first layer l The energy function on ( l=0,1,2,...,n-1 ) is:

Same as the parameters in the CV model, C _l represents the evolution curve on the L _l layer. Introducing the time variable t≥0, we can get the level set evolution equation on L _l :

3. Experiment with the multiphase CV model introduced above, which is a parallel model. It can avoid the overlapping coverage and the leakage coverage problem of the active area of ​​the level set function without constraints. However, multiple levelsets may eventually converge to a region, and the model assumes the number of regions in the image. The actual colony image is often complicated, and it is difficult to artificially delineate the area, so this method is more troublesome to use. The multi-level level set framework belongs to the horizontal set serial segmentation method. The contour curve obtained by each segmentation is located inside the contour curve obtained by the previous segmentation, which can ensure that the convergence targets of the respective level sets are different, and have a affiliation relationship, and the use is relatively simple. This experiment mainly uses a multi-level level set framework model.
Figure 5 shows the detection effect of a multi-level horizontal set frame model for multi-species, multi-featured colonies. Figure 5-a is the original image, Figure 5-b is the segmentation result using the multi-level horizontal set frame model, and Figure 5-c is used to color fill the inside of the segmentation profile for observation. In order to solve the problem of some colonies sticking to each other, the watershed adhesion segmentation was added in the experiment.
It is not difficult to see from Fig. 5 that since the horizontal set active contour model segmentation technique is in the process of minimizing the energy functional, the active contour is continuously approached to the target to achieve the segmentation of the target, so there is no microparticle on the surface of the mold. Segmentation phenomenon. In addition, the energy function is based on the gray-scale variance inside and outside the contour. Therefore, when the active contour is contracted, it is not affected by the different colors of various colonies, resulting in mis-segmentation. Most colonies have been accurately segmented except in a few places.

Figure 5 is based on the segmentation effect of the multi-level level set framework model

4, level set method for image segmentation Prospect active contour model, with strong anti-noise current, numerical solution stability, the divided continuous smooth boundary, the advantages can handle complex topologies other case, the image segmentation become international forefront One of the technologies. The multi-phase CV model and the multi-level level set framework model are based on the level set and have a good effect in solving the multi-objective and multi-feature segmentation problems. It provides a feasible method for colony detection in mixed cases.
However, each iterative calculation of the multi-phase CV model and the multi-level level set framework model needs to be initialized, resulting in a large amount of calculation and time-consuming segmentation. As the complexity of the problem increases, so does its computational complexity. In addition, the initial contours of these two methods are still contingent, and the size and shape of the pre-selected regions will affect the segmentation results, which needs further study.
5 , references
[1] Zheng Gang, Li Yuanlu, Wang Huinan. A new multi-phase level set framework for 3D medical image segmentation based on TPBG [C]. 27th Annual International Conference of IEEE EMBC05, Shanghai, 2005.
[2] Wang Xiaofeng. Level set method and its application in image segmentation[J]. University of Science and Technology of China, 2011(02).
Hangzhou Xunda Technology Co., Ltd. R & D Department