滤泡淋巴瘤分级

2021年1月21日发(作者：荣智健)
Histopathological Image Analysis Using Model- Based
Intermediate Representations and Color Texture:
Follicular Lymphoma Grading
滤泡淋巴瘤分级

——使用基于中间表示法模型的和颜色纹理信息分析组织病理图像


Olcay Sertel ·
Jun Kong ·
Umit V. Catalyurek ·

Gerard Lozanski ·
Joel H. Saltz ·
Metin N. Gurcan

Abstract

Follicular
lymphoma
(FL)
is
a
cancer
of
lymph
system
and
it
is
the
second
most

common
lymphoid
malignancy
in
the
western
world.
Currently,
the
risk
stratification
of
FL
relies
on
histological
grading
method,
where
pathologists
evaluate
hematoxilin
and
eosin
(H&E)
stained
tissue sections under a microscope as recommended by the World Health Organization.
This manual method requires
intensive labor in nature. Due to the
sampling bias, it also suffers
from
inter-
and
intra-reader
variability
and
poor
reproducibility.
We
are
developing
a
computer-assisted
system
to
provide
quantitative
assessment
of
FL
images
for
more
consistent
evaluation of FL. In this study, we proposed a statistical framework to classify FL images based on
their
histological
grades.
We
introduced
model-based
intermediate
representation
(MBIR)
of
cytological
components
that
enables
higher
level
semantic
description
of
tissue
characteristics.
Moreover, we introduced a
novel color-texture analysis approach that combines the MBIR with
low level texture features, which capture tissue characteristics at pixel level. Experimental results
on real follicular lymphoma images demonstrate that the combined feature space improved the
accuracy of the system significantly. The implemented system can identify the most aggressive FL
(grade III) with 98.9%
sensitivity and 98.7%
specificity and the
overall classification accuracy
of
the system is 85.5%.
摘要

滤泡淋巴瘤
(FL)
是一种淋巴系统癌症,
它是西方世界排名第二的恶性淋巴肿瘤。目前，
FL
的恶性分级依赖于组织病理 图像。目前，
FL
的恶性分级依赖于组织学分级方法，由世界卫
生组织建议病理学家在 显微镜下评估由苏木精
-
伊红染色法（
H
＆
E
）染色的组织 切片。
这种
人工方法需要很大的精力。
由于抽样误差，它也受到来自医生间和医生本身 的差异和不可重复
性的约束。
我们正在开发的电脑辅助系统对
FL
图像进行定 量评估以便提供更一致的
FL
评价。
在这项研究中，我们提出了一个统计框架，根据其 组织学分级分类
FL
图像。我们推出了基
于模型的中间表示
（
MBI R
）
的细胞学部件，
实现了更高层次的语义描述的组织特性。
此外，
我们引入了一个捕捉像素级的组织特征的新的彩色纹理分析方法，
它结合了
MBIR
和 低级别
的纹理特征。
在真正的滤泡性淋巴瘤图像上的实验结果表明，
该组合的特征空间 上显着改善
了系统的精度。
实现的系统可以识别的恶性的
FL
（
II I
级）
，
灵敏度为
98.9
％和特异度为
98.7
％，
系统的总分类精度为
85.5
％。

Keywords


Histopathological
image
analysis
·
Model-based
intermediate
representation
·
Color
texture
analysis ·
Follicular lymphoma
关键词：病理组织学图像分析；基于模型的中间表示；彩色纹理分析；滤泡性淋巴瘤。

1 Introduction
1
简介

Follicular
lymphoma
(FL)
is
a
cancer
of
lymph
system
and
it
is
the
second
most
common
lymphoid
malignancy in the western world. FL is a mature B lymphocyte malignancy
of follicular
center
cell
origin.
Diagnosis
of
FL
is
based
on
specific
morphologic,
immunophenotypic

and
cytogenetic findings in lymph node/tissue biopsy specimens. 滤泡性淋巴瘤（
FL
）是一种淋巴系统的癌症，这是西方世界排名第二的恶性淋巴肿瘤。< br>佛罗里达州是一个成熟的
B
淋巴细胞恶性的滤泡中心细胞来源。
诊断
F L
是基于特定淋巴结
/
组织活检标本形态学，免疫表型和细胞遗传学的研究结果。



FL has a highly variable clinical course ranging from an indolent to a highly aggressive disease.
Patients withindolent disease often live for many
decades and may never require therapy, while
the
patients
with
aggressive
FL
have
short
survival
if
not
treated
early
with
aggressive
chemotherapy. It is important to note that in contrast to aggressive FL, the indolent
FL patients
do not benefit from early chemotherapy and that treatment should be avoided in these patients
to
prevent
serious
side
effects.
This
variable
clinical
presentation
requires
an
accurate
risk
stratification of FL samples as a guidance for oncologist in making decisions on timing and type of

a
result,
it
can
contribute
to
reducing
the
likelihood
of
making
under
and
over
treatments.
FL
是一种具有高度可变从一个懒惰的 一个极具攻击性的临床过程的疾病。良性
FL
疾病
经常潜伏几十年或者可能从不需要治 疗
,
而恶性
FL
患者不及早治疗与化疗，则会很快夺去性
命。所以， 正确区别恶性
FL
和良性
FL
就非常关键，早期化疗对良性
FL病人没有好处，所
以良性
FL
病人应避免这些治疗以防止产生哪些严重的副作用。 这个变化的临床过程需要一
个准确的
FL
恶性分级以便为肿瘤医生们提供一个依据去决 对治疗的时间、类型和方法。所
以，它有助于减少做出错误疗法的可能性。

Currently, the most commonly used
FL risk
stratification method is histological grading (HG)
system adopted by the World Health Organization [1]. The HG method is based on average count
of large malignant cells called centroblasts (CB)
per standard microscopic high power field (HPF)
defined
as
0.159
mm2.
Follicular
lymphoma
cases
are
stratified
into
three
histological
grades:
Grade
I
(0
–
5
CB/HPF),
grade
II
(6
–
15
CB/HPF)
and
grade
III(>15CB/HPF).
Grades
I
and
II
are
considered low risk category while grade III is considered
high risk category. In this method the
average
centroblast
count
per
HPF
is
based
on
CB
count

in
ten
random
HPFs
representing
malignant

CB
count
is
performed
manually
by
the
pathologistusing
an
optical
microscope
and
hematoxilin
and
eosin
(H&E)
stained
tissue
section(s).
Since
this
is
a
highly
subjective method, the results
show well
documented inter- and intra- observer variability [2, 3]
for the various grades of FL even among the experts [4]. Moreover,since this method, for practical
reasons, uses only ten high power fields for
CB count, the results for specimens with
high tumor
heterogeneity
are
vulnerable
to
sampling
bias.
This
poor
reliability
and
reproducibility
of
FL
histological
grading
may
lead
to
inappropriate
clinical
decisions
on
timing
and
type
of
therapy
and
result
in
under
or
over
treatment
for
the
individual
FL

patient
with
many
serious
clinical
consequences.
Using
computerized
image
analysis,
it
is
possible
to
extract
more
objective
and
accurate prognostic clues, which may not be easily observed by qualitative analysis performed by
pathologists. Besides, instead of evaluating
only representative regions, a computerized system
can process the whole- slide and prevent the sampling bias.

目前，最常用的
FL
危险分层的方法是组织学分级系统
（
HG
）
通过世界卫生组织
[1]
。

HG
方法是基于大的恶性细胞称为平 均计数每标准微观高倍视野（
HPF
）中心母细胞（
CB
）定义
为< br>0.159
平方毫米。
滤泡性淋巴瘤病例分为三种组织学分级：
Ⅰ级
（
0-5 CB / HPF
）
，
Ⅱ级
（
6-15
CB / HPF
）和Ⅲ级（
>15CB/HPF
）
。
< br>I
和
II
级被认为是低风险类别，而Ⅲ级被认为是高风险类
别。在该方 法中，每
HPF
的的平均
centroblast
计数是根据代表恶性卵泡< br>CB
计数在
10
随机
住房公积金
CB
计数是手动执行 的由光学显微镜和苏木精
-
伊红染色法
（
H
＆
E
）
染色的组织切
片（）
。由于这是一个非常主观的方法，结果表明有据可查间和观察者内 的变异
[2
，
3]
，甚
至专家之间的各种档次的
FL[4]
。此外，由于这种方法中，由于实际原因，只使用了
10
个高
倍视野
CB
数为肿瘤的异质性高的标本，结果很容易受到抽样误差。这个可怜的可靠性和可
重复性的< br>FL
组织学分级可能会导致不适当的临床治疗，并导致了许多严重的临床后果的个
人FL
患者治疗过度或不足，时间和类型的决定。利用计算机图像分析技术，它可以提取更
客 观，
更准确的预后线索，
这可能不是很容易观察到的病理学家进行定性分析。
此外，< br>一个
计算机化的系统，而不是只代表性的地区进行评估，可以处理整个幻灯片，防止取样偏差。< br>



As reported by Meijer et al., the roots of image analysis for a more objective and reproducible
prognosis
date
back
to
seventeenth
century
[5].
Being
amazingly
precise,
Leeuwenhoek
had
developed a system to measure the size of human erytrocytes using sand grain and hairs from his
head.
However,
the
real
acceleration
in
histopathological
image
analysis
is
due
to
the
recent
developments
in
whole-slide
scanners.
Whole
slide
scanners
allow
digitization
of
whole
microscope
slides
at
high
magnifications
up
to
40×
and
provide
very
high
resolution
images.
Recently, several image analysis approaches have been proposed for diverse types of cancer such
as prostate [6], breast [7], brain [8] and neuroblastoma [9, 10]. These methods commonly exploit
texture,
color
or
morphological
properties
of
the
tissue
and
propose
quantitative
methods
to
differentiate different histological grades.
据报道
Meijer
等人，一个比较客 观的和可再生的预后追溯到十七世纪的根源，图像分
析
[5]
。令人惊讶的精确，列文 虎克已经开发了一个系统的大小来衡量的人
erytrocytes
的沙
粒和头发从他 的头上。
然而，
真正的加速是由于最近的事态发展在整个幻灯片扫描仪在病理
组织学图 像分析。整个幻灯片扫描仪，使整个显微镜载玻片数字化高倍率高达
40
倍，提供
了非 常高的分辨率的图像。
最近，
一些图像分析方法已经被提出了不同类型的癌症，
如前列
腺癌
[6]
，乳腺癌
[7]
，脑
[8]
和神经母细 胞瘤
[9
，
10]
。这些方法通常利用组织的质地，颜色或
形态特征 ，并提出定量的方法来区分不同的组织学分级。

In this study, our goal is to develop a computer-aided prognosis (CAP) system that will assist
the pathologists in the grading of FL. The flowchart of the proposed system is given in Fig. 1.
在这项研究中，
我们的目标是开发了计算机辅助预测
（
CAP
）
在佛罗里达州的分级系统，
这将有助于病理学家。所提出的系统的流程图给出在图
1
。


We
propose
a
novel
approach
that
semantically
describes
histology
images
using
model
based
intermediate
representation
(MBIR)
and
incorporates
low
level
color
texture
analysis.
In
this
approach,
we
first
identify
basic
cytological
components
in
the
image
and
model
the
connected
components
of
such
regions
using
ellipses.
An
extensive
set
of
features
can
be
constructed
from
this
intermediate
representation
to
characterize
the
tissue.
Using
this
representation,
we
measure
the
relative
amount
and
spatial
distribution
of
these
cytological
components. We observe that the spatial distribution of these regions vary considerably between
different
histological
grades
and
using
MBIR
provides
a
convenient
way
to
quantify
our
observations. Although this approach provides reasonable results especially identifying the most
aggressive
grade
of
FL,
it
is
relatively
less
successful
in
classification
of
low
grades.
The
tissue
samples
of
these
grades
are
better
characterized
by
low
level
color
texture
features.
Since
graylevel
features
or
other
color
texture
features
could
not
adequately
model
the
microscopic
tissue image content,we developed a non-linear color quantification based color texture feature
constructed method. Due to the staining of the tissue samples, the resulting digitized FL images
have considerably limited dynamic ranges in the color spectrum. Taking this fact into account, we
propose
the
use
of
a
non-linear
color
quantization
using
self-organizing
maps
(SOM).
We
used
the
quantized
image
to
construct
the
co-occurrence
matrix
that
is
used
to
compute
low
level
color texture features [12]. By combining the statistical features constructed from the MBIR with
the
low
level
color
texture
features,
the
classification
performance
of
the
system
is
improved
significantly.
我们提出了一个新的方法，组织 学图像语义描述模型的中间表示（
MBIR
）
，并采用低级
别的色彩纹理分析 。
在这种方法中，
我们首先确定基本的细胞学组件中的图像和模型等地区，
使用椭圆形 的连接组件。广泛的功能集，可以由这个中间表示，该组织的特征。这表示，我
们测量这些细胞学成分的 相对含量和空间分布。
我们观察到，
这些地区的空间分布，
不同病
理分级之间 有很大的不同和使用
MBIR
提供了一个方便的方法来量化我们的观察。
虽然这种方法提供了合理的结果，特别是确定最积极的档次
FL
，低等级的分类是相对不太成功的。
这些成绩的组织样本更好的特点是低级别的色彩纹理特征。由于
graylevel
功 能或其他颜色
的纹理特征不能充分模拟的微观组织图像内容，
我们开发了一个非线性颜色量化为 基础的色
彩纹理特征构造方法。由于染色的组织样本，所得到的数字化的
FL
图像已相 当有限的动态
范围中的色彩频谱。考虑到这一点，我们提出了使用非直线的颜色量化，使用自组织映射< br>（
SOM
）
。
我们使用的量化的图像构造的共生矩阵的，
被用 来计算低电平颜色纹理特征
[12]
。
通过结合从
MBIR
与低级别 的色彩纹理特征构成的统计特性，系统的分类性能显着提高。




The rest
of the paper is
organized as n 2 describes the image dataset used
in
this study;feature construction and extraction methods for the grading of FL images as well as the
statistical
n
3
presents
the
experimental
y,
in
Section
4,
we
conclude our results and point out the future research directions.



下面论文的结构如下。第
2
章描述了本研究中使用的图像数据集
;
FL
图像的分级以及统
计分类
.
第
3
章的特征构 造和提取方法呈现实验结果。最后，在第
4
章，我们总结我们的研
究结果，并指出了今 后的研究方向。

2 Feature Construction and Classification
2
功能建设和分类

We formulated the problem of grading FL images as a statistical pattern recognition problem.
Using a number
of image samples evaluated in consensus
by five
pathologists, we constructed a
set
of
discriminative
features
and
performed
classification.
We
first
segmented
the
image
into
basic
cytological
components
based
on
the
color
information
using
an
unsupervised
clustering

approach. This is followed by feature construction
using MBIR and the color texture analysis. We
used
the
combination
of
the
principal
components
analysis
(PCA)
and
the
linear
discriminant
analysis (LDA) methods to reduce the dimensionality of the feature space. This step is followed by
a Bayesian classifier based on maximum a posteriori decision rule.
< br>我们把
FL
图像分级问题设定为一个统计模式识别问题。我们有很多样本，它们是由五< br>位病理学家一致评估的，
我们设定了一系列的区别特征进行分类。
首先，
我们利 用一个对颜
色信息无监督的聚类方法把图像分割为基本的细胞成分。其次，是使用
MBIR和颜色纹理分
析的特征提取。再次，我们使用主成分分析（
PCA
）和线性鉴别分 析（
LDA
）方法结合的方
法，以减少的特征空间的维数。最后，利用基于最大后验概 率决策规则的贝叶斯分类器。

In this section, we describe the
dataset and the methods we used for automated grading of
FL through image analysis.

在本节中，我们将描述数据集及通过图像分析自动分级
FL
的方法。

2.1 Image Dataset
2.1
图像数据集

The
input
FL
images
to
our
system
are
H&E-stained
tissue
slides
digitized
using
a
Scope
XT
digitizer
(Aperio,
San
Diego,
CA,
USA)
at
40×

slides
are
collected
from
the
Department
of
Pathology,
The
Ohio
State
University
in
accordance
with
an
IRB
(Institutional
Review
Board)
approved
protocol.
The
image
dataset
consists
of
17
whole
-slide
FL
cases.
A
consensus of
five hematopathologists evaluated the grades of these samples. Six of the samples
were identified as grade I, eight were identified as grade II and three were identified as grade III.

经
H
＆
E-
染色组织切片
FL
图像，
使用范围
XT
数字化仪
(Aperio, San Diego, CA, USA)40
倍放大
后数字化
.
组织切片来 自于美国俄亥俄州立大学病理部，经
IRB
（机构审查委员会）批准。图
像数据集包含
17
个完整
FL
病例。这些样本是由五位病理学家一致评估的。其中

6
个样品
被认定为
I
级，
8
被确定为Ⅱ级，被确定 为Ⅲ级。

The
whole-slide
images
used
in
this
study
are
representative
samples
and
cover
most
of
tissue variations among all three histological grades. In order to train the computerized system
and test its accuracy, we asked three experienced pathologists to extract ten follicle regions that
are equivalent of one HPF from all these whole-slide cases. In the rest of the manuscript, we will
refer
to
these
three
sets
of
images
provided
by
distinct
pathologists
as
Sets
1,
2
and
3,
each
set of images, there are 170 images extracted from 17 whole
-slide
images.
This constructed a dataset of 510 images with a total of 180 grade I, 240 grade II and 90 grade III
images
each
of
which
has
a
spatial
resolution
of
2,165
×
1,365
pixels
that
is
equivalent
to
one
microscopic HPF. For our training and testing of the computerized image analysis system, we used
one set of images as training data and the remaining two sets
of images as testing data and we
repeat this three times each time using a different set of images.
这项研究中所使用的切片图像样品极具代表性，覆盖了大部分组织的所有三个组 织学分
级的变化。
为了训练计算机系统及测试其准确度，
我们邀请了三位经验丰富的病 理学家，
提
取
10
的卵泡地区，相当于一个
HPF
所有这些 整个滑动的情况下。在下面的文章中，我们会
参考这三组由不同的病理学家提供的分别
1
，
2
和
3
级的图像。
在每级图像中，
从
17完整切
片图像中提取
170
个图像。这样，就构建了一个总数为
510< br>幅图像的数据集，其中
180
幅
I
级图像，
240
幅Ⅱ级图像和
90
级幅
III
级图像，
每一幅图像其空间分辨率为< br>2,165
×
1,365
像素，等同于电子显微镜的高倍视野（
HPF
）
。
对于我们的计算机图像分析系统的训练和测
试中，
我们使用了一 组图像作为训练数据，
和剩余的两个图像作为测试数据集，
我们重复此
步骤三次，每次 使用一组不同的图像。

2.2 Image Segmentation
2.2
图像分割




There are five major cytological components in the FL tissue: nuclei, cytoplasm, extra-cellular
material,
red
blood
cells
(RBC)
and
background
regions.
Having

nuclei
and
cytoplasm
regions
dyed with hues of blue and purple, extra-cellular material dyed with hues of pink and RBCs dyed
with hues of red, H&E-stained FL images provides useful visual clues for addition
to
these
components,
there
are
also
background
regions
that
do
not
correspond
to
any
tissue


this
a
priori
knowledge
on
FL
images,
we
performed
the
segmentation
using
K-means clustering algorithm to identify these cytological components.

有五种主要的
FL
组织 ：细胞核，细胞质，细胞外的材料，红血细胞（
RBC
）和背景区域
的细胞学组件。有 色调的蓝色和紫色，细胞外的物质染色，染成红，
H
＆
E-
染色
FL
图像的色
彩
色
调
的
粉
红
色
和红
细
胞
染
色
的
细
胞
核
和细
胞
质
区
域
提
供
了
有
用的
视
觉
的
线
索

除了这些组件，有背景区域不对 应任何组织

的先验知识，
上
FL
影像，我们进行了分割，使用
K-means
聚类算法，以确定这些细胞学组件。




Instead of using the red
–
green
–
blue (RGB) color space, we converted the image to the La*b*
color
space,a
perceptually
uniform
color
space
developed
by
Commission
Internationale
d’Eclairage
(CIE).
Perceptually

uniform
means
that
the
same
amount
of
change
in
color
values
produces
the
same
amount
of
perceptualdifference
of
visual
importance.
This
property
of
the

La*b* color space allows us to use the Euclidean distance in comparing the colors [13]. Moreover,
the
La*b*
color
space
separates
the
luminance
and
the
chrominance
information
such
that
L
channels
corresponds
to
illumination
and
a*
and
b*
channels
correspond
to
color
opponent
dimensions.
Hence,
the
feature
vector
for
each
pixel
contains
intensity
and
color
information
separately.
而不是使用的红

-
绿

-
蓝（
RGB
）颜色空间中，我们的图像转换的
La* b *
色空间 ，听
觉均匀的颜色空间由国际照明委员会（
CIE
）的开发。感知均匀的装置，相同的 颜色值中的
变化量产生相同量的视觉重要
perceptualdifference
。
此属性的
La* b *
色空间的颜色进行比较
[13]
，使我们 能够使用的欧几里得距离。此外，在
La* b *
色空间分离的亮度和色度信息，使
得
L
个通道对应的照明和
*
和
b*
信道对应于颜色对手尺寸 。因此，对每一个像素的特征向量
分别包含强度和颜色信息。




We performed the clustering in the La*b* color space using the K-means algorithm. RBCs and
background regions show relatively uniform patterns; thus they are segmented
by thresholding
the
intensity
values
in
the
RGB
color
space.
The
segmentation
of
RBCs
is
relatively
straightforward
by
using
a
simple
threshold
operation
in
the
RGB
color
space;
therefore,
to
reduce the cost of the
iterations and accelerate the convergence of the algorithm, we identified
the
RBC
regions
prior
to
the
K-means
clustering
step
and
excluded
such
pixels.
A
pixelwise
thresholding is used to determine the RBC regions as follows:
我们进行的
La* b *
的色彩空间使用
K-means
算法的聚类。

RBC
和 背景区域的显示相对
均匀的图案，因此，它们是分割的阈值在
RGB
颜色空间中的强度 值。红细胞的分割是相对
简单的，通过使用一个简单的阈值在
RGB
颜色空间中的操作 ，因此，减少了迭代的成本和
加快算法的收敛性，我们确定的
RBC
区域的
K -means
聚类步骤之前，并排除这种像素。甲
pixelwise
阈值用于确定的
RBC
区域如下：


where i, j indicate the pixel coordinates; and r, g, and b are the corresponding red, green and blue
color channels of the image in the RGB color space. The threshold
τ

is experimentally chosen
as 0.45 after examining the histogram values of several representative regions.

Similarly, the background pixels were identified using a threshold as follows:

where ξ was determined to be 200.



Due to the possible variations in the color spectrum of whole
-slide images, we selected an
unsupervised segmentation method as opposed to a supervised segmentation method. Since we
are not limiting our system to training samples, using an unsupervised method,the segmentation
step will be more robust to intensity variations. K-means [14] is an unsupervised learning method.
It is an iterative process that divides samples into k partitions based on their attributes (e.g.,
La*b* color values) by minimizing a cost function:



where
k
is the number of components to be identified,
N
j
is the number of samples in cluster
j
,
x
ji
is the
i
th
feature data
(
i
∈

{
1
,
2
,
·

·

·

N
j
}
)
in the
j
th
cluster, and
μ
j
is the centroid of the cluster
j
.
Incorporating our prior knowledge of the cytological components in the FL slides, we set the number of
clusters to be three that correspond to nuclei, cytoplasm and extra-cellular
2
, shows
sample H&E-stained FL images associated with three different histological grades and their
segmentation results using the K-means algorithm as explained above.



2.3 Model Based Intermediate Representation(MBIR)

We observed that the spatial distribution of cytological components vary considerably with different
histological grades. This is mostly due to the fact that there are larger number of CBs in higher grade
FL are characterized by larger size, vesicular chromatin,accentuated nuclear membrane
and one to three prominent nuclei as opposed to centrocytes characterized by disposed chromatin,
inconspicuous nucleoli and scant cytoplasm. Therefore, images of higher histological grades have less
homogeneous organization of nuclei and cytoplasm components with respect to their relative spatial
distributions when compared to lower grade samples in which those regions are more compact and
evenly distributed. In an attempt to capture this information,we introduced MBIR to analyze the spatial
distribution of nuclei and cytoplasm components. Using the segmentation approach described in
Section
2.2
,we represented each nuclei and cytoplasm component using ellipses and computed a set of
features to differentiate histological grades of FL.

The use of high level representations has been studied for content based image retrieval
applications in remote sensing and medicine. A
ksoy et al. proposed a Bayesian framework to construct
a visual grammar to incorporate low-level features with high-level semantics [
15
]. They first
segmented the images using the spectral and texture information and decomposed the image scene into
prototype regions. Subsequently,they modeled the spatial relationships and interactions
between these
regions and performed the land cover classification. In a more recent study, Aksoy discussed several
representations such as smoothed polygons,convex hull, grid representation and minimum bounding
rectangle at different levels of complexity to simplify the computation of spatial relationships between
regions [
16
]. In [
17
] authors proposed the use of gridbased layouts for semantic content analysis for
histological images. Their system combines low-level image processing technology (i.e., segmentation)
with highlevel semantic analysis of medical image content similar to our abstraction using MBIR that
corresponds to different cytological components.

Different from the region representation methods
described in [
16
], we use ellipses to represent
different cytological regions. Ellipses are accurate representations
of 2D shapes and their computation
is relatively efficient. Besides we can easily construct features using their morphology such as the
lengths of major and minor axis, and area as well as using their relative spatial distribution such as the
number of neighbors and the minimum and average distances among , we explain how
we obtain the MBIR from the segmented images and construct morphological and topological features
to be used differentiating histological grades.

After the segmentation process, we constructed the MBIR of the cytological regions in the tissue for
further analysis. We only considered the nuclei and cytoplasm components since the variations in the
rest of the components are random and intuitively do not provide any discriminative information.
Segmentation step generates the binary representation of both the binary
representation, we first applied a morphological pre-processing step to remove noisy pixels. We first
removed insignificant regions by an area threshold (
≈
0.6
μ
m
2
). This is followed by separating the
touching components from each other for a more precise and accurate representation. For nuclei
components,we used the watershed transform, a mathematical morphology based on partitioning [
18
],
to separate touching nuclei. For the cytoplasm components, we incorporated our prior knowledge and
associated each cytoplasm component to the closest nuclei used the distance transform,
which computes, for every cytoplasm pixel, the distance to the nearest nuclei pixel. By regrouping the
connected components, we obtained the separated labels. It should be noted that there will also be some
cytoplasm regions that are not associated with any nuclei because the corresponding nuclei of the
particular cell associated with the cytoplasm component may be too scant, or the size of the nuclei
component can be very small so that it has been discarded in the pre-processing step. Such cytoplasm
components are still retained since they are important for the feature construction step.

Next, we fit an ellipse to represent each connectednuclei and cytoplasm region. We used the least
squares ellipse fitting approach introduced in [
19
]. The authors proposed a closed form solution, which
leads to a simple and robust solution with an efficient computational cost. Using this approach, we
represented each connected region with a few parameters, i.e., the
x
and
y
coordinates of the center
point, the length of major and minor axes and the angle between the x-axis and the major axis. Figure
3
shows the construction of MBIR for the nuclei component segmented in a sample grade II image.
Similarly, we obtained the MBIR for cytoplasm components in the FL images. Figure
4
shows the
MBIR for typical images associated with three different histological grades in each row.