5. OpenCV Tutorial - Learn Classic Computer Vision & Face Detection (OPTIONAL)/25. Segmentation and Contours - Extract Defined Shapes In Your Image.srt

1
00:00:01,210 --> 00:00:07,300
Hi and welcome to Section 4 we are going to learn about a very important topic in computer vision and

2
00:00:07,300 --> 00:00:09,960
that topic is image segmentation.

3
00:00:10,000 --> 00:00:17,250
So what exactly is image segmentation So simply put image segmentation is a process by which we partition

4
00:00:17,280 --> 00:00:19,440
images into different regions.

5
00:00:19,440 --> 00:00:22,030
So here we have a few simple examples.

6
00:00:22,050 --> 00:00:27,870
So looking at this Volkswagen Beetle sketch here we can see using an image segmentation technique called

7
00:00:27,870 --> 00:00:28,710
Kontos.

8
00:00:28,980 --> 00:00:33,110
We've actually extracted all the body parts so all the major body parts of the car.

9
00:00:33,390 --> 00:00:38,430
Maybe we've gone a little overboard in some areas but overall it's quite nice and looking at the domino

10
00:00:38,430 --> 00:00:44,430
over here we can see every bonded region in the Domino sketch has been extracted by image segmentation

11
00:00:44,430 --> 00:00:45,070
technique.

12
00:00:46,600 --> 00:00:51,790
And moving on to some handwritten digits we can see every number here has been isolated by Imman segmentation

13
00:00:51,790 --> 00:00:54,790
technique as well as in this bottle cap collection.

14
00:00:54,820 --> 00:01:01,700
We've identified the circles quite nicely here so in this section recovering quite a few image segmentation

15
00:01:01,700 --> 00:01:08,030
techniques starting first with the very important contours Contos format very important segmentation

16
00:01:08,030 --> 00:01:13,460
technique in open C-v so it would while we spend some time doing it it leads us to a second mini project

17
00:01:13,490 --> 00:01:15,890
where we identify ships by contorts.

18
00:01:16,190 --> 00:01:20,060
We then look at line the cicle and block detection which are all pretty important as well.

19
00:01:20,300 --> 00:01:25,670
And that leads to a project where we're actually counting the circles and ellipses in an image.

20
00:01:25,790 --> 00:01:30,090
So let's introduce this image segmentation technique and that's Contos.

21
00:01:30,140 --> 00:01:36,530
So Contos are a continuous lanes or curves that bound or covered full bunchy of an object in an image.

22
00:01:36,560 --> 00:01:39,610
So this picture here illustrates this concept quite nicely.

23
00:01:39,620 --> 00:01:45,020
Here we have that piece of people and a black on the people being the object in our image and the green

24
00:01:45,020 --> 00:01:46,550
lines being all contours.

25
00:01:46,550 --> 00:01:50,320
So these green lines here cover this people are totally in this image.

26
00:01:50,570 --> 00:01:53,830
So this is a very good illustration of what a CONTO is.

27
00:01:53,940 --> 00:01:57,360
And in case you're wondering why we actually bother to find Contos.

28
00:01:57,560 --> 00:02:03,700
Well Contos I extensively use an object detection and shape analysis.

29
00:02:03,710 --> 00:02:06,860
OK so let's begin implementing Contos using open Zeevi.

30
00:02:06,950 --> 00:02:13,190
So that's LoDo and the surrounding contours I buy them the book file selection 4.1 and it brings up

31
00:02:13,190 --> 00:02:14,620
the fold rate here.

32
00:02:14,630 --> 00:02:18,520
So this is a code for Kontos projec seems slightly lengthy.

33
00:02:18,530 --> 00:02:23,330
However I'll step through each line so make sure you get a good grasp of contorts.

34
00:02:23,330 --> 00:02:29,010
So let's look at the image of tree black squares here on a white background which we will see shortly.

35
00:02:29,020 --> 00:02:32,860
Second part of program you may notice is that we grayscale the image here.

36
00:02:33,020 --> 00:02:40,250
Now scaling is a key step when you finding Contos mainly because it opens to find Contos function here.

37
00:02:40,710 --> 00:02:46,790
When the process grayscale images if you with a pass a color image here it would actually retune an

38
00:02:46,790 --> 00:02:49,310
arrow.

39
00:02:49,530 --> 00:02:53,250
So the next set of image processing we do is finding Kenni edges.

40
00:02:53,250 --> 00:02:56,350
Now finding kenning edges of the image isn't necessary.

41
00:02:56,430 --> 00:03:01,040
However I find in practice it reduces a lot of the noise that we experience when are finding Contos

42
00:03:01,080 --> 00:03:03,600
using the C-v to find Contos function.

43
00:03:03,900 --> 00:03:05,600
So this is something you can play by ear.

44
00:03:05,750 --> 00:03:10,530
Maybe experiment with it or not just keep it in mind that it actually can help reduce the number of

45
00:03:10,680 --> 00:03:15,520
necessary Contos when using find Contos.

46
00:03:15,540 --> 00:03:19,080
OK so let's move on to find Contos function here.

47
00:03:19,080 --> 00:03:21,250
This is me and call this program.

48
00:03:21,250 --> 00:03:27,750
So dysfunction C-v to that fine Contos takes the input arguments it takes input image here which is

49
00:03:27,800 --> 00:03:29,250
a kind of edged image.

50
00:03:29,340 --> 00:03:33,320
It takes a retrieval mode and the approximation mode and there are several options.

51
00:03:33,330 --> 00:03:39,680
Each of these here which we will discuss shortly after which we display or edged image.

52
00:03:39,870 --> 00:03:44,490
And the reason I displayed for that I just wanted to tell you why I should tell you that fine contours

53
00:03:44,550 --> 00:03:46,170
actually edits this image.

54
00:03:46,350 --> 00:03:49,430
If you don't want to edit your image use this instead.

55
00:03:49,440 --> 00:03:55,770
Here edge dot copy the copies of a Python function that actually copies this image here.

56
00:03:55,920 --> 00:03:57,200
Well the variable here.

57
00:03:57,540 --> 00:03:59,810
So we don't actually edit the original variable

58
00:04:02,990 --> 00:04:09,030
so moving back to the outputs of find Contos returns to Kontos which I'll explain shortly.

59
00:04:09,030 --> 00:04:10,330
And the hierarchy.

60
00:04:10,680 --> 00:04:16,320
And later on we print number of Contos phunk image and then we draw the contours on the image.

61
00:04:16,320 --> 00:04:21,810
So if you remember now a drawing functions it takes the input image takes the contours which are points

62
00:04:21,810 --> 00:04:28,500
and explain again shortly in this minus one here tells it to draw all contours or we can actually index

63
00:04:28,500 --> 00:04:35,150
it to draw the first CONTO and deleted Camarda or second contour and vice versa.

64
00:04:35,200 --> 00:04:40,000
Some Let's keep it at minus 1 to draw all this here's a color and I believe this will be green.

65
00:04:40,000 --> 00:04:41,280
BGR Yes.

66
00:04:41,430 --> 00:04:43,080
And this is like line thickness here.

67
00:04:43,440 --> 00:04:46,340
So let's run this code and see what's actually happening here.

68
00:04:47,070 --> 00:04:49,290
Here we have over tree black squares.

69
00:04:49,670 --> 00:04:52,700
Is it a can the edges done very well.

70
00:04:53,100 --> 00:04:57,200
These are the this is exactly how find is at its image.

71
00:04:57,210 --> 00:04:59,460
It does some sort of really with the Contos.

72
00:04:59,490 --> 00:05:01,520
This is why it looks like this.

73
00:05:01,680 --> 00:05:03,560
You probably wouldn't ever need to use this.

74
00:05:03,560 --> 00:05:04,760
Use this image.

75
00:05:04,800 --> 00:05:10,310
But keep in mind if you have a thing if you're ever playing with your Contos function.

76
00:05:10,440 --> 00:05:10,720
All right.

77
00:05:10,740 --> 00:05:12,490
And let's actually see the Kontos now.

78
00:05:12,730 --> 00:05:13,360
Ah.

79
00:05:13,980 --> 00:05:15,600
So this is proof that you don't.

80
00:05:15,640 --> 00:05:20,680
These are tree contours that have probably segmented or black squares here.

81
00:05:20,790 --> 00:05:25,440
So you've just run in your first finding or Foose onto extraction called

82
00:05:28,360 --> 00:05:34,630
and as you can see here it actually opiated that we've created or found tree contorts which is exactly

83
00:05:34,630 --> 00:05:35,630
as we saw.

84
00:05:36,070 --> 00:05:38,980
So now let's do some other interesting things with Contos.

85
00:05:38,980 --> 00:05:41,470
Let's take a look at the actual contour file.

86
00:05:41,470 --> 00:05:45,120
So let's actually print the CONTO file here that we extract.

87
00:05:45,280 --> 00:05:53,010
So let's do Prince contorts run Kalid we go.

88
00:05:53,020 --> 00:05:58,090
So here we have a matrix it looks like coordinate points x y points of contours.

89
00:05:58,090 --> 00:06:00,670
So what exactly are we doing here.

90
00:06:00,790 --> 00:06:06,550
So open civies stores Kontos us a list of lists and get a list of Lister's is that remember when we

91
00:06:06,550 --> 00:06:11,130
run linked function on our CONTO file we saw a link being treated.

92
00:06:11,260 --> 00:06:14,470
That means they are trialist awfullest in that file.

93
00:06:14,500 --> 00:06:20,050
So imagine CONTO one being the first element in nuttery and that that element contains a list of all

94
00:06:20,050 --> 00:06:24,120
the coordinates and these coordinates here are the points along the contour.

95
00:06:24,180 --> 00:06:27,970
Now there are different ways we can store these points and that's called the approximation methods which

96
00:06:27,970 --> 00:06:28,960
we come to shortly.

97
00:06:30,170 --> 00:06:30,400
OK.

98
00:06:30,410 --> 00:06:34,230
So let's head back to Kuwait and I'll teach you a bit about approximation types.

99
00:06:34,310 --> 00:06:39,770
So as you saw here when we printed all CONTO file we saw a bunch of bunch of points here.

100
00:06:39,770 --> 00:06:45,620
So these points actually the points of lie around the Contos that we just saw those green squares or

101
00:06:45,680 --> 00:06:48,020
rectangles or angle blocks.

102
00:06:48,020 --> 00:06:53,930
So let's look at the notice an approximation methods NODIA to approximation methods here is ciii and

103
00:06:53,930 --> 00:06:54,320
approx.

104
00:06:54,320 --> 00:06:54,770
None.

105
00:06:54,800 --> 00:06:58,800
And Cheena proc simple no difference between these two is at approx.

106
00:06:58,800 --> 00:07:04,700
None actually stores or gives you back all the points along those green lines here.

107
00:07:05,000 --> 00:07:09,140
So it's going to be a ton of I think so be it a number of number of points.

108
00:07:09,140 --> 00:07:15,350
However Cheena proc simple is actually gives you the bounding endpoints so it gives you basically a

109
00:07:15,350 --> 00:07:16,190
polygon.

110
00:07:16,520 --> 00:07:21,420
So if it's a square it's just going to give you four points if it's a triangle tree points.

111
00:07:21,440 --> 00:07:24,630
So this is a much more efficient way of storing counter-points.

112
00:07:24,680 --> 00:07:29,430
So just keep that in mind when you're if you're finding Contos in large images or large sets of images.

113
00:07:30,540 --> 00:07:33,000
So we're next we're going to talk about retrieval mode.

114
00:07:35,410 --> 00:07:35,820
OK.

115
00:07:35,880 --> 00:07:40,740
So let's actually take a look at retrieval modes now and retrieval modes are sometimes a bit confusing

116
00:07:41,280 --> 00:07:41,930
actually to me.

117
00:07:41,930 --> 00:07:44,550
I find it a bit confusing the first time I looked at it.

118
00:07:44,610 --> 00:07:48,740
However they're actually not that confusing and they're only basically just two main types that will

119
00:07:48,770 --> 00:07:50,910
be using in most cases.

120
00:07:50,940 --> 00:07:54,340
So let's take a look at the types here and I'll pop one slide.

121
00:07:54,330 --> 00:08:00,360
So retrieval mode essentially defines the hierarchy of the console as so hierarchy being like do you

122
00:08:00,360 --> 00:08:04,800
want subclans to his or do you want external controllers or do you want all countries.

123
00:08:04,800 --> 00:08:08,130
So let's look at the four types of achievements here.

124
00:08:08,190 --> 00:08:15,350
Does retrieve modalists which returns all Contos which is good if you just want everything.

125
00:08:15,430 --> 00:08:19,290
There's the retrieve external which gives us only the external Contos.

126
00:08:19,330 --> 00:08:21,880
And I actually find this to be probably the most useful one.

127
00:08:21,880 --> 00:08:28,090
In most cases however there will be times you'd want a contour that's in a contour imagine like a different

128
00:08:28,180 --> 00:08:35,160
shape perhaps does retrieve comp and retrieve tree retrieve tree is actually also very useful as well.

129
00:08:35,290 --> 00:08:36,730
So the first two are most useful.

130
00:08:36,730 --> 00:08:43,240
However a tree is actually very useful tree Rutan's I'll tell you why tree returns the full hierarchy

131
00:08:43,240 --> 00:08:48,460
information of that no hierarchy information is a bit confusing and I'm not going to discuss it in length

132
00:08:48,460 --> 00:08:54,720
here unless you guys wish for me to do it I think in most cases you don't necessarily need to understand

133
00:08:54,720 --> 00:08:59,430
them too much but if you want you can click this link here and actually get a very detailed tutorial

134
00:08:59,910 --> 00:09:01,410
on the hierarchy modes here.

135
00:09:01,740 --> 00:09:07,530
So just to simply add something here by setting two different retrieval moods or a hierarchy file because

136
00:09:07,560 --> 00:09:10,460
if you remember correctly let's look at the code again.

137
00:09:10,590 --> 00:09:14,550
Define Contos function returns Contos hulky.

138
00:09:14,760 --> 00:09:20,830
And if you look at Harkee hierarchy would actually give you disgruntle presently.

139
00:09:20,910 --> 00:09:22,940
I guess you can consider it like a LEO.

140
00:09:23,160 --> 00:09:26,300
The next layer and the first child and the parent.

141
00:09:26,310 --> 00:09:30,450
So these are a bit confusing so I suggest you read this just to get some more information here.

142
00:09:32,310 --> 00:09:32,630
OK.

143
00:09:32,640 --> 00:09:37,320
So let's go back to Kuwait and illustrate the difference between the first two CONTO types retrieve

144
00:09:37,320 --> 00:09:43,980
lists which achieves all of extend external which achieves the outer continental as only.

145
00:09:44,090 --> 00:09:48,010
OK so let's stick a recap what listed here.

146
00:09:48,460 --> 00:09:50,040
So we see we've got the Contos idea.

147
00:09:50,050 --> 00:09:53,690
So let's change our file now to another file here.

148
00:09:53,830 --> 00:09:56,080
You know there are all doing that.

149
00:09:56,110 --> 00:09:59,620
It's a square doing that because there's a chip Square.

150
00:09:59,740 --> 00:10:02,010
Sorry about terminology.

151
00:10:02,290 --> 00:10:07,170
So that's a square and left as we can see.

152
00:10:07,260 --> 00:10:11,690
We've got the tree extension Contos but the contour inside has been totally ignored.

153
00:10:12,210 --> 00:10:16,680
So now let's change this now to do last

154
00:10:22,040 --> 00:10:23,260
Let's run this code again.

155
00:10:25,630 --> 00:10:26,410
And there we go.

156
00:10:26,410 --> 00:10:31,300
So we see no list receives all Kontos including to in a Quanto in the square.

157
00:10:31,300 --> 00:10:34,790
So that's a good example of harder different achievements work.

158
00:10:35,050 --> 00:10:39,430
So when you're implementing some projects that on please always consider whether you need to get external

159
00:10:39,490 --> 00:10:45,280
Contos only or what do you need to get all contours and in future if you did want to.

160
00:10:45,370 --> 00:10:51,670
If you do want to extract you know Autor Contos only you can use functions like retrieve all or retrieve

161
00:10:51,670 --> 00:10:51,970
list.

162
00:10:51,970 --> 00:10:57,640
Sorry I actually use hierarchy to actually identify different layers or levels of Contos which is quite

163
00:10:57,640 --> 00:10:58,090
cool.

164
00:10:59,610 --> 00:11:02,750
So that concludes a gentle introduction to CONTO here.

165
00:11:03,120 --> 00:11:05,240
Let's not look at sorting containers.

166
00:11:05,340 --> 00:11:07,560
We can sort from left to right.

167
00:11:07,610 --> 00:11:08,880
We can sort by size.

168
00:11:08,880 --> 00:11:09,620
So that's pretty cool.

169
00:11:09,660 --> 00:11:10,710
So let's take a look at at.