It’s been awhile since I did something in Flash, \u61d0\u304b\u3057\u3044\u301c\u3002<\/p>\n
I’ve been experimenting with eye tracking system in Flash lately, there are a few<\/a> demos<\/a> out there but none of them provide the source code.<\/p>\n<\/iframe><\/p>\n<p>Through Google I came across to <a title=\"ACCURATE EYE CENTRE LOCALISATION BY MEANS OF GRADIENTS\" href=\"http:\/\/www.inb.uni-luebeck.de\/publikationen\/pdfs\/TiBa11b.pdf\" target=\"_blank\">Fabian Timm and Erhardt Barth’s research<\/a>, which basically explains the face detection process as follows;<\/p>\n<ul>\n<li>Applying the face detector and extracting the eye region.<\/li>\n<li>Isolating the eye area by detecting the color gradient between iris and sclera<\/li>\n<\/ul>\n<p>It’s pretty simple and I thought the BitmapData class is robust enough to handle this type of algorithm<\/p>\n<p>Whilst tinkering I\u2019ve stumbled across <a title=\"Detecting blobs at the speed of light\" href=\"http:\/\/play.blog2t.net\/fast-blob-detection\/\" target=\"_blank\">Tomek\u2019s blog<\/a>, which describes a clever way of extracting color by juggling the <a title=\"threshold\" href=\"http:\/\/help.adobe.com\/en_US\/FlashPlatform\/reference\/actionscript\/3\/flash\/display\/BitmapData.html#threshold()\" target=\"_blank\">threshold<\/a> and <a title=\"BlendMode\" href=\"http:\/\/help.adobe.com\/en_US\/FlashPlatform\/reference\/actionscript\/3\/flash\/display\/BlendMode.html#HARDLIGHT\" target=\"_blank\">blend<\/a> method, that rang a few bells.<\/p>\n<p>So after a few custom adjustments I\u2019ve made my own eye tracking code, nothing too fancy since the requirement isn\u2019t too complicated.<\/p>\n<p>First add BlendMode<\/p>\n<pre class=\"brush: as3; title: ; notranslate\" title=\"\">\r\npublic function addBlendMode(s:BitmapData):BitmapData {\r\n\r\n\tvar r:BitmapData = new BitmapData(s.width,s.height);\r\n\tvar r2:BitmapData = new BitmapData(s.width,s.height);\r\n\tvar rect:Rectangle = new Rectangle(0,0,s.width,s.height);\r\n\tvar pt:Point = new Point(0,0);\r\n\tr.draw(s);\r\n\tr2.draw(s);\r\n\r\n\t\/\/ to get more contrast\r\n\tr.draw(r2, new Matrix(), new ColorTransform(), BlendMode.MULTIPLY);\r\n\treturn r;\r\n\r\n}\r\n<\/pre>\n<p>Use the threshold method, convert the bitmap into 1 bit color.<\/p>\n<pre class=\"brush: as3; title: ; notranslate\" title=\"\">\r\n\/\/ the threshold color is important\r\npublic function addThresholdColor(b:BitmapData, th:* = 0xff111111):BitmapData {\r\n\t\t\t\r\n\tvar bmd2:BitmapData = new BitmapData(b.width, b.height);\r\n\tvar pt:Point = new Point(0,0);\r\n\tvar rect:Rectangle = new Rectangle(0, 0, b.width, b.height);\r\n\tvar color:uint = 0xFF000000;\r\n\r\n\tbmd2.threshold(b, rect, pt, ">=", th, color, 0xFFFFFFFF, false);\r\n\r\n\treturn bmd2;\r\n}\r\n<\/pre>\n<p>Lastly, mark the eye, I’ve done it only for one eye since the movement will be symmetrical anyway.<\/p>\n<pre class=\"brush: as3; title: ; notranslate\" title=\"\">\r\npublic function getBound(b:BitmapData):Rectangle {\r\n\r\n\tvar maxBlobs:int = 40;\r\n\tvar i:int = 0;\r\n\t\r\n\tvar minX:int = 640; \/\/ video width\r\n\tvar maxX:int = 0;\r\n\r\n\tvar minY:int = 480; \/\/ video height\r\n\tvar maxY:int = 0;\r\n\r\n\tvar hx:int;\r\n\tvar hy:int;\r\n\r\n\twhile(i < maxBlobs) {\r\n\r\n\t\tvar bound:Rectangle = b.getColorBoundsRect(0xffffffff, 0xffffffff); \r\n\r\n\t\tif(bound.isEmpty()) break;\r\n\r\n\t\tvar bx:int = bound.x;\r\n\t\tvar by:int = bound.y;\r\n\t\tvar bwidth:int = bx + bound.width;\r\n\t\tvar bheight:int = by + bound.height;\r\n\r\n\t\tif(bwidth < minX) minX = bx;\r\n\t\tif(bwidth > maxX) maxX = bwidth;\r\n\r\n\t\tif(bheight < minY) minY = by; \r\n\t\tif(bheight > maxY) maxY = bheight;\r\n\r\n\t\tfor(var y:uint = by; y < bheight; y++) {\r\n\t\t\t\r\n\t\t\tif(b.getPixel32(bx,y) == 0xffffffff) {\r\n\r\n\t\t\t\t\/\/ fill color\r\n\t\t\t\tb.floodFill(bx,y, 0xffff0000);\r\n\t\t\t}\r\n\t\t}\r\n\r\n\t\ti++;\r\n\t}\r\n\t\r\n\treturn new Rectangle(minX, minY, maxX - minX, maxY - minY);\r\n\r\n}\r\n<\/pre>\n<p>At the moment the algorithm is not perfect, I need to add automatic light detector and stabilizer and due to the webcam resolution and environment lighting the gradient tracking seems impractical but the same principle applies, more or less, also I haven’t tested it against a person who has <a href=\"http:\/\/i.imgur.com\/ypLqggl.png\" title=\"Black gamer problem\" target=\"_blank\">dark colored skin<\/a> or light colored eyes, so I’ll post it later. <\/p>\n<h5>Resources<\/h5>\n<p><a href=\"http:\/\/www.beyond-reality-face.com\/app\" title=\"http:\/\/www.beyond-reality-face.com\/app\" target=\"_blank\">Beyond Reality Face Detection<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>It’s been awhile since I did something in Flash, \u61d0\u304b\u3057\u3044\u301c\u3002 I’ve been experimenting with eye tracking system in Flash lately, there are a few demos out there but none of them provide the source code. Through Google I came across to Fabian Timm and Erhardt Barth’s research, which basically explains the face detection process as […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[4,7,12,24,25,31],"tags":[64,75,95,96],"_links":{"self":[{"href":"https:\/\/rahmat-hidayat.com\/index.php?rest_route=\/wp\/v2\/posts\/1017"}],"collection":[{"href":"https:\/\/rahmat-hidayat.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/rahmat-hidayat.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/rahmat-hidayat.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/rahmat-hidayat.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=1017"}],"version-history":[{"count":1,"href":"https:\/\/rahmat-hidayat.com\/index.php?rest_route=\/wp\/v2\/posts\/1017\/revisions"}],"predecessor-version":[{"id":1636,"href":"https:\/\/rahmat-hidayat.com\/index.php?rest_route=\/wp\/v2\/posts\/1017\/revisions\/1636"}],"wp:attachment":[{"href":"https:\/\/rahmat-hidayat.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=1017"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/rahmat-hidayat.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=1017"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/rahmat-hidayat.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=1017"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
<\/iframe><\/p>\n<p>Through Google I came across to <a title=\"ACCURATE EYE CENTRE LOCALISATION BY MEANS OF GRADIENTS\" href=\"http:\/\/www.inb.uni-luebeck.de\/publikationen\/pdfs\/TiBa11b.pdf\" target=\"_blank\">Fabian Timm and Erhardt Barth’s research<\/a>, which basically explains the face detection process as follows;<\/p>\n<ul>\n<li>Applying the face detector and extracting the eye region.<\/li>\n<li>Isolating the eye area by detecting the color gradient between iris and sclera<\/li>\n<\/ul>\n<p>It’s pretty simple and I thought the BitmapData class is robust enough to handle this type of algorithm<\/p>\n<p>Whilst tinkering I\u2019ve stumbled across <a title=\"Detecting blobs at the speed of light\" href=\"http:\/\/play.blog2t.net\/fast-blob-detection\/\" target=\"_blank\">Tomek\u2019s blog<\/a>, which describes a clever way of extracting color by juggling the <a title=\"threshold\" href=\"http:\/\/help.adobe.com\/en_US\/FlashPlatform\/reference\/actionscript\/3\/flash\/display\/BitmapData.html#threshold()\" target=\"_blank\">threshold<\/a> and <a title=\"BlendMode\" href=\"http:\/\/help.adobe.com\/en_US\/FlashPlatform\/reference\/actionscript\/3\/flash\/display\/BlendMode.html#HARDLIGHT\" target=\"_blank\">blend<\/a> method, that rang a few bells.<\/p>\n<p>So after a few custom adjustments I\u2019ve made my own eye tracking code, nothing too fancy since the requirement isn\u2019t too complicated.<\/p>\n<p>First add BlendMode<\/p>\n<pre class=\"brush: as3; title: ; notranslate\" title=\"\">\r\npublic function addBlendMode(s:BitmapData):BitmapData {\r\n\r\n\tvar r:BitmapData = new BitmapData(s.width,s.height);\r\n\tvar r2:BitmapData = new BitmapData(s.width,s.height);\r\n\tvar rect:Rectangle = new Rectangle(0,0,s.width,s.height);\r\n\tvar pt:Point = new Point(0,0);\r\n\tr.draw(s);\r\n\tr2.draw(s);\r\n\r\n\t\/\/ to get more contrast\r\n\tr.draw(r2, new Matrix(), new ColorTransform(), BlendMode.MULTIPLY);\r\n\treturn r;\r\n\r\n}\r\n<\/pre>\n<p>Use the threshold method, convert the bitmap into 1 bit color.<\/p>\n<pre class=\"brush: as3; title: ; notranslate\" title=\"\">\r\n\/\/ the threshold color is important\r\npublic function addThresholdColor(b:BitmapData, th:* = 0xff111111):BitmapData {\r\n\t\t\t\r\n\tvar bmd2:BitmapData = new BitmapData(b.width, b.height);\r\n\tvar pt:Point = new Point(0,0);\r\n\tvar rect:Rectangle = new Rectangle(0, 0, b.width, b.height);\r\n\tvar color:uint = 0xFF000000;\r\n\r\n\tbmd2.threshold(b, rect, pt, ">=", th, color, 0xFFFFFFFF, false);\r\n\r\n\treturn bmd2;\r\n}\r\n<\/pre>\n<p>Lastly, mark the eye, I’ve done it only for one eye since the movement will be symmetrical anyway.<\/p>\n<pre class=\"brush: as3; title: ; notranslate\" title=\"\">\r\npublic function getBound(b:BitmapData):Rectangle {\r\n\r\n\tvar maxBlobs:int = 40;\r\n\tvar i:int = 0;\r\n\t\r\n\tvar minX:int = 640; \/\/ video width\r\n\tvar maxX:int = 0;\r\n\r\n\tvar minY:int = 480; \/\/ video height\r\n\tvar maxY:int = 0;\r\n\r\n\tvar hx:int;\r\n\tvar hy:int;\r\n\r\n\twhile(i < maxBlobs) {\r\n\r\n\t\tvar bound:Rectangle = b.getColorBoundsRect(0xffffffff, 0xffffffff); \r\n\r\n\t\tif(bound.isEmpty()) break;\r\n\r\n\t\tvar bx:int = bound.x;\r\n\t\tvar by:int = bound.y;\r\n\t\tvar bwidth:int = bx + bound.width;\r\n\t\tvar bheight:int = by + bound.height;\r\n\r\n\t\tif(bwidth < minX) minX = bx;\r\n\t\tif(bwidth > maxX) maxX = bwidth;\r\n\r\n\t\tif(bheight < minY) minY = by; \r\n\t\tif(bheight > maxY) maxY = bheight;\r\n\r\n\t\tfor(var y:uint = by; y < bheight; y++) {\r\n\t\t\t\r\n\t\t\tif(b.getPixel32(bx,y) == 0xffffffff) {\r\n\r\n\t\t\t\t\/\/ fill color\r\n\t\t\t\tb.floodFill(bx,y, 0xffff0000);\r\n\t\t\t}\r\n\t\t}\r\n\r\n\t\ti++;\r\n\t}\r\n\t\r\n\treturn new Rectangle(minX, minY, maxX - minX, maxY - minY);\r\n\r\n}\r\n<\/pre>\n<p>At the moment the algorithm is not perfect, I need to add automatic light detector and stabilizer and due to the webcam resolution and environment lighting the gradient tracking seems impractical but the same principle applies, more or less, also I haven’t tested it against a person who has <a href=\"http:\/\/i.imgur.com\/ypLqggl.png\" title=\"Black gamer problem\" target=\"_blank\">dark colored skin<\/a> or light colored eyes, so I’ll post it later. <\/p>\n<h5>Resources<\/h5>\n<p><a href=\"http:\/\/www.beyond-reality-face.com\/app\" title=\"http:\/\/www.beyond-reality-face.com\/app\" target=\"_blank\">Beyond Reality Face Detection<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>It’s been awhile since I did something in Flash, \u61d0\u304b\u3057\u3044\u301c\u3002 I’ve been experimenting with eye tracking system in Flash lately, there are a few demos out there but none of them provide the source code. Through Google I came across to Fabian Timm and Erhardt Barth’s research, which basically explains the face detection process as […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[4,7,12,24,25,31],"tags":[64,75,95,96],"_links":{"self":[{"href":"https:\/\/rahmat-hidayat.com\/index.php?rest_route=\/wp\/v2\/posts\/1017"}],"collection":[{"href":"https:\/\/rahmat-hidayat.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/rahmat-hidayat.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/rahmat-hidayat.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/rahmat-hidayat.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=1017"}],"version-history":[{"count":1,"href":"https:\/\/rahmat-hidayat.com\/index.php?rest_route=\/wp\/v2\/posts\/1017\/revisions"}],"predecessor-version":[{"id":1636,"href":"https:\/\/rahmat-hidayat.com\/index.php?rest_route=\/wp\/v2\/posts\/1017\/revisions\/1636"}],"wp:attachment":[{"href":"https:\/\/rahmat-hidayat.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=1017"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/rahmat-hidayat.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=1017"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/rahmat-hidayat.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=1017"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}