MuhammadFakhriZuhair - 1903421009 - BM6B - Deteksi Dan Pelacakan Warna

ESP32-CAM WEB SERVER DENGAN OPENCV.
JS
DETEKSI DAN PELACAKAN WARNA
Disusun untuk memenuhi Nilai Mata Kuliah
KOMPUTASI PERVASIVE
DISUSUN OLEH:
Muhammad Fakhri Zuhair (1903421009)
William Immanuel (1903421042)
BM-6B
PROGRAM STUDI BROADBAND MULTIMEDIA
JURUSAN TEKNIK ELEKTRO
POLITEKNIK NEGERI JAKARTA
2022
Tujuan Praktikum
Pada praktikum ini akan dikenalkan alat/tool OpenCV.js dan OpenCV untuk lingkungan
Server Web Kamera ESP32. Sebagai contoh, akan dibuat Server Web Kamera ESP32
sederhana yang menyertakan deteksi warna dan pelacakan objek bergerak. Diharapkan
pengenalan ini akan menginspirasi kerja OpenCV tambahan dengan kamera ESP32.
Teori Singkat
1. ESP32 Cam
ESP32-CAM merupakan salah satu mikrokontroler yang memiliki fasilitas tambahan berupa
bluetooth, wifi, kamera, bahkan sampai ke slot mikroSD. ESP32-CAM ini biasanya digunakan
untuk project IoT (Internet of Things) yang membutuhkan fitur kamera. Modul ESP32CAM
memiliki lebih sedikit pin I/O dibandingkan modul ESP32 produk sebelumnya, yaitu ESP32
Wroom. Hal ini dikarenakan sudah banyak pin yang digunakan secara internal untuk fungsi
kamera dan fungsi slot kartu microSD. Selain itu, modul ESP32CAM juga tidak memiliki port
USB khusus (mengirim program dari port USB komputer). Jadi untuk memprogram modul ini
Anda harus menggunakan USB TTL atau kita dapat menambahkan modul tambahan berupa
downloader khusus untuk ESP32-CAM.
Sumber : https://indobot.co.id/blog/mengenal-esp32-cam-dan-bagaimana-cara-
menggunakannya/
2. OpenCV.js
OpenCV.js adalah pengikatan JavaScript untuk subset terpilih dari fungsi OpenCV
untuk platform web. Hal ini memungkinkan aplikasi web yang muncul dengan
pemrosesan multimedia untuk mendapatkan keuntungan dari berbagai macam fungsi
visi yang tersedia di OpenCV. OpenCV.js memanfaatkan Emscripten untuk
mengkompilasi fungsi OpenCV ke dalam target asm.js atau WebAssembly, dan
menyediakan API JavaScript untuk aplikasi web untuk mengaksesnya. Versi
perpustakaan mendatang akan memanfaatkan API akselerasi yang tersedia di Web
seperti SIMD dan eksekusi multi-threaded
Sumber : https://docs.opencv.org/3.4/df/d0a/tutorial_js_intro.html.
Diagram Rangkaian
Langkah Percobaan
I. Menambahkan aplikasi ke dalan proyek firebase
1. Buka Konsol proyek Firebase Anda dan tambahkan aplikasi ke proyek yang Anda buat di
tutorial sebelumnya dengan mengeklik tombol + Tambahkan aplikasi.
2. Pilih ikon aplikasi web.
3. Beri nama pada aplikasi . Kemudian, centang kotak di sebelah Juga siapkan Firebase
Hosting untuk Aplikasi ini. Klik Daftar aplikasi.
4. Kemudian, salin objek firebaseConfig dan simpan karena Anda akan membutuhkannya
nanti.
5. Klik Next pada langkah selanjutnya, dan terakhir pada Continue to console.
II. Menyiapkan Kode VS Proyek Aplikasi Web Firebase

1. Buat folder di komputer Anda untuk menyimpan proyek Firebase misalnya, Proyek-
Firebase di Desktop.
2. Buka VS Code. Buka File > Open Folder… dan pilih folder yang baru saja Anda buat.
3. Buka Terminal > New terminal. Jendela Terminal baru akan terbuka di jalur proyek Anda.
4. Pada jendela Terminal sebelumnya, ketik berikut ini:
firebase login
5. Anda akan diminta untuk mengumpulkan penggunaan CLI dan informasi pelaporan
kesalahan. Masukkan "n" dan tekan Enter untuk menolak.
6. Setelah ini, akan muncul jendela baru di browser Anda untuk masuk ke akun firebase
Anda.
7. Izinkan Firebase CLI untuk mengakses akun google Anda.
8. Setelah ini, login Firebase CLI seharusnya berhasil. Anda dapat menutup jendela browser.
9. Setelah berhasil masuk, jalankan perintah berikut untuk memulai direktori proyek Firebase
di folder saat ini.
10. Anda akan ditanya apakah ingin menginisialisasi proyek Firebase di direktori saat ini.
Masukkan Y dan tekan Enter.
11. Kemudian, gunakan panah atas dan bawah dan tombol Spasi untuk memilih opsi. Pilih
opsi berikut:
Hosting : Konfigurasikan file untuk Firebase Hosting dan (opsional) siapkan penerapan
GitHub Action
Storage : Konfigurasikan file aturan keamanan untuk Cloud Storage
Opsi yang dipilih akan muncul dengan tanda bintang hijau. Kemudian, tekan Enter.
12. Pilih opsi "Use an existing project” harus disorot dengan warna biru, lalu tekan Enter.
13. Setelah itu, pilih proyek Firebase untuk direktori ini itu harus menjadi proyek yang dibuat
di tutorial sebelumnya. Dalam kasus saya, ini disebut esp-firebase-demo. Kemudian tekan
Enter.
14. Kemudian, pilih opsi hosting seperti yang ditunjukkan di bawah ini:
Apa yang ingin Anda gunakan sebagai direktori publik Anda? Tekan Enter untuk memilih
publik.
Konfigurasikan sebagai aplikasi satu halaman (tulis ulang url ke /index.html)? Tidak
Siapkan pembuatan dan penerapan otomatis dengan GitHub? Tidak
15. Kemudian, tekan Enter pada pertanyaan berikut untuk memilih file default untuk aturan
penyimpanan.
File apa yang harus digunakan untuk Aturan Penyimpanan? (storage.rules). Tekan Enter.
16. Proyek Firebase sekarang harus berhasil diinisialisasi. Perhatikan bahwa VS Code
membuat beberapa file penting di bawah folder proyek Anda.
17. Untuk memeriksa apakah semuanya berjalan seperti yang diharapkan, jalankan perintah
berikut di jendela Terminal VS Code.
18. Salin URL hosting dan tempel ke jendela browser web. Anda akan melihat halaman web
berikut. Anda dapat mengakses halaman web itu dari mana saja di dunia.
Hasil Praktikum
OCV_ColorTrack
/***
The include file, index_OCV_ColorTrack.h, the Client, is an intoduction of

OpenCV.js to the ESP32 Camera environment. The Client was
developed and written by Andrew R. Sass. Permission to reproduce the

index_OCV_ColorTrack.h file is granted free of charge if this
entire copyright notice is included in all copies of the

index_OCV_ColorTrack.h file.
Complete instructions at https://RandomNerdTutorials.com/esp32-cam-

opencv-js-color-detection-tracking/
Permission is hereby granted, free of charge, to any person obtaining a

copy of this software and associated documentation files.
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
***/
#include <WiFi.h>
#include <WiFiClientSecure.h>
#include "esp_camera.h"
#include "soc/soc.h"
#include "soc/rtc_cntl_reg.h"
#include "index_OCV_ColorTrack.h"
// Replace with your network credentials
const char* ssid = "HAYUK!";
const char* password = "bismillahkuy";

String Feedback="";
String
Command="",cmd="",P1="",P2="",P3="",P4="",P5="",P6="",P7="",P8="",P9="";
byte
ReceiveState=0,cmdState=1,strState=1,questionstate=0,equalstate=0,semicolon
state=0;
//ANN:0
// AI-Thinker
#define PWDN_GPIO_NUM 32
#define RESET_GPIO_NUM -1
#define XCLK_GPIO_NUM 0
#define SIOD_GPIO_NUM 26
#define SIOC_GPIO_NUM 27
#define Y9_GPIO_NUM 35
#define VSYNC_GPIO_NUM 25
#define HREF_GPIO_NUM 23
#define PCLK_GPIO_NUM 22
WiFiServer server(80);
//ANN:2
void ExecuteCommand() {
if (cmd!="colorDetect") { //Omit printout
//Serial.println("cmd= "+cmd+" ,P1= "+P1+" ,P2= "+P2+" ,P3= "+P3+" ,P4=

"+P4+" ,P5= "+P5+" ,P6= "+P6+" ,P7= "+P7+" ,P8= "+P8+" ,P9= "+P9);
//Serial.println("");
if (cmd=="resetwifi") {
WiFi.begin(P1.c_str(), P2.c_str());
Serial.print("Connecting to ");
Serial.println(P1);
long int StartTime=millis();
while (WiFi.status() != WL_CONNECTED)
delay(500);
if ((StartTime+5000) < millis()) break;
Serial.println("");
Serial.println("STAIP: "+WiFi.localIP().toString());
Feedback="STAIP: "+WiFi.localIP().toString();
else if (cmd=="restart") {
ESP.restart();
else if (cmd=="cm"){
int XcmVal = P1.toInt();
int YcmVal = P2.toInt();
Serial.println("cmd= "+cmd+" ,VALXCM= "+XcmVal);
Serial.println("cmd= "+cmd+" ,VALYCM= "+YcmVal);

}
else if (cmd=="quality") {
sensor_t * s = esp_camera_sensor_get();
int val = P1.toInt();
s->set_quality(s, val);
else if (cmd=="contrast") {
s->set_contrast(s, val);
else if (cmd=="brightness") {
s->set_brightness(s, val);
else {
Feedback="Command is not defined.";
if (Feedback=="") {
Feedback=Command;
void setup() {
WRITE_PERI_REG(RTC_CNTL_BROWN_OUT_REG, 0);
Serial.begin(115200);
Serial.setDebugOutput(true);
Serial.println();
camera_config_t config;
config.ledc_channel = LEDC_CHANNEL_0;
config.ledc_timer = LEDC_TIMER_0;
config.pin_d0 = Y2_GPIO_NUM;
config.pin_xclk = XCLK_GPIO_NUM;
config.pin_pclk = PCLK_GPIO_NUM;
config.pin_vsync = VSYNC_GPIO_NUM;
config.pin_href = HREF_GPIO_NUM;
config.pin_sscb_sda = SIOD_GPIO_NUM;
config.pin_sscb_scl = SIOC_GPIO_NUM;
config.pin_pwdn = PWDN_GPIO_NUM;
config.pin_reset = RESET_GPIO_NUM;
config.xclk_freq_hz = 20000000;
config.pixel_format = PIXFORMAT_JPEG;
//init with high specs to pre-allocate larger buffers
if(psramFound()){
config.frame_size = FRAMESIZE_UXGA;
config.jpeg_quality = 10; //0-63 lower number means higher quality

config.fb_count = 2;
} else {
config.frame_size = FRAMESIZE_SVGA;
config.jpeg_quality = 12; //0-63 lower number means higher quality
config.fb_count = 1;
// camera init
esp_err_t err = esp_camera_init(&config);
if (err != ESP_OK) {
Serial.printf("Camera init failed with error 0x%x", err);
delay(1000);
ESP.restart();
//drop down frame size for higher initial frame rate
s->set_framesize(s, FRAMESIZE_CIF);
//UXGA|SXGA|XGA|SVGA|VGA|CIF|QVGA|HQVGA|QQVGA 設定初始化影像解析度
WiFi.mode(WIFI_AP_STA);
WiFi.begin(ssid, password);
delay(1000);
long int StartTime=millis();
while (WiFi.status() != WL_CONNECTED) {
delay(500);
if ((StartTime+10000) < millis())
break;
if (WiFi.status() == WL_CONNECTED) {
Serial.print("ESP IP Address: http://");
Serial.println(WiFi.localIP());
server.begin();
void loop() {
Feedback="";Command="";cmd="";P1="";P2="";P3="";P4="";P5="";P6="";P7="";P8=
"";P9="";
ReceiveState=0,cmdState=1,strState=1,questionstate=0,equalstate=0,semicolon
state=0;
WiFiClient client = server.available();
if (client) {
String currentLine = "";
while (client.connected()) {
if (client.available()) {
char c = client.read();
getCommand(c);
if (c == '\n') {
if (currentLine.length() == 0) {
if (cmd=="colorDetect") {
camera_fb_t * fb = NULL;
fb = esp_camera_fb_get();
if(!fb) {
Serial.println("Camera capture failed");
delay(1000);
ESP.restart();
//ANN:1
client.println("HTTP/1.1 200 OK");
client.println("Access-Control-Allow-Origin: *");
client.println("Access-Control-Allow-Headers: Origin, X-
Requested-With, Content-Type, Accept");
client.println("Access-Control-Allow-Methods:
GET,POST,PUT,DELETE,OPTIONS");
client.println("Content-Type: image/jpeg");
client.println("Content-Disposition: form-data;
name=\"imageFile\"; filename=\"picture.jpg\"");
client.println("Content-Length: " + String(fb->len));
client.println("Connection: close");
client.println();
uint8_t *fbBuf = fb->buf;

size_t fbLen = fb->len;
for (size_t n=0;n<fbLen;n=n+1024) {
if (n+1024<fbLen) {
client.write(fbBuf, 1024);
fbBuf += 1024;
else if (fbLen%1024>0) {
size_t remainder = fbLen%1024;
client.write(fbBuf, remainder);
esp_camera_fb_return(fb);
else {
//ANN:1
client.println("HTTP/1.1 200 OK");
client.println("Access-Control-Allow-Headers: Origin, X-
Requested-With, Content-Type, Accept");
client.println("Access-Control-Allow-Methods:
GET,POST,PUT,DELETE,OPTIONS");
client.println("Content-Type: text/html; charset=utf-8");
client.println("Access-Control-Allow-Origin: *");
client.println("Connection: close");
client.println();
String Data="";
if (cmd!="")
Data = Feedback;
else {
Data = String((const char *)INDEX_HTML);

}
int Index;
for (Index = 0; Index < Data.length(); Index = Index+1000) {
client.print(Data.substring(Index, Index+1000));
client.println();
Feedback="";
break;
} else {
currentLine = "";
else if (c != '\r') {
currentLine += c;
if ((currentLine.indexOf("/?")!=-1)&&(currentLine.indexOf("
HTTP")!=-1)) {
if (Command.indexOf("stop")!=-1) {
client.println();
client.println();
client.stop();
currentLine="";
Feedback="";
ExecuteCommand();
}
}
delay(1);
client.stop();
void getCommand(char c){
if (c=='?') ReceiveState=1;
if ((c==' ')||(c=='\r')||(c=='\n')) ReceiveState=0;
if (ReceiveState==1) {
Command=Command+String(c);
if (c=='=') cmdState=0;
if (c==';') strState++;
if ((cmdState==1)&&((c!='?')||(questionstate==1))) cmd=cmd+String(c);
if ((cmdState==0)&&(strState==1)&&((c!='=')||(equalstate==1)))
P1=P1+String(c);
if ((cmdState==0)&&(strState==2)&&(c!=';')) P2=P2+String(c);
if ((cmdState==0)&&(strState>=9)&&((c!=';')||(semicolonstate==1)))
P9=P9+String(c);
if (c=='?') questionstate=1;
if (c=='=') equalstate=1;
if ((strState>=9)&&(c==';')) semicolonstate=1;
index_OCV_ColorTrack
/**********
This include file, index_OCV_ColorTrack.h, the Client, is an intoduction

of OpenCV.js to the ESP32 Camera environment. The Client was
developed and written by Andrew R. Sass. Permission to reproduce the

index_OCV_ColorTrack.h file is granted free of charge if this
entire copyright notice is included in all copies of the

index_OCV_ColorTrack.h file.
Complete instructions at https://RandomNerdTutorials.com/esp32-cam-

opencv-js-color-detection-tracking/
***********/
static const char PROGMEM INDEX_HTML[] = R"rawliteral(
<!DOCTYPE html>
<html>
<head>
<title>ESP32-CAMERA COLOR DETECTION</title>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width,initial-scale=1">

<script async src=" https://docs.opencv.org/master/opencv.js"

type="text/javascript"></script>
</head>
<style>
html {
font-family: Arial, Helvetica, sans-serif;

}
body {
background-color: #F7F7F2;
margin: 0px;
h1 {
font-size: 1.6rem;
color:white;
text-align: center;
.topnav {
overflow: hidden;
background-color: #0A1128;
.main-controls{
padding-top: 5px;
h2 {
color: #0A1128;
font-size: 1rem;
.section {
margin: 2px;
padding: 10px;
.column{
float: left;
width: 50%
}
table {
margin: 0;
width: 90%;
border-collapse: collapse;
th{
text-align: center;
.row{
margin-right:50px;
margin-left:50px;
#colorDetect{
border: none;
color: #FEFCFB;
padding: 15px;
text-align: center;
display: inline-block;
font-size: 16px;
border-radius: 4px;
#restart{
border: none;
color: #FEFCFB;
background-color: #7B0828;
padding: 15px;
text-align: center;
font-size: 16px;
border-radius: 4px;
button{
border: none;
color: #FEFCFB;
padding: 10px;
text-align: center;
border-radius: 4px;
</style>
<body>
<div class="topnav">
<h1>ESP32-CAM Color Detection and Tracking</h1>
</div>
<div class="main-controls">
<table>
<tr>
<td><center><input type="button" id="colorDetect"

value="COLOR DETECTION"></center></td>
<td><center><input type="button" id="restart" value="RESET

BOARD"></center></td>
</tr>
</table>
</div>
<div class="container">
<div class = "row">
<div class = "column">
<div class="section">
<div class ="video-container">
<h2>Video Streaming</h2>
<center><img id="ShowImage" src=""

style="display:none"></center>
<center><canvas id="canvas"
style="display:none"></canvas></center>
</div>
</div>
<table>
<tr>
<td>Quality</td>
<td><input type="range" id="quality" min="10" max="63"

value="10"></td>
</tr>
<tr>
<td>Brightness</td>
<td><input type="range" id="brightness" min="-2" max="2"

value="0"></td>
</tr>
<tr>
<td>Contrast</td>
<td><input type="range" id="contrast" min="-2" max="2"

value="0"></td>
</tr>
</table>
</div>

<h2>RGB Color Trackbars</h2>
<table>
<tr>
<td>R min:   <span
id="RMINdemo"></span></td>
<td><input type="range" id="rmin" min="0" max="255"

value="0" class = "slider"></td>
<td>R max:   <span
id="RMAXdemo"></span></td>
<td><input type="range" id="rmax" min="0" max="255"

</tr>
<tr>
<td>G min:   <span
id="GMINdemo"></span></td>
<td><input type="range" id="gmin" min="0" max="255"

<td>G max:   <span
id="GMAXdemo"></span></td>
<td><input type="range" id="gmax" min="0" max="255"

</tr>
<tr>
<td>B min:   <span id
="BMINdemo"></span></td>
<td><input type="range" id="bmin" min="0" max="255"
value="0" class = "slider"> </td>
<td>B max:   <span
id="BMAXdemo"></span></td>
<td> <input type="range" id="bmax" min="0" max="255"

</tr>
</table>
</div>
<h2>Threshold Minimum-Binary Image</h2>
<table>
<tr>
<td>Minimum Threshold:   <span
id="THRESH_MINdemo"></span></td>
<td><input type="range" id="thresh_min" min="0" max="255"

</tr>
</table>
</div>

<h2>Color Probe</h2>
<table>
<tr>
<td>X probe:   <span
id="X_PROBEdemo"></span></td>
<td><input type="range" id="x_probe" min="0" max="400"

<td>Y probe:   <span
id="Y_PROBEdemo"></span></td>
<td> <input type="range" id="y_probe" min="0" max="296"

</tr>
</table>
</div>
</div> 
<div class = "column">
<h2>Image Mask</h2>
<canvas id="imageMask"></canvas>
</div>
<h2>Image Canvas</h2>
<canvas id="imageCanvas"></canvas>
</div>
<table>
<tr>
<td><button type="button" id="invertButton" class="btn

btn-primary">INVERT</button></td>
<td><button type="button" id="contourButton" class="btn

btn-primary">SHOW CONTOUR</button></td>
<td><button type="button" id="trackButton" class="btn

btn-primary">TRACKING</button></td>
</tr>
<tr>
<td>Invert: <span id="INVERTdemo"></span></td>
<td>Contour: <span id="CONTOURdemo"></span></td>
<td>Track: <span id="TRACKdemo"></span>
</td>
</tr>
</table>
</div>
<table>
<tr>
<td><strong>XCM:</strong> <span
id="XCMdemo"></span></td>
<td><strong>YCM:</strong> <span
id="YCMdemo"></span></td>
</tr>
</table>
</div>
<canvas id="textCanvas" width="480" height="180" style= "border:

1px solid #black;"></canvas>
<iframe id="ifr" style="display:none"></iframe>
<div id="message"></div>
</div>
</div> 
</div> 
</div> 


<div class="modal"></div>
<script>
var colorDetect = document.getElementById('colorDetect');
var ShowImage = document.getElementById('ShowImage');
var canvas = document.getElementById("canvas");
var context = canvas.getContext("2d");
var imageMask = document.getElementById("imageMask");
var imageMaskContext = imageMask.getContext("2d");
var imageCanvas = document.getElementById("imageCanvas");
var imageContext = imageCanvas.getContext("2d");
var txtcanvas = document.getElementById("textCanvas");
var ctx = txtcanvas.getContext("2d");
var message = document.getElementById('message');
var ifr = document.getElementById('ifr');
var myTimer;
var restartCount=0;
const modelPath = 'https://ruisantosdotme.github.io/face-api.js/weights/';
let currentStream;
let displaySize = { width:400, height: 296 }
let faceDetection;
let b_tracker = false;
let x_cm = 0;
let y_cm = 0;
let b_invert = false;
let b_contour = false;

var RMAX=50;
var RMIN=0;
var GMAX=50;
var GMIN=0;
var BMAX=50;
var BMIN=0;
var THRESH_MIN=120;
var X_PROBE=200;
var Y_PROBE=196;
var R=0;
var G=0;
var B=0;
var A=0;
colorDetect.onclick = function (event) {
clearInterval(myTimer);
myTimer = setInterval(function(){error_handle();},5000);
ShowImage.src=location.origin+'/?colorDetect='+Math.random();
//ANN:READY
var Module = {
onRuntimeInitialized(){onOpenCvReady();}
function onOpenCvReady(){
//alert("onOpenCvReady");
console.log("OpenCV IS READY!!!");
drawReadyText();
document.body.classList.remove("loading");
function error_handle() {
restartCount++;
if (restartCount<=2) {
message.innerHTML = "Get still error. <br>Restart ESP32-CAM

"+restartCount+" times.";
myTimer = setInterval(function(){colorDetect.click();},10000);
ifr.src = document.location.origin+'?restart';
else
message.innerHTML = "Get still error. <br>Please close the page and check
ESP32-CAM.";
colorDetect.style.display = "block";
ShowImage.onload = function (event) {
//alert("SHOW IMAGE");
console.log("SHOW iMAGE");
restartCount=0;
canvas.setAttribute("width", ShowImage.width);
canvas.setAttribute("height", ShowImage.height);
canvas.style.display = "block";
imageCanvas.setAttribute("width", ShowImage.width);
imageCanvas.setAttribute("height", ShowImage.height);
imageCanvas.style.display = "block";
imageMask.setAttribute("width", ShowImage.width);
imageMask.setAttribute("height", ShowImage.height);
imageMask.style.display = "block";
context.drawImage(ShowImage,0,0,ShowImage.width,ShowImage.height);
DetectImage();
restart.onclick = function (event) {
fetch(location.origin+'/?restart=stop');
quality.onclick = function (event) {
fetch(document.location.origin+'/?quality='+this.value+';stop');
brightness.onclick = function (event) {
fetch(document.location.origin+'/?brightness='+this.value+';stop');
contrast.onclick = function (event) {
fetch(document.location.origin+'/?contrast='+this.value+';stop');
async function DetectImage() {
//alert("DETECT IMAGE");
console.log("DETECT IMAGE");
/******opencv***********/
//ANN:4
let src = cv.imread(ShowImage);
arows = src.rows;
acols = src.cols;
aarea = arows*acols;
adepth = src.depth();
atype = src.type();
achannels = src.channels();
console.log("rows = " + arows);
console.log("cols = " + acols);
console.log("pic area = " + aarea);
console.log("depth = " + adepth);
console.log("type = " + atype);
console.log("channels = " + achannels);
/*******COLOR DETECT***********/
//ANN:6
var RMAXslider = document.getElementById("rmax");
var RMAXoutput = document.getElementById("RMAXdemo");
RMAXoutput.innerHTML = RMAXslider.value;
RMAXslider.oninput = function(){
RMAXoutput.innerHTML = this.value;
RMAX = parseInt(RMAXoutput.innerHTML,10);
console.log("RMAX=" + RMAX);
}
console.log("RMAX=" + RMAX);
var RMINslider = document.getElementById("rmin");
var RMINoutput = document.getElementById("RMINdemo");
RMINoutput.innerHTML = RMINslider.value;
RMINslider.oninput = function(){
RMINoutput.innerHTML = this.value;
RMIN = parseInt(RMINoutput.innerHTML,10);
console.log("RMIN=" + RMIN);
console.log("RMIN=" + RMIN);
var GMAXslider = document.getElementById("gmax");
var GMAXoutput = document.getElementById("GMAXdemo");
GMAXoutput.innerHTML = GMAXslider.value;
GMAXslider.oninput = function(){
GMAXoutput.innerHTML = this.value;
GMAX = parseInt(GMAXoutput.innerHTML,10);
console.log("GMAX=" + GMAX);
var GMINslider = document.getElementById("gmin");
var GMINoutput = document.getElementById("GMINdemo");
GMINoutput.innerHTML = GMINslider.value;
GMINslider.oninput = function(){
GMINoutput.innerHTML = this.value;
GMIN = parseInt(GMINoutput.innerHTML,10);
}
console.log("GMIN=" + GMIN);
var BMAXslider = document.getElementById("bmax");
var BMAXoutput = document.getElementById("BMAXdemo");
BMAXoutput.innerHTML = BMAXslider.value;
BMAXslider.oninput = function(){
BMAXoutput.innerHTML = this.value;
BMAX = parseInt(BMAXoutput.innerHTML,10);
console.log("BMAX=" + BMAX);
var BMINslider = document.getElementById("bmin");
var BMINoutput = document.getElementById("BMINdemo");
BMINoutput.innerHTML = BMINslider.value;
BMINslider.oninput = function(){
BMINoutput.innerHTML = this.value;
BMIN = parseInt(BMINoutput.innerHTML,10);
console.log("BMIN=" + BMIN);
var THRESH_MINslider = document.getElementById("thresh_min");
var THRESH_MINoutput = document.getElementById("THRESH_MINdemo");
THRESH_MINoutput.innerHTML = THRESH_MINslider.value;
THRESH_MINslider.oninput = function(){
THRESH_MINoutput.innerHTML = this.value;
THRESH_MIN = parseInt(THRESH_MINoutput.innerHTML,10);
}
console.log("THRESHOLD MIN=" + THRESH_MIN);
//ANN:9A
var X_PROBEslider = document.getElementById("x_probe");
var X_PROBEoutput = document.getElementById("X_PROBEdemo");
X_PROBEoutput.innerHTML = X_PROBEslider.value;
X_PROBEslider.oninput = function(){
X_PROBEoutput.innerHTML = this.value;
X_PROBE = parseInt(X_PROBEoutput.innerHTML,10);
console.log("X_PROBE=" + X_PROBE);
var Y_PROBEslider = document.getElementById("y_probe");
var Y_PROBEoutput = document.getElementById("Y_PROBEdemo");
Y_PROBEoutput.innerHTML = Y_PROBEslider.value;
Y_PROBEslider.oninput = function(){
Y_PROBEoutput.innerHTML = this.value;
Y_PROBE = parseInt(Y_PROBEoutput.innerHTML,10);
console.log("Y_PROBE=" + Y_PROBE);
document.getElementById('trackButton').onclick = function(){
b_tracker = (true && !b_tracker)
console.log("TRACKER = " + b_tracker );
var TRACKoutput = document.getElementById("TRACKdemo");
TRACKoutput.innerHTML = b_tracker;
//var XCMoutput = document.getElementById("XCMdemo");
//XCMoutput.innerHTML = x_cm;
document.getElementById('invertButton').onclick = function(){
b_invert = (true && !b_invert)
console.log("TRACKER = " + b_invert );
var INVERToutput = document.getElementById("INVERTdemo");
INVERToutput.innerHTML = b_invert;
/**/
document.getElementById('contourButton').onclick = function(){
b_contour = (true && !b_contour)
console.log("TRACKER = " + b_contour );
var CONTOURoutput = document.getElementById("CONTOURdemo");
CONTOURoutput.innerHTML = b_contour;
/**/
let tracker = 0;
var TRACKoutput = document.getElementById("TRACKdemo");
TRACKoutput.innerHTML = b_tracker;
var XCMoutput = document.getElementById("XCMdemo");
var YCMoutput = document.getElementById("YCMdemo");
XCMoutput.innerHTML = 0;
YCMoutput.innerHTML = 0;
var INVERToutput = document.getElementById("INVERTdemo");
INVERToutput.innerHTML = b_invert;
var CONTOURoutput = document.getElementById("CONTOURdemo");
CONTOURoutput.innerHTML = b_contour;
//ANN:8
let M00Array = [0,];
let orig = new cv.Mat();
let mask = new cv.Mat();
let mask1 = new cv.Mat();
let mask2 = new cv.Mat();
let contours = new cv.MatVector();
let hierarchy = new cv.Mat();
let rgbaPlanes = new cv.MatVector();
let color = new cv.Scalar(0,0,0);
clear_canvas();
orig = cv.imread(ShowImage);
cv.split(orig,rgbaPlanes); //SPLIT
let BP = rgbaPlanes.get(2); // SELECTED COLOR PLANE
let GP = rgbaPlanes.get(1);
let RP = rgbaPlanes.get(0);
cv.merge(rgbaPlanes,orig);
// BLK BLU GRN RED
let row = Y_PROBE //180//275 //225 //150 //130
let col = X_PROBE //100//10 //100 //200 //300
drawColRowText(acols,arows);
console.log("ISCONTINUOUS = " + orig.isContinuous());
//ANN:9C
R = src.data[row * src.cols * src.channels() + col * src.channels()];
G = src.data[row * src.cols * src.channels() + col * src.channels() + 1];
B = src.data[row * src.cols * src.channels() + col * src.channels() + 2];
A = src.data[row * src.cols * src.channels() + col * src.channels() + 3];
console.log("RDATA = " + R);
console.log("GDATA = " + G);
console.log("BDATA = " + B);
console.log("ADATA = " + A);
drawRGB_PROBE_Text();
//ANN:9b
//****draw probe point********

let point4 = new cv.Point(col,row);
cv.circle(src,point4,5,[255,255,255,255],2,cv.LINE_AA,0);
//****end draw probe point********
//ANN:7
let high = new cv.Mat(src.rows,src.cols,src.type(),[RMAX,GMAX,BMAX,255]);
let low = new cv.Mat(src.rows,src.cols,src.type(),[RMIN,GMIN,BMIN,0]);
cv.inRange(src,low,high,mask1);
//inRange(source image, lower limit, higher limit, destination image)
cv.threshold(mask1,mask,THRESH_MIN,255,cv.THRESH_BINARY);
//threshold(source image,destination image,threshold,255,threshold

method);
//ANN:9
if(b_invert==true){
cv.bitwise_not(mask,mask2);
/*******start contours***************/
//ANN:10
if(b_tracker == true){
try{
if(b_invert==false){
//ANN:11
cv.findContours(mask,contours,hierarchy,cv.RETR_CCOMP,cv.CHAIN_APPROX_SIMPL
E);
//findContours(source image, array of contours found, hierarchy of
contours
// if contours are inside other contours, method of contour data

retrieval,
//algorithm method)
else{
cv.findContours(mask2,contours,hierarchy,cv.RETR_CCOMP,cv.CHAIN_APPROX_SIMP
LE);
console.log("CONTOUR_SIZE = " + contours.size());
//draw contours
if(b_contour==true){
for(let i = 0; i < contours.size(); i++){
cv.drawContours(src,contours,i,[0,0,0,255],2,cv.LINE_8,hierarchy,100)
//ANN:12
let cnt;
let Moments;
let M00;
let M10;
//let x_cm;
//let y_cm;
//ANN:13
for(let k = 0; k < contours.size(); k++){
cnt = contours.get(k);
Moments = cv.moments(cnt,false);
M00Array[k] = Moments.m00;
// cnt.delete();
//ANN13A
let max_area_arg = MaxAreaArg(M00Array);
console.log("MAXAREAARG = "+max_area_arg);
//let TestArray = [0,0,0,15,4,15,2];
//let TestArray0 = [];
//let max_test_area_arg = MaxAreaArg(TestArray0);
//console.log("MAXTESTAREAARG = "+max_test_area_arg);
let ArgMaxArea = MaxAreaArg(M00Array);
if(ArgMaxArea >= 0){
cnt = contours.get(MaxAreaArg(M00Array)); //use the contour with biggest

MOO
//cnt = contours.get(54);
Moments = cv.moments(cnt,false);
M00 = Moments.m00;
M10 = Moments.m10;
M01 = Moments.m01;
x_cm = M10/M00; // 75 for circle_9.jpg

y_cm = M01/M00; // 41 for circle_9.jpg
XCMoutput.innerHTML = Math.round(x_cm);
YCMoutput.innerHTML = Math.round(y_cm);
console.log("M00 = "+M00);
console.log("XCM = "+Math.round(x_cm));
console.log("YCM = "+Math.round(y_cm));
//fetch(document.location.origin+'/?xcm='+Math.round(x_cm)+';stop');
fetch(document.location.origin+'/?cm='+Math.round(x_cm)+';'+Math.round(y_cm
)+';stop');
console.log("M00ARRAY = " + M00Array);
//ANN:14
//*****min area bounding rect*******
//let rotatedRect=cv.minAreaRect(cnt);
//let vertices = cv.RotatedRect.points(rotatedRect);
//for(let j=0;j<4;j++){
// cv.line(src,vertices[j],
// vertices[(j+1)%4],[0,0,255,255],2,cv.LINE_AA,0);
//}
//******end min area bounding rect************

//******bounding rect**********
let rect = cv.boundingRect(cnt);
let point1 = new cv.Point(rect.x,rect.y);
let point2 = new cv.Point(rect.x+rect.width,rect.y+rect.height);
cv.rectangle(src,point1,point2,[0,0,255,255],2,cv.LINE_AA,0);
//****end bounding rect**********
//****draw center point********
let point3 = new cv.Point(x_cm,y_cm);
cv.circle(src,point3,2,[0,0,255,255],2,cv.LINE_AA,0);
//****end draw center point********
}//end if(ArgMaxArea >= 0)
else{
if(ArgMaxArea==-1){
console.log("ZERO ARRAY LENGTH");
else{ //ArgMaxArea=-2
console.log("DUPLICATE MAX ARRAY-ELEMENT");
cnt.delete();
/*******end contours note cnt line one up**************/
drawXCM_YCM_Text();
}//end try
catch{
console.log("ERROR TRACKER NO CONTOUR");
clear_canvas();
drawErrorTracking_Text();
}//end b_tracking if statement
else{
XCMoutput.innerHTML = 0;
YCMoutput.innerHTML = 0;
if(b_invert==false){
cv.imshow('imageMask', mask);
else{
cv.imshow('imageMask', mask2);
//cv.imshow('imageMask', R);
cv.imshow('imageCanvas', src);
//ANN:8A
src.delete();
high.delete();
low.delete();
orig.delete();
mask1.delete();
mask2.delete();
mask.delete();
contours.delete();
hierarchy.delete();
//cnt.delete();
RP.delete();
/*******END COLOR DETECT*********/
/******end opencv***********/
setTimeout(function(){colorDetect.click();},500);
}//end detectimage
function MaxAreaArg(arr){
if (arr.length == 0) {
return -1;
}
var max = arr[0];
var maxIndex = 0;
var dupIndexCount = 0; //duplicate max elements?
if(arr[0] >= .90*aarea){
max = 0;
for (var i = 1; i < arr.length; i++) {
if (arr[i] > max && arr[i] < .99*aarea) {
maxIndex = i;
max = arr[i];
dupIndexCount = 0;
else if(arr[i]==max && arr[i]!=0){
dupIndexCount++;
if(dupIndexCount==0){
return maxIndex;
else{
return -2;
}//end MaxAreaArg
function clear_canvas(){
ctx.clearRect(0,0,txtcanvas.width,txtcanvas.height);
ctx.rect(0,0,txtcanvas.width,txtcanvas.height);
ctx.fillStyle="red";
ctx.fill();
function drawReadyText(){
ctx.fillStyle = 'black';
ctx.font = '20px serif';
ctx.fillText('OpenCV.JS READY',txtcanvas.width/4,txtcanvas.height/10);
function drawColRowText(x,y){
ctx.fillText('ImageCols='+x,0,txtcanvas.height/10);
ctx.fillText('ImageRows='+y,txtcanvas.width/2,txtcanvas.height/10);
function drawRGB_PROBE_Text(){
ctx.fillText('Rp='+R,0,2*txtcanvas.height/10);
ctx.fillText('Gp='+G,txtcanvas.width/4,2*txtcanvas.height/10);
ctx.fillText('Bp='+B,txtcanvas.width/2,2*txtcanvas.height/10);
ctx.fillText('Ap='+A,3*txtcanvas.width/4,2*txtcanvas.height/10);
function drawXCM_YCM_Text(){
ctx.fillText('XCM='+Math.round(x_cm),0,3*txtcanvas.height/10);
ctx.fillText('YCM='+Math.round(y_cm),txtcanvas.width/4,3*txtcanvas.height/1
0);
function drawErrorTracking_Text(){
ctx.fillText('ERROR TRACKING-NO CONTOUR',0,3*txtcanvas.height/10);
</script>
</body>
</html>
)rawliteral";
Penjelasan Bagaimana Kode Program Bekerja
Di <head> file HTML, kita harus menambahkan semua metadata yang diperlukan.
Judul halaman web adalah Aplikasi Firebase ESP, tetapi Anda dapat mengubahnya di baris
berikut.
<title>ESP Firebase App</title>

Baris berikut memungkinkan kita untuk menggunakan ikon fontawesome:
<link rel="stylesheet"
href="https://use.fontawesome.com/releases/v5.7.2/css/all.cs
s" integrity="sha384-
fnmOCqbTlWIlj8LyTjo7mOUStjsKC4pOpQbqyi7RrhN7udi9RwhKkMHpvLbH
G9Sr" crossorigin="anonymous">
Referensikan file style.css eksternal untuk memformat halaman HTML.
<link rel="stylesheet" type="text/css" href="style.css">
Anda harus menambahkan baris berikut untuk dapat menggunakan Firebase dengan aplikasi
Anda.
<script
src="https://www.gstatic.com/firebasejs/8.10.0/firebase-
app.js"></script>
Anda juga harus menambahkan produk Firebase yang ingin digunakan. Dalam contoh ini, kami
menggunakan Penyimpanan dan Otentikasi.
<script
auth.js"></script>
<script
storage.js"></script>
Kemudian, ganti objek firebaseConfig dengan yang Anda dapatkan dari langkah ini.
const firebaseConfig = {
apiKey: "AIzaSyA2sJiXEcOWVZqdrh348eUO3aHothDz7sE",
authDomain: "esp-firebase-demo.firebaseapp.com",
databaseURL: "https://esp-firebase-demo-default-
rtdb.europe-west1.firebasedatabase.app",
projectId: "esp-firebase-demo",
storageBucket: "esp-firebase-demo.appspot.com",
messagingSenderId: "1086300631316",
appId: "1:1086300631316:web:316410846aaeb0bb8c9201"
};
Terakhir, Firebase diinisialisasi, dan kami membuat dua variabel global storage, dan
storageRef, yang merujuk ke Firebase Storage dan referensi ke layanan penyimpanan.
// Initialize Firebase
firebase.initializeApp(firebaseConfig);
// Get a reference to the storage service, which is used to

create references in your storage bucket
var storage = firebase.storage();
// Create a storage reference from our storage service

var storageRef = storage.ref();
Kitai selesai dengan metadata. Sekarang, mari masuk ke bagian HTML yang terlihat oleh
pengguna beralih di antara tag <body> dan </body>. Kita membuat bilah "navigasi" teratas
dengan nama aplikasi kami dan ikon kecil dari fontawesome.
<div class="topnav">
<h1>ESP32-CAM Web App <i class="fas fa-camera"></i></h1>
</div>
Baris berikut membuat paragraf dengan tag gambar di mana kita akan menampilkan gambar
nanti. Tag gambar memiliki id tertentu (“img”) sehingga kita dapat merujuk ke elemen
tersebut dalam file JavaScript dengan id-nya.
<p><img id="img" width="500px"></p>

Selanjutnya, kita membuat paragraf untuk menampilkan tanggal dan waktu pengambilan
gambar. Ada tag <span> dengan id tanggal-waktu di mana kami akan menampilkan informasi
itu nanti.
<p>Last picture taken: <span id="date-time"></span></p>

Terakhir, ada tombol untuk menyegarkan halaman web (untuk menampilkan gambar baru,
jika demikian).
<button onclick="window.location.reload();">Refresh</button>
Setelah membuat perubahan yang diperlukan (memasukkan objek firebaseConfig Anda),
Anda dapat menyimpan file HTML.

MuhammadFakhriZuhair - 1903421009 - BM6B - Deteksi Dan Pelacakan Warna

Diunggah oleh

Informasi Dokumen

Judul Asli

Hak Cipta

Format Tersedia

Bagikan dokumen Ini

Bagikan atau Tanam Dokumen

Opsi Berbagi

Apakah menurut Anda dokumen ini bermanfaat?

Apakah konten ini tidak pantas?

Hak Cipta:

Format Tersedia

MuhammadFakhriZuhair - 1903421009 - BM6B - Deteksi Dan Pelacakan Warna

Diunggah oleh

Hak Cipta:

Format Tersedia

ESP32-CAM WEB SERVER DENGAN OPENCV.

Disusun untuk memenuhi Nilai Mata Kuliah

Muhammad Fakhri Zuhair (1903421009)

William Immanuel (1903421042)

PROGRAM STUDI BROADBAND MULTIMEDIA

JURUSAN TEKNIK ELEKTRO

POLITEKNIK NEGERI JAKARTA

II. Menyiapkan Kode VS Proyek Aplikasi Web Firebase

The include file, index_OCV_ColorTrack.h, the Client, is an intoduction of

developed and written by Andrew R. Sass. Permission to reproduce the

entire copyright notice is included in all copies of the

Complete instructions at https://RandomNerdTutorials.com/esp32-cam-

Permission is hereby granted, free of charge, to any person obtaining a

// Replace with your network credentials

const char* ssid = "HAYUK!";

const char* password = "bismillahkuy";

//Serial.println("cmd= "+cmd+" ,P1= "+P1+" ,P2= "+P2+" ,P3= "+P3+" ,P4=

long int StartTime=millis();

while (WiFi.status() != WL_CONNECTED)

if ((StartTime+5000) < millis()) break;

int XcmVal = P1.toInt();

int YcmVal = P2.toInt();

Serial.println("cmd= "+cmd+" ,VALXCM= "+XcmVal);

Serial.println("cmd= "+cmd+" ,VALYCM= "+YcmVal);

int val = P1.toInt();

int val = P1.toInt();

int val = P1.toInt();

Feedback="Command is not defined.";

//init with high specs to pre-allocate larger buffers

config.jpeg_quality = 10; //0-63 lower number means higher quality

config.jpeg_quality = 12; //0-63 lower number means higher quality

esp_err_t err = esp_camera_init(&config);

Serial.printf("Camera init failed with error 0x%x", err);

//drop down frame size for higher initial frame rate

long int StartTime=millis();

while (WiFi.status() != WL_CONNECTED) {

Serial.print("ESP IP Address: http://");

WiFiClient client = server.available();

String currentLine = "";

Serial.println("Camera capture failed");

client.println("HTTP/1.1 200 OK");

client.println("Content-Length: " + String(fb->len));

uint8_t *fbBuf = fb->buf;

for (size_t n=0;n<fbLen;n=n+1024) {

size_t remainder = fbLen%1024;

client.println("HTTP/1.1 200 OK");

client.println("Content-Type: text/html; charset=utf-8");

Data = String((const char *)INDEX_HTML);

for (Index = 0; Index < Data.length(); Index = Index+1000) {

void getCommand(char c){

if ((c==' ')||(c=='\r')||(c=='\n')) ReceiveState=0;

This include file, index_OCV_ColorTrack.h, the Client, is an intoduction

developed and written by Andrew R. Sass. Permission to reproduce the

entire copyright notice is included in all copies of the

Complete instructions at https://RandomNerdTutorials.com/esp32-cam-

static const char PROGMEM INDEX_HTML[] = R"rawliteral(

<title>ESP32-CAMERA COLOR DETECTION</title>

<meta name="viewport" content="width=device-width,initial-scale=1">

<script async src=" https://docs.opencv.org/master/opencv.js"

font-family: Arial, Helvetica, sans-serif;

<h1>ESP32-CAM Color Detection and Tracking</h1>

<td><center><input type="button" id="colorDetect"

<td><center><input type="button" id="restart" value="RESET