ESP32-CAM(ardunio)拍照auto take picture、串流stream及物件辨識object detection

ESP32-CAM是ESP32為主版並外掛相機,ESP32本身就已經內建WIFI跟藍芽,而ESP32-CAM竟然還加了相機給你,而這樣的東西竟然不到三百台幣,這樣的好東西當然要發揮他最大效能。

我的ESP32實做書籍:我出書了 ESP32 物聯網專題
博客來網址:https://www.books.com.tw/products/0010901195

ESP32-CAM is an extended dev board based on ESP32 which already built-in WIFI and BT4.0. Additionally, ESP32-CAM combines ESP32 and OV2640, and only costs about US 7$.

ESP32介紹: 第一篇 ESP32 Arduino開發環境架設
ESP32-CAM介紹: ESP32-CAM (arduino)影像伺服器及臉部辨識教學原始檔Video Stream Server

本次要介紹的主題是當作門禁,透過人體感測器,感測到活動時自動拍照,並存在SD卡中。
This post introduces how to use ESP32-CAM and PIR(SR501) as a surveillance camera and will take pictures to save to sd card when people active.


ESP32-CAM環境架設(ESP32-CAM Arduino IDE environment setup)

關於環境架設,請參考這篇前作ESP32-CAM介紹: ESP32-CAM (arduino)影像伺服器及臉部辨識教學原始檔Video Stream Server,依照步驟完成環境安裝並測試。
About ESP32-CAM arduino IDE environment setup, you can follow this post.

材料準備及接線(Material prepare )

本次須準備的材料包括

ESP32-CAM dev board主版,這次本人使用廣角鏡頭,視野真的變得很廣
露天搜尋: https://goods.ruten.com.tw/item/show?21910115309507

ESP32-CAM dev board主版
廣角鏡頭畫面

USB-TTL(本人使用Prolific USB-to-Serial TTL: ESP32-CAM使用Prolific USB-to-Serial TTL ,但個人不推這款)
人體感測器(SR501)

ESP32-CAM材料連接方式

先依照上圖將USB TTL與ESP32-CAM連接,紅->5V、黑->GND、白->U0T、綠->U0R。connect USB TTL to ESP32-CAM by this table.

USB TTLESP32-CAM
5V
GND
U0T
U0R
USB TTL與ESP32-CAM連接腳位

將人體感測器與ESP32-CAM連接,VCC->5V或3.3V、OUT->IO13、GND->GND。connect SR501 to ESP32-CAM by this table.

SR-501ESP32-CAM
VCC5V或3.3V
OUTIO13
GNDGND
人體感測器與ESP32-CAM連接腳位

另外準備一條母母杜邦線,燒錄時,需要將ESP32-CAM的GND->IO0對接,燒錄完後拆除。you also need a line for connecting GND and IO0 while uploading.

程式撰寫(Source code)

本程式修改於(My codes mainly referred to this post): https://github.com/v12345vtm/CameraWebserver2SD
修改的地方包括My codes have those advantages

刪除所有臉部辨識部份,主要是效能差,且無法記憶,重開機就須重新設定
Delete all face regnition function
將程式縮小到可以OTA的狀態
reduce codes for OTA
刪除不必要的列印
delete redundant print codes
註解相關程式說明
remark codes
加入拍照函數:getPic
add new function : getPic to take pictures
完成的內容以放置git,放在本文最下方(The source codes were shown below)

程式上傳與測試(Upload and test)

  1. 複製下方的程式碼 (copy source code to your arduino IDE)
  2. 上傳程式前,先將GND與IO0對接 (connect GND and IO0 while uploading)
  3. 選項請依照下方規定,除了port之外 (upload options follow this picture)
arduino IDE 開發板型號設定
  1. 上傳時,出現「connecting……….………..」時,按下reset (press reset whlie “connecting……….………..”)
上傳中畫面
  1. 等候更新成功訊息出現 (done)
上傳完成畫面
  1. 查看Serial視窗出現的內容 (message in serial monitor )
上傳完成後Serial視窗內容
  1. 每當偵測到人體活動時,就會直接拍照,並存放在SD卡內,並依序編號 (pictures will auto take while people active in front of the camera)
自動拍照至SD卡中
  1. 若要直接查看影像,可透過http://ip,直接查看串流內容 (you can also watch the video stream in http://ip)
http://ip查看串流畫面

目前研究到利用.net讀取mjepg stream資料,並利用雲端分析物件,但這又是一個很長的故事。
And now, I try to get ESP32-CAM stream in .net framework and do some object recognition function.

教學、研習邀約請洽LineID:youjunjer
可開課程: 貨架電子紙標籤,IoT Arduino esp32 教學,esp32com 影像辨識pos機,藍牙beacon偵測,unity ARVR互動,AI臉部辨識,機器學習。

//Change Tools/Partition/Scheme to Minimal SPIFFS (Large APPS with OTA)
//IR sensor set at GPIO13
//Must Insert SD card
#include "esp_camera.h"
#include <WiFi.h>
#include "esp_timer.h"
#include "img_converters.h"
#include "Arduino.h"
#include "fb_gfx.h"
#include "fd_forward.h"
#include "fr_forward.h"
#include "FS.h" //sd card esp32
#include "SD_MMC.h" //sd card esp32
#include "soc/soc.h" //disable brownour problems
#include "soc/rtc_cntl_reg.h" //disable brownour problems
#include <WiFi.h> //used for internet time
#include "dl_lib.h"
#include "esp_http_server.h"
#define CAMERA_MODEL_AI_THINKER
#define PART_BOUNDARY "123456789000000000000987654321"
#define PWDN_GPIO_NUM 32
#define RESET_GPIO_NUM -1
#define XCLK_GPIO_NUM 0
#define SIOD_GPIO_NUM 26
#define SIOC_GPIO_NUM 27
#define Y9_GPIO_NUM 35
#define Y8_GPIO_NUM 34
#define Y7_GPIO_NUM 39
#define Y6_GPIO_NUM 36
#define Y5_GPIO_NUM 21
#define Y4_GPIO_NUM 19
#define Y3_GPIO_NUM 18
#define Y2_GPIO_NUM 5
#define VSYNC_GPIO_NUM 25
#define HREF_GPIO_NUM 23
#define PCLK_GPIO_NUM 22
static const char* _STREAM_CONTENT_TYPE = "multipart/x-mixed-replace;boundary=" PART_BOUNDARY;
static const char* _STREAM_BOUNDARY = "\r\n--" PART_BOUNDARY "\r\n";
static const char* _STREAM_PART = "Content-Type: image/jpeg\r\nContent-Length: %u\r\n\r\n";
typedef struct {
size_t size; //number of values used for filtering
size_t index; //current value index
size_t count; //value count
int sum;
int * values; //array to be filled with values
} ra_filter_t;
static ra_filter_t ra_filter;
httpd_handle_t stream_httpd = NULL;
httpd_handle_t camera_httpd = NULL;
static mtmn_config_t mtmn_config = {0};
static int8_t recognition_enabled = 0;
static int8_t is_enrolling = 0;
static face_id_list id_list = {0};
typedef struct {
httpd_req_t *req;
size_t len;
} jpg_chunking_t;
const char* ssid = "YourSSID"; //remember change your SSID
const char* password = "YourPW";// remember change your SSID PW
static int8_t detection_enabled = 0;
//Sd card function--------------------
void listDir(fs::FS &fs, const char * dirname, uint8_t levels) {
Serial.printf("Listing directory: %s\n", dirname);
File root = fs.open(dirname);
if (!root) {
Serial.println("Failed to open directory");
return;
}
if (!root.isDirectory()) {
Serial.println("Not a directory");
return;
}
File file = root.openNextFile();
while (file) {
if (file.isDirectory()) {
Serial.print(" DIR : ");
Serial.println(file.name());
if (levels) {
listDir(fs, file.name(), levels - 1);
}
} else {
Serial.print(" FILE: ");
Serial.print(file.name());
Serial.print(" SIZE: ");
Serial.println(file.size());
}
file = root.openNextFile();
}
}
void createDir(fs::FS &fs, const char * path) {
Serial.printf("Creating Dir: %s\n", path);
if (fs.mkdir(path)) {
Serial.println("Dir created");
} else {
Serial.println("mkdir failed");
}
}
void removeDir(fs::FS &fs, const char * path) {
Serial.printf("Removing Dir: %s\n", path);
if (fs.rmdir(path)) {
Serial.println("Dir removed");
} else {
Serial.println("rmdir failed");
}
}
void readFile(fs::FS &fs, const char * path) {
Serial.printf("Reading file: %s\n", path);
File file = fs.open(path);
if (!file) {
Serial.println("Failed to open file for reading");
return;
}
Serial.print("Read from file: ");
while (file.available()) {
Serial.write(file.read());
}
}
void writeFile(fs::FS &fs, const char * path, const char * message) {
Serial.printf("Writing file: %s\n", path);
File file = fs.open(path, FILE_WRITE);
if (!file) {
Serial.println("Failed to open file for writing");
return;
}
//fwrite(fb->buf, 1, fb->len, file);
if (file.print(message)) {
Serial.println("File written");
} else {
Serial.println("Write failed");
}
}
void appendFile(fs::FS &fs, const char * path, const char * message) {
Serial.printf("Appending to file: %s\n", path);
File file = fs.open(path, FILE_APPEND);
if (!file) {
Serial.println("Failed to open file for appending");
return;
}
if (file.print(message)) {
Serial.println("Message appended");
} else {
Serial.println("Append failed");
}
}
void renameFile(fs::FS &fs, const char * path1, const char * path2) {
Serial.printf("Renaming file %s to %s\n", path1, path2);
if (fs.rename(path1, path2)) {
Serial.println("File renamed");
} else {
Serial.println("Rename failed");
}
}
void deleteFile(fs::FS &fs, const char * path) {
Serial.printf("Deleting file: %s\n", path);
if (fs.remove(path)) {
Serial.println("File deleted");
} else {
Serial.println("Delete failed");
}
}
void testFileIO(fs::FS &fs, const char * path) {
File file = fs.open(path);
static uint8_t buf[512];
size_t len = 0;
uint32_t start = millis();
uint32_t end = start;
if (file) {
len = file.size();
size_t flen = len;
start = millis();
while (len) {
size_t toRead = len;
if (toRead > 512) {
toRead = 512;
}
file.read(buf, toRead);
len -= toRead;
}
end = millis() - start;
Serial.printf("%u bytes read for %u ms\n", flen, end);
file.close();
} else {
Serial.println("Failed to open file for reading");
}
file = fs.open(path, FILE_WRITE);
if (!file) {
Serial.println("Failed to open file for writing");
return;
}
size_t i;
start = millis();
for (i = 0; i < 2048; i++) {
file.write(buf, 512);
}
end = millis() - start;
Serial.printf("%u bytes written for %u ms\n", 2048 * 512, end);
file.close();
}
//Sd card function--------------------
static size_t jpg_encode_stream(void * arg, size_t index, const void* data, size_t len) {
jpg_chunking_t *j = (jpg_chunking_t *)arg;
if (!index) {
j->len = 0;
Serial.println("lijn459-jpg-encode-stream !index (met FACE recog. passeren we hier");
}
if (httpd_resp_send_chunk(j->req, (const char *)data, len) != ESP_OK) {
Serial.println("lijn462 httpsendchunk");
return 0;
}
j->len += len;
return len;
}
static int rgb_printf(dl_matrix3du_t *image_matrix, uint32_t color, const char *format, ...) {
char loc_buf[64];
char * temp = loc_buf;
int len;
va_list arg;
va_list copy;
va_start(arg, format);
va_copy(copy, arg);
len = vsnprintf(loc_buf, sizeof(loc_buf), format, arg);
va_end(copy);
if (len >= sizeof(loc_buf)) {
temp = (char*)malloc(len + 1);
if (temp == NULL) {
return 0;
}
}
vsnprintf(temp, len + 1, format, arg);
va_end(arg);
rgb_print(image_matrix, color, temp);
if (len > 64) {
free(temp);
}
return len;
}
static ra_filter_t * ra_filter_init(ra_filter_t * filter, size_t sample_size) {
memset(filter, 0, sizeof(ra_filter_t));
filter->values = (int *)malloc(sample_size * sizeof(int));
if (!filter->values) {
return NULL;
}
memset(filter->values, 0, sample_size * sizeof(int));
filter->size = sample_size;
return filter;
}
static int ra_filter_run(ra_filter_t * filter, int value) {
if (!filter->values) {
return value;
}
filter->sum -= filter->values[filter->index];
filter->values[filter->index] = value;
filter->sum += filter->values[filter->index];
filter->index++;
filter->index = filter->index % filter->size;
if (filter->count < filter->size) {
filter->count++;
}
return filter->sum / filter->count;
}
static void rgb_print(dl_matrix3du_t *image_matrix, uint32_t color, const char * str) {
fb_data_t fb;
fb.width = image_matrix->w;
fb.height = image_matrix->h;
fb.data = image_matrix->item;
fb.bytes_per_pixel = 3;
fb.format = FB_BGR888;
fb_gfx_print(&fb, (fb.width - (strlen(str) * 14)) / 2, 10, color, str);
}
//HTML-----------------------------------
static const char PROGMEM INDEX2_HTML[] = R"rawliteral(
<!doctype html>
<html>
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width,initial-scale=1">
<title>ESP32-CAM Stream and Save to SD</title>
<style>
body{font-family:Arial,Helvetica,sans-serif;background:#181818;color:#EFEFEF;font-size:16px}h2{font-size:18px}section.main{display:flex}#menu,section.main{flex-direction:column}#menu{display:none;flex-wrap:nowrap;min-width:340px;background:#363636;padding:8px;border-radius:4px;margin-top:-10px;margin-right:10px}#content{display:flex;flex-wrap:wrap;align-items:stretch}figure{padding:0;margin:0;-webkit-margin-before:0;margin-block-start:0;-webkit-margin-after:0;margin-block-end:0;-webkit-margin-start:0;margin-inline-start:0;-webkit-margin-end:0;margin-inline-end:0}figure img{display:block;width:100%;height:auto;border-radius:4px;margin-top:8px}@media (min-width: 800px) and (orientation:landscape){#content{display:flex;flex-wrap:nowrap;align-items:stretch}figure img{display:block;max-width:100%;max-height:calc(100vh - 40px);width:auto;height:auto}figure{padding:0;margin:0;-webkit-margin-before:0;margin-block-start:0;-webkit-margin-after:0;margin-block-end:0;-webkit-margin-start:0;margin-inline-start:0;-webkit-margin-end:0;margin-inline-end:0}}section#buttons{display:flex;flex-wrap:nowrap;justify-content:space-between}#nav-toggle{cursor:pointer;display:block}#nav-toggle-cb{outline:0;opacity:0;width:0;height:0}#nav-toggle-cb:checked+#menu{display:flex}.input-group{display:flex;flex-wrap:nowrap;line-height:22px;margin:5px 0}.input-group>label{display:inline-block;padding-right:10px;min-width:47%}.input-group input,.input-group select{flex-grow:1}.range-max,.range-min{display:inline-block;padding:0 5px}button{display:block;margin:5px;padding:0 12px;border:0;line-height:28px;cursor:pointer;color:#fff;background:#ff3034;border-radius:5px;font-size:16px;outline:0}button:hover{background:#ff494d}button:active{background:#f21c21}button.disabled{cursor:default;background:#a0a0a0}input[type=range]{-webkit-appearance:none;width:100%;height:22px;background:#363636;cursor:pointer;margin:0}input[type=range]:focus{outline:0}input[type=range]::-webkit-slider-runnable-track{width:100%;height:2px;cursor:pointer;background:#EFEFEF;border-radius:0;border:0 solid #EFEFEF}input[type=range]::-webkit-slider-thumb{border:1px solid rgba(0,0,30,0);height:22px;width:22px;border-radius:50px;background:#ff3034;cursor:pointer;-webkit-appearance:none;margin-top:-11.5px}input[type=range]:focus::-webkit-slider-runnable-track{background:#EFEFEF}input[type=range]::-moz-range-track{width:100%;height:2px;cursor:pointer;background:#EFEFEF;border-radius:0;border:0 solid #EFEFEF}input[type=range]::-moz-range-thumb{border:1px solid rgba(0,0,30,0);height:22px;width:22px;border-radius:50px;background:#ff3034;cursor:pointer}input[type=range]::-ms-track{width:100%;height:2px;cursor:pointer;background:0 0;border-color:transparent;color:transparent}input[type=range]::-ms-fill-lower{background:#EFEFEF;border:0 solid #EFEFEF;border-radius:0}input[type=range]::-ms-fill-upper{background:#EFEFEF;border:0 solid #EFEFEF;border-radius:0}input[type=range]::-ms-thumb{border:1px solid rgba(0,0,30,0);height:22px;width:22px;border-radius:50px;background:#ff3034;cursor:pointer;height:2px}input[type=range]:focus::-ms-fill-lower{background:#EFEFEF}input[type=range]:focus::-ms-fill-upper{background:#363636}.switch{display:block;position:relative;line-height:22px;font-size:16px;height:22px}.switch input{outline:0;opacity:0;width:0;height:0}.slider{width:50px;height:22px;border-radius:22px;cursor:pointer;background-color:grey}.slider,.slider:before{display:inline-block;transition:.4s}.slider:before{position:relative;content:"";border-radius:50%;height:16px;width:16px;left:4px;top:3px;background-color:#fff}input:checked+.slider{background-color:#ff3034}input:checked+.slider:before{-webkit-transform:translateX(26px);transform:translateX(26px)}select{border:1px solid #363636;font-size:14px;height:22px;outline:0;border-radius:5px}.image-container{position:relative;min-width:160px}.close{position:absolute;right:5px;top:5px;background:#ff3034;width:16px;height:16px;border-radius:100px;color:#fff;text-align:center;line-height:18px;cursor:pointer}.hidden{display:none}
</style>
</head>
<body>
<section class="main">
<div id="logo">
<label for="nav-toggle-cb" id="nav-toggle">&#9776;&nbsp;&nbsp;Toggle settings</label>
</div>
<div id="content">
<div id="sidebar">
<input type="checkbox" id="nav-toggle-cb" checked="checked">
<nav id="menu">
<div class="input-group" id="framesize-group">
<label for="framesize">Resolution</label>
<select id="framesize" class="default-action">
<option value="10">UXGA(1600x1200)</option>
<option value="9">SXGA(1280x1024)</option>
<option value="8">XGA(1024x768)</option>
<option value="7">SVGA(800x600)</option>
<option value="6">VGA(640x480)</option>
<option value="5" selected="selected">CIF(400x296)</option>
<option value="4">QVGA(320x240)</option>
<option value="3">HQVGA(240x176)</option>
<option value="0">QQVGA(160x120)</option>
</select>
</div>
<div class="input-group" id="quality-group">
<label for="quality">Quality</label>
<div class="range-min">10</div>
<input type="range" id="quality" min="10" max="63" value="10" class="default-action">
<div class="range-max">63</div>
</div>
<div class="input-group" id="brightness-group">
<label for="brightness">Brightness</label>
<div class="range-min">-2</div>
<input type="range" id="brightness" min="-2" max="2" value="0" class="default-action">
<div class="range-max">2</div>
</div>
<div class="input-group" id="contrast-group">
<label for="contrast">Contrast</label>
<div class="range-min">-2</div>
<input type="range" id="contrast" min="-2" max="2" value="0" class="default-action">
<div class="range-max">2</div>
</div>
<div class="input-group" id="saturation-group">
<label for="saturation">Saturation</label>
<div class="range-min">-2</div>
<input type="range" id="saturation" min="-2" max="2" value="0" class="default-action">
<div class="range-max">2</div>
</div>
<div class="input-group" id="special_effect-group">
<label for="special_effect">Special Effect</label>
<select id="special_effect" class="default-action">
<option value="0" selected="selected">No Effect</option>
<option value="1">Negative</option>
<option value="2">Grayscale</option>
<option value="3">Red Tint</option>
<option value="4">Green Tint</option>
<option value="5">Blue Tint</option>
<option value="6">Sepia</option>
</select>
</div>
<div class="input-group" id="awb-group">
<label for="awb">AWB</label>
<div class="switch">
<input id="awb" type="checkbox" class="default-action" checked="checked">
<label class="slider" for="awb"></label>
</div>
</div>
<div class="input-group" id="awb_gain-group">
<label for="awb_gain">AWB Gain</label>
<div class="switch">
<input id="awb_gain" type="checkbox" class="default-action" checked="checked">
<label class="slider" for="awb_gain"></label>
</div>
</div>
<div class="input-group" id="wb_mode-group">
<label for="wb_mode">WB Mode</label>
<select id="wb_mode" class="default-action">
<option value="0" selected="selected">Auto</option>
<option value="1">Sunny</option>
<option value="2">Cloudy</option>
<option value="3">Office</option>
<option value="4">Home</option>
</select>
</div>
<div class="input-group" id="aec-group">
<label for="aec">AEC SENSOR</label>
<div class="switch">
<input id="aec" type="checkbox" class="default-action" checked="checked">
<label class="slider" for="aec"></label>
</div>
</div>
<div class="input-group" id="aec2-group">
<label for="aec2">AEC DSP</label>
<div class="switch">
<input id="aec2" type="checkbox" class="default-action" checked="checked">
<label class="slider" for="aec2"></label>
</div>
</div>
<div class="input-group" id="ae_level-group">
<label for="ae_level">AE Level</label>
<div class="range-min">-2</div>
<input type="range" id="ae_level" min="-2" max="2" value="0" class="default-action">
<div class="range-max">2</div>
</div>
<div class="input-group" id="aec_value-group">
<label for="aec_value">Exposure</label>
<div class="range-min">0</div>
<input type="range" id="aec_value" min="0" max="1200" value="204" class="default-action">
<div class="range-max">1200</div>
</div>
<div class="input-group" id="agc-group">
<label for="agc">AGC</label>
<div class="switch">
<input id="agc" type="checkbox" class="default-action" checked="checked">
<label class="slider" for="agc"></label>
</div>
</div>
<div class="input-group hidden" id="agc_gain-group">
<label for="agc_gain">Gain</label>
<div class="range-min">1x</div>
<input type="range" id="agc_gain" min="0" max="30" value="5" class="default-action">
<div class="range-max">31x</div>
</div>
<div class="input-group" id="gainceiling-group">
<label for="gainceiling">Gain Ceiling</label>
<div class="range-min">2x</div>
<input type="range" id="gainceiling" min="0" max="6" value="0" class="default-action">
<div class="range-max">128x</div>
</div>
<div class="input-group" id="bpc-group">
<label for="bpc">BPC</label>
<div class="switch">
<input id="bpc" type="checkbox" class="default-action">
<label class="slider" for="bpc"></label>
</div>
</div>
<div class="input-group" id="wpc-group">
<label for="wpc">WPC</label>
<div class="switch">
<input id="wpc" type="checkbox" class="default-action" checked="checked">
<label class="slider" for="wpc"></label>
</div>
</div>
<div class="input-group" id="raw_gma-group">
<label for="raw_gma">Raw GMA</label>
<div class="switch">
<input id="raw_gma" type="checkbox" class="default-action" checked="checked">
<label class="slider" for="raw_gma"></label>
</div>
</div>
<div class="input-group" id="lenc-group">
<label for="lenc">Lens Correction</label>
<div class="switch">
<input id="lenc" type="checkbox" class="default-action" checked="checked">
<label class="slider" for="lenc"></label>
</div>
</div>
<div class="input-group" id="hmirror-group">
<label for="hmirror">H-Mirror</label>
<div class="switch">
<input id="hmirror" type="checkbox" class="default-action" checked="checked">
<label class="slider" for="hmirror"></label>
</div>
</div>
<div class="input-group" id="vflip-group">
<label for="vflip">V-Flip</label>
<div class="switch">
<input id="vflip" type="checkbox" class="default-action" checked="checked">
<label class="slider" for="vflip"></label>
</div>
</div>
<div class="input-group" id="dcw-group">
<label for="dcw">DCW (Downsize EN)</label>
<div class="switch">
<input id="dcw" type="checkbox" class="default-action" checked="checked">
<label class="slider" for="dcw"></label>
</div>
</div>
<div class="input-group" id="colorbar-group">
<label for="colorbar">Color Bar</label>
<div class="switch">
<input id="colorbar" type="checkbox" class="default-action">
<label class="slider" for="colorbar"></label>
</div>
</div>
<section id="buttons">
<button id="get-still">Get Still</button>
<button id="toggle-stream">Start Stream</button>
</section>
</nav>
</div>
<figure>
<div id="stream-container" class="image-container hidden">
<div class="close" id="close-stream">×</div>
<img id="stream" src="">
</div>
</figure>
</div>
</section>
<script>
document.addEventListener('DOMContentLoaded',function(){function b(B){let C;switch(B.type){case'checkbox':C=B.checked?1:0;break;case'range':case'select-one':C=B.value;break;case'button':case'submit':C='1';break;default:return;}const D=`${c}/control?var=${B.id}&val=${C}`;fetch(D).then(E=>{console.log(`request to ${D} finished, status: ${E.status}`)})}var c=document.location.origin;const e=B=>{B.classList.add('hidden')},f=B=>{B.classList.remove('hidden')},g=B=>{B.classList.add('disabled'),B.disabled=!0},h=B=>{B.classList.remove('disabled'),B.disabled=!1},i=(B,C,D)=>{D=!(null!=D)||D;let E;'checkbox'===B.type?(E=B.checked,C=!!C,B.checked=C):(E=B.value,B.value=C),D&&E!==C?b(B):!D&&('aec'===B.id?C?e(v):f(v):'agc'===B.id?C?(f(t),e(s)):(e(t),f(s)):'awb_gain'===B.id?C?f(x):e(x):'face_recognize'===B.id&&(C?h(n):g(n)))};document.querySelectorAll('.close').forEach(B=>{B.onclick=()=>{e(B.parentNode)}}),fetch(`${c}/status`).then(function(B){return B.json()}).then(function(B){document.querySelectorAll('.default-action').forEach(C=>{i(C,B[C.id],!1)})});const j=document.getElementById('stream'),k=document.getElementById('stream-container'),l=document.getElementById('get-still'),m=document.getElementById('toggle-stream'),n=document.getElementById('face_enroll'),o=document.getElementById('close-stream'),p=()=>{window.stop(),m.innerHTML='Start Stream'},q=()=>{j.src=`${c+':9601'}/stream`,f(k),m.innerHTML='Stop Stream'};l.onclick=()=>{p(),j.src=`${c}/capture?_cb=${Date.now()}`,f(k)},o.onclick=()=>{p(),e(k)},m.onclick=()=>{const B='Stop Stream'===m.innerHTML;B?p():q()},n.onclick=()=>{b(n)},document.querySelectorAll('.default-action').forEach(B=>{B.onchange=()=>b(B)});const r=document.getElementById('agc'),s=document.getElementById('agc_gain-group'),t=document.getElementById('gainceiling-group');r.onchange=()=>{b(r),r.checked?(f(t),e(s)):(e(t),f(s))};const u=document.getElementById('aec'),v=document.getElementById('aec_value-group');u.onchange=()=>{b(u),u.checked?e(v):f(v)};const w=document.getElementById('awb_gain'),x=document.getElementById('wb_mode-group');w.onchange=()=>{b(w),w.checked?f(x):e(x)};const y=document.getElementById('face_detect'),z=document.getElementById('face_recognize'),A=document.getElementById('framesize');A.onchange=()=>{b(A),5<A.value&&(i(y,!1),i(z,!1))},y.onchange=()=>{return 5<A.value?(alert('Please select CIF or lower resolution before enabling this feature!'),void i(y,!1)):void(b(y),!y.checked&&(g(n),i(z,!1)))},z.onchange=()=>{return 5<A.value?(alert('Please select CIF or lower resolution before enabling this feature!'),void i(z,!1)):void(b(z),z.checked?(h(n),i(y,!0)):g(n))}});
</script>
</body>
</html>
)rawliteral";
//HTML-----------------------------------
//handle mjpeg stream from http://ip and http://ip/stream
static esp_err_t stream_handler(httpd_req_t *req){
camera_fb_t * fb = NULL;
esp_err_t res = ESP_OK;
size_t _jpg_buf_len = 0;
uint8_t * _jpg_buf = NULL;
char * part_buf[64];
dl_matrix3du_t *image_matrix = NULL;
bool detected = false;
int face_id = 0;
int64_t fr_start = 0;
int64_t fr_ready = 0;
int64_t fr_face = 0;
int64_t fr_recognize = 0;
int64_t fr_encode = 0;
static int64_t last_frame = 0;
if(!last_frame) {
last_frame = esp_timer_get_time();
}
res = httpd_resp_set_type(req, _STREAM_CONTENT_TYPE);
if(res != ESP_OK){
return res;
}
while(true){
detected = false;
face_id = 0;
fb = esp_camera_fb_get();
if (!fb) {
Serial.println("Camera capture failed");
res = ESP_FAIL;
} else {
fr_start = esp_timer_get_time();
fr_ready = fr_start;
fr_face = fr_start;
fr_encode = fr_start;
fr_recognize = fr_start;
if(!detection_enabled || fb->width > 400){
if(fb->format != PIXFORMAT_JPEG){
bool jpeg_converted = frame2jpg(fb, 80, &_jpg_buf, &_jpg_buf_len);
esp_camera_fb_return(fb);
fb = NULL;
if(!jpeg_converted){
Serial.println("JPEG compression failed");
res = ESP_FAIL;
}
} else {
_jpg_buf_len = fb->len;
_jpg_buf = fb->buf;
}
} else {
image_matrix = dl_matrix3du_alloc(1, fb->width, fb->height, 3);
if (!image_matrix) {
Serial.println("dl_matrix3du_alloc failed");
res = ESP_FAIL;
} else {
if(!fmt2rgb888(fb->buf, fb->len, fb->format, image_matrix->item)){
Serial.println("fmt2rgb888 failed");
res = ESP_FAIL;
} else {
fr_ready = esp_timer_get_time();
box_array_t *net_boxes = NULL;
if(detection_enabled){
net_boxes = face_detect(image_matrix, &mtmn_config);
}
/*
fr_face = esp_timer_get_time();
fr_recognize = fr_face;
if (net_boxes || fb->format != PIXFORMAT_JPEG){
if(net_boxes){
detected = true;
if(recognition_enabled){
face_id = run_face_recognition(image_matrix, net_boxes);
}
fr_recognize = esp_timer_get_time();
draw_face_boxes(image_matrix, net_boxes, face_id);
free(net_boxes->box);
free(net_boxes->landmark);
free(net_boxes);
}
if(!fmt2jpg(image_matrix->item, fb->width*fb->height*3, fb->width, fb->height, PIXFORMAT_RGB888, 90, &_jpg_buf, &_jpg_buf_len)){
Serial.println("fmt2jpg failed");
res = ESP_FAIL;
}
esp_camera_fb_return(fb);
fb = NULL;
} else {
_jpg_buf = fb->buf;
_jpg_buf_len = fb->len;
}
fr_encode = esp_timer_get_time();
*/
}
dl_matrix3du_free(image_matrix);
}
}
}
if(res == ESP_OK){
size_t hlen = snprintf((char *)part_buf, 64, _STREAM_PART, _jpg_buf_len);
res = httpd_resp_send_chunk(req, (const char *)part_buf, hlen);
}
if(res == ESP_OK){
res = httpd_resp_send_chunk(req, (const char *)_jpg_buf, _jpg_buf_len);
}
if(res == ESP_OK){
res = httpd_resp_send_chunk(req, _STREAM_BOUNDARY, strlen(_STREAM_BOUNDARY));
}
if(fb){
esp_camera_fb_return(fb);
fb = NULL;
_jpg_buf = NULL;
} else if(_jpg_buf){
free(_jpg_buf);
_jpg_buf = NULL;
}
if(res != ESP_OK){
break;
}
int64_t fr_end = esp_timer_get_time();
int64_t ready_time = (fr_ready - fr_start)/1000;
int64_t face_time = (fr_face - fr_ready)/1000;
int64_t recognize_time = (fr_recognize - fr_face)/1000;
int64_t encode_time = (fr_encode - fr_recognize)/1000;
int64_t process_time = (fr_encode - fr_start)/1000;
int64_t frame_time = fr_end - last_frame;
last_frame = fr_end;
frame_time /= 1000;
uint32_t avg_frame_time = ra_filter_run(&ra_filter, frame_time);
/* show frame rate
Serial.printf("MJPG: %uB %ums (%.1ffps), AVG: %ums (%.1ffps), %u+%u+%u+%u=%u %s%d\n",
(uint32_t)(_jpg_buf_len),
(uint32_t)frame_time, 1000.0 / (uint32_t)frame_time,
avg_frame_time, 1000.0 / avg_frame_time,
(uint32_t)ready_time, (uint32_t)face_time, (uint32_t)recognize_time, (uint32_t)encode_time, (uint32_t)process_time,
(detected)?"DETECTED ":"", face_id
);
*/
}
last_frame = 0;
return res;
}
//handle change option from http://ip
static esp_err_t cmd_handler(httpd_req_t *req){
char* buf;
size_t buf_len;
char variable[32] = {0,};
char value[32] = {0,};
buf_len = httpd_req_get_url_query_len(req) + 1;
if (buf_len > 1) {
buf = (char*)malloc(buf_len);
if(!buf){
httpd_resp_send_500(req);
return ESP_FAIL;
}
if (httpd_req_get_url_query_str(req, buf, buf_len) == ESP_OK) {
if (httpd_query_key_value(buf, "var", variable, sizeof(variable)) == ESP_OK &&
httpd_query_key_value(buf, "val", value, sizeof(value)) == ESP_OK) {
} else {
free(buf);
httpd_resp_send_404(req);
return ESP_FAIL;
}
} else {
free(buf);
httpd_resp_send_404(req);
return ESP_FAIL;
}
free(buf);
} else {
httpd_resp_send_404(req);
return ESP_FAIL;
}
int val = atoi(value);
sensor_t * s = esp_camera_sensor_get();
int res = 0;
Serial.println(val);
if(!strcmp(variable, "framesize")) {
if(s->pixformat == PIXFORMAT_JPEG) res = s->set_framesize(s, (framesize_t)val);
}
else if(!strcmp(variable, "quality")) res = s->set_quality(s, val);
else if(!strcmp(variable, "contrast")) res = s->set_contrast(s, val);
else if(!strcmp(variable, "brightness")) res = s->set_brightness(s, val);
else if(!strcmp(variable, "saturation")) res = s->set_saturation(s, val);
else if(!strcmp(variable, "gainceiling")) res = s->set_gainceiling(s, (gainceiling_t)val);
else if(!strcmp(variable, "colorbar")) res = s->set_colorbar(s, val);
else if(!strcmp(variable, "awb")) res = s->set_whitebal(s, val);
else if(!strcmp(variable, "agc")) res = s->set_gain_ctrl(s, val);
else if(!strcmp(variable, "aec")) res = s->set_exposure_ctrl(s, val);
else if(!strcmp(variable, "hmirror")) res = s->set_hmirror(s, val);
else if(!strcmp(variable, "vflip")) res = s->set_vflip(s, val);
else if(!strcmp(variable, "awb_gain")) res = s->set_awb_gain(s, val);
else if(!strcmp(variable, "agc_gain")) res = s->set_agc_gain(s, val);
else if(!strcmp(variable, "aec_value")) res = s->set_aec_value(s, val);
else if(!strcmp(variable, "aec2")) res = s->set_aec2(s, val);
else if(!strcmp(variable, "dcw")) res = s->set_dcw(s, val);
else if(!strcmp(variable, "bpc")) res = s->set_bpc(s, val);
else if(!strcmp(variable, "wpc")) res = s->set_wpc(s, val);
else if(!strcmp(variable, "raw_gma")) res = s->set_raw_gma(s, val);
else if(!strcmp(variable, "lenc")) res = s->set_lenc(s, val);
else if(!strcmp(variable, "special_effect")) res = s->set_special_effect(s, val);
else if(!strcmp(variable, "wb_mode")) res = s->set_wb_mode(s, val);
else if(!strcmp(variable, "ae_level")) res = s->set_ae_level(s, val);
/* disable face detect function
else if(!strcmp(variable, "face_detect")) {
detection_enabled = val;
if(!detection_enabled) {
recognition_enabled = 0;
}
}
else if(!strcmp(variable, "face_enroll")) is_enrolling = val;
else if(!strcmp(variable, "face_recognize")) {
recognition_enabled = val;
if(recognition_enabled){
detection_enabled = val;
}
}
*/
else {
res = -1;
}
if(res){
return httpd_resp_send_500(req);
}
httpd_resp_set_hdr(req, "Access-Control-Allow-Origin", "*");
return httpd_resp_send(req, NULL, 0);
}
static esp_err_t status_handler(httpd_req_t *req){
static char json_response[1024];
sensor_t * s = esp_camera_sensor_get();
char * p = json_response;
*p++ = '{';
p+=sprintf(p, "\"framesize\":%u,", s->status.framesize);
p+=sprintf(p, "\"quality\":%u,", s->status.quality);
p+=sprintf(p, "\"brightness\":%d,", s->status.brightness);
p+=sprintf(p, "\"contrast\":%d,", s->status.contrast);
p+=sprintf(p, "\"saturation\":%d,", s->status.saturation);
p+=sprintf(p, "\"special_effect\":%u,", s->status.special_effect);
p+=sprintf(p, "\"wb_mode\":%u,", s->status.wb_mode);
p+=sprintf(p, "\"awb\":%u,", s->status.awb);
p+=sprintf(p, "\"awb_gain\":%u,", s->status.awb_gain);
p+=sprintf(p, "\"aec\":%u,", s->status.aec);
p+=sprintf(p, "\"aec2\":%u,", s->status.aec2);
p+=sprintf(p, "\"ae_level\":%d,", s->status.ae_level);
p+=sprintf(p, "\"aec_value\":%u,", s->status.aec_value);
p+=sprintf(p, "\"agc\":%u,", s->status.agc);
p+=sprintf(p, "\"agc_gain\":%u,", s->status.agc_gain);
p+=sprintf(p, "\"gainceiling\":%u,", s->status.gainceiling);
p+=sprintf(p, "\"bpc\":%u,", s->status.bpc);
p+=sprintf(p, "\"wpc\":%u,", s->status.wpc);
p+=sprintf(p, "\"raw_gma\":%u,", s->status.raw_gma);
p+=sprintf(p, "\"lenc\":%u,", s->status.lenc);
p+=sprintf(p, "\"vflip\":%u,", s->status.vflip);
p+=sprintf(p, "\"hmirror\":%u,", s->status.hmirror);
p+=sprintf(p, "\"dcw\":%u,", s->status.dcw);
p+=sprintf(p, "\"colorbar\":%u,", s->status.colorbar);
/*
p+=sprintf(p, "\"face_detect\":%u,", detection_enabled);
p+=sprintf(p, "\"face_enroll\":%u,", is_enrolling);
p+=sprintf(p, "\"face_recognize\":%u", recognition_enabled);
*/
*p++ = '}';
*p++ = 0;
httpd_resp_set_type(req, "application/json");
httpd_resp_set_hdr(req, "Access-Control-Allow-Origin", "*");
return httpd_resp_send(req, json_response, strlen(json_response));
}
//handle get still pic from http://ip/capture
static esp_err_t capture_handler(httpd_req_t *req){
//Serial.print(String(req));
camera_fb_t * fb = NULL;
esp_err_t res = ESP_OK;
int64_t fr_start = esp_timer_get_time();
fb = esp_camera_fb_get(); //get picture from cam
if (!fb) {
Serial.println("Camera capture failed");
httpd_resp_send_500(req);
return ESP_FAIL;
}
httpd_resp_set_type(req, "image/jpeg");
httpd_resp_set_hdr(req, "Content-Disposition", "inline; filename=capture.jpg");
size_t out_len, out_width, out_height;
uint8_t * out_buf;
bool s;
bool detected = false;
int face_id = 0;
if(!detection_enabled || fb->width > 400){
Serial.println("lijn493");
size_t fb_len = 0;
if(fb->format == PIXFORMAT_JPEG){
Serial.println("lijn496 pixformatjpg for httpd send");
fb_len = fb->len;
res = httpd_resp_send(req, (const char *)fb->buf, fb->len);
} else {
jpg_chunking_t jchunk = {req, 0};
Serial.println(jchunk.len);//
res = frame2jpg_cb(fb, 80, jpg_encode_stream, &jchunk)?ESP_OK:ESP_FAIL;
Serial.println("frame2jpg lijn507");
httpd_resp_send_chunk(req, NULL, 0);
fb_len = jchunk.len;
}
Serial.println ("fb lengte=");
Serial.println ( fb->len );//jpg filesize
const char * path = "/capture.jpg";
fs::FS &fs = SD_MMC;
Serial.printf("Writing file: %s\n", path);
File file = fs.open(path, FILE_WRITE);
if(!file){
Serial.println("Failed to open file for SDwriting547");
}
else
{
file.write(fb->buf , fb->len); //payload , lengte vd payload
Serial.println("succes to open file for SDwriting552");
}
writeFile(SD_MMC, "/info.txt", "saved 1 jpgfile");
esp_camera_fb_return(fb);
int64_t fr_end = esp_timer_get_time();
//Serial.println(res);// is 1 if jpg convertion worked
Serial.printf("JPG: %uB %ums\n\n\n\n", (uint32_t)(fb_len), (uint32_t)((fr_end - fr_start)/1000));
return res;
}
}
static esp_err_t index_handler(httpd_req_t *req){
httpd_resp_set_type(req, "text/html");
Serial.printf("webpage loading \n");
return httpd_resp_send(req, (const char *)INDEX2_HTML, strlen(INDEX2_HTML));
}
//-----get Pic
void getPic(String filename){
Serial.println("Camera capturing:"+ filename);
camera_fb_t * fb = NULL;
//esp_err_t res = ESP_OK;
fb = esp_camera_fb_get(); //get picture from cam
if (!fb) {
Serial.println("Camera capture failed");
}
Serial.println ("fb lengte=");
Serial.println ( fb->len );//jpg filesize
fs::FS &fs = SD_MMC;
Serial.printf("Writing file: %s\n", filename);
File file = fs.open(filename, FILE_WRITE);
if(!file){
Serial.println("Failed to open file for SD");
} else {
file.write(fb->buf , fb->len); //payload , lengte vd payload
Serial.println("succes to open file for SD");
}
esp_camera_fb_return(fb);
}
void startCameraServer(){
httpd_config_t config = HTTPD_DEFAULT_CONFIG();
config.server_port = 80;//overwrite
httpd_uri_t index_uri = {
.uri = "/",
.method = HTTP_GET,
.handler = index_handler,
.user_ctx = NULL
};
httpd_uri_t status_uri = {
.uri = "/status",
.method = HTTP_GET,
.handler = status_handler,
.user_ctx = NULL
};
httpd_uri_t cmd_uri = {
.uri = "/control",
.method = HTTP_GET,
.handler = cmd_handler,
.user_ctx = NULL
};
httpd_uri_t capture_uri = {
.uri = "/capture",
.method = HTTP_GET,
.handler = capture_handler,
.user_ctx = NULL
};
httpd_uri_t stream_uri = {
.uri = "/stream",
.method = HTTP_GET,
.handler = stream_handler,
.user_ctx = NULL
};
Serial.printf("Starting web server on port: '%d'\n", config.server_port);
if (httpd_start(&camera_httpd, &config) == ESP_OK) {
httpd_register_uri_handler(camera_httpd, &index_uri);
httpd_register_uri_handler(camera_httpd, &cmd_uri);
httpd_register_uri_handler(camera_httpd, &status_uri);
httpd_register_uri_handler(camera_httpd, &capture_uri);
}
config.server_port = 9601;//overwrite
config.ctrl_port = 9601;
Serial.printf("Starting stream server on port: '%d'\n", config.server_port);
if (httpd_start(&stream_httpd, &config) == ESP_OK) {
httpd_register_uri_handler(stream_httpd, &stream_uri);
}
}
void setup() {
//1. setup camera param
//2. init camera
//3. init sd card
//4. Wifi connect
//5. start Camera Websever
WRITE_PERI_REG(RTC_CNTL_BROWN_OUT_REG, 0); //disable brownout detector
Serial.begin(115200);
Serial.println();
//1. setup camera param
camera_config_t config;
config.ledc_channel = LEDC_CHANNEL_0;
config.ledc_timer = LEDC_TIMER_0;
config.pin_d0 = Y2_GPIO_NUM;
config.pin_d1 = Y3_GPIO_NUM;
config.pin_d2 = Y4_GPIO_NUM;
config.pin_d3 = Y5_GPIO_NUM;
config.pin_d4 = Y6_GPIO_NUM;
config.pin_d5 = Y7_GPIO_NUM;
config.pin_d6 = Y8_GPIO_NUM;
config.pin_d7 = Y9_GPIO_NUM;
config.pin_xclk = XCLK_GPIO_NUM;
config.pin_pclk = PCLK_GPIO_NUM;
config.pin_vsync = VSYNC_GPIO_NUM;
config.pin_href = HREF_GPIO_NUM;
config.pin_sscb_sda = SIOD_GPIO_NUM;
config.pin_sscb_scl = SIOC_GPIO_NUM;
config.pin_pwdn = PWDN_GPIO_NUM;
config.pin_reset = RESET_GPIO_NUM;
config.xclk_freq_hz = 20000000;
//Format of the pixel data: PIXFORMAT_ + YUV422|GRAYSCALE|RGB565|JPEG
config.pixel_format = PIXFORMAT_JPEG;
// frame_size set as FRAMESIZE_ + QVGA|CIF|VGA|SVGA|XGA|SXGA|UXGA
config.frame_size = FRAMESIZE_SVGA;
//jpeg_quality set 10-63
config.jpeg_quality = 10;
// fb_count set 2 for jpg
config.fb_count = 2;
//2. init camera
esp_err_t err = esp_camera_init(&config);
if (err == ESP_OK) {
Serial.println("Camera init passed");
// setup stream ------------------------
sensor_t * s = esp_camera_sensor_get();
int res = 0;
//res=s->set_brightness(s, 0);//-2 to 2
//res=s->set_contrast(s, 0);//-2 to 2
//res=s->set_saturation(s, 0);//-2 to 2
//res=s->set_special_effect(s, 0);//0=no effect,1 to 6=negative to sepia
//res=s->set_whitebal(s, 1);//disabled=0,enabled=1
//res=s->set_awb_gain(s, 1);//disabled=0,enabled=1
//res=s->set_wb_mode(s, 0);//0=default,1 to 4=sunny to home
//res=s->set_lenc(s, 1);//disabled=0,enabled=1
//res=s->set_hmirror(s, 0);//disabled=0,enabled=1
res=s->set_vflip(s, 1);//disabled=0,enabled=1
} else {
Serial.printf("Camera init failed with error 0x%x", err);
return;
}
//3. init sd card
if(!SD_MMC.begin()){
Serial.println("Card Mount Failed");
return;
} else {
uint8_t cardType = SD_MMC.cardType();
if(cardType == CARD_NONE){
Serial.println("No SD_MMC card attached");
return;
}
Serial.print("SD_MMC Card Type: ");
if(cardType == CARD_MMC){
Serial.println("MMC");
} else if(cardType == CARD_SD){
Serial.println("SDSC");
} else if(cardType == CARD_SDHC){
Serial.println("SDHC");
} else {
Serial.println("UNKNOWN");
}
uint64_t cardSize = SD_MMC.cardSize() / (1024 * 1024);
Serial.printf("SD_MMC Card Size: %lluMB\n", cardSize);
}
/*3.1 Sd card testing.....
listDir(SD_MMC, "/", 0);
createDir(SD_MMC, "/mydir");
listDir(SD_MMC, "/", 0);
removeDir(SD_MMC, "/mydir");
listDir(SD_MMC, "/", 2);
writeFile(SD_MMC, "/hello.txt", "Hello ");
appendFile(SD_MMC, "/hello.txt", "World!\n");
readFile(SD_MMC, "/hello.txt");
deleteFile(SD_MMC, "/foo.txt");
renameFile(SD_MMC, "/hello.txt", "/foo.txt");
readFile(SD_MMC, "/foo.txt");
//SD card speed testing
testFileIO(SD_MMC, "/test.txt");
//Show SD card infomation
Serial.printf("Total space: %lluMB\n", SD_MMC.totalBytes() / (1024 * 1024));
Serial.printf("Used space: %lluMB\n", SD_MMC.usedBytes() / (1024 * 1024));
*/
//4. Wifi connect
WiFi.begin(ssid, password);
while (WiFi.status() != WL_CONNECTED) {
delay(500);
Serial.print(".");
}
Serial.println("");
Serial.println("WiFi connected");
//5. start Camera Websever
Serial.println("startCameraServer");
startCameraServer();
//Show server info
Serial.print("Camera Ready! Use 'http://");
Serial.print(WiFi.localIP());
Serial.println("' to connect , de stream zit op een andere poortkanaal 9601 ");
Serial.print("stream Ready! Use 'http://");
Serial.print(WiFi.localIP());
Serial.println(":9601/stream ");
Serial.print("image Ready! Use 'http://");
Serial.print(WiFi.localIP());
Serial.println("/capture ");
}
int i=0; //pic count as filename
void loop() {
// put your main code here, to run repeatedly:
if (digitalRead(13)==1){
i++;
getPic("/pic" + String(i) +".jpg");
delay(5000);
}
//
delay(1000);
}

Leave a Comment

發佈留言必須填寫的電子郵件地址不會公開。 必填欄位標示為 *