Personal Object Detection End-point

Deploying object detection on Flask web server

In this post, I will detail out the steps I take to run an object detection model (Yolo v3) on a web server (Flask) then how to interface with it from a website using Javascript

To deploy your Yolo v3 model on flask, follow this very nice tutorial until he jumped into his obsession with dockers and make it a full deployable solution.

Yolo v3 on Flask

If you are into that kind of thing, you can follow him through, but we are not here to create full produciton level code right? The hackier the better, it's called hackathon for a reason (and not like... productionathon.... which is less catchy and I'm pretty sure that's the main reason :D )

And congratulation, you've deployed your yolo v3 model on the web using flask (wohoo)

Thank you for reading this post that just basically points you to another source :D

But wait, you don't know what to do with the server? It's like.. running but don't know how to send data to it without using curl? Well if you do, good job and this post is useless for you, if you do, read on

Before showing you the code to perform the above, lemme explain what it is trying to do

BUT EVEN BEFORE I EXPLAIN THAT

Small code change from the tutorial above

There's this function that encodes the image to byte array? The one shown here

def image_to_byte_array(image: Image): imgByteArr = io.BytesIO() image.save(imgByteArr, format="JPEG") imgByteArr = imgByteArr.getvalue() return imgByteArr

Yes that one, change the final line to so:

def image_to_byte_array(image: Image): imgByteArr = io.BytesIO() image.save(imgByteArr, format="JPEG") imgByteArr = imgByteArr.getvalue() return base64.b64encode(imgByteArr)

Okay, back to what I was talking about

Basically... You use get image input using the input tag with attribute file. You then get the file using javascript, convert it to a blob then send it to the flask server using Javascript

In other words

HTML

<input type="file" name="file" id="file" /> <img src=""alt="" id="imagePreview" /> <button id="upload-file-btn">Upload</button> <img src=""alt="" id="received-image" />

Javascript

//If file change, read load file to img $("#file").change(() => { readURL($("#file")[0]); }); $(function () { $('#upload-file-btn').click(function () { var form_data = new FormData(); console.log(form_data); var img_blob = $("#imagePreview")[0].src; img_blob = base64ToBlob(img_blob); form_data.append('image', img_blob, "clientImage.jpg"); console.log(form_data); $.ajax({ type: 'POST', url: 'https://192.168.1.69:5000/api/test', data: form_data, contentType: false, cache: false, processData: false, success: function (data) { console.log('Success!'); console.log(data); $('#received-image').attr('src', `data:image/jpeg;base64,${data}`); }, }); }); }); function readURL(input) { if (input.files && input.files[0]) { var reader = new FileReader(); reader.onload = function (e) { $('#imagePreview') .attr('src', e.target.result) }; reader.readAsDataURL(input.files[0]); } } function base64ToBlob(imgInBase64) { var data = imgInBase64.split(";base64,"); var dataType = data[0]; var base64Arr = data[1]; console.log("Data before encoding: " + imgInBase64); const byteCharacter = atob(base64Arr); console.log("Data after encoding: " + byteCharacter); const byteNumber = new Array(byteCharacter.length); for (let i = 0; i < byteCharacter.length; ++i) { byteNumber[i] = byteCharacter.charCodeAt(i); } const byteArray = new Uint8Array(byteNumber); var blob = new Blob([byteArray], { type: dataType }); console.log(blob); return blob; }
Change the destination ip address to the ip address of your flask server

The above code is pretty self-explanatory (imo) but for the 5% out there (hopefully, or my code is just bad TT)

*deep breath* The javascript code watch for change in the input element, if there is a change (user selected an image) the script will load the image to an image. When the user press the upload button, the script will take the base64 source of the image from the src component of the img element previously loaded and convert it to a blob (it's a thing, don't worry about it) the blob is then sent to the flask server on the ip address given (don't forget to change that to yours). Then, when the script receive the response from the server, put the response data to an img element to display the returned image

But wait I hear, you want the user to take picture using their camera? Well

Here you go!

Take picture in website

Incase you want the bad code that I wrote it, you can have it to you masochist.

HTML

<div id="screenshot"> <video autoplay></video> <img src="" alt="" id="imagePreview" height="500px" width="auto"> <button class="capture-button">Capture Video Button</button> <button id="screenshot-button">Screen Shot Button</button> <button id="stop-button">Stop</button> <select name="" id=""></select> <canvas style="display:none" id="canvas"></canvas> </div> <form id="upload-file" method="post" enctype="multipart/form-data"> <fieldset> <label for="file">Select a file</label> <input type="file" name="image" id="file" accept="image/gif, image/jpeg, image/png"> </fieldset> <fieldset> <button id="upload-file-btn" type="button">Upload</button> </fieldset> </form> <h1 id="ItemPreview"></h1> <img src="" alt="" id="received-image">

Javascript

const captureVideoButton = document.querySelector("#screenshot .capture-button"); const screenshotButton = document.querySelector("#screenshot-button"); const img = document.querySelector("#screenshot img"); const video = document.querySelector("#screenshot video"); const videoSelect = document.querySelector('#screenshot select') const canvas = document.getElementById("canvas"); if (hasGetUserMedia()) { navigator.mediaDevices.enumerateDevices().then(gotDevices).then(getStream).catch(handleError); videoSelect.onchange = getStream; function gotDevices(deviceInfos) { for (let i = 0; i !== deviceInfos.length; ++i) { const deviceInfo = deviceInfos[i]; const option = document.createElement('option'); option.value = deviceInfo.deviceId; if (deviceInfo.kind == 'videoinput') { option.text = deviceInfo.label || 'camera ' + (videoSelect.length + 1); videoSelect.appendChild(option); } else { console.log('Found another kind of device: ', deviceInfo); } } } function getStream() { if (window.stream) { window.stream.getTracks().forEach( function (track) { track.stop(); }); } const constraints = { video: { deviceId: { exact: videoSelect.value } } }; captureVideoButton.onclick = function () { navigator.mediaDevices.getUserMedia(constraints).then(gotStream).catch(handleError); } } function gotStream(stream) { window.stream = stream; video.srcObject = stream; } screenshotButton.onclick = video.onclick = function () { canvas.width = video.videoWidth; canvas.height = video.videoHeight; canvas.getContext('2d').drawImage(video, 0, 0); img.src = canvas.toDataURL('image/jpeg'); $("#file")[0] }; } else { alert("getUserMedia() is not suppoerted by your browser"); } function hasGetUserMedia() { return !!(navigator.mediaDevices && navigator.mediaDevices.getUserMedia); } function handleSuccess(stream) { screenshotButton.disabled = false; video.srcObject = stream; } function handleError(error) { console.log("Error: ", error); }

The above code directly above only contains the javascript to take picture and put it to an img elem (i think) so you will still need the all the javascripts in this post to make the thing work. The directly above html is complete though, so just that html will suffice

With that, you are done! yay...

Please like, subscribe and yeah.... do.... something...